Heating on when thermostat is removed

I was recently redecorating my hallway and thought I might as well remove my thermostat and just leave the back plate on the wall instead of painting around it but after removing the thermostat I noticed that my radiators all started getting warm? I don't know much about heating systems but I know the thermostat is basically just a switch that calls for heat when the room is below a set temperature or will not call for heat when it's above the room is above the set temperature. So my question is why would my heating system turn on when I removed the thermostat faceplate? Surly with the thermostat removed there is no chance the circuit can be completed and a call for heat could be received.