The fuse is a protection. It's best to put it between the item you want to protect and the source voltage.
There are different types of fuses... most commonly are slow blow, and quick blow fuses.
The slow blow can handle a rise in current, often called a surge, and not blow. This type of fuse would be good to use on the lamps. As the lamp is turned on there is a momnetary spike of current. The fuse would not react to the spike. If the lamp connector shorted, as example, then the fuse would blow and remove the voltage, protecting the devices.
Now the relay that you'd use has a failry small coil of wire in that's part of an electro-magnet. If you use a quick blow fuse the relay is fully protected in case of failure, as is the rest of the electrics. A slow blow fuse would work but maybe not before the relay were destroyed.
