Depending on the age of this switch, the light-bulb may be just that: An incandescent bulb (i.e., not an LED light).
On my rig, the light-bulb can be accessed by gently pulling the rocker-switch cover towards you. With luck, the pivot-point of the rocker switch will release....and fall off in your hands.
But I digress. When my bulb burned out, I never replaced it. Although minor, it is still yet another current-draw on the battery.
Robi, Dale, and "The Duchess" (lap kitty)
2004 Monaco Diplomat 40' PST
Tow: '08 Buick Lucerne