It depends on the make and style of rocker switches. Many can have the light replaced with a seed bulb
. Some can be replaced by just 'popping' the switch cover off and the bulb is accessed without removing the switch from the dash. Others need to be disassembled to replace the bulb, with step one being remove the switch from the dash and disconnect wiring from the switch. You might look up your switch type and check with the manufacturer.
I've removed bulbs from some rocker switches and replaced with an LED and resistor to make them longer lasting and reduce heat build-up.