It is not supposed to stay in. It is a momentary push button normally open.
If you are saying it will stay in OD if you hold the button in then no there is no fuse or solenoid to cause your problem as they would both be in working condition then.
The computer starts out in the OD mode and the light will be out. When you push the button in and release it then the light will be on and you will be in third gear for going down hills or anything else you would need third gear for.
Is the light on when you start out after starting the engine?
If so you might have a bad PCM Computer.
/