I have the same vehicle as a toad, only AWD. My first trip to Myrtle Beach, I had the same thing happen.
Instead of putting it in the accessory position, I turn the key back another click toward the off position.
Since I did that, the battery has been fine.
I have the Nav radio, so my guess is, even with the radio off, it's still pulling juice from the battery.
09 Newmar Ventana 3942 (bunkbeds for the munchkins)
08 Ford Taurus X AWD Toad
69 Road Runner Convertible 528 Hemi 4 Speed