Funny story! I had something similar happen in my garage shop years ago. One winter I started my air compressor to fill up some tires and it made a heck of a racket like a bearing in the motor had failed. I looked down and shredded black oil sunflower seeds were blowing out the end of the motor. It looked like a wood chipper running. We had a bird feeder in the front yard and guess what we filled it with?
One other tip is is described by Ben Franklin's famous saying - "An ounce of prevention is worth a pound of cure." Mice mark their runs with urine, so if you can keep them out to start with, there won't be a trail of - ahem, "crumbs", for others to follow.
For those still following the thread, using the block style bait in a bait station is prefered over the bags full of pellets for just the reason you state. They have to chew on the blocks which makes it less likely they'll just carry it off and hoard it like they can with the pellets.
To add another funny story, a couple weeks ago I was working in my shop when something caught my eye. Leaning against one of my cabinets was the 5 foot tall remains of an old fencepost that rotted off at the ground and sleeping on top, a little deer mouse. I'd been walking back and forth past him for quite some time and my movement and noise didn't seem to bother him.
On a more serious note, as cute as the little guy is in the picture, this species of mouse is known for carrying the Hantavirus. Although rare, the nasty form of the virus can sicken and kill humans. If you are cleaning up mouse poop, don't just sweep it up or vacuum. Spray it first with a bleach solution and let it sit for a few minutes. Wear gloves and wipe or pick it up with paper towels. The virus is excreted in the urine, feces and saliva of infected mice. If you find droppings on your counters or in your cabinets remember, you see the poop, but you don't see the urine trails so disinfect the whole area even if there were a few droppings.