As a matter of fact I really don't know of any other roads from West Yellowstone that would be useful to travel to Colter Bay. To go US 20 or US 191 take you in the wrong direction, your only route is through Yellowstone and down the J.D. Rockefeller Parkway to Grand Teton NP. In YNP, when you get to Madison Junction, turn right (S) towards Old Faithful. This will take you over Craig's Pass then down to Yellowstone Lake. It's not a bad road, speed is 45 mph in the park. On long upgrades, downshift to bring engine RPMs back into best power/torque range. At West Thumb Junction, turn right again (S) towards Lewis Lake and the South Entrance. West Yellowstone to Colter Bay is 87 miles, about 2 hr and 10 minutes with no stops or major BHM jams. (Big Hairy Mammal)
When I was there 2 years ago in September, Craig's Pass was closed for snow plowing the day we wished to leave West Yellowstone, we had to go north at Madison to Norris to Canyon to West Thumb. It added 28 miles to the trip.