I think you're thinking of Hwy 550 from Durango to Ouray. That's the steep part and we love it!
Hwy 550 south from Durango into New Mexico is not scary. You mentioned Hwy 50 - a different highway and is not scary but it goes west to east and it would begin at Montrose, CO.
So... from Bryce to Durango I'd recommend heading south on UT89 to Kanab, UT and on to Page, AZ then pick up AZ 98 to AZ 160 heading east to CO 491 north to Cortez, CO then east on 160 to Durango.... a easy drive.
You could then work in a stop at Monument Valley and Mesa Verde Nat'l Park on this route.
From Bryce you could also take Hwy 12 east but then, you probably wouldn't like it either.
- beautiful scenery though.