I towed travel trailers and 5th wheels for years and have experienced the same thing you describe. The best thing that cured it for me was getting a really big truck (crew cab dually). I always found myself watching the rear view mirror for trucks overtaking me in the left lane. You are in the right lane, correct?
Years ago, the cabover semi's were the worst. You don't see many of those on the road anymore.
Also, if you drive in back of big-rigs, you can get caught in a little vortex thingy that'll swirl you around. Nothing to worry about, just back off a little.