Conventional wisdom is you need 6" clearance for pavement and 8" or more clearance for off road.
I had about 5" clearance, but I also had dings on the top of the bed on both sides. Didn't hurt the trailer, but definitely bent the top of the bed. Caused by turning while crosssing a ditch or dip.
If you stay on the pavement and don't turn while crossing dips or ditches, then 6" should be good. But lots of streets have a drainage dip between the roadway and the service station or shopping center lot, so turning into one of those places is when I needed more than 5".
If you don't have at least 6" while the trailer is level front to rear, then you must either raise the trailer or lower the truck. I had one trailer that had to be raised by "flipping the axles" so it would ride level behind a 4x2 SuperDuty. You don't literally turn the axles over, but you move the spring perches from under the axle to over the axle. That raised the trailer about 3 to 4 inches. And caused us to have to buy one step more so old folks could climb into the 5er.
If your tow vehicle is a 4x4, you may have 4" blocks in the rear suspension. On a SuperDuty, you can change those blocks for 2" blocks and lower the back of the truck about 2". Experience says you cannot have less than 2' blocks in a SuperDuty 4x4 because of driveline alignment, so you can't lower the truck to make it the same height as a 4x2.
Grumpy ole man with over 50 years towing experience. Now my heaviest trailer is a 7,000-pound enclosed cargo trailer, RV is a 5,600 pound Skyline Nomad Joey 196S, and my tow vehicle is a 2012 F-150 EcoBoost SuperCrew.