Was the towing pin in place? If not that would allow the dolly to wander.
There should not be any play in the drawbar pivot, there isn't on ours. I almost have to drive the pin in our out. That would also cause problems.
I bought a locally made dolly some years ago, it would shake you all over the road unless loaded, once loaded it was OK. Found that the alignment was off when not loaded. Sold it to a friend and got the KK-460.
2009 45' Magna 630 w/Cummins ISX 650 HP/1950 Lbs Ft
Charter Good Sam Lifetime Member, FMCA, SKP
RV'ing since 1957, NRA Benefactor Life, towing '14 CR-V