Was the towing pin in place? If not that would allow the dolly to wander.
There should not be any play in the drawbar pivot, there isn't on ours. I almost have to drive the pin in our out. That would also cause problems.
I bought a locally made dolly some years ago, it would shake you all over the road unless loaded, once loaded it was OK. Found that the alignment was off when not loaded. Sold it to a friend and got the KK-460.
2002 Newmar Dutch Star 4090 ISC 350/1050 with Banks Kit, now at least 435/1200
Charter Good Sam Lifetime Member, FMCA f47302s, SKP 118205
RV'ing since 1957, NRA Benefactor Life Member