I don't find any need to separately secure the rack to the ladder. I carry a folded ladder hanging on my installed ladder, on a rack similar to yours. I just hang the rack, then the folded ladder, then bungee the whole thing to the installed ladder as it sits. I only use three bungees, just to prevent wobbling. The folded ladder weighs 40Lbs, so it doesn't tend to bounce much, and I haven't had any problem in the three years I've been doing it. Your chairs will be much lighter, easier to bungee, and so should be no factor.
John & Diane, Fulltimers. RVM103 NHSO
On the road since June '12 with Lincoln, the guard cat.
2002 Dutch Star 40, Freightliner, Cat 3126, 2004 Element