Don't worry about wear from walking on the roof - normal foot traffic won't bother EDPM a bit. It's just sharp edges and continuous wear points that are a concern.
I used to carry a Port-A-Bote on a roof rack I constructed for it. Used it on two Rvs (one 5W, one MH) without a problem. A Port-A-Bote isn't heavy - only about 60 Lbs plus the oars I had strapped on with it - so my rack was 2x2 cross pieces sitting on 2x4 blocks at each end. Those I attached to the roof with lag screws and caulked well. When I removed the rack, I merely reseated short lag screws in the holes and caulked over it, basically leaving a rounded nubbin of caulk. A good rubber roof caulk/sealer like C10 FLo Seal did the job nicely.
Those caulked bolt heads are no worse than any of the other dozens of protruding screws and bolts holding things onto your roof.
Former owner of 2004 American Tradition and several other RVs
Home is in the Ocala Nat'l Forest near Ocala, FL
Summers in Black Mountain, NC