If you have a special interest group in mind try Googling that group with the word 'Quartzsite' in it and perhaps something will pop up. There are hundreds of groups scattered all around the desert - some of friend get-togethers, many are specific RV manufacturer brands (look on their owner forums for info), there are Jeeping groups, ATV groups and yes, even nudist groups - waaaaay out in the boonies.
Here's a bloggers write-up on various groups you might see:
Quartzsite, Arizona - The RV Gathering Place | Roads Less Traveled
The majority of groups meet up in January, specifically mid-January for the 'RV Show' time. That's when the desert is the fullest.