This place is also close, but spensive
RV Parking
I drive by Churchill Downs every day. As Gocoffeer state, there are sites at the Kentucky Fair and Expo Center (which is not much more than a parking lot). There is a KOA in Jeffersonville, IN which is right across the river, the KOA in Shepherdsville and Grandma's in Shepherdsville. On Derby Day, you'll want to find a shuttle in and out as traffic is a complete nightmare.
I leave Louisville on Derby Day
Anything close to Louisville will be hard to get into on Derby Day if you don't already have reservations.