Sometimes dirt and debris gets crammed into the mounting hole and removing the ball can be quite difficult. I would use a spray lubricant to soak the ball to help release it from the mounting hole.
Which bed mount is this ball hitch.....??
Hide-A-Goose (handle/lever in drivers side wheel well that has to be pulled and locked back to release pin)
B&W also uses a handle/lever to release pin..drivers wheel well
Other brands made...same hidden handle/lever needs pulled to release pin
Sometimes they get installed backwards and handle ends up in passenger side wheel well
Linear Mode
