I like it to be in the 70's before caulking. I always leave the blue tape up as long as the instructions for the caulking says it takes to cure and develop a film. Then play it by ear, if pulling the tape is bringing caulking with it, I wait longer.
But I rarely caulk any more. What I do now is the seams get Eternabond tape covering them that lasts for years and years whereas caulking is supposed to be checked every 6 months and repaired as required...which is often unless everything goes perfectly during application.
So I just avoid that caulking business all together. Some of the E-bond tape seams have been fine for years so far: