Butyl tape is the favored caulking to reseal windows. It does require removing the window. Home Depot, Lowes and many hardware stores carry the tape at a much better price than RV dealers. It is also advisable to seal across the top and down the sides of the window about 2" with polturethane caulk.
When removing the window be sure to watch for any spacers that may be present and replace them in the same position when the window is reinstalled.
Not all that hard a project but much easier with two people.
Harold & Linda
2009 CT coachworks siena 35V
W22 Workhorse 8.1L. Explorer Sport toad,