Not 100% sure but I think the weatherstrip holds the window. There is probably some adhesive but I'm not sure I'd trust it.
I'm getting ready to replace my weatherstrip for the same reason.
This company should have what you need. Order the sample kit to be sure you're getting the correct size.
https://www.pellandent.com/RV-Window-Seal