Window Chalk Removal?

I have some of that Window Chalk (the stuff people use to write on cars for graduation, weddings, etc.) on the window seal of my car and can not get it off. Moon Shine window cleaner took it right off the glass but not the rubber seal thing that prevetns water from leaking in that goes around the outside of the window. Is there a certain cleaner or something that will take it off without damaging anything or ruining the seal?