thermostat housing removal

I tried changing my thermostat this past weekend and had trouble removing the thermostat housing. I believe it has the silicone gasket and it feels like it is stuck on there pretty good. Does anyone know how I should remove the housing??? It was pretty frustrating since I was expecting a simple job.

Any help would be greatly appreciated.

Thanks,
Eric