The back of my evap core was also covered with oil from a leak in the air conditioning system somewhere above the core. The front was clean so I decided to try to use some engine degeaser and acetone to clean the back. That was a mistake, it was a dirty smelly job, I'm not sure if it helped, and now my car smells like solvent. Hopefully driving around with the heater on and the windows open for a few days will fix the solvent smell.
Sometimes the "easy" rode turns out harder... I think removing the evap core it definitely the way to go when it has any kind of oil on it.
Going to wait until winter though and see if my cleaning job did and good, if not I'll take it to a shop and get them to take the evap core out. Not going to bother replacing it and getting the AC running again, I've been told that's a $1000 job and I don't really care about AC that much.