Upper brake light

My upper brake light is out and went to pull it out but I am uncertain of how to get it out. I see the outside trim panel that is held on by 4 bolts. I removed those bolts and can't get the panel off. I used a lot of force but it doesn't seem to move and I don't want to break anything. How do I remove that panel so I can get to the light?