2000 Avalon Rear Speaker Removal

I got the speaker loose from the back deck but I can't get the assembly out of the back deck covering. It looks like I have to remove the back deck covering. How do I do this and then how do I replace it after I have replaced the rear speaker? If I really don't have to remove the rear deck covering please advise. Thank you.