1994 Escort CD Player problems

I have a 1994 escort and i am trying to install a Cd player into it. I have the wiring correct (i think!) because the player turns on and i hear the radio etc...the problem is that if i turn off the key i.e the unit looses power, when i turn it on again the unit powers up but no sound through speakers. If I remove the block connector with the speaker connections and replace it again then the sound comes back. So every time I turn the key I have to remove the speaker connection and replace it again to get sound...Any suggestions???.