Any of you had this problem?

when i start my 92 everything's fine but then i hear the compressor cause you can hear that hissing noise in the dash when u turn off the airconditioning .. the clutch on the compressor would jerk like it's trying to come on and the switch on the dash is on the off position ...i turn on the switch and it sorts it'self out... any ideas on how to fix this problem switch could be worn out? i wanna pin point the problem before i go ripping out stuff :)