93 Tbird rear window motor

My rear view mirrior motor works going left and right but not up or down. Any ideas on whats wrong?

Do you mean your side mirrors? Its probably the switch.

Yes its the side
How would I go about replacing the switch?

there are certain occasions when mine dont go up and down either and I just push them in the direction that I want them to go in and they work again. Almost like they came off a track or something.