Thudding noise whenever AC is turned on

Whenever I turn on the AC I get a loud thudding noise coming from the passenger side of the engine compartment. It starts as soon as I press the AC button and goes on until a few seconds after I shut the AC off. I am guessing it is the AC compressor. Any other ideas? If it is the compressor, how likely is it that it will go out completely anytime soon? The car has a lot of miles so I don't want to put too much money into it. I don't use the AC much. And if it does go out, will I have any other problems other than the AC not working? Thanks!