I have a 1998 Contour 24V with 92k on it. The car runs and drives great but there is an ugly grinding noise coming from somwhere by the belt area. I did take the belt off and started the car and the noise was gone. Is there a way to figue out which compononet is making all the noise???

which belt? accessory belt or the serpentine belt?

serpentine belt

Any advice on how to remove the serpentine belt? Is the idler pulley spring loaded?

Yes.