The load of being in gear could take the idle speed down to the threshold where the noise can be heard.....what kind of noise is it (chatter, clicking, sqeal, hiss, etc)?
Could just be the belt itself too...they get stuff imbedded in them that produces weird sounds at low rpm's...little flecks of rock and other grit stuck in the ribbed side will make light clicking chatter....when it idles, does the belt tensioner arm stay fairly still, or does it hop around excessively? The arm will usually start to jump around pretty wildly when for example the bearing in the tensioner pulley starts going bad, and you can also look straight down at the belt and make sure there is no tip-in where the arm is making the belt run slightly off-track, this will produce a constant hiss or squeal at low speeds...in that case, the tensioner arm itself would need replacement.