Heater problems with '01 Cougar

I just noticed a few days ago that the heater in my car isn't working like it normally does. When I am at a stop light, the air gets cold, and then it takes a mile or so going at least 35-40mph before the heater air turns hot. What can I do to fix this? Also, my coolant light is on, meaning that my coolant is probably low. That is the first thing that I am going to fix to see if that will help. If it's the water pump (some have suggested this as the problem) what other signs or problems should I be having with the car? My car isn't overheating. Thanks.
:confused:

if it was your water pump, your car WOULD overheat, plain and simple,

if your coolant is low enough to make the light come on, thats probably your problem