Strange RPM problems...

Recently I've been having some interesting issues crop up with my '97 3.5L. It starts fine, idles fine, but when I start to drive and let off the accelerator, the RPMs drop very suddenly (under 500, idles around 1000 in park, 800 in drive). This isn't generally a problem, unless the heat is on (it's been about 20 degrees F around here lately) and then the engine starts to die, lights dim, heat blower slows, etc...a little gas and it smoothes out, untill you depress the accelerator again. Any ideas on what/where to start? No check engine light, but I did have the voltage light come on twice in the past two weeks, one time I drove through a rather large puddle, assumed a short or something along that line, they are no longer on during this problem. Any help is greatly appreciated.

have your charging system checked out for voltage and amperage. Your alternator could be getting weak not being able to push out the necessary power to keep all your electrical accessories on. Also, have your battery checked out as well; just as a rule of thumb, if your battery is more than 3 years old, it's time to go on and replace it.

also check the egr valve you can tell if the valve is open by either momentarily touching egr tubes at the rear of the intake plenum or spray water on them if tubes steam or boil water right away valve is open now don't get me wrong these tubes do get fairly warm but not enough to steam water

good suggestion, thunder, I didn't think about that.