This is a classic sign of an Idle Air Control (IAC) valve problem.
The engine computer uses the IAC to adjust how much air the engine gets in order to maintain the correct idle speed. If the valve is dirty or gets stuck, the engine can't idle properly. As you've discovered, manually pushing the gas pedal supplies the needed air to keep the engine running.
The IAC is attached with two bolts to the throttle body (look for what moves under the hood when someone presses the gas pedal), and has a connector with two wires attached to it. The valve is easy to remove, and cleans up nicely with a quality fuel injector cleaner (Berryman's is my favorite). This only takes 5 or 10 minutes to do this and costs almost nothing. See how to do it here:
http://leckemby.net/windstar/windstar02.html
You can also replace the valve, which isn't terribly expensive.
I hope this helps!