Take the EGR off and verify that it is seated shut and sealed (spray some cleaner on the pintle and make sure it doesn't disappear instantly), then reinstall but don't hook it up, see if it'll start...it's gonna code for EGR but should start if it's getting hung open on startup.
I think it does a self-test on startup that cycles the valve to check for problems. If it's getting stuck open, it could act like this. What happens if you try to start with partial throttle applied? That would overcome the problem if it was related to a vacuum leak or bad cold idle setup. Ususally if the EGR or IACV are sticking, they will start fine like this, just won't idle when you release the throttle.
No telling though...a bad MAF will act like this too. But won't start w/throttle applied.