98 Jimmy Problems

Got in it this morning and barely starts it idle up to about 4,000 rpm's for about 5-10 seconds then dropped back down to 500 drove about 10 mins. to work then tried to start it back up. Didn't think it was going to start finally started and it got bad enough it ejected the cd and when it started it died about as fast. Tried it again and gave it a little bit of gas and started and idled fine could this be a battery problem its seems to getting worse to start the colder it gets terminals seems to look good.

Do a battery load test and alternator out put test make sure all the cable connections and grounds are good.