should I replace my starter immediately?

over the last 2-3 days, when I turn the key to start my 96 3.8, it doesnt click, it doesnt do anything for about 1/2 to 1 second, then it turns over. Yesterday before work it turned over 11 times before it started and it usually only takes 7-8. Last night when I got off work, it was freezing out, and there was ice on my car, and I tried starting it, but it didnt do anything at all for about 4-5 times. I opened up my door and yelled a profanity because I thought my starter was toast, and it was midnight and 28 degrees out, but at the same time I was trying it again and it turned over. This has never ever ever happened to me before. Is my starter going bad, should I have it replaced immediately? or is it possibly something else?

Put a load test on your battery and see if it can hold at least 9.6 volts under load for at least 10 seconds. If it can't, replace it. The voltage should stay fairly steady as you test it. If it drops slowly in the test and doesn't drop below 9.6v it should be fine for awhile. Or go to your local parts house and have them test it for you.