My 95 jetta takes awhile to actually turn over. It seems when i havent started or drove it for a long time it starts up with a delay. But if i drive it around for awhile, turn it off and then start it up again, it starts perfectly without any hickups. Could that be just a battery issue? Or is it the starter? Alternater?

That was a common problem on the automatics. It sounds like the starter to me.

Well, mine is a 5-speed. Does that matter?

No, it's just more common on the automatics. Make sure all the connections are corrosion free & that it doesn't seem to be in the igition switch before you go and replace the starter. One check is when it delays get a test light & see if there is 12 Volts getting to the starter solenoid (small red/black wire) when you turn to the start position. If it's getting 12V & the starter is delaying, it's the starter.