97 Rodeo starter removal?

I see on some other Isuzu posts that the Exhaust manifold needs to be off to get the starter out. Is this true on a 97 Rodeo 6 cyl. 4X4.
The problem is sometimes it clicks then starts fine for awhile, put in new battery still happens? Any other tips?
Thanks!
Sorry wrong room. I'l put it in the rodeo room.