my brothers 4runner used to do that...we took the wires off the starter and just cleaned them up a bit to make sure the connection was good and he never had another problem... my buddy also had the same problem, but his was the starter solenoid...we had to beat it with a bar every once in a while on our campin trip to keep playin in the mtns.
just start trouble shootin, man. if it doesnt start, hit the solenoid real quick and hard with a bar or something and see if that does anything. then clean the wires up a bit. if you disconnect it, you can take it to autozone, schucks, checkers etc. and usually get it tested for free. good luck