That's good. If you would be getting a whirrr, I would be warning you to replace the starter immediately so you wouldn't grind the teeth off the flywheel in one spot and have a bigger repair on your hands.
If I was to guess at your problem, I would say the solenoid on top of the starter or the relay. A little testing with a multi-meter or test light may help you narrow it down. Also check what aleekat said. Good luck.