Good observation Curtis!... I worked on one of the early hybrid car projects at the University I was attending back in the late 60's. There are actually two other key points that do make the hybrid car viable:
1. Because you change the way the vehicle is powered(electric motor), you can now take advantage of regenerative braking which is "free" considering it was being thrown away as heat previously.
2. You can run the internal combustion engine only when you need to and only where it's most efficient. If the IC Engine was the prime mover, you don't get that choice because you give the operator a throttle pedal and he/she decides. If you have a hybrid, the computer decides how the engine runs-- it is most efficient at wide open throttle and under higher loads.