I took off and inspected the ignition module. It was not very tight and there was very little silicone grease (the dielectric stuff I believe).
I simply took it off, cleaned it, added new grease, and bolted it back down snug-tight.
It works much better now and starts almost instantly every time.
Thanks for the help!