Car will push start but Electrical system won't work

After hitting a deer in my 1986 Camry, I stopped to look at the damage and my car stalled out. I mannaged to push start my car a drive home. My headlights still work as well as radio, horn, hazards. Upon further inspection all of the fuses are fine and it seems like the fuse box in the kicker pannel is not getting any power. I checked all of the wiring and it is also fine. The fuse box right beside the battery is getting power( all of the functions in there work fine) The weirdest thing is that when I turn the ignition to on, the charge light sometimes comes on. Does anyone know what is going on?????