Yes, the fuel pressure should stay up. Ours stays up for at least a weekend - and it never sits longer than that, so it may stay up much longer, I dont know.
When you first turn on the ignition, the fuel pump should run for almost 2 seconds. In a quiet location you can hear the soft hum. This would normally bring the fuel pressure up to the proper amount for starting if it wasnt still pressurized from the previous day. Assuming your problem is back at the fuel tank: If your fuel system will hold pressure for even a few seconds, you can turn the ignition on once to bring the pressure up, and do it again a couple of seconds later - and hopefully the engine will start quickly. (Thus not wearing out the starter.)
The fuel pump could have lost its 'anti-backflow' feature, or the pressure regulator could have a 'leakdown' problem'. On my 97 Neon, both the pressure regulator and the pump are located on/in the fuel tank, on a module that comes out after the (emptied) fuel tank is lowered. I think that if the problem is the fuel pressure regulator (which is also the fuel filter) leaking-down, it may lower your fuel pressure generally. If it lowers it enough, the computer wont be able to compensate by pulsing the injectors longer, and the engine could be running 'lean' at higher rpm and large throttle openings.
Loss of fuel pressure can also happen if one or more injectors are leaking - but I think this is very rare on Neons. It sometimes happens on the older VW Jettas if they have high mileage (like 200k miles).