Yes there should be fuel pressure in the lines, even if the engine is not running. Fuel pressure should be 9-13 psi. Sounds like the check valve is leaking in the fuel pump.
Here is a site for testing fuel pressure:
http://www.autozone.com/az/cds/en_us...rInfoPages.htm