96 Jetta engine lacks power. Fuel pump?

My 96 Jetta runs a little rough and lacks power when I push the gas pedal to the floor. I put new plugs, wires, distributor cap and rotor on it and there is no change.

A little more history on the car:
When I first got it a few days ago, it would crank and not start. After trying a bunch of stuff, I finally resorted to spraying a bit of starter fluid in the air box and it sputtered to life. Since then it has started pretty easily so long as the battery had enough juice. It has a 2.0 liter gas engine.

The next thing I was going to try is the fuel filter. But before I do, I wanted to ask you guys on here how likely it is that the fuel filter could be causing this lack of power. It seems to make sense since the car wouldn't start before I tried the starter fluid, maybe that meant that there was low fuel pressure.

What do you guys think?

I would start by changing the fuel filter then go from there.