Replaced the fuel pump, or more accurately, the fuel pump assembly. The mechanic said the assembly sits in a baffle, and that was secure.
He said the fuel pump is inside a canister. Fuel from the canister is what fuels the engine, not directly from the tank. He said the canister only had a little fuel in the bottom, he said they are usually full.
Anyhow, no noise from the back now. I've only driven it about 20 miles, but it has not had any problems.