If the Spec V has 180ft-lb of torque and the SVT has only 142 ft-lb of torque, then why are their 1/4 mile times and 0-60 times so close (assuming they're both stock)??? Shouldn't the Spec V have much better times due to the massive torque?

depends on the driver

Check the weight of the two.

Acceleration is a function of power/weight.

A lighter car with less power can keep up with a heavier car with more power.

Yes, but aside from the torque, the HP numbers are pretty close and the SVT's gearing is much better than the 02 and 03 Spec-V's. But in reality, the Spec-V will win a drag race by about 5-6 car lengths.