Ive owned and driven a lot of different vehicles including full size Z-71's, Jettas, Volvo's and a Saturn
etc.. (I like the Saturn SL1 - It appeals to the Utility side of my character).
The X is an interesting ride - it is predictable, and a moderate perfomer on road. It foots the bill well if you need to do some "truck" type duties like pulling trailers and hauling stuff but can also be cleaned up and quite acceptable for a night out on the town. Acceleration is moderate, Brakes are pretty good. Handling is better than it feels. Initially a lot of XTerra owners feel the truck to be "joggly" on the highway and scary - some have even taken theirs back. This is because of the truck's high center of gravity. I can attest to pushing that as far as it can possibly go and it is suprisingly good. I have over 50,000 miles on mine and it runs as well as when I bought it even if it is a "Gray Market" rig (long story). I used it for downtown onsite work, hauling, long distance runs, supermarket trips and just about anything else you can think of and it hasnt let me down. Its build quality is very good compared to american vehicles even if it is basic and it competes well with Toyota and outshines Mitsubishi etc. A word of warning though.. if you go offroad you WILL get sucked into it and show signs of a worsening problem called "modaholicism" in which you will want to suddenly put and ARB, winch, Tires and all sorts of goodies on your rig - you will be amazed at the excuses you come up with to do such things too.
A note: even if you dont plan to go off-road dont get the 4X2 - you will regret it. The X is a different creature in 4X4 and you will be very glad of it if you ever hit mountain passes or other low-traction areas. Most of the people I know with 4X2 XTerras regret not getting the 4X4. It also holds up resale.
Cheers and good luck car hunting!