The basic wrangler is about the same to an Xterra stock. The Rubicon is a whole different story, it will drive rings around a stock Xterra. Any Jeep has much more aftermarket support to improve its offroad capabilities. So if this is the only area you are considering go with a Jeep. If you worry about creature comforts, more space and reliability the Xterra should be a lot better fit.
I lifted my girlfriend's XJ for Xmas and this weekend was the first chance we had to take it out. For half the money of my Xterra lift we got an inch and a half more lift, double the flex and several inches better ground clearance. Once her confidence is worked up, she will be kicking my ass.