I'm no engineer, but you might want to try this experiment. Remove your tire (is it a "light" stocker?) and pick it up. Now, imagine driving over the rough, bumping up and down...holding that tire.
Next, you might want to remove your rear tail light just to peer into that area to see what's there.
Long story short, it might work for a while and then destroy your Xterra over time.
Now, here is my idea for you do it yourselfer's. Take the rear bumper off and you will find a channel in the structural support of the bumper that runs from one side to the other. My thought would be to place a heavy bar in there, weld it in, and on the end of the bar have a heavy hinge. You would need to cut out a section of the plastic bumper for the hinge, but just mount your swing away there. Dunno what would be the stop for the tire and any details, but its something to think about.
Course....your probably gonna spend more in time and frustration than you would with the Calmini?!?!?!