troublesome transfer case

I have an '04 Xterra with 10K miles and have been having problems shifting from 2hi to 4hi. While driving, without adding power, below 25 mph and not on any slick surface, when I try to shift into 4hi it grinds and doesnt engage smoothly. The other day it wouldn't even go into 4hi so I shifted it back to 2 hi and the right auto hub started to grind. After about 5 seconds, before I could pull over, there was a loud clunk. After I came to a stop, I shifted it into 4 hi and it worked with no problems. The dealer had no comment. Has anyone had this type of problem?

I don't like shift on the fly. I always come to a stop(got to get out and lock my Warn hubs anyway).