Grinding noise in park or neutral

1st time Jeep owner, bought a 95 Cherokee 4.0 Automatic, it has been running perfectly, but today a loud grinding noise started coming from the Transmission (I think)when it is in Park or Neutral, but as soon as I put it in gear the grinding noise quits.
Someone said my transmission linkage could be out of adjustment or I could have loose Flywheel bolts, but it only occurs in Park and Neutral.
Any help or Ideas?
Thanks