It came out the usual way. When I put the bolt back in it threads all the way back in against the tensioner but will not tighten. The belt is running tight on the pulleys. Will this work itself back out and cause the belt to come off while I am driving?

I put the belt tensioner back on,got the belt back on. The only problem at all is that the bolt going back in holding the tensioner will not tighten back up tight. I even tried put the old bolt back in and it will not tighten up. What could cause this?