Well the HB may be causing the problem...or maybe the torn rubber is hitting something as it spins?
The best way to figure out if something is causing a problem is to fix it then see if the symptoms go away... I would go to a pick-a-part salvage yard and pick up a HB there, then put that on and try that out($30 at most), if that wasnt the cause then at least you have eliminated a possibility and fixed a serious problem at the same time.
Or if that isnt an option, then take some additional things off and check for other damaged parts. I have a suggestion....
I suggest you check the timming chain, tensioner/guide, and sprockets...they can make quite a bit of noise, especially if the chain is loose...
If it aint the chain, then I would check the flywheel...I had mine break on my camaro one time and the car still drove find and the engine seemed to work fine too, but sounded just like you described as it was banging around(only at low rpms!)
BTW...How many miles are on the engine?