Weird grinding in 1st gear....

While taking off, toward the upper part of the RPM band I hear a stranger grinding slash clunking noise coming from the tranny. After I shift into second the noise goes away and this only happens in 1st gear. Any ideas? I have a shoft through shifter and I have bumped the shift linkage on a speed bump before (but who has not done that.). I just need to know if I should be worried. Thanks, and keep movin.
:)