Possible shifting problem in trans

On occasion the trans will not shift into second gear or any other gear. I would then shut the car off and wait for like 20 seconds and then restart the car and it works fine. What might be the problem with the car. I only have less than 56k on the car and have replaced the trans filter, and complete fluid exchange. The fluid is red and full when cold and also when hot. The car is a 96 lx with the 3.8 v6 engine. Any help would be greatly appreciated. Thank you.