2000 Sienna Front Door Window problem

The glass has intermittent issues going up. Sometimes it won't go up but after couple minutes it operates normally.

Anyone has this problem and what's the culprit?

Yes, I had to replace the window motor