I have a 2000 Sienna with the same problem. Sometimes by repeatedly hitting the driver's button, I can finally make it close. Sometimes I have to fight the motor and just force it to close. That must be damaging the motor and/or drive system.
Since the door never fails to open, I am guessing that something is telling the door that it is being blocked, and so it reopens every time I try to close it.
The dealer says that it's the controller, and wants $500 to repair it. I really hate that.
Ray