Tire Mystery

I have an '06 Highlander. I have this persistent issue where the rear tires lose a bunch of pressure when I go for longer trips on the highway, eg 100-200 miles or more. I will fill up my tires before I leave and when I stop at a rest stop or gas station, the pressure in the rear tires has gone down by 15-20 psi. This has been happening to only the rear tires despite previous tire rotations. The loss during normal in town driving is not noticeable. Only on long trips. Does anyone have any insight? I need help!