when to port and polish

just wondering when you come to the point where your heads are holding you back in the fow department. i know the 6.0l heads are the same for all 3 ls based engines (4.8, 5.3, 6.0) and i am loosing already but when does it come time to pull your heads off and get them machined? i have a huge volant intake, 1.75" LT headers, 2.5" exhaust (magnaflow duels), and high flow cats. am i hinderng these parts by not putting any money into my heads? the main concern about not doing my heads already is i have nowhere to take them to locally, so it would be weeks before i got them back. is it better to just buy new ones and forget the factory ones? i don't really race or anything i just play a lot in the dirt and whatnot. what do the brains think, port and polish my existing heads or i that more money than what it's worth?