The only problem with driving with worn synchros is that when you grind the gears, the engagement teeth on the gear itself are getting worn down. Then you have to replace the whole gear, which is not cheap.
As mentioned, switching the fluid to Synchromesh might help, along with learning to shift it without grinding.