84 Accord Brake Issues

Here is the symtom.....When you press on the brakes it is really hard to press down at first like the first inch or so but is normal after that. Also it doesn't do it every time either. Every 4th or 5th braking. Anyone every had the problem before?