I'm a bit confused - can you not sand the enamel you used as a primer? Or can you not sand the acrylic you applied OVER the primer?
I don't have any experience with Xorust primer, but I do have enough experience with enamels that I avoid them unless I'm doing some specialist technique (like washes over lacquers). I find that they effectively never dry, and a not fully set enamel will just gum up any sort of sanding you try to do. Is this the problem you're having? It may be worth stripping the enamel off and starting again - if you got to it early enough, it shouldn't be too difficult.