As long as the rpm is low (and the tool head moves nice and slow) it shouldn't be a problem. I'd use chunks of 2 pack polyurethane resin moulded vaguely to the shape you want first. I'm guessing that the usual kit polystyrene may get too hot and clog (has a lower glass transition temp).
Polyurethane cuts beautifully with milling tools designed for metal (and they don't go blunt too quick either)!
Have a look at my RS60 thread - there are shots of me milling some polyurethane brake drums using a 1mm milling flute - around 300 rpm. I cast round resin shapes inside shaving can caps etc. ready to fit in the mill/lathe chuck.
Good luck and let us know how you get on!