If I removed the catilytic converter from a 80s accord, would it be worth the effort for the power?

Probably not, unless it's blocked or something. The stock monolithic cats are pretty free flowing. Unless you have improvements to the intake, I doubt you'd notice any difference. Plus you couldn't pass inspection anywhere without it.