Don't believe you can delete a Incident Category. Or make it Inactive.
You could try playing around with Access Rights to the Incident Category to see if you can remove it from the majority of Incidents.
Failing that, you can rename them, including Category Name and ID. Why don't you rename them as 'ZZZ - Do not use', so they appear at the bottom of the list. If you need a new Incident Category in the future, rename one of the 'ZZZ' ones.