Upgrading Agents is very much manual, difficult, error-prone, long, time consuming, tedious and boring.
It's probably one of the top reasons why customers don't upgrade and keep running old releases of Introscope for several years.
Since upgrading EMs is overall a lot simpler and cheaper (although there's still room for improvement, for example I would like to upgrade only my MOM and have it distribute updates to all of its Collectors), let's assume that I just upgraded my entire cluster to the latest release.
Now it's time to upgrade my Agents.
I would like to be able to select a group of Agents (there could be a new "Agent groups" object, or I"m eventually willing to select them one by one based on a regexp search results displayed in a table), click on the "Upgrade" button, be offered to upgrade Agents now or at a scheduled time (next Saturday night while I'm setting the dance floor on fire for example), and have the EM cluster automatically distribute those upgrades to all my selected Agents.
A few notes:
_It might not even have to restart the JVMs, that could still be my responsibility
_It should preserve my existing Agent configurations as much as possible