Side by side upgrade cutover strategies and experience.

Discussion created by comfortably_nim on Dec 5, 2014
Latest reply on Jan 10, 2017 by gpolk

We have a very large Nimbus environment with nearly a decade and a half of history. Upgrades have been done in place over that time and we're currently on a modified 5.1 environment with a beta of what became the first multi-threaded data engine with partitioning on MSSQL.


We are migrating to 8.x and our strategy is to build a new environment and cut over to it. Several strategies have been discussed. Uptime and the ability to work out the bugs before cutover is essential in our environment. Last time we copied our database from the old systems and installed old versions over the database in an isolated network. Then stepped through upgrades to get to the version we wanted, then cross posted production data to prove out and refine the system. Then shut down the old root infrastructure and switched out the domain on the new with secedit and some scripting.


Various changes appear to make that potentially more complicated and we've found that there are domain related keys in the new database. We're looking for recommendations and information on potential pitfalls to various strategies.


One strategy we're considering is to put both pieces of core infrastructure on the same domain. We believe we can do this by disabling all but the hub and robot probes on the new environment. Shutting the new robots down. Then switching the domain in the various configs, removing security.cfg, and restarting the robot so it syncs with the other hubs security.cfg. We may also have to merge new default ACLs from 8.0 into that security.cfg. We already hard code root probe addresses in various UMP components and most things that don't have it hard coded expect it on the local robot or hub anway. We intend to use the firewall to ensure new and old systems can't talk to the wrong database if something accidentally discovers the wrong data_engine.


Another approach being considered is leaving new infrastructure on a new domain, then changing the domain on all down stream hubs and robots in a phased cutover.


Does running two sets of root infrastructure in the same security domain work? What are the risks. I recall one unknown was that the install process could discover the old components and want to upgrade them, but I believe we have that mitigated.  What have others done?  Of course this is well outside of what we've found documented.


-Ray Ferguson