CA Tuesday Tip: APM Database: Owner's Manual - Upgrades and Migrations
Introduction
This month's Tip continues to explore the APM database lifecycle by looking at upgrades and migrations. Future months will include database optimization and administration.
Planning
An upgrade or migration offers a chance to revisit existing choices and expand existing capabilities. This could include any and all of the following decision points:
-
- Migrating from Postgres to Oracle (There is no supported migration tool to go the other way.)
-
- Moving to new hardware/operating system & software
-
- Upgrading database version only
-
- Upgrading APM and database
-
- Install database then restore statistics and configuration
-
- Install later version of database and import definitions only
-
- Keep existing APM database as a historical database only.
-
- Upgrade manually or using the installer.
-
- Undergoing capacity planning to handle current and future database needs. This includes determining how many future business applications will be monitored.
Some guidelines in this process include the following:
-
- Starting with a fresh database rather than migrating an existing database ensures typically a smoother implementation.
-
- If a database is being migrated/upgraded, then it should be cleaned up first. This includes vacuuming for Postgres, reducing data retention settings, removing unused definitions, and cleaning up triage map data. This is a very important but often overlooked step.
-
- Have three different types of backup before migration/upgrade
-
* Backup (Configuration and Statistics)
-
* Configuration Export (Configuration and Definitions Only)
-
* Screenshots and Business Transaction Export (Definitions only). Screens shots are needed of the Business Application and Business Services so they can be recreated before the Business Transaction is imported.
-
This way you can use the appropriate restoration/export tool. Such as if you want one particular transaction definition, then just import only that definition. Note that having screenshots of the configuration and definitions is a good fallback on what was in the pre-upgrade/migration database.
-
Not having a backup means that all data is lost and there is no fallback plan. So it is something that should be avoided.
-
- Do a dry run of restoration/import.
-
- The documentation has special procedures for APM databases with versions earlier than 9.0.6, schema updates only, migrations, etc.
-
- Revisit changing database settings such as the connection pool (c3p0) and Postgres max connections in postgresql.conf file. These may need to be changed to handle increased traffic.
Installing
A successful planning process means a healthy database migration/upgrade. The following should be done during the actual install of software
- Before installing, make sure that you have:
* the needed values for the input items requested during the database installation.
* met all the upgrade pre-reqs (such as having enough disk/table space/memory etc.)
* a successful backup, config export, and business transaction export.
* Use an account with correct permissions to install.
- During/After the software install:
* Check the install and schematools.log
* Check that the permissions and account of database files are correct
* See if you can connect to the database via the appropriate tool.
* See if data is showing up in APM CE GUI.
* Verify that the database is doing aggregation correctly after an upgrade. Otherwise, aggregation issues may lie undetected for weeks or months. I previously distributed a tech note on this topic..
* Review Collector and APM database logs to see if database is logging any errors such as connection
Issues. This will eliminate future issues.
* Review APM Database Supportability metrics/tessperflog to see if the APM database is performing as expected.
* Review the triage map in the APM Investigator to see that all is working as expected
- One area of confusion is doing a Postgres upgrade manually. The basic steps to follow are:
* Drop the database - drops all the tables in the database and deletes the database
* Create the database - Creates the APM CE database
* Create the schema - Creates the schema, tables, etc in the new database.
Questions for Discussion:
1) What best practices have worked for you in terms of APM database and upgrades.
2) What would you like to see changed in APM database upgrades?
3) What other database topics would you like to be covered in Tuesday Tips?