Haroon Ahmed

10.5 SmartStor Metadata Changes Improve EM performance

Blog Post created by Haroon Ahmed Employee on Mar 17, 2017

SmartStor (SS) is CA APM’s high performance time series database, it acts like a flight data recorder and records every metric reported to the Enterprise Manager (EM).  SmartStor is purpose built and finely tuned to handle a very high volume of data storage and retrieval.  With SmartStor APM users can store historical metrics for an extended period of time for historical reporting and capacity planning.  

 

SmartStor is a very disk intensive process and demands a guaranteed high performance I/O channel.  The many advantages of SmartStor include:

  1. Fast data storage and retrieval
  2. Highly compressed data storage to limit storage requirements
  3. An out of the box embedded database requiring limited maintenance
  4. No need to maintain or procure a 3rd party database repository to store a high volume of performance metrics

 

However, with all these advantages one area of contention in large deployments is with how changes to the metadata files are managed.  Metadata files are kept in memory and act like indexes that maintain pointers for very fast data retrieval.  Whenever an agent or metric name changes the entire metadata file gets flushed to disk to maintain data integrity.  And can eventually impact performance in situations where applications experience a metric explosion, have very large metric names or have a metric leak where new metric names are constantly being created.  In this scenario, the metadata file size can grow over time and eventually start impacting the disk write duration if it gets very large and there is disk contention.  To get around this problem our customers either avoid monitoring the offending components like large SQL statements, fiddle around with cluster load balancing rules to reduce agent sprawl or flush the metadata file on a regular basis henceforth loosing access to historical data.  These workarounds are stop gap solutions that limit application visibility and require constant ongoing maintenance.

 

Good news is that with APM 10.5 SmartStor enhancements these issues have been addressed.  The new approach to processing the Metadata file includes:

  • Only writing the changes to agent or metric names to disk, this significantly reduces the disk I/O requirements when there is an agent or metric name change. 
  • Only keeping live agents and metric names in memory.  This reduces the memory footprint by not hogging memory for agent or metric names that are not accessed frequently or are only kept around for historical reporting.
  • Allowing agents to change security domains and still preserve historical data access. 

The 10.5 SmartStor Metadata enhancements are already showing significant improvements to EM performance in customer environments. 

 

Outcomes