rodma08

Grafana Time Series dashboards for CA UIM

Blog Post created by rodma08 Employee on Apr 17, 2017

Here at the Agile Operations team @CA technologies, we are always looking for opportunities to bring new technology that would complement our Agile Operations Software portfolio and today we will explore how to display metrics collected from CA UIM in Grafana. The integration gives customers more flexibility in the way the data can be displayed and shared. 

We will be exploring two opensource projects, opentsdb which is a Time Series database, built on top of the popular Apache HBASE project.

And the second project is Grafana, the open source time series dashboard.

Grafana provides easy out of the box integration with opentsdb and both solutions provides an easy and scalable way to get insights from your monitoring data.

See our example dashboard below:

grafana-dashboard-correlation

One of the most useful features of Grafana is the ability to customize and correlate metrics dashboards in the same graph panel. It allows the user to get different metrics insights and quick create actionable and interactive dashboards.

The combination of opentsdb and Grafana provides out of the box metric statistics calculations,  (avg, min, max and etc...) available right from the User interface.

One example usage of Grafana and opentsdb features can be demonstrated below. Let's try to answer the question below.
What is the average of CPU Usage from all my monitored servers?

Getting this information from Grafana is very easy, check out our query example:

And the result:

 

If you are not familiar with CA UIM QOS (Quality of Service) data, please revisit our architecture documentation, but QOS data is just the metrics collected by CA UIM probes(CPU Usage, Memory Usage and etc...). You must consider QOS as our Time Series data collected and reported by the CA UIM install.
The diagram below shows the opentsdb concept integration flow diagram. This is how we are envisioning the use of the probe and opentsdb as CA UIM backed server.

The integration needs a limited release CA UIM probe (opentsdbgtw) to use opentsdb as the final destination for all QOS metrics collected. opentsdbgtw is similar to the current data_engine probe in the sense that is able to capture QOS messages from CA UIM hub queue and write to opentsdb.
Note: opentsdbgtw probe is current under active development and is distributed to selected customers / partners only. If you are interested in the probe, Contact your sales representative or send us an e-mail with your use case details and we will try to help. marcio.rodrigues@ca.com


The probe is distributed as a stand-alone zip archive to be imported to CA UIM directly. Once the probe is imported you can deploy it to your primary hub.

 

Once the probe is running and configured properly, it will start to send QOS metric data to your opentsdb instance. 

opentsdb-cauim-metrics

In order to complete the integration we need to plug Grafana to opentsdb. Go to the data Sources menu in your Grafana server and add a new opentsdb data source connection by selecting the proper ip/port and correct opentsdb version.

 

Now, create a test dashboard and verify the query UI shows CA UIM metrics:

You will see that as soon as you start to type, Grafana will suggest the metric names accordingly to what is found in opentsdb. Also, note the list of available statistical aggregations for the Data.

ca-uim-grafana-search-metric

This is a quick demonstration of the potential Grafana and opentsdb has to add to Infrastructure Management.


We are launching a Grafana dashboard central in order to share some of our development dashboards with the community.

Visit, rate it and comment. Your feedback is really appreciated and certainly make us improve.

 

At this time there is limited support for the dashboards and we expect the community to post contributions as well.
Feel free to jump in and create your own but don't forget to share back with the community.


Visit our dashboards page:

CA UIM QOS Data Dashboard dashboard for Grafana | Grafana Labs 

 

And for convenience there is a public github where we will share updates on the JSON file. 

GitHub - marciokugler/ca-uim-qos-grafana-opentsdb-dashboard: CA UIM JSON Dashboard. 

 

Check out our example dashboard that can be found in the links above.

grafana dashboard

 

We are currently selecting specific use cases for alpha / beta tests. If you are interested, reach out to us!!!
marcio.rodrigues@ca.com

 

Have a great dashboard idea to share? Did you like the topic, want to see more? Comment out!!!


Happy hacking!
Marcio Kugler Rodrigues - Agile Ops team

Outcomes