EPAgent 101: Use the SDK to Send Data to APM

Document created by Chris Kline Employee on Nov 6, 2014Last modified by Guenter_Grossberger on Mar 31, 2017
Version 10Show Document
  • View in full screen mode

So you want to send some outside data into CA APM?  Great, the good news is that it's super easy.  The CA APM version of an SDK is called the EPAgent.  It stands for Environment Performance Agent, but most folks just call it EPA.

 

 

GET THE STUFF

 

Before we go any further, you can find the (CA wiki at http://wiki.ca.com) documentation for (the 9.7 version) EPA here.  EPA doc is in a section titled "Implementing Agents" and is titled something like "CA APM Environment Performance Agent Implementation Guide".  It's a short doc and pretty quick to get through.  But we'll provide some tips here as well.

 

You can download the EPA using your ca.com account (and entitlement).  The exact link varies by version, but you can start here.  Find the right version and then search for a download titled something like "Introscope Environment Performance-EP-Agent Files."

 

NOTE: As of April 2015, the EPA with REST interface is now available from product downloads as part of APM 9.7.1.  Since the API is new, you can find the REST interface documentation here until it gets merged into the core APM doc set.

          

 

ABOUT EPA

 

This information does not replace official documentation for EPA.  It's just a quick-start guide, so don't try to find all the answers within.

 

The CA APM EPA is a stand-alone Java application with a wide range of features for collecting data to send to the Enterprise Manager.  EPA was designed so that customers could, as easily as possible, integrate any data into CA APM – even information about the state of the garage door, lego robots, etc.

 

Introscope EPA was originally designed for customers to register script “plugins”, either in “stateful” (e.g. log reading) or “stateless” (e.g. CPU or OS stats.)  Some APM customers had needs that weren’t met by creating an EPA and running it local to an environment – the footprint of the agent and scripts were too expensive, or the process of installing an entire EPA to collect the data was inconvenient.

 

A single EPA can be configured to listen to a network port for metric data from arbitrary remote systems.  Specifically, one port can be set to listen for XML-formatted metric data, and a second can listen for data sent using HTTP requests (HTTP GET or RESTful).

  • New in version 9.7, a REST-compliant interface.  Metrics can be submitted in batches over a RESTful PUT operation.  This is the preferred interface (more efficient).
  • HTTP GET requests are super simple for quick-and-dirty problems and can often be sent from common programs like ‘wget’ or 'curl', which are often already available at the command prompt.    This is used for single-metric submission and is less efficient than the newer RESTful interface.  This interface can be quickly used via a browser to illustrate sending metrics.  This interface will be deprecated and maintained only for backwards compatibility.
  • XML format provides maximum flexibility, but is slightly more complex and requires manual management of the socket.

 

Refer to the more detailed discussion of the formats below.

 

WHY EPA?

 

EPA is a great way to get started because you can take your existing monitoring scripts (which I often refer to as "poor-man's monitoring") and simply wrapper them without making additional changes.  Using this technique you can continue sending current alerts to email (or wherever) and fork the data to EPA until you're ready to make a single stream.  This can be done very quickly, so you can be the hero before lunchtime.

 

When run locally, output printed to STDOUT will be sent to the EPA and output printed to STDERR will be sent to the EPA log.  So the easiest way to wrap existing script work is to create a short wrapper that takes existing script output and parses into the correct format (simple or XML; both described below) and then prints the formatted output to STDOUT (or to HTTP GET).  In this manner, existing scripts can be ported to EPA in a matter of minutes!  Libraries for formatting output are available in Perl.

 

NOTE: EPA takes string or integer data only.  If you have floating-point data, it should be converted to integer values prior to sending to EPA.

 

DETAILED TRAINING MATERIALS (INCLUDING RESTFUL-API DETAILS)

 

In addition to product documentation, CA recently provided updated training materials in support of the RESTful API update.  This training includes details on error codes, syntax highlights, etc.  Please review.  The information on this quick-start page will not be as exhaustive.

SETUP INSTRUCTIONS

 

Download the EPA as directed above.  Then unzip it to some fancy directory.  It can be on any machine (versions exist for Windows and UNIX but the only difference is startup files).  You will need an available JRE to run EPA (it does not ship with one embedded.)

 

Edit the config/IntroscopeEPAgent.properties

Find and uncomment the following lines:


# Network port on which to receive simple or XML formatted data. If commented


# out or left unconfigured then EPAgent will not listen for data.


#


introscope.epagent.config.networkDataPort=8000






# Network port on which to listen for HTTP GET


# left unconfigured then EPAgent iwll not list


#


introscope.epagent.config.httpServerPort=8080













 

You will probably want to reconfigure the network ports to be appropriate to your environment.  You should also edit the file for any other properties, such as the correct location of the EM to which you'll be sending data.

 

You should have a bin/ directory with a start script:

 


[root@centos64rt2 ~]# cd epagent/bin


[root@centos64rt2 bin]# ./EPACtrl.sh start


./EPACtrl.sh start: EPAgent started


[root@centos64rt2 bin]# ./EPACtrl.sh status


./EPACtrl.sh status:  Agent is running


[root@centos64rt2 bin]#













 

On Windows, install and start the service (there's an executable that will do this for you).  From a unix command-line, you can launch Introscope EPA manually with the following:

 

java -jar lib/EPAgent.jar

 

You may want to give the Introscope EPA some additional JVM memory, depending on the processing load of metrics you expect it to handle:

 

java –Xms256m –Xmx256m -jar lib/EPAgent.jar

 

You should see confirmation that the agent started successfully:

 

4/23/08 10:23:01 AM MDT [INFO] [EPAgent] Introscope EPAgent started.

 

Now the listener should be active (assuming you're going to use one of the network options and not STDOUT wrapping).

 

SENDING DATA TO EPA

 

There are three ways to send data.  Data is formatted with pipes (|) dilineating folders/resources and a colon (:) marking the metric name:

Folder1|Another Folder With A Space:Metric1

 

RESTful API (HTTP POST)

 

This is the modern, preferred method.  Once running, create a sample text file like the one below.  API Documentation can be found here.  Then pipe it into an HTTP command with the curl statement provided:

 


michaels-macbook-pro:tmp mwalker$ cat epagent_rest_sample.json

{



    metrics : [



        {type: "IntCounter", name: "Sample Data|Ints:IntCounter", value : "1234"},



        {type: "IntAverage", name: "Sample Data|Ints:IntAverage", value : "3"},

        {type: "IntAverage", name: "Sample Data|Ints:IntAverage", value : "4"},

        {type: "IntAverage", name: "Sample Data|Ints:IntAverage", value : "8"},



        {type: "LongCounter", name: "Sample Data|Longs:LongCounter", value : "1234"},



        {type: "LongAverage", name: "Sample Data|Longs:LongAverage", value : "3"},

        {type: "LongAverage", name: "Sample Data|Longs:LongAverage", value : "4"},

        {type: "LongAverage", name: "Sample Data|Longs:LongAverage", value : "8"},



        {type: "StringEvent", name: "Sample Data|Strings:StringEvent", value : "The quick brown fox jumps over the lazy dog"}

    ]

}

michaels-macbook-pro:tmp mwalker$ curl -H "Content-Type: application/json" --data @epagent_rest_sample.json
http://centos64rt2.ca.com:8080/apm/metricFeed


{"validCount":9}

michaels-macbook-pro:tmp mwalker$












 

Metrics will show up in your WebView UI within 15 seconds:

 

webview.png

 

RESTful CODE SAMPLES

We have a number of samples to assist you in actual coding.

d     

 

HTTP GET

As an HTTP GET request (deprecated), EPA metrics take the following form:

 

http://localhost:8080/?metricType=IntCounter&metricName=LarryA|MoeA:CurlyB&metricValue=60

 

You will need to change the host and port as appropriate to your environment.  You can test by putting that into a browser because it's a simple GET request (not a POST).  After pasting this URL into a browser, your WebView/Workstation UI will show the new metric within 15 seconds (as shown above).

 

Metric names with spaces should be quoted when using command-line tools, but do not need quoting when testing with a live browser session.

 

‘wget’ is an excellent, free command-line tool for generating such requests from shell scripts and similar.  NOTE: When using wget, you may need to quote the URL for correct operation; this will also allow the use of spaces in metric names.

 

Send XML Data Directly to the Socket

The XML format of output is much more robust and allows for the sending of errors and transaction traces in addition to time-series metrics.  Connect to the XML network port, report a snippet of XML like what follows, and disconnect from the port (or leave the port open synchronously).

 


<metric type="LongCounter" name="Resource Usage|File IO:diskWrites" value="37" />


<metric type="StringEvent" name="Apache Errors:LastErrorString" value="ERROR: Apache shutdown unexpectedly" />













 

NOTE: The entire XML Schema is documented in the official documentation.

 

METRIC TYPES

 

When sending data use the following metric types:

 

  1. PerIntervalCounter (useful for rate metrics, like “miles per hour” or “errors per interval”; resets to zero at each new interval)
  2. IntCounter (useful for tally metrics, like “msgs in queue”, and does not change until a new value is reported)
  3. IntAverage (useful for response times, like “average time in seconds”; you don’t calculate the average yourself; just report all the applicable metrics (like in a loop) and the calculation will be performed automatically at the end of the interval)
  4. LongCounter  (same as above, but for very large numbers)
  5. LongAverage (same as above, but for very large numbers
  6. StringEvent (use to report string values, like “startup command-line” or a log entry).  NOTE: StringEvents are not stored historically; only current values are used.

For more information see Metric data types.

 

That's all you need to get started.  Now go script something cool!

3 people found this helpful

Attachments

    Outcomes