Skip navigation
All Places > CA Unified Infrastructure Management > Blog > Authors BryanKMorrow

The function of this probe will be to provide a UIM administrator with small callback utilities to make the management of the UIM infrastructure a little easier. This probe will be under constant development with new utilities being added regularly. If you have suggestions for new automated utilities or feedback on the current state, please post in the comments of this post.



PLEASE NOTE: When using the automation_device_wiper callback with the delete_qos option enabled, this will perform a delete on all of the Raw and Summary tables, this could lead to significant load on the database server and possibly some deadlocks.


Probe New Features
This probe is a collection of useful callback utilities to help a UIM administrator manage their system. You can accomplish the following tasks currently with this probe:


NEW: First attempt at a UIM topology map (hubs with large robot counts won't work OOTB currently, still working on the dynamic sizing).
UPDATED: HTML 5 Report for top probes in use , now supports a top_n parameter to adjust the default of 15

UPDATED: MYSQL connectivity when special characters are used should now be fixed.

Probe Existing Features
• Encrypt passwords for profile usage in almost all current probes
• Update the interface alias for a specified device->interface (Usually only available in USM)
• Reset a probe’s security on a specified robot
• Retrieve a list of source->targets where data has not been received in a specific time frame
• Remove a device or list of devices from discovery with an option to delete QOS

• Delete QOS by providing a list of targets
• Clean niscache of provided list of robot names
• Generate HTML Reports for the following: License Pack Counts, UIM Users, Account Contacts, Hub Subscribers, CDM/Processes/NTServices Thresholds.

      Threshold reports can now take a USM Group Name parameter. NOTE: Needs to be child group that contains                 devices
• Generate HTML Report for UMP database health. Based on TechDoc
• Manually configure probes by providing the following information: probe name, section, key and value. Can provide a       comma-separated list of robots. Multi-threaded
• Modify single probe configuration by providing JSON
• Collect and ZIP a probe’s log files and configuration files. Could be used for support.
• Collect thresholding information based on configured probe profiles, multi-threaded (This is the migration of the     threshold_archive probe that is currently on the Communities). Multi-threaded

      Can now be limited by providing a hublist parameter. Hub name, not hub address is required.
• Generate HTML SVG report of VMware topology. This features uses the Vsphere API to collect parent-child relationships for your infrastructure. It ‘should’ generate one HTML page for each configured resource for each vmware probe instance. Multi-threaded.
      o NOTE: This connects to vSphere directly so the probe will need to be on the same physical network as the Vcenter          or ESX host. The vmware probe doesn’t collect the needed attributes out of the box, so that is why it connects                  directly to vSphere.
• Retrieve list of current USM Groups (Will be needed for future USM group creation feature)




• Create USM groups from JSON
• Store data in H2 database for license usage over time
• Improve configuration archive and diff reports (configuration_archive functionality)
• Add more probe threshold reports (logmon)
• Improve probe configuration through JSON to support multiple robots and probes
• Continue testing the VMware topology feature









July 22nd, 2016


Initial Draft

July 22nd, 2016


Added devices_with_no_data

July 25th, 2016


Devices with no data always creates a CSV, no matter the size. Initial merge of healthcheck probe with just list_subscribers creating a CSV.

July 28th, 2016


Added automation_device_wiper callback

September 9th, 2016


Removed decrypt password option, added list_accounts/users, can now provide a CSV list of devices for removal

October 4th, 2016


Added ability to manually configure probe configurations and new licensing report.

October 26th, 2016


Added niscache_clean, modify probe configurations from json, UMP/database health report, threshold gathering and reporting and vmware topology

November 8th, 2016


Added support file retrieval and processes/ntservices threshold reports

November 21st, 2016


Delete QOS by target, inactive probe report and added filtering for threshold gathering and reporting

December 7th, 2016


Top probe usage reports, configuration_archive custom probe integration

January 3rd, 2017


mysql connectivity fixed, added uim topology map, top_n parameter to top_probes report

This is a newer version of the probe_configuration_archive that was available on the CA Marketplace. This probe offers the same features, but does so with multi-threading capabilities. This post will hopefully be a living document where people will comment on the current state and also additional features they would like to see implemented. I will attach the latest documentation and probe package to this post, so please subscribe for updates.


Probe Features

This multi-threaded probe will archive every probe configuration in your UIM environment and automatically do configuration differentials between the current and previous configurations. This will allow an administrator to track configuration changes in their environment automatically. This probe base is derived from the previous single threaded probe_configuration_archive.



  • UIM Server version 8.0 or greater
  • Java_jre version >= 1.7


Probe Configuration

This probe can only be configured using Raw Configure mode.


ignored_probes: Use a comma separated list of probe configurations you wish to be excluded from collection

archive_path: File system location where you wish to store the configuration files

threads: The number of threads used while doing probe configuration gathering



The current version is driven from the rebuild_configuration_database callback, please run this whenever you want to archive your configurations. Future version will re-implement an interval based archive.




The probe uses an internal JDBC capable database ( to store the probe configurations and also the configuration change data. This allows the data to still be accessible for custom dashboards and reports, but also avoids using the UIM database as a datastore. I chose to stick with the raw_configure option as opposed to the newer Probe Framework SDK because of the multi-threading that was implemented. The multi-threading should speed up the configuration gathering quite significantly compared to the older probe_configuration_archive.


This table contains a record of each probe configuration. It stores the origin, hub, robot, probe, probe version, filesystem path to configuration file, complete configuration file in a column, a record id and archive date.


This table contains a record of each probe configuration change. It stores the hub, robot, probe, configuration section, configuration key, previous and current configuration value, previous and current archive date and previous and current record ids.


The probe uses the native callbacks provided in many probes to get the probe configurations.

  1. It uses the gethubs callback on the hub probe to get a list of available hubs
  2. Loops through each of those hubs to get a list of robots using the hub’s getrobots callback
  3. (Multi-threaded) Grabs a list of probes and information on each probe from the controller via the probe_list callback
  4. (Multi-threaded) Saves the configuration file from the robot by using the text_file_get callback
  5. (Multi-threaded) Inserts information about the configuration into the database
  6. Compares the current and previous configuration to determine if there were any changes
  7. If there are changes, writes those changes to the database



In order to query the data from the UMP you will need to copy the h2 database driver jar file into the <Nimsoft_installation_directory>\probes\service\wasp\libs directory. The driver can be found Here. You will then need to create the data source in the Dashboard Designer.


  • Name: Logical name for the connection
  • Driver Class: org.h2.Driver
  • JDBC URL: jdbc:h2:tcp://<IPADDRESS_OF_PROBE_CONFIGURATION_ROBOT/./configuration_archive
  • User: uim
  • Password: uim



  • Ignore List function is currently not working
  • Sometimes when re-deploying you may need to deactivate->reactivate the probe for it to start properly


  • Enable file and table retention
  • Add callbacks for adhoc differential between configurations based on record/date
  • Add callback option to replace current probe configuration with archived configuration


Again, please feel free to comment directly on this post with any feedback or suggestions for the probe. I plan to continue development on this for quite some time.





As a follow-up to the Executive Level Scorecard dashboard example posted previously, this is a summary of a network device. It contains a context selector that pulls from devices being polled by SNMP Collector. It also has some predefined parameters that allow you to modify the interface utilization metrics that are collected in bytes, allowing you to convert to kilobytes, megabytes or gigabytes.






As you can see its the same type of information that can be found in USM, but in what I feel is a more presentable format.


  • All the data is driven by the context selector at the top right.
  • All of the datasources are SQL driven to use the context selector.
  • The KiloBytes In and Kilobytes Out columns are modified using a parameter






select distinct source from s_Qos_Data where probe = 'pollagent' order by source asc  


This query retrieves all devices that are collecting data from the snmp_collector probe






As you can see its driven by a SQL datasource, pulling from the S_QOS_SNAPSHOT table.


select s.samplevalue from s_Qos_data d join s_qos_snapshot s on d.table_id = s.table_id where d.source = '${device}' and d.qos = 'qos_cpu_utilization'  







Here I am just mapping the column headers to the column output and adjusting the width so the information fits without a horizontal scrollbar


I am by no means an SQL expert so this query may not be very optimized but it works!


NOTE: This is for MSSQL, queries for MYSQL and Oracle will most likely need to be adjusted.


select i.interface, i.utilization_in, gg.bytes_in / ${kilo} as bytes_in, i.utilization_out, gg.bytes_out / ${kilo} as bytes_out, i.errors_in, i.errors_out, j.discards_in, j.discards_out from (select g.interface, g.utilization_in, g.utilization_out, h.errors_in, h.errors_out from   (select a.interface, a.utilization_in, b.utilization_out from   (select as interface, y.samplevalue as utilization_in from s_Qos_data z   join s_qos_snapshot y   on z.table_id = y.table_id   where z.source = '${device}' and z.qos = 'qos_interface_utilizationin') a     INNER JOIN     (select as interface, w.samplevalue as utilization_out from s_Qos_data x   join s_qos_snapshot w   on x.table_id = w.table_id   where x.source = '${device}' and x.qos = 'qos_interface_utilizationout') b   on a.interface=b.interface ) g   INNER JOIN     (select c.interface, c.errors_in, d.errors_out from   (select as interface, u.samplevalue as errors_in from s_Qos_data v   join s_qos_snapshot u   on v.table_id = u.table_id   where v.source = '${device}' and v.qos = 'qos_interface_pcterrorsin') c     INNER JOIN     (select as interface, s.samplevalue as errors_out from s_Qos_data t   join s_qos_snapshot s   on t.table_id = s.table_id   where t.source = '${device}' and t.qos = 'qos_interface_pcterrorsout') d   on c.interface=d.interface) h   on g.interface=h.interface) i   INNER JOIN     (select e.interface, e.discards_in, f.discards_out from   (select as interface, o.samplevalue as discards_in from s_Qos_data p   join s_qos_snapshot o   on o.table_id = p.table_id   where p.source = '${device}' and p.qos = 'qos_interface_pctdiscardsin') e     INNER JOIN     (select as interface, m.samplevalue as discards_out from s_Qos_data n   join s_qos_snapshot m   on n.table_id = m.table_id   where n.source = '${device}' and n.qos = 'qos_interface_pctdiscardsout') f   on e.interface=f.interface) j   on i.interface=j.interface   INNER JOIN     (select cc.interface, cc.bytes_in, ff.bytes_out from   (select as interface, bb.samplevalue as bytes_in from s_Qos_data aa   join s_qos_snapshot bb   on bb.table_id = aa.table_id   where aa.source = '${device}' and aa.qos = 'qos_interface_bytesin') cc     INNER JOIN     (select as interface, ee.samplevalue as bytes_out from s_Qos_data dd   join s_qos_snapshot ee   on dd.table_id = ee.table_id   where dd.source = '${device}' and dd.qos = 'qos_interface_bytesout') ff   on ff.interface=cc.interface) gg   on j.interface=gg.interface  


As you can see I am using ${device} to pull in the device.  As for the metric conversion you can see where I'm using the front slash to mark the division and ${kilo} as the conversion.


select i.interface, i.utilization_in, gg.bytes_in / ${kilo} as bytes_in, i.utilization_out, gg.bytes_out / ${kilo} as bytes_out, i.errors_in, i.errors_out, j.discards_in, j.discards_out 






Here I'm just entering the conversion numbers and assigning them a parameter.



Please let me know your thoughts on the dashboard and any ideas on improving it.



Recently I have completed a dashboard that I feel provides both technical and business users with a very high level overview of your Infrastructure that is managed by UIM. Not only does it have the Executive level information (SLAs, Health Index, Alarm State) but it also allows a more technical user to drill down into a more detailed set of data (USM, other dashboards, etc). The Scorecard also shows very well in a NOC environment on the projector or tv. You can download the file and import it into the HTML 5 Dashboard Designer.






As you can see there are different sets of information that each 'badge' contains.


  • The numbered gauge is used to represent the health of the service or application.
    • I prefer either an SLA or Health Index value.
  • The A icon represents the alarm status of the service or application.
    • A simple alarm filter is best.
  • The P icon represents a drill down dashboard (Some examples of these will follow in later posts).
    • This could also just link to a Unified Dashboard or any other direct link in the UMP.







In order to make the dashboard generic I have created some simple SQL based datasources to be used at the time of import. These are just hard coded numbers that each query represents. You can just as easily select the SLA datasource that is appropriate for the service or application.
















Please let me know your thoughts on the dashboard and any ideas on improving it.


TO DO: I still need to make the different sections modular, currently its a fixed background.





As some people are still new to HTML 5 Dashboard options in UIM, I wanted to give a brief example of using the Line Chart Widget.


First we will start with the Line Chart; this will provide you the ability to create a time series chart based on either a QOS metric or an SQL query. In this example I will be providing the SQL example.


Step 1 – Drag the Line Chart widget onto the dashboard canvas. This will leave give you a blank widget.




Step 2 – Configure the Chart

  • Give it a Y-Axis Label. In my case that is Messages Per Minute
  • Give it a Series Duration, I will be using 1 hour




Step 3 – Configure the Series

  • Highlight the Line Chart widget on the canvas (will have a red border).
  • Press the + icon under the Series navigation section on the right column.
  • In the Series Data Source Type dropdown, select SQL.
  • Press the + icon under the SQL data source to create the datasource.
  • Give the Query a name, select the NIS option from the database dropdown.
  • Paste and Test your query.

In this example I am returning the last hour with the following query:


select d.sampletime, d.samplevalue as posted from S_QOS_DATA s join RN_QOS_DATA_1730 d on s.table_id=d.table_id where = 'WorldWideHQ' and d.sampletime > DATEADD(HOUR, -1, GETDATE())



I like to go back and rename the Series 1 that was created earlier, this will help if you are creating your own legend.


       Repeat the steps required to create multiple series if needed.


NOTE: The format of the SQL query needs to return two columns, the first being the sampletime and the second being the value.


Step 4 – Create a custom legend. The current version of the dashboard does not have a legend associated with the Line Chart, so we will need to create one.

  • Drag a Rectangle Widget onto the canvas underneath your Line Chart.
  • Change the size to be appropriate to your Line Chart.
  • Change the color to match the Series.
    • Series 1: #1F77B4
    • Series 2: #FF7F0E
    • Series 3: #2CA02C
    • Series 4: #D62728
    • Series 5: #9467BD
    • Series 6: #8C564B
  • Give the Widget a Label to match the Series.


Clone the Rectangle Widget and repeat for each Series.