Skip navigation
All Places > CA Unified Infrastructure Management > Blog > 2016 > July
2016

The function of this probe will be to provide a UIM administrator with small callback utilities to make the management of the UIM infrastructure a little easier. This probe will be under constant development with new utilities being added regularly. If you have suggestions for new automated utilities or feedback on the current state, please post in the comments of this post.

 

 

PLEASE NOTE: When using the automation_device_wiper callback with the delete_qos option enabled, this will perform a delete on all of the Raw and Summary tables, this could lead to significant load on the database server and possibly some deadlocks.

 

Probe New Features
This probe is a collection of useful callback utilities to help a UIM administrator manage their system. You can accomplish the following tasks currently with this probe:

 

NEW: First attempt at a UIM topology map (hubs with large robot counts won't work OOTB currently, still working on the dynamic sizing).
UPDATED: HTML 5 Report for top probes in use , now supports a top_n parameter to adjust the default of 15

UPDATED: MYSQL connectivity when special characters are used should now be fixed.


Probe Existing Features
• Encrypt passwords for profile usage in almost all current probes
• Update the interface alias for a specified device->interface (Usually only available in USM)
• Reset a probe’s security on a specified robot
• Retrieve a list of source->targets where data has not been received in a specific time frame
• Remove a device or list of devices from discovery with an option to delete QOS

• Delete QOS by providing a list of targets
• Clean niscache of provided list of robot names
• Generate HTML Reports for the following: License Pack Counts, UIM Users, Account Contacts, Hub Subscribers, CDM/Processes/NTServices Thresholds.

      Threshold reports can now take a USM Group Name parameter. NOTE: Needs to be child group that contains                 devices
• Generate HTML Report for UMP database health. Based on TechDoc http://search.ca.com/assets/SiteAssets/TEC1405477_External/UMPUSMSlowPerformanceGuideandTroubleshootingChecklist1.1.pdf
• Manually configure probes by providing the following information: probe name, section, key and value. Can provide a       comma-separated list of robots. Multi-threaded
• Modify single probe configuration by providing JSON
• Collect and ZIP a probe’s log files and configuration files. Could be used for support.
• Collect thresholding information based on configured probe profiles, multi-threaded (This is the migration of the     threshold_archive probe that is currently on the Communities). Multi-threaded

      Can now be limited by providing a hublist parameter. Hub name, not hub address is required.
• Generate HTML SVG report of VMware topology. This features uses the Vsphere API to collect parent-child relationships for your infrastructure. It ‘should’ generate one HTML page for each configured resource for each vmware probe instance. Multi-threaded.
      o NOTE: This connects to vSphere directly so the probe will need to be on the same physical network as the Vcenter          or ESX host. The vmware probe doesn’t collect the needed attributes out of the box, so that is why it connects                  directly to vSphere.
• Retrieve list of current USM Groups (Will be needed for future USM group creation feature)

 

 

 

TODO LIST
• Create USM groups from JSON
• Store data in H2 database for license usage over time
• Improve configuration archive and diff reports (configuration_archive functionality)
• Add more probe threshold reports (logmon)
• Improve probe configuration through JSON to support multiple robots and probes
• Continue testing the VMware topology feature

 

 

 

 

REVISION HISTORY

Date

Version

Change

July 22nd, 2016

1.00

Initial Draft

July 22nd, 2016

1.01

Added devices_with_no_data

July 25th, 2016

1.02

Devices with no data always creates a CSV, no matter the size. Initial merge of healthcheck probe with just list_subscribers creating a CSV.

July 28th, 2016

1.03

Added automation_device_wiper callback

September 9th, 2016

1.04

Removed decrypt password option, added list_accounts/users, can now provide a CSV list of devices for removal

October 4th, 2016

1.05

Added ability to manually configure probe configurations and new licensing report.

October 26th, 2016

1.10

Added niscache_clean, modify probe configurations from json, UMP/database health report, threshold gathering and reporting and vmware topology

November 8th, 2016

1.11

Added support file retrieval and processes/ntservices threshold reports

November 21st, 2016

1.12

Delete QOS by target, inactive probe report and added filtering for threshold gathering and reporting

December 7th, 2016

1.13

Top probe usage reports, configuration_archive custom probe integration

January 3rd, 2017

1.14

mysql connectivity fixed, added uim topology map, top_n parameter to top_probes report

This is a newer version of the probe_configuration_archive that was available on the CA Marketplace. This probe offers the same features, but does so with multi-threading capabilities. This post will hopefully be a living document where people will comment on the current state and also additional features they would like to see implemented. I will attach the latest documentation and probe package to this post, so please subscribe for updates.

 

Probe Features

This multi-threaded probe will archive every probe configuration in your UIM environment and automatically do configuration differentials between the current and previous configurations. This will allow an administrator to track configuration changes in their environment automatically. This probe base is derived from the previous single threaded probe_configuration_archive.

 

Prerequisites

  • UIM Server version 8.0 or greater
  • Java_jre version >= 1.7

 

Probe Configuration

This probe can only be configured using Raw Configure mode.

raw_config_v1.png

ignored_probes: Use a comma separated list of probe configurations you wish to be excluded from collection

archive_path: File system location where you wish to store the configuration files

threads: The number of threads used while doing probe configuration gathering

 

RUNNING THE ARCHIVE

The current version is driven from the rebuild_configuration_database callback, please run this whenever you want to archive your configurations. Future version will re-implement an interval based archive.

probe_utility.png

 

TECHNICAL INFORMATION

The probe uses an internal JDBC capable database (http://www.h2database.com) to store the probe configurations and also the configuration change data. This allows the data to still be accessible for custom dashboards and reports, but also avoids using the UIM database as a datastore. I chose to stick with the raw_configure option as opposed to the newer Probe Framework SDK because of the multi-threading that was implemented. The multi-threading should speed up the configuration gathering quite significantly compared to the older probe_configuration_archive.

CONFIGURATION_ARCHIVE_CURRENT TABLE

This table contains a record of each probe configuration. It stores the origin, hub, robot, probe, probe version, filesystem path to configuration file, complete configuration file in a column, a record id and archive date.

CONFIGURATION_CHANGES TABLE

This table contains a record of each probe configuration change. It stores the hub, robot, probe, configuration section, configuration key, previous and current configuration value, previous and current archive date and previous and current record ids.

HOW IT WORKS

The probe uses the native callbacks provided in many probes to get the probe configurations.

  1. It uses the gethubs callback on the hub probe to get a list of available hubs
  2. Loops through each of those hubs to get a list of robots using the hub’s getrobots callback
  3. (Multi-threaded) Grabs a list of probes and information on each probe from the controller via the probe_list callback
  4. (Multi-threaded) Saves the configuration file from the robot by using the text_file_get callback
  5. (Multi-threaded) Inserts information about the configuration into the database
  6. Compares the current and previous configuration to determine if there were any changes
  7. If there are changes, writes those changes to the database

 

JDBC CONNECTIVITY

In order to query the data from the UMP you will need to copy the h2 database driver jar file into the <Nimsoft_installation_directory>\probes\service\wasp\libs directory. The driver can be found Here. You will then need to create the data source in the Dashboard Designer.

database_configuration.png

  • Name: Logical name for the connection
  • Driver Class: org.h2.Driver
  • JDBC URL: jdbc:h2:tcp://<IPADDRESS_OF_PROBE_CONFIGURATION_ROBOT/./configuration_archive
  • User: uim
  • Password: uim

 

KNOWN BUGS

  • Ignore List function is currently not working
  • Sometimes when re-deploying you may need to deactivate->reactivate the probe for it to start properly

TODO LIST

  • Enable file and table retention
  • Add callbacks for adhoc differential between configurations based on record/date
  • Add callback option to replace current probe configuration with archived configuration

 

Again, please feel free to comment directly on this post with any feedback or suggestions for the probe. I plan to continue development on this for quite some time.

 

Thanks,

 

Bryan