We are in the process of going live with our new ticketing system, ServiceNow. Everything is working as expected, except for 1 very important piece, which is when we Resolve a Ticket in ServiceNow, that alert is not being Acknowledged in Nim Alert Console.
We have 2 environments, one for testing and one for production. The alerts that we are sending over to production ServiceNow from test Nim environment, are being Acknowledged when the ticket is Resolved. So everything works as expected there within Test. But when the same alert is sent from production Nim environment, to the production instance of ServiceNow, arlarms are NOT being Acknowledged when ticket is resolved.
After digging through the Trace logs of the sdgtw probe, I found the issue, or at least something pointing to what the issue may be, which is:
Lab/Test environment log file for sdgtw probe, which Acknowledges the alert in Nim properly----------
Oct 19 12:43:22:970 [ServiceNow, sdgtw] user idadministrator
Oct 19 12:43:22:993 [ServiceNow, sdgtw] alarmlist[com.nimsoft.events.api.model.Alarm@7fd43795]
Oct 19 12:43:23:117 [ServiceNow, sdgtw] [clearAlarmsForClosedIncidents ()]: Alarms acknowledgment successfully using AlarmService API aId[DV11899587-80892]
Oct 19 12:43:23:117 [ServiceNow, sdgtw] Clearing the cache for alarm with Id - DV11899587-80892 and incidentId 62d2aa74db55e780995a791c8c9619fd
Production log file for sdgtw probe, which IS NOT Acknowledging the alert in Nim properly----------
Oct 19 14:59:17:981 [ServiceNow, sdgtw] user idadministrator
Oct 19 14:59:18:101 [ServiceNow, sdgtw] alarmlist
Oct 19 14:59:18:101 [ServiceNow, sdgtw] Clearing the cache for alarm with Id - SV16579396-19936 and incidentId 32537abcdb55e780995a791c8c9619ae
As you can see, the alarmlist is empty, and we don't get the successful Acknowledgement message. After digging some more, I found the following Error in the log file for production Nim:
Error while connecting AlarmService API Reason: (11) command not found, Received status (11) on response (for sendRcv) for cmd = 'dispatcher' ST
Its worth noting that I am currently on our Secondary Hub within our Nim UIM environment. But the Secondary hub is acting as our primary, which houses the following probes to support the sdgtw to servicenow integration:
- TRELLIS - When I noticed that AlarmServiceAPI error, I thought for sure I found the issue. So I checked the Probe Utility for the trellis and it only HAD: alarm-routing-service / nas services / trellis container core services. I then dropped the following probes: nas-api-service / das access services
Unfortunately, that did not resolve the issue.
Perhaps there is another probe that is needed for handling dispatching (maybe the UDM_Manager)?
Additionally, I did open up a case to CA Support and they said that, per the documentation, the Secondary hub does not support the sdgtw probe. But being that are secondary is acting as the primary, I suspect we may just be missing a probe to facilitate the communications for Acknowledging an alert.
Any assistance would be greatly appreciated!