DX NetOps

Struggling with setting up automated alarm email notifications. Mail relay works through OneClick, I can select an alarm from the contents pane and manually mail it to the desired address. SANM Application and Policy has been set up against a specific global collection, security string matches, running AlarmNotifier from bash produces results. It's not an aggressive filter either, set against the landscape and that global collection. everything else is default other than the specific notification email address. The same address One Click successfully delivers to when doing it manually. The emails just don't go out from AlarmNotifier.

Smail registry entry was set to the same smtphost as OneClick.  Used the FQDN, and tried swapping that out for just the IP. Neither has worked so far. How important are the username and hostname fields within the Smail registry entry?

Processd is set to autobootstart and autorestart.

What else do I need to look at? I've been pouring over documentation and so far it looks like what I have setup SHOULD work, but it doesn't. Need help. Lot of pressure from up the chain to get this done.

Jayakrishna,

This is on windows. I have tried the mail command. An item I'm unsure of is does the -u and -m parameters have to be on the same domain as the mailserverhost? I ask because notifications will be going out to an enterprise email system separate from our domain, the mail server here is functioning as a relay agent to that system. If so, we can set up a local email account on this domain for that to match.

We don't want the notifications to go to a spectrum user, rather to techs on other teams and leadership personnel. So the notification  data in the policy filter will end up pointing at an enterprise email distro list. Which I believe is possible right?

Hello,

That is functional now. Played around with the settings some and the -u and -m parameters do not seem to be a major factor. having the FQDN in there did seem to be a factor here. Using either the IP or just the hostname would not work.  So the server itself has the ability to send the messages. What remains now is finding what is not configured properly.

Automated email notifications are still not going out.

That it is. The same one I manually sent mail to from OneClick on my host, and, using the mail command in bash on the server itself just before. When running AlarmNotifier in bash, it is not presenting any errors.

However, I also do not see anything like "sending mail to, or sending notification to" within the entries that populate. What other files can we check to make sure everything is setup properly to automate AlarmNotifier  running continuously and sending email notifications?

With the Notification Data field filled in for your filter, if the alarm passes the filter, the e-mail address in the Notification Data field should be passed to AlarmNotifier and you should see a message in the $SPECROOT/Notifier/NOTIFIER.OUT file stating it is sending e-mail.

One thing I did notice are the scripts you attached have a ".txt" extension. What is the SET_SCRIPT parameter configured for in the $SPECROOT/Notifier/.alarmrc file? Out of the box it is set to "SET_SCRIPT=/app/spectrum/Notifier/SetScript". If that is what yours is set to and the file name you want to use is SetScript.txt, then that could be your issue. Rename your script to SetScript (remove the txt), restart AlarmNotifier and see if that works.

Not sure why the system put the .txt on those, but on the server that is not present. They all show as "File" as type within the directory.

I cut out all of the alarm data from the notifier file after the error because I can't share that specific data on here.

What did you use to edit the AlarmNotifier scripts?

Open a bash shell by running "bash -login". cd to the $SPECROOT/Notifier directory and enter "ls -al". Is there a file named "SetScript.txt?

I used notepad++

After running the command there are no .txt files in the directory at all.

That worked when manually running the instance of AlarmNotifier in bash. I created a unique named instance for the global collection (ex - AlarmNotifier -n test_123), so, to have that running in the background consistently do I need to create a new .idb file for it in the SDPM/partslist directory? Would I also need to create new instances of the scripts for each uniquely named AlarmNotifier file? Leadership wants to have a few separately monitored items, so, trying to figure out the correct route for that to happen.

Correct me if I'm wrong, but, it looks like I would need a unique AlarmNotifier file and a unique Setscript/Updatescript/Clearscript....then also a unique .idb file set for them to autorestart/autobootstart so it will be running in the background to send out email notifications as they happen. For each set of required alarms leadership wants. I think I should be able to create a custom global collection for each set of those and just set the policies against those to get that to work. Please advise...and thank you for all of your help so far. You all have been great.

One instance of AlarmNotifier means, dedicate SetScript, ClearScript and UpdateScripts. It means also dedicated Processd ticket (.idb). I usually have 2 AlarmNotifier instances: one used for sending emails, another one for sending SMS. In theory you can use a single AlarmNotifier instance for doing both, but it easier to setup and process the notification if you have dedicated instances. It also works faster that way.

What do you mean by "have a few separately monitored items". What's the exact requirement?. You don't have to have separate scripts for notifying differently. Maybe different filters set in the Policy will be enough.

CatalinF,

I figured each .alarmrc file would need unique script files that it needs to point to within it. Is there a field I need to modify in the new Processd ticket to tell it to point at the other .alarmrc file?

So the requirement is they want a specific link to only email notify when it either goes down or changes something like OSPF status, while other links, on other devices, they want similar notifications but with specific criteria. But they do not want those email notifications, as of yet, for all of the other links in our AOR. So I thought the best way to do that is to make 2 separate GC's for each of those requirements and them apply custom policy filters to them to capture what they want. Just add in the interfaces on each side of that one link in one GC and then the devices/interfaces for the other links they want into the other GC.

The .alarmrc resource file is read automatically from the AlarmNotifier $WORKPATH. Actually that defines the location from with the executable in the ticket should be launched. If you want to specify your own .alarmrc, you have to do it by launching the AlarmNotifier with the -r switch and specify the resource file to be used.

Your requirement will be done using filters in the policy. Basically you create a new AlarmNotifier instance when the notification process changes dramatically. And by that I mean all the processes that you involve in sending a notification, the message of the notification and so on. Filtering of notification receipts and sent alarms is done based in filters in the policy. Create your own GC and specify your criterias in the filters.