DX Application Performance Management

APM 10.5.1.8 - This breach did not send an email.  I expect this to be considered breached and a email or any action configured to be fired.   I vaguely remember working on something similar and don't remember if this was resolved with a hotfix or deemed working 'poorly' as intended. 

Can someone help Chris?

Please change "trigger Alert Notification" to Whenever Severity Changes from When Severity Increases

that may help. its all about when/what condition you want alert

Chris,

The problem here is the "Trigger Alert Notification".  You have it set to "Whenever Severity Increases" and in your case, the state has not increase in state (good-caution, good-error, caution-error) so the alert was not triggered.

As Ranganathan noted, if you change to "Whenever Severity Changes", when the state moves across a threshold, then you would get an alert.

Now if another case, if you set the trigger to "Each Period While Problem Exists", if the metric was above threshold after the "resolution" cycle, you would have gotten another message.

Hope this helps,

Billy

I'm not so sure that is correct. 

From Level 2 I received the following states of alerts 

The state of the alert may be Not Reporting (0), Normal (1), Caution (2) and Danger (3)

My screenshot shows my metric going from a Not Reporting (0) to a Danger (3).  This should trigger my alert considering the severity increased. 

Hi Chris,

The screen shot shows that the first reported metric value was 1 at 12:16:45 then at 12:17:15 it was at zero. 

Before 12:16:45, there was no value and no metric meta data reported.  This is a different state than not reporting with meta data, state 0.  Since there was no meta data (agent/metric label), the metric grouping the alert is based on could not pick up the non-presence of the metric.  The first matching metric value, based on the alert metric grouping would have been the value 1 at 12:16:30.  The next value, being at 12:17:15, was reported as a zero, which would have been a decreased value and would not have triggered the alert.

I usually tell someone that asks if the APM will alert if the agent stops reporting, that if there is no data, there is no alert.

There are a number of tricks you can do to get an alert on a data point that does not report data, but you have to use a javascript calculator or a sum calculator to retrieve the metric and if there is no results of the query, then publish a metric with a value, then alert on the calculator value that represents that the original value is not reporting.

Hope this helps,

Billy

I completely agree with what you said.   And this is exactly right. 

 if the agent stops reporting, that if there is no data, there is no alert.

However as I mentioned when I once investigated a behavior like this with Level 2 they revealed that a no metric represents a 0 and if I am breaching for the first metric this should 'technically' be considered a increase in severity.    In my case i'm ok with the behaviour since my agents should never be disconnected.   In my clients case I do remember they had a different scenario where there where metrics that would often have aged out and not reporting but then come to life.   They wanted to be able to capture a breached threshold in this scenario.    I think in the end they did in fact have to change the operator despite not wanting to.   ( to severity changes I think ) 

Thanks for your help on this.

0 means no data to report so no, it will trigger an alert.