DX NetOps

Expand all | Collapse all

Spectro Watch to alert on a % change in traffic on a port - help

  • 1.  Spectro Watch to alert on a % change in traffic on a port - help

    Posted Feb 27, 2012 05:18 PM
    Hi
    I've only just got onto Spectrum 9.2 from 8.1 (Linux) so am new to the Watch modelling
    I need to create a watch on a number of ports that alerts us on a drop in total traffic on that port of 50% or more in a 5 minute period

    I've tryed to create the watch from scratch and am failing miserably - I can't seem to get beyond the "syntax error in the watch description" no matter what I try

    I think I need to head down this path of
    INTEGER ( ( COUNTER_DELTA( ifHCInOctets ) * 100 ) / ( INTEGER( ifSpeed ) * 300 ) )


    to return the value but as I say not getting far

    Any assistance appreciated


  • 2.  RE: Spectro Watch to alert on a % change in traffic on a port - help

    Posted Feb 28, 2012 01:43 PM
    I can try to help you if u want, but u will need to provide me for info.


    First, which attribute contains the utilization of the port?

    Second, when u want to generate an event/alarm?
    -You take one value every 5 minutes and if in between those 5 minutes it drops below 50% you generate an alarm?
    Or maybe take the maximun value in a period of 5 minutes, and then if it drops below 50% for more than 140 seconds generate an alarm.
    U need to be more explicit in the conditions u want to make.


    As i can see from your watch expresion u are not referencing the instance.
    Almost sure that ifHCInOctets and ifSpeed are tables, this means, you must do 1 watch per instance and call it like: INTEGER ( ( COUNTER_DELTA( ifHCInOctets.x ) * 100 ) / ( INTEGER( ifSpeed.x ) * 300 ) ) where x is the number of the instance (1,2,3,...,x)

    or make a watch for all the instance (or a range) and call it like: INTEGER ( ( COUNTER_DELTA( ifHCInOctets.# ) * 100 ) / ( INTEGER( ifSpeed.# ) * 300 ) )


  • 3.  RE: Spectro Watch to alert on a % change in traffic on a port - help

    Posted Feb 28, 2012 02:48 PM

    dzuleta wrote:

    I can try to help you if u want, but u will need to provide me for info.


    First, which attribute contains the utilization of the port?

    Second, when u want to generate an event/alarm?
    -You take one value every 5 minutes and if in between those 5 minutes it drops below 50% you generate an alarm?
    Or maybe take the maximun value in a period of 5 minutes, and then if it drops below 50% for more than 140 seconds generate an alarm.
    U need to be more explicit in the conditions u want to make.


    As i can see from your watch expresion u are not referencing the instance.
    Almost sure that ifHCInOctets and ifSpeed are tables, this means, you must do 1 watch per instance and call it like: INTEGER ( ( COUNTER_DELTA( ifHCInOctets.x ) * 100 ) / ( INTEGER( ifSpeed.x ) * 300 ) ) where x is the number of the instance (1,2,3,...,x)

    or make a watch for all the instance (or a range) and call it like: INTEGER ( ( COUNTER_DELTA( ifHCInOctets.# ) * 100 ) / ( INTEGER( ifSpeed.# ) * 300 ) )
    Hi
    Thanks for the info
    Interestingly I think I have a bug with the Watcheditor - no matter what I do - including following the manuals examples - I get a "there is a syntax error in the watch expression" - so am checkign with CA

    Right - on to the questions
    As to the generation of the alarm it will be
    take one reading (a), 5 minutes later take another reading (b)
    if there is a 50% drop from (a) to (b) then generate an event
    What I will do then is use EventDisp to create an escalation process

    As to the watch expression - so even if the watch is against the port model it needs the instance number? I was going off the manual (which ia a bit vague ) and using some examples of port types to work out the syntax (badly of course)
    Adding an instance is fine as its only a subset of ports
    Wondering is one could use a variable to pick up the ifindex value by defining a watch to get the index of the port

    Anyway its only a small number of ports so I can create a limited specific watch for each port it nescessary

    As to the exact syntax to work out load - just working out the best attribute there, trouble is with the watch editor giving me borken values tricky to confirm the exact value.


  • 4.  RE: Spectro Watch to alert on a % change in traffic on a port - help

    Posted Feb 28, 2012 03:51 PM
    About the bug, i think the same thing happended to me once or twice. Restarting the Spectro server fix it. Hopefully u are doing the configurations in a laboratory enviroment.
    The bug happended to me after a few bad configurations in the watch editor while i was learning, then no matter what i put it would keep giving me that error.

    Maybe before restarting the spectro server try restarting the tomcat service or closing and opening the oneclick console. To really check if it is a bug or just configuration problem do an easy test like monitoring an attribute. Just put in the expresion section the name of the attribute and thats it (watch the image attached).

    If you make the watch in the port model, then you dont need to reference the instance. Check the attribute editor to be sure, if the attribute u want to use is an array then u will need to use the instance, otherwise you can just put the attribute. In my opinion, if u need to do the watch for more than 1 port on the same device it would be easier to make only one watch for a range of instance.

    I couldnt understand what u tried to say with this: "Wondering is one could use a variable to pick up the ifindex value by defining a watch to get the index of the port"


    To achieve the generation of the alarm i would do something like this:

    watch_a:
    expresion: %utilization
    evaluation: by polling 300 seconds


    watch_b:
    expresion: watch_a
    evaluation: On demand


    watch_utilization_drop
    expresion: %utilization/2 <= watch_a
    evaluation: by polling 300 seconds


    My only issue about this idea is that i am not sure which watch will run first. If watch_utilization_drop runs first then it will work fine, but if watch_a runs first the results will not be the one expected.
    I will think another solution to be able to avoid this issue


  • 5.  RE: Spectro Watch to alert on a % change in traffic on a port - help

    Posted Feb 28, 2012 04:30 PM
    Hi Daniel:

    I justed wanted to take a moment to thank you for you for all of the help you provide to the community.

    Thanks so much!!!!!!!!!!!!!!!!!!!

    Mary


  • 6.  RE: Spectro Watch to alert on a % change in traffic on a port - help

    Posted Feb 29, 2012 08:10 AM
    You're welcome Mary,

    I think Spectrum is a very good software and i am looking forward to improve my expertise in the subject.

    Being a tight community and helping each other will make us all become better professionals.
    I have noticed that lately more people has been participating in this forum so i am glad about it, hopefully we can solve each others problem as a community


  • 7.  RE: Spectro Watch to alert on a % change in traffic on a port - help

    Posted Mar 22, 2012 02:42 PM

    MaryGreening wrote:

    Hi Daniel:

    I justed wanted to take a moment to thank you for you for all of the help you provide to the community.

    Thanks so much!!!!!!!!!!!!!!!!!!!

    Mary
    Me too, Daniel. It is very much appreciated!

    -Scott (SDBase)


  • 8.  RE: Spectro Watch to alert on a % change in traffic on a port - help

    Posted Mar 26, 2012 08:36 AM
    Glad that i can help :)

    By the way, how does SSLOGGER works and/or in what documentation i can find information about it?

    Thanks,
    Daniel


  • 9.  RE: Spectro Watch to alert on a % change in traffic on a port - help

    Posted Mar 27, 2012 06:41 PM
    SPECTRUM_SSLOGGER_User :)

    Just look inside your docs folder.

    Been using SSLOGGER for years for graphs - I love it.


  • 10.  RE: Spectro Watch to alert on a % change in traffic on a port - help

    Posted Mar 27, 2012 07:01 PM
    @jasonheb

    I believe you need to refine a little bit your idea because at this moment is a little bit to vague for a successful watch.
    As Daniel pointed out in his previous threads - you will need a kind of referrence.

    Are you expecting a -quite- constant traffic through those ports? (let's say they are Gigabit ports and the usual traffic is around 500Mb/s ... then any drop under 250Mb for more than 5 minutes should generate an alarm)
    OR
    The traffic is in no way constant and you need to "know" when the traffic drops more than 50% over 5 mintes.

    These are two scenarios but the difference in the watches is more than "significant"

    Thinking about "instances" - I am not 100% sure but I would say that you don't need any instance here - the watch is active on a model and it will use the attributes relevant to that model only.

    Jason, give it a little bit of thinking and maybe you can try a different approach.


  • 11.  RE: Spectro Watch to alert on a % change in traffic on a port - help
    Best Answer

    Posted Mar 27, 2012 07:16 PM
    Hi

    I've built a good working solution outside of spectrowatch using the SSLOGGER tool - we use that anyway for graphing and traffic revenue generation processes.
    What I have done is parsed through the SSLOGGER generated logs and pulled the last x in and out stats for a group of ports (using Global Collecitons to define which ports we care about)
    I then take the minute readings and do some basic calcutaitons to get the diff values minute to minute (with additional checking to avoid issues of polls being too far apart)
    I use a value from 5 minute ago and 10 minutes ago as the bas lines then do usage comparisons
    The rules I use are
    if 1min ago is 50% or less than 5 min ago AND 3 min ago is 50% or less than 5 min ago (these take care of down spikes) AND 1 min ago is 50% or less than 10 min ago then generate alarms

    All testing is showing it as woreking well and has the added advantage of having the raw data for checking stats later if questioned

    Re the question of SSLOGGER - its now a retired component that was part of the SRG - see older version docs - but its still downloadable and usable.
    If you don't have EHEALTH it gives you a basic tool set that you can manipulate to provide some useful info


  • 12.  RE: Spectro Watch to alert on a % change in traffic on a port - help

    Posted Mar 27, 2012 07:32 PM
    Edited: what I posted (before editing this) won't work.
    My bad - sorry about that. (just in case you read the post in the past 5 minutes)

    Glad you found a solution with SSLOGGER.


  • 13.  RE: Spectro Watch to alert on a % change in traffic on a port - help

    Posted Mar 27, 2012 07:46 PM
    If anyone wants the basic bash script that does this give me a yell (thats the SSLOGGER solution)


  • 14.  RE: Spectro Watch to alert on a % change in traffic on a port - help

    Posted Mar 28, 2012 09:56 AM
    Nice that you could achieve ur goal Jason, congrats.

    I will take your offer of the script tho, I have just started reading the SSLOGGER documentation since you guys mentioned it, so it could be helpfull to have your script as an example. It seems like a good tool to know about.

    Thanks,
    Daniel

    PD: My email is dzuleta@neosecure.com if you wanna contact me


  • 15.  RE: Spectro Watch to alert on a % change in traffic on a port - help

    Posted Feb 28, 2012 05:40 PM
    Thanks for the feedback - yep I thought it may be a restart spectroserver to fix issue
    Yesterday I tried creating the simplest of watches (which I had done previously) and that also failed, even editing existing attribute check watches returned a failure. Oh well, time to restart SpectroServer.

    "I couldnt understand what u tried to say with this: "Wondering is one could use a variable to pick up the ifindex value by defining a watch to get the index of the port"
    As to that statement - I've noticed it seems possible to return a value from one watch into a second watch - so my musing was whether I can do a simple watch to ascertain the ifindex on a port then feed it into a following watch - but this is somewhat irrelecany



    "To achieve the generation of the alarm i would do something like this:

    watch_a:
    expresion: %utilization
    evaluation: by polling 300 seconds


    watch_b:
    expresion: watch_a
    evaluation: On demand


    watch_utilization_drop
    expresion: %utilization/2 <= watch_a
    evaluation: by polling 300 seconds


    My only issue about this idea is that i am not sure which watch will run first. If watch_utilization_drop runs first then it will work fine, but if watch_a runs first the results will not be the one expected.
    I will think another solution to be able to avoid this issue"


    Thanks for that - thats kind of what I thougth mauy need to happen. I've built the modeling using a shell script combined with SSLOGGER to return values (old unsupport SSLOGGER is such a useful tool) and worked out a method that way - but doing something in oneclick is much better and will make life easier for the end users


  • 16.  RE: Spectro Watch to alert on a % change in traffic on a port - help

    Posted Feb 29, 2012 08:13 AM
    I havent use the SSLOGGER, but i have combined watchs invoking shell scripts that updates attributes through the VNMSH. (watch -> shell script -> VNMSH -> watch).

    Not sure what SSLOGGER does, but pretty sure u can achieve the same thing with a script (any language) and VNMSH.


  • 17.  RE: Spectro Watch to alert on a % change in traffic on a port - help

    Posted Mar 21, 2012 07:20 PM
    Hi
    SSLOGGER is an older tool set that allows polling of models and recording that polled data into text files
    What I've done is created a process that polls various port traffic attributes using SSLOGGER and then uses some basic scritping to perform a series of comparisons of traffic stats and bandwitch utilisations over various time periods.
    The fun part has ben working out the best way of avoiding alerting on Spikes up and very short term drops in traffic. The scripts then use the CLI to create events against the various port models and escalate to alarms as appropriate to the port criticality.
    All seems to be working well now, still tuning it a bit


  • 18.  RE: Spectro Watch to alert on a % change in traffic on a port - help

    Posted Mar 22, 2012 12:57 PM
    Hi Jason:

    It looks like Daniel was able to help you out. Make sure check the "Mark as Accepted Solution" box in the posts that answered your questions.

    Thanks!
    Mary