DX Unified Infrastructure Management

Expand all | Collapse all

Max. restarts reached for probe 'wasp' (command =

  • 1.  Max. restarts reached for probe 'wasp' (command =

    Posted Mar 01, 2016 07:28 PM

    After starting up the system, most of the probes start up, except 'wasp refuses to start.

    The wasp log shows only:

     

    Mar 1 17:58:31:397 [140439341786944] Controller: Max. restarts reached for probe 'wasp' (command = <startup java>)

     

    I've restarted everything a couple of times, and nothing seems to help.

     

    I'm running UIMServer, version 8.31.  The wasp probe is version 8.31.

     

    Any help would be appreciated, as I don't know how to get the wasp probe up and running.  Thanks!



  • 2.  Re: Max. restarts reached for probe 'wasp' (command =

    Broadcom Employee
    Posted Mar 01, 2016 07:33 PM

    Lets try to see if the issue is resolved after deploying "java_jre" archive to your robot where "wasp" probe is installed.



  • 3.  Re: Max. restarts reached for probe 'wasp' (command =

    Posted Mar 02, 2016 01:58 PM

    Thanks for the reply!

    I've installed the "java_jre" package, version 1.71, build 126, and restarted the UIM server.  Unfortunately, the wasp probe shows the same error after the restart.

     

    Any other ideas that I can try?



  • 4.  Re: Max. restarts reached for probe 'wasp' (command =

    Posted Mar 02, 2016 02:29 PM

    Is anything being logged in the wasp.log file? If the probe is actually starting but then stopping/crashing, there should be something in the log (even if not helpful). If the controller can't start the Java process at all, then nothing would get added to wasp.log.



  • 5.  Re: Max. restarts reached for probe 'wasp' (command =

    Posted Mar 02, 2016 02:41 PM

    Hi Keith,

     

    Yea, in my original post I show what's in the wasp.log file.  That one line is the only thing that shows up there.

     

    Thanks.



  • 6.  Re: Max. restarts reached for probe 'wasp' (command =

    Posted Mar 02, 2016 03:41 PM

    Ah, that's the same as what would be in the controller.log file. If that's all you see, then the controller is not successfully starting java.exe. You should be able to test that outside of the controller by running the java.exe process manually, but you have to nail down the command line arguments and probably some of the environment. You would have to run it from the wasp directory and use the command line arguments listed on the wasp probe entry. The environment variables should be displayed somewhere in the controller properties, although I only know how to get them from Infrastructure Manager off the top of my head. I'm not sure how that works in Admin Console.

     

    I think I noticed in the java_jre release note recently that having more than one version of Java installed could be problematic, and it recommended removing old versions. You might want to check if that could be the case.



  • 7.  Re: Max. restarts reached for probe 'wasp' (command =

    Posted Mar 02, 2016 03:54 PM

    In UIM 8.2, the command line arguments for our wasp probe are:

     

    -server -cp "lib/*;i18n" -Djava.util.logging.config.file=conf/logging.properties -Djava.security.auth.login.config=conf/NimJAAS.config -Djava.library.path=../../../lib -Djs.license.directory=conf com.nimsoft.nimbus.probe.service.wasp.Probe

     

    The environment variables specific to Java are:

     

    NIM_JRE_HOME = jre/jre7
    NIM_JRE_HOME_1_7 = jre/jre7

     

    There are a bunch of other environment variables specific to Nimsoft, but I'm not sure which ones will matter. That's a bit trickier to determine. Running from the command line can sometimes be a great way to see why a probe won't start at all, but it is not always helpful.



  • 8.  Re: Max. restarts reached for probe 'wasp' (command =

    Broadcom Employee
    Posted Mar 03, 2016 02:31 AM

    Can you deactivate / activate WASP only and see any change ?



  • 9.  Re: Max. restarts reached for probe 'wasp' (command =

    Posted Mar 03, 2016 11:53 AM

    Deactivating/reactivating, restarting the server, all give the same result, unfortunately.



  • 10.  Re: Max. restarts reached for probe 'wasp' (command =

    Posted Mar 03, 2016 11:52 AM

    If I start wasp from its own directory, this is what I get:

     

    [root@nimsoft-hub-1 wasp]# /opt/nimsoft/jre/jre7/bin/java -server -classpath "lib/*" -Djava.util.logging.config.file=conf/logging.properties -Djava.security.auth.login.config=conf/NimJAAS.config -Djava.library.path=../../../lib -Djs.license.directory=conf com.nimsoft.nimbus.probe.service.wasp.Probe
    Loading jar:file:/opt/nimsoft/probes/service/wasp/webapps/ROOT/WEB-INF/lib/portal-impl.jar!/system.properties
    log4j:WARN No appenders could be found for logger (org.apache.commons.configuration.ConfigurationUtils).
    log4j:WARN Please initialize the log4j system properly.
    log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
    Loading jar:file:/opt/nimsoft/probes/service/wasp/webapps/ROOT/WEB-INF/lib/portal-impl.jar!/portal.properties
    Loading file:/opt/nimsoft/probes/service/wasp/webapps/ROOT/WEB-INF/classes/portal-ext.properties
    

     

    It just stops at that last Loading file line.  I don't know if it'll continue on after some time, but at the very least it's starting.

    Is it possible that the wasp probe is using the wrong JRE somehow?  I don't have Java installed on this machine other than the JRE(s) installed with the UIM Server.  Do I need to tell the probe(s) which JRE to use or something?  Seems strange that only wasp would have an issue with finding the right Java, but I can sure point it to the right one if that's the issue.

     

    Thanks again for all the help.



  • 11.  Re: Max. restarts reached for probe 'wasp' (command =

    Broadcom Employee
    Posted Mar 03, 2016 12:41 PM

    Hi there,

     

    What memory do you have allocated to the wasp probe?

    Open the wasp configuration GUI and check on the Setup-->General tab.

    What do you see configured for the initial and maximum memory fields?

     

    I recommend setting these values to at least 1024 initial and 2048 max - you can allocate more, as needed.

    Those setting should give most wasp configurations adequate resources to get up and continue running.

     

    Phil



  • 12.  Re: Max. restarts reached for probe 'wasp' (command =

    Posted Mar 03, 2016 03:54 PM

    Philip,

     

    I've bumped the memory for Xms and Xmx to your recommended values.  Unfortunately I receive the same line in the log file.  I'm not sure it's getting far enough to even run into a memory problem.  Basically as soon as I start or restart wasp, this message appears.

     

    Thanks for the recommendation.



  • 13.  Re: Max. restarts reached for probe 'wasp' (command =
    Best Answer

    Posted Mar 03, 2016 11:59 PM

    I was informed by customer support that the wasp probe is not supported on CentOS 7, which is what I was using.  Fair enough.

     

    I have since reinstalled everything on CentOS 6, and things are working for now.

     

    Thanks everyone for your suggestions.



  • 14.  Re: Max. restarts reached for probe 'wasp' (command =

    Posted Mar 04, 2016 12:26 AM

    Ugh. CentOS 7 has only been around for almost 20 months, so I guess you're an early adopter!

     

    Thanks for sharing the insight!