IT Process Automation

Expand all | Collapse all

Alto consumo en cpu en PAM / High cpu usage PAM

  • 1.  Alto consumo en cpu en PAM / High cpu usage PAM

    Posted Apr 04, 2016 05:39 PM

    Hola,

     

    Tengo un problema con el proceso de java, actualmente esta consumiendo el 100% del cpu, Heemos seguido la indicaciones como se indican en el siguiente enlace High CPU usage in ITPAM and Job progress slow  y no se resuelve el problema.

     

    Adicionalmente se ha probado con herramientas como jps, jconsole y jvisualvm, no muestra el pid java asociado al PAM, ¿existe forma de identificar el hilo que esta consumiendo los recursos del cpu?.

     

    Saludos.

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Hello,

     

    I have a problem with java process currently is consuming 100% cpu, followed the directions as indicated in the following link High CPU usage in ITPAM and Job progress slow  and the problem is not resolved.

     

    Additionally, it has been proven tools such as jps, jconsole and jvisualvm, not shows the java pid associated with PAM, ¿there is way to identify the thread that is consuming CPU resources?.

     

    Greetings.



  • 2.  Re: Alto consumo en cpu en PAM / High cpu usage PAM
    Best Answer

    Broadcom Employee
    Posted Apr 06, 2016 09:16 AM

    For a tool like jvisualvm to show PAM threads, you must perform the following steps:

     

    Add the following at the end of c2osvcw.conf file, so we can monitor the threads and memory. (c2osvcw.conf is located in the PAM install directory under server\c2o\bin)

     

    wrapper.java.additional.14=-Dcom.sun.management.jmxremote

    wrapper.java.additional.15=-Dcom.sun.management.jmxremote.port=40001

    wrapper.java.additional.16=-Dcom.sun.management.jmxremote.ssl=false

    wrapper.java.additional.17=-Dcom.sun.management.jmxremote.authenticate=false

                   

    Note: the numbers (14, 15, 16, 17) should be consecutive to what you already have in the file.

     

    Save the file and restart the orchestrator.

     

    On the ITPAM server, go to a command prompt and type JVisualVM. If it returns an error that it can't be found, you may need to browse to the directory where it is located.  It is under your JDK install folder and in the bin directory.

     

    It may prompt you for some callibration which is fine. Click ok to go through that.

     

    Once you are in the tool, go to the File menu. Select-> Add JMX Connection and add the JMX Connection for hostname:port which would be your hostname for ITPAM and the port 40001 which we added to the conf file above.

     

    Once you have done this you will see your server name listed on the left screen. Double click that to see details on threads and memory.



  • 3.  Re: Alto consumo en cpu en PAM / High cpu usage PAM

    Posted Apr 08, 2016 10:27 AM

    Hello Andrew,

     

    Thanks for help, apply argument on config file and can look the thread what consume cpu.

     

    I think is a bug, but I not idea.

    Attachment(s)

    zip
    ITPAM LOG.txt.zip   1 KB 1 version


  • 4.  Re: Alto consumo en cpu en PAM / High cpu usage PAM

    Broadcom Employee
    Posted Apr 08, 2016 11:02 AM

    Miguel,

    This should probably be investigated by the support organization in more detail, but we maybe able to help if you can provide some additional detail.

    1. What version of Process Automation are you using?  Open Help / About and provide the build and version.  

     

    2. Is this a clustered environment or a single Orchestrator?

     

    3. What hardware is the Orchestrator installed on?   CPU and Memory

     

    4. How much is process automation doing?  How many running / waiting instances are currently showing on the operations tab?

     

    5. Are you keeping up with Archiving and Purging? 

         a.)  How many Completed / Failed / Aborted isntances show in the Current view on the operations tab?

         b.)  How many instances show in the Archive view in the Archive view on the operations tab?



  • 5.  Re: Alto consumo en cpu en PAM / High cpu usage PAM

    Posted Apr 08, 2016 11:55 AM

    Hello Michael,

     

    My Answers:

    1. What version of Process Automation are you using?  Open Help / About and provide the build and version.

         4.2 SP02

    2. Is this a clustered environment or a single Orchestrator?

         Single Orchestrator

    3. What hardware is the Orchestrator installed on?   CPU and Memory

         CPU Xeon ES-2665 2.4

         Memory 16GB

    4. How much is process automation doing?  How many running / waiting instances are currently showing on the operations tab?

         Approximately 430 on waiting

     

    5. Are you keeping up with Archiving and Purging?

         a.)  How many Completed / Failed / Aborted isntances show in the Current view on the operations tab?

              Approximately 10 completed

         b.)  How many instances show in the Archive view in the Archive view on the operations tab?

              Approximately 200 completed and 15 canceled.

     

    Space disk 8GB

     

    Thanks.



  • 6.  Re: Alto consumo en cpu en PAM / High cpu usage PAM

    Broadcom Employee
    Posted Apr 08, 2016 12:13 PM

    Miguel,

    It doesn't sound like it is load related unless it is due to what your processes are doing.

     

    The Orchestrator starts 2 Java.exe processes - how much memory is the larger of the 2 using?

    Then check \pam\server\c2o\bin for the memory settings:

     

    wrapper.java.initmemory=2048

    wrapper.java.maxmemory=2048

     

    These settings can be increased to 4096 (and should match) without other modifications.  If the larger of the two java.exe is using more than 4gb of memory then it maybe time to consider a cluster node to share the load.   We can increase the memory beyond 4gb, but from what you have indicated your load is not all that large so 4gb should be sufficient.

     

    Changing the memory will require PAM to be shut down.   While Pam is off, have your DBA review the connections to ensure there are no hung connections at the database.   For example on SQL if you run sp_who2 with PAM shut down there should be no connections from the orchestrator server.  If any are found the DBA should kill them before you start PAM back up.



  • 7.  Re: Alto consumo en cpu en PAM / High cpu usage PAM

    Posted Apr 08, 2016 12:27 PM

    Hello,

     

    The setting memory is:

     

    # Initial Java Heap Size (in MB)

    wrapper.java.initmemory=4096

    # Maximum Java Heap Size (in MB)

    wrapper.java.maxmemory=8096

     

    actually PAM process memory on windows is:

    java 2,844,572K

    java 256,628K

     

    I found error on log ActiveMQ, any relationship?

    2016-04-07 18:13:51,133 | WARN  | Store limit is 102400 mb, whilst the data directory: C:\Program Files\CA\PAM\activemq\activemq-data\Node1 only has 8645 mb of usable space | org.apache.activemq.broker.BrokerService | main

    2016-04-07 18:13:51,135 | ERROR | Temporary Store limit is 51200 mb, whilst the temporary data directory: C:\Program Files\CA\PAM\activemq\activemq-data\Node1\tmp_storage only has 8645 mb of usable space | org.apache.activemq.broker.BrokerService | main

    2016-04-07 18:13:51,137 | WARN  | Job Schedular Store limit is 50000 mb, whilst the data directory: C:\Program Files\CA\PAM\activemq\activemq-data\Node1\scheduler only has 8645 mb of usable space | org.apache.activemq.broker.BrokerService | main

    2016-04-07 18:14:13,663 | WARN  | Transport Connection to: tcp://172.26.109.249:56787 failed: java.net.SocketException: Software caused connection abort: recv failed | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///172.26.109.249:56787@61618

     

    Regards.



  • 8.  Re: Alto consumo en cpu en PAM / High cpu usage PAM

    Broadcom Employee
    Posted Apr 08, 2016 12:41 PM

    # Initial Java Heap Size (in MB)

    wrapper.java.initmemory=4096

    # Maximum Java Heap Size (in MB)

    wrapper.java.maxmemory=8096

     

    these values should match.  If PAM is only using 2.8gb of memory setting both to 4096 should be sufficient and will cut down on the load on java somewhat.

     

    I do not believe the other messages are anything more than warnings and should not be causing this problem.



  • 9.  Re: Alto consumo en cpu en PAM / High cpu usage PAM

    Posted Apr 08, 2016 01:49 PM

    Hello Michael,

     

    Ty for help, I later set to parameter memory to 4096, this server is production and not have permission for reboot PAM. 

     

    Show new graphics use cpu and memory and table thread and soo many message running MQ ¬¬



  • 10.  Re: Alto consumo en cpu en PAM / High cpu usage PAM

    Posted Jun 22, 2017 06:59 AM

    Was facing the similar issue, take a look at some of these solutions:

     

    Clear Event Viewer Logs
    Download and install all available Windows updates
    Scan your computer for viruses and malware
    Find and Disable the service that causes the “svchost” high CPU usage problem

     

    See brief solution here: http://errorcodespro.com/fix-svchost-exe-netsvcs-high-cpu-memory-usage/