AutoSys Workload Automation

  • 1.  Which System Agent log provides more details delayed processing problem

    Posted Jan 13, 2016 09:05 PM

    AutoSys Scheduler processes events as soon as it receives it example STARTING event at 02:03:58

    But the Agent for some reason is slow and does not send the RUNNING event until 02:09:24  

     

    Can someone point me to a System Agent log or something which defines the cause of the problem.

    I have looked at the System Activity at this time and looks CPU% usage was high.

    But is there a System Agent log which logs helps in the confirmation and give more details if possible.


    Status/[Event]     Time                Ntry ES      ProcessTime           Machine


    --------------  --------------------- --  --      --------------------- ----------------------------------------

    STARTING       01/13/2016      02:03:58    1  PD  01/13/2016     02:03:58        auggvdsp001.sys.au.eds.com

    RUNNING         01/13/2016     02:09:24    1  PD  01/13/2016      02:09:24       auggvdsp001.sys.au.eds.com



  • 2.  Re: Which System Agent log provides more details delayed processing problem
    Best Answer

    Posted Jan 13, 2016 09:58 PM

    Hi Ahmed,

     

     

    ·         The receiver.log shows all job-run requests from Scheduler and transmitter.log shows job status events sent to the Scheduler.

     

     

     

    receiver.log

     

     

     

    01/02/2016 06:05:00.462 IST+0530 2 TCP/IP Controller Plugin.Receiver pool thread <Regular:2>.CybReceiverSession.accept[:276] - Message received: 20160102 06050043-0530 WA_AGENT LNX_SCH 104.15003_1/WAAE_WF0.1/MAIN RUN . Data(Command="date") Env(AUTOSERV=LNX,AUTO_JOB_NAME="22041247_exclude_cal",AUTORUN="15003-1") User(autosys) Password(XXXX) MFUser(autosys)

     

     

     

    transmitter.log

     

     

     

    01/02/2016 06:05:01.903 IST+0530 5 TCP/IP Controller Plugin.Transmitter pool thread <Regular:1>.CybTargetHandlerChannel.call[:742] - Attempting to send:    20160102 06050189-0530 LNX_SCH WA_AGENT 104.15003_1/WAAE_WF0.1/MAIN State EXEC SetStart Status(Executing at WA_AGENT) Jobno(14536) ChildPid(14669) JobLogId(715757BB6FC0272A3F7A357096F5582682772997)  User(WA_AGENT) Host(XXXX)

     

     

     

    01/02/2016 06:05:02.189 IST+0530 2 TCP/IP Controller Plugin.Transmitter pool thread <Regular:1>.CybTargetHandlerChannel.call[:776] - Message has been sent: 20160102 06050189-0530 LNX_SCH WA_AGENT 104.15003_1/WAAE_WF0.1/MAIN State EXEC SetStart Status(Executing at WA_AGENT) Jobno(14536) ChildPid(14669) JobLogId(715757BB6FC0272A3F7A357096F5582682772997)  User(WA_AGENT) Host(XXXX)

     

     

     

    01/02/2016 06:05:02.196 IST+0530 5 TCP/IP Controller Plugin.Transmitter pool thread <Regular:2>.CybTargetHandlerChannel.call[:742] - Attempting to send:    20160102 06050195-0530 LNX_SCH WA_AGENT 104.15003_1/WAAE_WF0.1/MAIN State COMPLETE Cmpc(0) SetEnd JobLogId(715757BB6FC0272A3F7A357096F5582682772997)  User(WA_AGENT) Host(XXXX)

     

     

     

    01/02/2016 06:05:02.315 IST+0530 2 TCP/IP Controller Plugin.Transmitter pool thread <Regular:2>.CybTargetHandlerChannel.call[:776] - Message has been sent: 20160102 06050195-0530 LNX_SCH WA_AGENT 104.15003_1/WAAE_WF0.1/MAIN State COMPLETE Cmpc(0) SetEnd JobLogId(715757BB6FC0272A3F7A357096F5582682772997)  User(WA_AGENT) Host(XXXX)

     

     

     

    ·         Check the initiatormanager.log to get an idea of how many initiators were available when the job was active. If there are too many jobs active on the system, then the agent might run out of the default 1000 initiators and so any new job requests will be queued.

     

     

     

    ·         The simple_health_monitor.log shows the Java memory consumption by the Agent.

     

     

     

    Thanks & Regards,

    Chandru



  • 3.  Re: Which System Agent log provides more details delayed processing problem

    Posted Jan 14, 2016 11:30 AM

    Hello,

       To build on what Chandru said.

      The .joblog file in the spool directory will give the time the job was launched and when it completes.  If the delay is in launching the job or in starting the 'launch' process you will see it in the .joblog.  Comparing the times from the receiver.log, the .joblog and the transmitter.log will narrow down where the delay is coming from.

    --

    To get the joblog for a successful run these need to be set in the agentparm.txt:

    oscomponent.joblog.success.autocleanup=false

    agent.spool.success.autocleanup=false

    oscomponent.joblog=true

    The spool directory is under the agent directory.  Usually at a location like:

    /opt/CA/WA_Agent_R11_3/spool/BKE_SCH/MAIN/WAAE_WF0.1

     

    Brad E



  • 4.  Re: Which System Agent log provides more details delayed processing problem

    Posted Jan 14, 2016 01:13 PM

    Hi Ahmed,

     

    I would start with what Chandru and Brad have suggested.  An other area where I have seen performance problem like this is with entropy levels on Linux vm.  If you are running the agent on Linux VM, I would also verify that you have sufficient level of entropy.  See the KD below for details.

     

    http://www.ca.com/us/support/ca-support-online/product-content/knowledgebase-articles/tec1428135.aspx

     

    Regards,

    Pradeep



  • 5.  Re: Which System Agent log provides more details delayed processing problem

    Posted Jan 14, 2016 06:06 PM


  • 6.  Re: Which System Agent log provides more details delayed processing problem

    Posted Jan 14, 2016 11:01 PM

    Hi Pradeep,

    This is a AIX physical box.

     

    Thanks,

    Ahmed



  • 7.  Re: Which System Agent log provides more details delayed processing problem

    Posted Jan 17, 2016 05:49 PM

    Hi Ahmed,

     

    Entropy/RNG is applicable to AIX as well.

    http://www-01.ibm.com/common/ssi/rep_ca/9/877/ENUSZP12-0499/ENUSZP12-0499.PDF

     

    "High-quality random numbers: AIX exploits the POWER7+ processor random

    number generator (RNG), providing high-quality random numbers at high

    performance. At the core of cryptography is the need for high-quality entropy

    for random numbers. AIX and POWER7+ ensures high demand for entropy

    on heavily loaded systems that yields high quality for random numbers. With

    the POWER7+ RNG offload providing this entropy, the security is ensured and

    processor performance remains focused on your business applications."

     

    Thanks,

    Chandru