Automic Workload Automation

Expand all | Collapse all

Look up the details of the job that wrote a particular log file

Luu_Le_185

Luu_Le_185Mar 16, 2016 02:55 AM

Judith Gitahi

Judith GitahiNov 02, 2016 12:10 PM

  • 1.  Look up the details of the job that wrote a particular log file

    Posted Mar 10, 2016 05:54 AM
      |   view attached

    We occasionally discover that the /var file system on a server is filling up due to large UC4 job log files in the agent/out directory. I wrote an Oracle SQL statement to list the details of the original job that wrote a log, and the last user who edited this job.

    with orig_job as
    (
    select AH_OH_Idnr as Job_Run_ID, AH_Alias as Job_Alias, OH_Name as Job_Name,
    OH_ModUserIDNr
    from AH,OH
    where AH_OH_Idnr = OH_Idnr
    and AH_Idnr =
    (
    select AH_ParentAct from AH
    where AH_OType = 'REPORT'
    and AH_HostSrc = ?
    and AH_FileNameSrc like ?
    )
    ),

    user_details as
    (
    select OH_Idnr, OH_Name as User_ID, USR_FirstName, USR_LastName,
    USR_EMail1,USR_DstngshdName,USR_LastSession
    from USR,OH
    where USR_OH_IDNr = OH_Idnr
    )

    select Job_Run_ID, Job_Alias, Job_Name, User_ID, USR_FirstName, USR_LastName,
    USR_EMail1,USR_DstngshdName,USR_LastSession
    from orig_job,user_details
    where orig_job.OH_ModUserIDNr = user_details.OH_Idnr

    The bind parameters are:

    1. Agent name, e.g.,CHRVL360
    2. Job log file with leading SQL wildcard, e.g.,%OADWUPSA.TXT

    I put this into an SQLI, and called it from a UC4 script that parses and prints the output. I then put the script into a workflow with a prompt set that prompts for the two input parameters. Our support team can now use this workflow to quickly identify both the job that created a given log file, and the job’s likely owner.

     

    Here is an example of the output of this script:

    Job that wrote report file OADWUPSA.TXT:
    Run ID       : 2323174
    Task alias   : UC0.MAL.TEST1#1.JOBS_UNIX
    Object name  : UC0.MAL.TEST1#1.JOBS_UNIX

    User who last edited this job:
    User ID      : XYZ789/CORP
    User name    : John Smith
    Email addr.  : John_Smith@mycompany123.com
    LDAP DN      : CN=XYZ789,OU=XYZ,OU=EMEA,OU=Staff,DC=corp,DC=mycompany123,DC=com
    Last session : 2016-03-10 06:13:33.000

    I’ve attached an XML export of this workflow and its constituent objects.

    Attachment(s)



  • 2.  Look up the details of the job that wrote a particular log file

    Posted Mar 16, 2016 02:55 AM
    Thanks for sharing Michael :)


  • 3.  Look up the details of the job that wrote a particular log file

    Posted Apr 11, 2016 12:15 PM

    I like this.. but this means you have to know the agent name.. I'd like to find out before I get an alert that the agent log is filling up the filesystem. Is there a way to know what agent is involved when you know the output file name?  So I get this message in my logs.. how would I know what agent it ran on?

    U0029007 Not all blocks (Limit '240') have been transferred from file 'C:\UC4\v9\Agents\Windows\TEMP\OAAKUSPU.TXT'.
     

     



  • 4.  Look up the details of the job that wrote a particular log file

    Posted Apr 11, 2016 12:31 PM
    HiRonGates608562 ,
    To discover the host from the temp file, you should first discover its runid converting the name file to it and then getting the statistics detail from the execution. I use something like this (example - change the &file to whatever your file name is):

    :set &file='OAACJGEF.TXT'
    :set &run=str_cut(&file,2)
    :set &run=str_sub(&run,'.TXT','')
    :set &run=alpha2runnr(&run)
    :set &host=get_statistic_detail(&run,DST_HOST)

    Hope it helps!


  • 5.  Look up the details of the job that wrote a particular log file

    Posted Apr 12, 2016 04:49 AM
    Here’s an idea:
    1. Set up a process that monitors the server logs, and looks for instances of messageU00029007.
    2. From the file name in the message, look up the agent name (e.g., usingSimone’s approach).
    3. Using mySQL queryabove, look up the details of the job that wrote the log, and its likely owner.
    4. Automatically notify the job owner that his/her job is writing logs that are too large to be transferred to the AE DB.
    5. Automatically notify the server owner that large UC4 job logs may be accumulating on his/her machine.


  • 6.  Look up the details of the job that wrote a particular log file

    Posted Apr 12, 2016 07:58 AM
      |   view attached

    I had forgotten all about ALPHA2RUNNR until reading Simone’s comment above. I updated my script so that it looks up the agent name automatically. Now, the only input required is the 7-letter job name, with or without the .TXT file extension.

    An XML export of the workflow and constituent objects is attached.

    By the way, it would be interesting to know in what DB table these 7-letter job names are stored — that is, to know how ALPHA2RUNNR maps from these names to run IDs.



  • 7.  Look up the details of the job that wrote a particular log file

    Posted Apr 29, 2016 04:29 AM

    By the way, it would be interesting to know in what DB table these 7-letter job names are stored — that is, to know how ALPHA2RUNNR maps from these names to run IDs.
    For my opinion the seven digit alpha values aren't stored in any database tables. The script command RUNNR2ALPHA is mapped to an internal server function that generates a hash value. You can test this by running the command with a fictitious runid that has never been used by the automation engine.

    Example 1:
    :SET &RUNNR# = ALPHA2RUNNR("XXXXXXY")
    :PRINT "RUNNR: &RUNNR#"
    RUNNR: 0000001200669230

    Example 2:
    :SET &RUNNR# = ALPHA2RUNNR("XXXXXXX")
    :PRINT "RUNNR: &RUNNR#"
    RUNNR: 0000001200669231

    If these values would be stored in the database, there wouldn't be a value, as there wouldn't be a row in the database.

    //Carsten


  • 8.  Look up the details of the job that wrote a particular log file

    Posted Apr 29, 2016 05:12 AM
     
    For my opinion the seven digit alpha values aren't stored in any database tables. The script command RUNNR2ALPHA is mapped to an internal server function that generates a hash value. You can test this by running the command with a fictitious runid that has never been used by the automation engine.
    Astute observation, Carsten!  Would someone Automic be willing to share the details of the function that maps from a run number to an alphabetic job ID?


  • 9.  Look up the details of the job that wrote a particular log file

    Posted Jun 28, 2016 10:33 AM
    Carsten Berberich wrote:
    For my opinion the seven digit alpha values aren't stored in any database tables. The script command RUNNR2ALPHA is mapped to an internal server function that generates a hash value. You can test this by running the command with a fictitious runid that has never been used by the automation engine.
    Hi Carsten. I found the answer.


  • 10.  Look up the details of the job that wrote a particular log file

    Posted Nov 02, 2016 09:48 AM
    Hi, Is there a way to maximize the limit? Or what do I need to do to stop getting this error: "Not all blocks (Limit '240') have been transferred from file 'D:\Automic\Agents\windows\TEMP\OAAYWNCL.TXT'?"


  • 11.  Look up the details of the job that wrote a particular log file

    Posted Nov 02, 2016 10:08 AM
    Hi Judith,
    This can be adjusted in UC_HOSTCHAR_DEFAULT (or whatever system variable your agent takes its settings from). The key is 'MAX_REPORT_SIZE'.

    More details can be found here : https://docs.automic.com/documentation/WEBHELP/English/AWA/11.2/AE/11.2/All%20Guides/help.htm#ucaabu.htm

    Best regards,
    Antoine



  • 12.  Look up the details of the job that wrote a particular log file

    Posted Nov 02, 2016 10:17 AM

    Judith Gitahi wrote

    Hi, Is there a way to maximize the limit? Or what do I need to do to stop getting this error: "Not all blocks (Limit '240') have been transferred from file 'D:\Automic\Agents\windows\TEMP\OAAYWNCL.TXT'?"

    The report block size in bytes is specified in REPORT_BLKSIZE in UC_SYSTEM_SETTINGS or UC_HOSTCHAR_DEFAULT. The maximum value is 8000. (This is also the default.)

    The maximum number of blocks of a report that will be saved in the AE database is specified in MAX_REPORT_SIZE in UC_HOSTCHAR_DEFAULT. The maximum value is 10000. (The default is 120.)

    REPORT_BLKSIZEMAX_REPORT_SIZESize of largest report that
    can be saved to the AE DB
    8000120960000 bytes / 937.5 KiB
    80001000080000000 bytes / ~76 MiB


  • 13.  Look up the details of the job that wrote a particular log file

    Posted Nov 02, 2016 12:10 PM


  • 14.  Look up the details of the job that wrote a particular log file

    Posted Nov 02, 2016 02:05 PM
    I also use this SQLServer SQL statement to identify reports that are getting close to our block count limit, so we can proactively work with the applications folks to shrink their fat reports before they hit our thresholds.

    select oh_name as job_name      --, ah_timestamp1 as activation_time
         , dateadd(hour, datediff(hour, getutcdate(), getdate()), ah_TimeStamp2) as Start_time
         --, ah_runtime as runtime
         , CONVERT(varchar, DATEADD(ms, ah_runtime * 1000, 0), 114) as runtime
         , count(*) as report_size
    from uc4.dbo.rt
       , uc4.dbo.ah
       , uc4.dbo.oh
    where ah_timestamp1 > cast('20161020 00:00:00:000' as DATETIME) -- how far back in time
    and   ah_oh_idnr = oh_idnr
    and   rt_ah_idnr = ah_idnr
    and   not oh_name in ('JOB1', 'JOB2') -- Objects to exclude from the selection
    group by oh_name, ah_timestamp1, ah_timestamp2, ah_runtime
    having count(*) > 500 -- MAX_REPORT_SIZE is set in UC_HOSTCHAR_DEFAULT
    order by 2;


  • 15.  Look up the details of the job that wrote a particular log file

    Posted Dec 20, 2016 10:58 AM

    Hi All.

    As you all seem to know how to use the ALPHA2RUNNR option, I thought I'd ask my question here.

    We run a Batch in UC4 and all job reports are recorded on the Unix server but with the file name as Alpha.  Either where do I change this within UC4 so these reports appear on the server with the job Run ID ad the file name, or where and how can I include the ALPHA2RUNNR command in all jobs so that the reports are filed with the run ID as the filename?

    Thanks



  • 16.  Look up the details of the job that wrote a particular log file

    Posted Dec 20, 2016 11:10 AM
    Hi IanGoddard606749 ,

    If I understood correctly, you want to change the report file name to RunId. Is that right? I think that this is not a possible configuration. To "translate" the alpha file name to runid, you just have to use the alpha2runnr function in a sequential job to rename the files.

    Here's an example if you use a job inside a workflow:

    :set &runid#=sys_act_prev_nr() >> to get the predecessor job runid
    :set &alpha#=runnr2alpha(&runid#)
    :set &filename#='O&alpha#.TXT'
    mv &filename# &runid.txt

    Or you can use the get_att function to get the filename directly (on the job itself):

    :set &filename#=get_att(FILENAME_JOB)

    Hope this helps.





  • 17.  Look up the details of the job that wrote a particular log file

    Posted Dec 20, 2016 12:16 PM


  • 18.  Look up the details of the job that wrote a particular log file

    Posted Aug 25, 2017 05:46 AM
    Ron Gates wrote:
    I like this.. but this means you have to know the agent name.. I'd like to find out before I get an alert that the agent log is filling up the filesystem. Is there a way to know what agent is involved when you know the output file name?  So I get this message in my logs.. how would I know what agent it ran on?

    U0029007 Not all blocks (Limit '240') have been transferred from file 'C:\UC4\v9\Agents\Windows\TEMP\OAAKUSPU.TXT'.

    Good point, Ron. I updated the workflow so that it looks up the agent on which the original job ran, using GET_STATISTIC_DETAIL or an SQL query. I will post the new workflow, which contains several other updates, shortly.


  • 19.  Look up the details of the job that wrote a particular log file

    Posted Aug 26, 2017 09:44 AM
    It turns out that it was never necessary to look up the agent name in the first place, because the name of the report file is an representation of the run ID of the task that created it. I have posted two variants of a new version of this workflow. They are much improved over the version posted in this thread.