ESP Workload Automation

  • 1.  Linux Space problems

    Posted May 11, 2015 12:13 PM

    Hello,

     

    We were wondering if ESP has a max number of spool/log files that ESP can keep track of? We had a problem over the weekend that we needed to delete logs and spool files due to jobs going into a ready state or failing with not enough space.

     

    This runs the clear spool file every 12, deleting anything over 3 days, we alos have the log.level=5, I think we will recommand that we change it to 2.

     

    This is very busy server; some jobs run every 5 mins, some every 15, some every 20 and some on the 20 mins and 40 mins past the hours.

     

    Thanks,

    Mary K Stevens



  • 2.  Re: Linux Space problems

    Posted May 11, 2015 05:08 PM

    The limit is on the Linux file system not the ESP Agent.


    We have encountered 2 types of "space" issues.

    1. Used space/capacity is at 100%
    2. Directory file limit (this is dependent on the file system type, ext2, ext3, ext4 . . . )

     

    To determine which of the conditions are occurring I use the Linux 'df' command with the options listed below:

     

    Space (-k or -h with -T)
    df -kT <ESP Agent Directory>

    Filesystem           Type 1K-blocks   Used Available Use% Mounted on

    <filesystem>

                         ext4   3096336 657004   2282048  23% <ESP Agent Directory>

     

    Number of files (-i)
    df -i <ESP Agent Directory>

    Filesystem           Inodes IUsed  IFree IUse% Mounted on
    <filesystem>
                         196608  9803 186805    5% <ESP Agent Directory>


    We have some busy servers that create over 3000+ APPL.APPLGEN directories in the spool directory.

     

    Our file system (ext3) had a limit of 32K, so after 10 days we experienced the "too many files" problem.

     

    Initially we modified the spool cleanup interval from 30 days to 10, then to 7 days.

     

    Was your issue “space/capacity” or “number of files?

    If it is “space” you could also check out the DISK_MON Workload Object and run a job to cleanup when the threshold is exceeded.

     

    Do you know what the average number of APPL.APPLGEN run per day?

     

    Do you know what the file system type is?

     

    Do you have access to the Linux Server?



  • 3.  Re: Linux Space problems

    Posted May 12, 2015 09:19 AM

    Thanks for the info. It seems to be space, I just wanted to make sure it wasn't ESP. Could figure out how n=many files a day, not sure of the file system type, and no we do not have access to the Linux systems here.

     

    We do have a Linux person writing a script to archive the spool files, keeping just 48hours worth, I have also change the log files to 2 we had it a 5.



  • 4.  Re: Linux Space problems

    Posted May 12, 2015 09:44 AM

    If you do not have the ability to logon to the linux server, y

     

    ou can create a simple ESP PROC to get the information.

     

    You will need to replace the <>

     

    with the correct values for your environment.

     

    For the CMDNAME /bin/bash should work.

     

    PROCEDURE_SECTION:                                               
                                                                     
    APPL LINUXDF                                                     
                                                                     
    Agent=%USER1                                         
    HDR1="File System Space and Type"                               
    HDR2="File System Number of Files"  

    SpoolDir="<ESP Spool Dir>"    

     

    LINUX_JOB LINUXDF                                                   
      AGENT %Agent                                                     
      CMDNAME <valid Linux shell>                                             
      USER <valid Linux ID>                                                 
      ARGS -c 'echo %HDR1;df -Th %SpoolDir;echo %HDR2;df -i %SpoolDir' 
      RUN ANYDAY                                                 
    ENDJOB

     

     

    Create an Event and Trigger with the Agent Name in USER1.

     

    After the jobs runs, use WSF to display output.



  • 5.  Re: Linux Space problems

    Posted May 12, 2015 09:46 AM

    Can you ask the Linux System Adnmin if the file system on the Linux server be made larger?

     

    If the server has a large number of jobs running that may be a solution.

     




  • 6.  Re: Linux Space problems

    Posted May 12, 2015 10:15 AM

    dont keep old spool files.. or zip them up after awhile.

    there's many ways to trim it.

    what u may want to do is .. find a larger file system .. and make a system link log to point there..

    ;-)



  • 7.  Re: Linux Space problems

    Posted May 12, 2015 08:07 PM

    Why not put the System Agent's built-in housekeeping services to work?

     

    1. Specify the following parameters:
    2. runnerplugin.spool.expire

         Specifies the file expiration time. The agent deletes spool files that are older than this value. This parameter uses the following syntax:

         n[D|H|M|S]

         n

        Specifies the time period.

         D

         Specifies the time period unit as days. This is the default.

         H

         Specifies the time period unit as hours.

         M

         Specifies the time period unit as minutes.

         S

         Specifies the time period unit as seconds.

         Default: 10D (10 days)

         Note: You cannot specify combinations of time periods. For example, 12D3H is not valid.

     

    1. runnerplugin.spool.sleep

         Specifies the sleep interval. At every interval, the agent checks for spool files that meet the expiration time and deletes them.

         Default: 1D (1 day)

     

    Example:

    Suppose that you want to configure the agent to review the spool files every 50 minutes and delete spool files that are older than 50 minutes as specified by runnerplugin.spool.expire.

    Add the indicated values to the following parameters in the agentparm.txt file:

    runnerplugin.spool.clean.enable=true

    runnerplugin.spool.expire=50M

    runnerplugin.spool.sleep=2H

    Note: The agent ignores the two hour sleep interval set by runnerplugin.spool.sleep.

     

     

    Thanks,

    Chandru



  • 8.  Re: Linux Space problems

    Posted May 13, 2015 10:27 AM

    I find this ability a double edged sword.

    on the one hand its helpful on the other.. well. it isnt.

    this deletes. what if what u want is to zip. then archive and then delete.

    I think each environment will want a different way to apply it. In the real world sometimes you need to have a separation of church and state. or in this case.. automated agent tools versus applied action..