ESP Workload Automation

CA Tuesday Tip by Lucy Zhang, Principal Support Engineer for 05/11/2015 and updated on 07/21/2016

Agent related:

RO72049: Support AES 256 encryption of communications with agents. AES 256 encryption keys are defined via the CRYPTKEY command, and the KEYNAME can be specified in an agent definition.

RO70124: Support for the MICROFOCUS Agent.       

RO81658: Values of JCLContent or JCLReference Application statements of MicroFocus Jobs (MF_JOB) will be displayed by LISTAPPL and APPLJOB commands, including the LJE command.

RO68947: Support for the INFORMATICA Agent.

RO67952: Support for the MS SQL Agent.     

RO63031:  The TANDEM_JOB job type should be updated for the CCI-based NonStop Agent. This agent adds new optional keywords and it also can create a spool file, job log file, output file and error file to be retrieved and displayed by CA Workload Automation ESP Edition.      

RO81392: Display of ChildPid values sent by the Tandem (Nonstop) Agent will be added to LISTAPPL and APPLJOB commands, including LJE command.

RO59906: PeopleSoft DISTRLISTROLES and DISTRLISTUSERS statements have maximum lengths of 256; that will be increased to 4078 to accomodate longer lists.      

RO70180:  DYNAMIC AGENT CONFIGURATION, see Tuesday tip “Tuesday Tips: (CA ESP) RFE - Change agent definition dynamically & Retrieve Agent Info”:

https://communities.ca.com/docs/DOC-231149338

RO74243: Support for USS File trigger. You can use a USS file path and USS keyword instead of data set name on the event level DSTRIG command or in the DSNAME statement for a DSTRIG workload object.  Note: please apply RO80514 together.       

RO77157: Retrieve Spool File command (WSF) is enabled in Informatica Job. Also, the Restart Workflow command now has an option of restarting from a selected Task.        

RO82303: Application statement OSUSER is added to FTP_JOB, SCP_JOB and SFTP_JOB. New application statement KEYFILE is created for SCP_JOB and SFTP_JOB. Application statement TRANSFERCODETYPE (alias FTPFORMAT) is added to SFTP_JOB, but only values of A and B will be accepted, everything else will be ignored.

RO81816: Web Services DOCUMENT/LITERAL Job Type.

RO82898: This enhancement adds SAF authorization checking for AGENT and AGENTUSR resource profiles in Simulation. All relevant error messages will have the application name and the job name displayed. Also, all "invalid agent" messages will have the job name displayed as well.

RO86662: Support the Hadoop Advanced Intergation for System Agent. Hadoop file operation (HDFS_JOB), Oozie (OOZIE_JOB), Sqoop (SQOOP_JOB) and Hive (HIVE_JOB) job types are added.   

RO87041: Cancel Job command has been added to Tandem_Job, NT_Job and all UNIX job types: AIX_Job, HPUX_Job, LINUX_Job, SUN_Job and UNIX_Job.       

RO88778: The maximum lengths of application statements CMDNAME and SCRIPTNAME will be increased from 100 characters to 256 characters.                                                 

Report/Display related:

RO72634: A NEW FIELD 'EVENT' HAS BEEN ADDED TO HISTORY REPORTING FIELDS. It represents the Event Name.                                

RO76366: This enhancement adds two optional fields to the DISPLAY operand of the JOBMAP command: AGENT_TYPE and AGENT_DESCR.       

RO63641: This enhancement makes the following fields available in JOBMAP reports: DUEOUT, SAPJOBNAME, STARTMODE, JOBCOPY, CHILDMONITOR and CHAIN. When DUEOUT field is requested, JOBMAP reports all DUEOUT and LATESUB settings for each job. DUEOUT applies to all job types. The other new fields are available in SAP jobs. To populate the new fields, a MAPGEN job must run with this fix applied.         

RO76602: New field Application Description has been added to the APPL statement. It is displayed in Simulation and by LISTAPPL commands. It is displayed by JOBMAP commands if DISPLAY(APPL_DESCR) is specified.

RO76447: This enhancement adds IP address of sender to the error message ESP507E.

RO77765: This enhancement enables users to display USER1-4 variables in the output of LISTAPPL(LAP) or LISTSCH(LSCH) commands.

RO77978: This enhancement changes the occurrence of informational message       

"ESP6106I Next execution of Event e not scheduled: unsuccessful CYCLING"

The message ESP6106I will be issued only when a real problem occurred during cycling of an event to avoid user confusion.           

RO83855: The LAPX command output is limited to 99 jobs. This enhancement extends the output limitation 99 jobs to APPLBUF size. And now only reports the first job when a dependency loop is found.

RO84691: In messages 6055 and 6062, add previous job priority information.

RO83247: With this PTF assignment statement becomes in line with other statements and is not printed when under a false branch of IF-THEN-ELSE.

RO81833: When the SIMULATE command is entered in page mode, INFOMSG SET is internally executed so that informational, warning, and error messages display within the body of the screen instead of in the upper right corner of the screen. 

RO87310: Show information about the user ID and date/time for an event being held/suspended last time.                                         

RO89239: This enhancement provides the information about resource which caused the security error:

ESP1883E "Insufficient access authority for requested function on resource <resource name>"     

ESP internal function related:

RO72017: The timing of access to the shared QUEUE data set will be set by ESP internally.  The initialization parameters that used to control this timing, MINHOLD, MINDORM and MAXDORM, will be accepted for compatibility, but will be ignored.                                

RO67513: Corruption in the resource file can cause repeated abends until the file is reformatted. This fix introduces automatic verification and repair of the resource file after an ESP abend.

RO84847: This PTF enables Workstation logon with a single-use passticket in place of a regular password.

RO88003: This enhancement makes specifying the JESNODE optional because it will set it automatically, if JESNODE is not specified.

RO79802: PREVENT SHARING OF RSVLOGIC BY MULTIPLE MASTERS

RO81067: Agent communications use the checkpoint file to notify application manager about sent, delayed or flushed messages. Spikes in the number of these notifications can fill the checkpoint. This fix eliminates the use of checkpoint for agent traffic feedback.                                     

Others:

RO70061: To coordinate workload between ESP masters, if a specific resource count is updated, a notification message will be sent to all relevant ESP masters.  See attached doc.

RO70737: Full support for Restful web service that executes an ESP command. To enable this feature, the new WEBPORT parameter must be added to the WSSPARM configuration data set.          

RO72998: This enhancement enables users of the RERUNM command and of the RM CSF line command to exclude a job from a rerun multiple request by placing a minus sign before the job name in the list of root jobs.                

RO78451: The AUDITLOG initparm statement will accept a new parameter, DEST, to specify the destination of the log.  Once the auditlog sysout file is deallocated, for example by a SPINLOG 

command, JES will send the file to the specified destination.

RO75777: This enhancement provides the ability to use ESP command processor as a host command environment in REXX. It is intended for use in REXX programs under CA OPS/MVS. The module name representing the command environment is CYBJS052.

RO78156: Improved integration with CA OPS/MVS: new command CALLOPS creates an API event in OPS/MVS with a user-defined event id suffix. It is available for use in ESP procedures.

CALLOPS MESSAGE('message text') [RULESUF(suffix)]                                                                                                     

RO77695: Jobs inserted into an active application via IW or APPLINS command are defined using ESP procedure language. The text of the definitions is stored in the application record but the BP command in CSF does not show it.  This enhancement enables display of inserted text by BP (Browse Procedure) command.                                                                 

RO69717:  Enhancement for CYBESUT3:

1. The user will not have to start ESP quiesced in order to format a new application file
2. The user will be able to update the existing JOBINDEX file in-place instead of creating a new JOBINDEX file, if requested. The new optional parameter NONEWINDEX will be introduced to support this feature. The sample member CYBESS18 describing the CYBESUT3 JCL has been updated to reflect these changes.        

RO78041: This enhancement enables users to see the calculated values of SCHEDULED or SCOPE parameter for an external job. These values can be displayed by the LJE command in CSF or using AJ LIST command to display detailed information about the specified external job.            

RO73721: A new optional SUBAPPLS keyword operand has been added to the TRIGGER and SIMULATE commands.

RO75729: This enhancement enables users to see jobs with HOLD attribute in simulation output along with a hold reason if it was provided. The new section of simulation output starts with the label          MANUAL HOLD JOBS and it is present only if there is at least one job with HOLD attribute selected for submission.         

RO77815: Jobs inserted into an active application via IW or APPLINS command are defined using ESP procedure language. The text of the definitions is stored in the application record and cannot be modified. This enhancement enables editing of inserted text by EP (Edit Procedure) command in CSF.

RO81648: For the SIMULATE command, check and issue a message for RESOURCEs that are not defined. Resources will only be checked on jobs selected for submission.  Issue message 4437I for each undefined resource.    

RO89165: When user mode 196 is turned on, ESP doesn't check existence of resources in simulation.

RO78611: This enhancement allows users to see the correct data set name from which the job's JCL was pulled if there is a DD name used on JCLLIB or TEMPLIB statement.                                    

The new format of JCL FROM line in the submitted job's log follows:

'JCL FROM DD NAME (if any) - DATA SET NAME'                             

RO81837: For the SIMULATE command, check and issue a message for ALERTs and EVENTS associated with a NOTIFY statement, that are not defined.        

NOTIFY statements will only be checked on jobs selected for submission. Issue message 4438I for each undefined event and message 4439I for each undefined alert.          

RO82002: This enhancement enables users to retrieve job spool files archived in CA View database.       

RO88215: EXTERNAL JOB can now have up to 64 characters in its job name.  You can now use TASK or any non-MVS job with the name of up to 64 characters as a home job for an EXTERNAL JOB.                                                      

Following were just added to the original post:

Agent related

RO74243: Support for USS File trigger. You can use a USS file path and USS keyword instead of data set name on the event level DSTRIG command or in the DSNAME statement for a DSTRIG workload object.          

RO77157: Retrieve Spool File command (WSF) is enabled in Informatica Job. Also, the Restart Workflow command now has an option of restarting from a selected Task.         

Others

RO78041: This enhancement enables users to see the calculated values of SCHEDULED or SCOPE parameter for an external job. These values can be displayed by the LJE command in CSF or using AJ LIST command to display detailed information about  the specified external job.              

RO73721: A new optional SUBAPPLS keyword operand has been added to the TRIGGER and SIMULATE commands.

RO77765: This enhancement enables users to display USER1-4 variables in the output of LISTAPPL(LAP) or LISTSCH(LSCH) commands. 

Hi Lucy, I was looking at RO74168 which makes some changes regarding IF-THEN-ELSE-DO-ENDDO logic.  I need to put on cumulative maintenance and was trying to make sure I understand the changes with this PTF.

It looks like, for the most part, it will correct some erroneous error messages that might be received when you have an ELSE statement to go with the IF-THEN statement, but I wanted to make sure I understand what else to look for.

It says something about an IF statement after an nested IF-THEN-ELSE construct, then the IF statement may get ignored, so the only change here would be that the IF statement after the IF-THEN-ELSE construct might now start working, correct? 

Am I understanding that correctly? What else should I be testing if I put this PTF on and has anyone else had problems after putting it on?

Hi Stan,

Sorry for my late response. It seems that there are more impact for the IF-THEN-ELSE change than it's described. Our next level is considering to mark RO74168 as PTF in error. Please open a support ticket if you want to apply following test fixes.

-  Critical errors are fixed by TR81748 for ESP 11.4 (T5HA209 in 11.3):

++APAR (TR81748) REWORK(20151680)

/*******************************************************************

BUGS IN IF-THEN-ELSE IMPLEMENTATION

PROBLEM DESCRIPTION:

This PTF fixes 2 bugs in the implementation of IF-THEN-ELSE logic.

1. ESP initiator can enter an endless loop when assignment

  statements are coded in a line and positioned in "no-process"

  environment. Here's an example of such coding:

  IF 1=1 THEN ...

  ELSE DO; A=1;B=2; ENDDO

  The error is due to incorrect parsing. This PTF fixes it.

2. Under certain conditions statements inside a job in ESP

  application are not executed. This happens when:

  - ENDJOB is missing

  - Followed missing ENDJOB is another job definition coded

  conditionally under IF-THEN-ELSE.

  Here's an example:

  JOB JOB1

  RUN ANY

  IF 1=1 THEN DO

  JOB JOB2

  RUN ANY

  DATASET '<dataset_name>'

  ENDJOB

  ENDDO

  The problem happens when JOB2 is being processed (submitted).

  DATASET statement will not be executed and JOB2 may not be

  found leading to error message (or wrong JOB2 may be submitted

  from JCLLIB).

  Note, this error can occur only when usermod 190 is off.

  When usermod 190 is on, there will be an error message about

  missing ENDJOB issued at application generation time.

  This PTF fixes the bug.

- Two other fixes, RO80370 in 11.4 (RO81955 in 11.3) and TR82219 in 11.4 (T5HA210 in 11.3) are solving compatibility problems:

Note: RO80370 and RO81955 are GA'd.

++PTF (RO80370)

/**********************************************************

DO STATEMENT AFTER THEN OR ELSE

PROBLEM DESCRIPTION:

Recent changes (see RO74168 PTF) made treatment of THEN

or ELSE statements without operands consistent, namely,

it is unconditionally considered as empty branches.

Prior to these changes there was one exception, if

THEN or ELSE without operands was followed by a DO

statement on the next line, that DO was considered as an

operand of the preceding THEN or ELSE, meaning that

branches were not actually empty. As a result it was

possible to code like this:

  IF <logical_expr> THEN

  DO

  .....

  ENDDO

  ELSE

  DO

  .....

  ENDDO

Now, without that exception, the above logic does not work

any more. THEN and ELSE are treated as empty branches and

subsequent DO-ENDDO blocks are considered to be outside of

IF-THEN-ELSE construction. In particular, in the above

example, ELSE would be treated as an out of sequence.

To preserve compatibility with the previous behavior

this fix restores the old way and allows the above

exception. But this is true only when usermod 190 is off.

When usermod 190 is on, new consistent treatment of THEN

or ELSE without operands is enforced.

++APAR (TR82219) REWORK(20151691)

/*********************************************************

ESLE TO CLOSE DO BLOCK COMPATIBILITY

PROBLEM DESCRIPTION:

This PTF fixes compatibility issue. Previously issued

RO74168 PTF fixed many bugs in IF-THEN-ELSE-DO-ENDDO logic,

but made impossible constructs like this one:

IF <logical_expr> THEN DO

R='1'

ELSE R='0'

ENDDO

Prior to RO74168, this construct was allowed because ELSE

statement would, by default, close DO block scope, and

subsequent ENDDO would be considered as a non-matching one,

but no error messages would be issued.

RO74168 disallowed such constructs unconditionally, with

ESLE statement reported to be out of sequence. To preserve

compatibility with the old behavior, this PTF allows ELSE

to close DO block scope as before, but this works only when

usermod 190 is off. When usermod is on, more restrictive

behavior is enforced.

Thank you,

Lucy

Following are also added to the post:

RO75729: This enhancement enables users to see jobs with HOLD attribute in simulation output along with a hold reason if it was provided. The new section of simulation output starts with the label          MANUAL HOLD JOBS and it is present only if there is at least one job with HOLD attribute selected for submission.         

RO81658: Values of JCLContent or JCLReference Application statements of MicroFocus Jobs (MF_JOB) will be displayed by LISTAPPL and APPLJOB commands, including the LJE command.

RO81392: Display of ChildPid values sent by the Tandem (Nonstop) Agent will be added to LISTAPPL and APPLJOB commands, including LJE command.

RO82303: Application statement OSUSER is added to FTP_JOB, SCP_JOB and  SFTP_JOB. New application statement KEYFILE is created for SCP_JOB and  SFTP_JOB. Application statement TRANSFERCODETYPE (alias FTPFORMAT) is added to SFTP_JOB, but only values of A and B will be accepted, everything else will be ignored.

RO76447: This enhancement adds IP address of sender to the error message ESP507E.

RO77815: Jobs inserted into an active application via IW or APPLINS command are defined using ESP procedure language. The text of the definitions is stored in the application record and cannot be modified. This enhancement enables  editing of inserted text by EP (Edit Procedure) command in CSF.

RO77978: This enhancement changes the occurrence of informational message       

"ESP6106I Next execution of Event e not scheduled: unsuccessful CYCLING"

The message ESP6106I will be issued only when a real problem occurred during cycling of an event to avoid user confusion.           

RO79802: PREVENT SHARING OF RSVLOGIC BY MULTIPLE MASTERS  

RO81067: Agent communications use the checkpoint file to notify application manager about sent, delayed or flushed messages. Spikes in the number of these notifications can fill the checkpoint. This fix eliminates the use of checkpoint for agent traffic feedback.                                     

RO81816: Web Services DOCUMENT/LITERAL Job Type.

RO81648: For the SIMULATE command, check and issue a message for RESOURCEs  that are not defined.  Resources will only be checked on jobs selected for submission.  Issue message 4437I for each undefined resource.         

RO78611: This enhancement allows users to see the correct data set name from which the job's JCL was pulled if there is a DD name used on JCLLIB or TEMPLIB statement.                                    

The new format of JCL FROM line in the submitted job's log follows:

'JCL FROM DD NAME (if any) - DATA SET NAME'                             

RO81837: For the SIMULATE command, check and issue a message for ALERTs and EVENTS associated with a NOTIFY statement, that are not defined.        

NOTIFY statements will only be checked on jobs selected for submission. Issue message 4438I for each undefined event and message 4439I for each undefined alert.       

RO82002: This enhancement enables users to retrieve job spool files archived in CA View database.                                      

RO82898: This enhancement adds SAF authorization checking for AGENT and AGENTUSR resource profiles in Simulation. All relevant error messages will have the application name and the job name displayed. Also, all "invalid agent" messages will have the job name displayed as well.

RO83855: The LAPX command output is limited to 99 jobs. This enhancement extends the output limitation 99 jobs to APPLBUF size.

RO84691: In messages 6055 and 6062, add previous job priority information.

RO84847: This PTF enables Workstation logon with a single-use passticket in place of a regular password.

RO83247: With this PTF assignment statement becomes in line with other statements and is not printed when under a false branch of IF-THEN-ELSE.

RO86662: Support the Hadoop Advanced Intergation for System Agent. Hadoop file operation (HDFS_JOB), Oozie (OOZIE_JOB), Sqoop (SQOOP_JOB) and Hive (HIVE_JOB) job types are added.   

RO81833: When the SIMULATE command is entered in page mode, INFOMSG SET is internally executed so that informational, warning, and error messages display within the body of the screen instead of in the upper right corner of the screen. 

RO87310: Show information about the user ID and date/time for an event being held/suspended last time.   

RO88215: EXTERNAL JOB can now have up to 64 characters in its job name.  You can now use TASK or any non-MVS job with the name of up to 64 characters as a home job for an EXTERNAL JOB.      

RO88003: This enhancement makes specifying the JESNODE optional because it will set it automatically, if JESNODE is not specified.         

RO87041: Cancel Job command has been added to Tandem_Job, NT_Job and all UNIX job types: AIX_Job, HPUX_Job, LINUX_Job, SUN_Job and UNIX_Job.                               

RO89239: This enhancement provides the information about resource which caused the security error :

ESP1883E "Insufficient access authority for requested function on resource <resource name>"       

RO89165: When user mode 196 is turned on, ESP doesn't check existence of resources in simulation.

Hi Lucy,  this type of post is very valuable for us to see what new functionality is available via ptfs (such as the new uss file trigger capability).  Are there any new ptfs  like the ones you cited in July that you can report on for this 3rd quarter?

Thanks

Hi Mike,

Following RFE is added:

RO88778: The maximum lengths of application statements CMDNAME and SCRIPTNAME will be increased from 100 characters to 256 characters.    

And the ESP online document can be used to review existing enhancements:

https://docops.ca.com/ca-workload-automation-esp-edition/11-4/en/release-information

As you may know, ESP team has sprint review meetings when development will explain/present newly published enhancements. Let me know if you are interested and I will get you to the invitation list.

Thank you,

Lucy

Thanks Lucy

Michael Bieganski

Technical Analyst

Exelon C&I Network, Infra & Cloud Ops -Mainframe

Chase Towers

10 S. Dearborn, 45th Floor

Chicago, IL 60603

Office:312 394-7707

Michael.Bieganski@exeloncorp.com<mailto:Michael.Bieganski@exeloncorp.com>

Following RFE is added:

RO88778: The maximum lengths of application statements CMDNAME and SCRIPTNAME will be increased from 100 characters to 256 characters.