ESP Workload Automation

I have noticed , few jobs in our system are keep on failing with SUBMIT Error (Submit Failed, Event error ) these are all the mainframe jobs and scheduling is looking correct to me.

- JCL Library/JOB / Member Name / Resources  everything is defined correctly but still jobs are failing . can anyone please help me how to solve this issue.

I have checked the ESP log but not getting anything .

JOB flow is here 

Here IRHSSAVE is running fine but problem with below 2 jobs. 

Hi Kumar, 

Normally there is a 528E message that is produced in the ESP log when this error happens.  The messages in the 528E or in the messages just prior to the 528E message provide some detail on what the issue is. 

If there is not a message in the ESP logs I would suggest opening a case with CA. 

2¢

Thank for Quick response,

I can't see any such 528E message . 

Here is the log details:  

ESP6260I ResoMGR: Job IRHSAUSL in Appl IRHPSSTD.461 Resources Satisfied
ESP6633I APPLMGR: APPL IRHPSSTD.461 JOB IRHSAUSL READIED
ESP6260I ResoMGR: Job IRHSTINH in Appl IRHPSSTD.461 Resources Satisfied
ESP6633I APPLMGR: APPL IRHPSSTD.461 JOB IRHSTINH READIED
ESP6108I EVENTEX: PFSTR1.IRH_IRHPSSTD_XX scheduled, for Job IRHSAUSL in Appl I
ESP6125I EVENTEX: PARMS: ESPTRUSR(T49DPJ1)ESPTRTYPE(T)RESUB_USER()USER1(UPDATE
ESP6108I EVENTEX: PFSTR1.IRH_IRHPSSTD_XX scheduled, for Job IRHSTINH in Appl I
ESP6125I EVENTEX: PARMS: ESPTRUSR(T49DPJ1)ESPTRTYPE(T)RESUB_USER()USER1(UPDATE
ESP6633I APPLMGR: APPL LI1PCU04.170205 JOB LI1PCU04 READIED
ESP6108I EVENTEX: PFLIP1.LIP_LI1PCU04_GB scheduled, for Job LI1PCU04 in Appl L
ESP6125I EVENTEX: PARMS: ESPTRTYPE(P)RESUB_USER()
ESP6220I MgrMsg: . LI1PCU04/LI1PCU04.170205/MAIN STATE COMPLETE SetEnd Cmpc(0
ESP6640I APPLMGR: APPL LI1PCU04.170205 JOB LI1PCU04 Now Complete
ESP6629I APPLMGR: APPL LI1PCU04.170205 NOW COMPLETE
ESP6220I MgrMsg: . INTDAMON.PLYSSQL339P/INTPCMON.109/MAIN State READY Status(
ESP6220I MgrMsg: ESPPPLYSSQL339P INTDAMON.PLYSSQL339P/INTPCMON.109/MAIN RESPO

Hi Kumar, 

It looks like the log that you sent is the SYS* log that is normally referred to as the auditlog. Take a look in the JESMSGLOG from the started task and see if you find a 528E message. 

2¢

Hi,

No such message in the log. You can see the JESMSGLOG here 

IRR010I USERID PFSTR1 IS ASSIGNED TO THIS JOB.
ICH70001I PFSTR1 LAST ACCESS AT 12:08:10 ON THURSDAY, JANUARY 19, 2017
$HASP373 IRHSAUSL STARTED - INIT 32 - CLASS 9 - SYS SI01
IEF403I IRHSAUSL - STARTED - TIME=12.08.16
IEF233A M 0AC7,B99715,,IRHSAUSL,S010,IRH.SAV.XF71R.M12161
IEF233A M 0AC8,PRIVAT,SL,IRHSAUSL,S010,VLC2.IRH.XF71R.M12161
IECTMS9 0AC8,C28302,IRHSAUSL,SYSUT2 ,2021/358 ,00001,.IRH.XF71R.M12161
IEC705I TAPE ON 0AC8,C28302,SL,COMP,IRHSAUSL,S010.IRHAUSL,VLC2.IRH.XF71R.M12161
IEC205I SYSUT2,IRHSAUSL,S010,FILESEQ=1, COMPLETE VOLUME LIST, 906
DSN=VLC2.IRH.XF71R.M12161,VOLS=C28302,TOTALBLOCKS=801
IEF234E K 0AC8,C28302,PVT,IRHSAUSL,S010
TMS014 IEF234E K 0AC8,C28302,PVT,IRHSAUSL,S010
IEF233A M 0ABF,PRIVAT,SL,IRHSAUSL,S020,VLC2.IRH.XF72R.M12161
IECTMS9 0ABF,B99719,IRHSAUSL,SYSUT2 ,2021/358 ,00001,.IRH.XF72R.M12161
IEC705I TAPE ON 0ABF,B99719,SL,COMP,IRHSAUSL,S020.IRHAUSL,VLC2.IRH.XF72R.M12161
IEC205I SYSUT2,IRHSAUSL,S020,FILESEQ=1, COMPLETE VOLUME LIST, 921
DSN=VLC2.IRH.XF72R.M12161,VOLS=B99719,TOTALBLOCKS=691
IECTMS9 0ABF,B99719,IRHSAUSL,SYSUT2 ,2021/358 ,00002,.IRH.XF70R.M12161
IEC205I SYSUT2,IRHSAUSL,S030,FILESEQ=2, COMPLETE VOLUME LIST, 923
DSN=VLC2.IRH.XF70R.M12161,VOLS=B99719,TOTALBLOCKS=1
IECTMS9 0ABF,B99719,IRHSAUSL,SYSUT2 ,2021/358 ,00003,.IRH.FU70R.M12161
IEC205I SYSUT2,IRHSAUSL,S040,FILESEQ=3, COMPLETE VOLUME LIST, 930
DSN=VLC2.IRH.FU70R.M12161,VOLS=B99719,TOTALBLOCKS=56
IECTMS9 0ABF,B99719,IRHSAUSL,SYSUT2 ,2021/358 ,00004,2.IRH.FU50.M12161
IEC205I SYSUT2,IRHSAUSL,S060,FILESEQ=4, COMPLETE VOLUME LIST, 932
DSN=VLC2.IRH.FU50.M12161,VOLS=B99719,TOTALBLOCKS=36
IEF233A M 0AC6,C26172,,IRHSAUSL,S070,IRH.SAV.FU51.M12161
IECTMS9 0ABF,B99719,IRHSAUSL,SYSUT2 ,2021/358 ,00005,2.IRH.FU51.M12161
IEC205I SYSUT2,IRHSAUSL,S070,FILESEQ=5, COMPLETE VOLUME LIST, 940
DSN=VLC2.IRH.FU51.M12161,VOLS=B99719,TOTALBLOCKS=684
IEF234E K 0AC7,B99715,PVT,IRHSAUSL
TMS014 IEF234E K 0AC7,B99715,PVT,IRHSAUSL
IEF234E K 0ABF,B99719,PVT,IRHSAUSL
TMS014 IEF234E K 0ABF,B99719,PVT,IRHSAUSL
IEF234E K 0AC6,C26172,PVT,IRHSAUSL
TMS014 IEF234E K 0AC6,C26172,PVT,IRHSAUSL
IEF404I IRHSAUSL - ENDED - TIME=12.08.25
$HASP395 IRHSAUSL ENDED - RC=0000

Hi Kumar,

I believe Don has asked about the ESP STC JESMSGLG, NOT JOB IRHSAUSL.

Thanks,

Lucy

Hi Lucy,

Sorry for delay respond . Please see here and Error Job is ISASB70B . 

ESP_528E ERROR OCCURRED PROCESSING EVENT PFSTR1.ISA_ISAPSSTD_XX IN APPL ISAPSUPD
ALLOCX VERSUCHE ISAUPDRX = 1 LASTCC = 0
IRX0555E The input or output file SYS27193 is not allocated. It cannot be opened
IRX0670E EXECIO error while trying to GET or PUT a record.
ESP_528E ERROR OCCURRED PROCESSING EVENT PFSTR1.ISA_ISAPSSTD_XX IN APPL ISAPSUPD
ALLOCX VERSUCHE ISAUPDRX = 1 LASTCC = 0
IRX0555E The input or output file SYS27194 is not allocated. It cannot be opened
ESP_528E ERROR OCCURRED PROCESSING EVENT PFSTR1.ISA_ISAPSSTD_XX IN APPL ISAPSUPD
IRX0670E EXECIO error while trying to GET or PUT a record.
ALLOCX VERSUCHE ISAUPDRX = 1 LASTCC = 0
IRX0555E The input or output file SYS27195 is not allocated. It cannot be opened
IRX0670E EXECIO error while trying to GET or PUT a record.
ICH70001I PFPTR2 LAST ACCESS AT 19:35:40 ON TUESDAY, FEBRUARY 14, 2017
ICH70001I PFPTR2 LAST ACCESS AT 19:35:42 ON TUESDAY, FEBRUARY 14, 2017
ICH70001I PFPTR2 LAST ACCESS AT 19:35:42 ON TUESDAY, FEBRUARY 14, 2017
$HASP100 ISASSAVE ON INTRDR STR-GROUP FROM STC09003 ESPIP
IRR010I USERID PFSTR1 IS ASSIGNED TO THIS JOB.
ALLOCX VERSUCHE ISAUPDRX = 1 LASTCC = 0
EXECIO WITH RETURN CODE = 0
ICH70001I PFPTR2 LAST ACCESS AT 19:35:42 ON TUESDAY, FEBRUARY 14, 2017
$HASP375 ESPIP ESTIMATE EXCEEDED BY 4,020,000 LINES
ESP_2502I EVENT TRIGGERED
ESP_1159I Application P49PCRES Generation 88676 is now Complete
ESP_2502I EVENT TRIGGERED
ESP_1159I Application P49PBRES Generation 91704 is now Complete

19.38.36 JOB07738 $HASP100 ISASB70B ON INTRDR STR1-GROUP FROM
19.38.36 JOB07738 IRR010I USERID PFSTR1 IS ASSIGNED TO THIS JOB.
19.38.36 STC09003 ALLOCX VERSUCHE ISAUPDRX = 1 LASTCC = 0
19.38.36 STC09003 EXECIO WITH RETURN CODE = 0

Hello,

Could you provide the JESMSGLG for JOB IRHSAUSL and/or IRHSTINH? Or information about JOB ISASB70B, like ESP Proc, ESP auditlog etc?

The IRX0555E and IRX0670E errors are related to ESP REXX code. Is it possible that the same REXX code with ALLOCX VERSUCHE statement ran concurrently or the paired FREEX was not issued somehow? This could cause following ALLOCX failure as it's opened already.

If more help is needed, I suggest that you open a support ticket.

Thank you,

Lucy