What I do is have an event invoked by a NOTIFY that invokes a ESP procedure that extracts various monitor variables about the failed job and puts them into a string that is passed via MQSERIES (a java messaging service) into a problem and incident tool. The problem and incident tool we use is CA Service Desk but before that we used iPortal and before that we used Remedy. The CA Service Desk documentation suggests that you can also input a problem via email but I have never tried that as it would have required changing the process.
Below is an example of a monitor event invoked as a result of a NOTIFY
-IPT- BROWSE DX.D952.ESP.PROCLIB(SERVICE) - 01.00 Line 00000000 Col 001 080
Command ===> Scroll ===> PAGE
********************************* Top of Data **********************************
SERVICE: ESPPROC
/***********************************************************/
/* ESP PROC EXECUTED ON CNTR2 */
/* BY ESP EVENT DX950GRP.DSALRT */
/***********************************************************/
IF %MNNXN = 'ESP_CNTR' THEN EXIT
INVOKE 'DX.D952.ESP.PROCLIB(CAPTURE)'
/***************************************************************/
/** LOGIC FOR OVERDUE - FAILURES IN THE FUTURE **/
/***************************************************************/
IF %MNPOINT = 'OVERDUE' THEN +
DO
COMPCODE = 'OVERDUE '
ENDDO
ELSE +
DO
COMPCODE = '%MNMXCMPC'
ENDDO
STR1=' %MNAPPL(8)%MNJOB(8)%MNJOBNO(F5)%COMPCODE(8)'
STR2='%TIME(5)%DATE(7)%STEPNAME(8)%PROGNAME(8)%CENTER(8)'
STR ='%STR1%STR2'
JCLLIB 'DX.D952.ESP.CNTL'
JOB DXEXAMIN
MEMBER SERVICE
RUN NOW
ENDJOB
EXIT
******************************** Bottom of Data ********************************
Notice that the CAPTURE routine tries to discern the name of the MVS jobstep and program name that has abended and would be a much shorter routine if one did not desire to do this
-IPT- BROWSE DX.D952.ESP.PROCLIB(CAPTURE) - 01.07 Line 0000 Nesting level 2
Command ===> Scroll ===> PAGE
********************************* Top of Data **********************************
CAPTURE: ESPPROC
INTEGER JOBNO
STEPNAME = ' '
PROGNAME = ' '
/* */
/* STEPNAME AND PROGRAM NAME WILL NOT BE VALID FOR ALL ERRORS */
/* THEREFORE WE INITIALIZE THEM TO NULL STRINGS INITIALLY */
/* */
REXXON
X = TRAPOUT('LINE.')
ADDRESS ESP "ESP LJ " CLANGVAR('%MNJOBNO')
X = TRAPOUT('OFF')
Y = CLANGVAR('%MNMXCMPC')
IF Y = 'CCFAIL' THEN +
DO
Y = CLANGVAR('%MNHICMPC')
END
/* */
/* IN THE CASE OF A CCFAIL CONDITION YOU HAVE TO USE THE */
/* HIGHEST RETURN CODE FROM A STEP IN THE JOB INSTEAD */
/* */
IF Y = 'JCLERROR' OR Y = 'SYSERROR' OR Y = 'RUNNING' THEN +
DO
EXIT
END
/* */
/* STEPNAME AND PROGRAM NAME HAVE NO MEANING IN THE CASE OF */
/* JOBS THAT GET COMPLETION CODES OF RUNNING AND SYSERROR */
/* AND THERE IS NO OBVIOUS WAY TO FIND OUT WHAT THE ERRONEOUS */
/* STEP IS IN THE CASE OF A JCLERROR CONDITION */
/* */
DO I=6 TO LINE.0
N = WORDS(LINE.I)
IF N = 0 THEN ITERATE
U = SUBWORD(LINE.I,N)
IF U = Y THEN
DO
IF N = 10 THEN M = 2
IF N = 11 THEN M = 3
/* THE STEPNAME IS NORMALLY IN WORD 2 BUT IF THERE */
/* IS A PROCSTEP NAME IT WILL BE IN WORD 3. */
/* IT IS VERY RARE FOR THERE TO BE A PROCSTEP NAME. */
D = WORD(LINE.I,1)
"STEPNAME ='"D"'"
E = WORD(LINE.I,M)
"PROGNAME ='"E"'"
F = WORD(LINE.I,N)
"CONDCODE ='"F"'"
LEAVE
END
ELSE
ITERATE
END
REXXOFF
/*
/* ALL THE VARIABLES ARE CONVERTED TO STRING VARIABLES
/* EXCEPT JOBNUMBER WHICH IS PADDED WITH ZEROS ON THE LEFT
/*
IF %MNNETID = '' THEN NETID = ' '; ELSE NETID = '%MNNETID'
IF %MNAPPL = '' THEN APPL = ' '; ELSE APPL = '%MNAPPL'
JOBNO = %MNJOBNO /* ONLY INTEGER VARIABLE IN DISPLAY */
JOBNAME = '%MNJOB'
COMPCODE = '%MNMXCMPC'
IF %MNMXCMPC = 'CCFAIL' THEN COMPCODE = '%MNHICMPC'
CENTER = 'CENTER2 '
TIME = '%ESPAHH..%ESPAMN'
DATE = '%ESPADD%ESPAMMM%ESPAYY'
STR1='%NETID(8)%APPL(8)%JOBNAME(8)%JOBNO(F5)'
STR2='%COMPCODE(8)%TIME(5)%DATE(7)%STEPNAME(8)%PROGNAME(8)%CENTER(8)'
STR ='%STR1%STR2'
/* ECHO 'STRING IS %STR' */
******************************** Bottom of Data ********************************
The string (STR) is feed into a batch job that puts the abend information on an MQSERIES queue to be picked up and turned into an inicident in CA Service Desk.
//DXEXAMN2 JOB (90948P,34SW,AHD000,B00000,34339),WINSTON,
// MSGLEVEL=1,NOTIFY=&SYSUID,PRTY=14,MSGCLASS=T,TIME=1
//*EXEC CNTR3
//*MAIN ORG=LOCAL,USER=CH34339,CLASS=DLI30
//* STRING BEGINS IN COLUMN FOUR ON THE NEXT CARD
//* %STR
//STEP01 EXEC PGM=ESPTOMQ
//STEPLIB DD DISP=SHR,DSN=DX.D952.JOBLIB
// DD DISP=SHR,DSN=DXX.MQ39.SCSQAUTH
// DD DISP=SHR,DSN=DXX.MQ30.SCSQAUTH
//SYSUDUMP DD SYSOUT=(,)
//SYSABOUT DD SYSOUT=(,)
//SYSOUT DD SYSOUT=(,)
//DSNTRACE DD SYSOUT=(,)
//SYSTSPRT DD SYSOUT=(,)
//* SYSIN SHOULD BE QM-NAME
//* MQOD-OBJECTNAME
//* MQOD-OBJECTQMGRNAME
//********************************************************************
//SYSIN DD DSN=DX.D952.ESP.CNTL(MQPARMS),DISP=SHR
//ESPEXCP DD DSN=DX.D952.ESP.EXCEPTN,DISP=OLD
//ESPHIST DD *
%STR
/*
//