Hello!
I've used two approaches to do this. We use ESP here, but I'm sure you can adapt either approach to your scheduler.
1. Use an event file sensor to trigger an event when the file comes in. This looks something like this:
/*************************************************/
/* DEFINED BY J94949B AT 13.23 ON WED 21MAR2018 */
/* LAST RUN AT 05.50 ON WED 2JAN2019 */
/*************************************************/
EVENT ID(ESPPROD.CW325X01) CLASS(PRODJOBS) SYSTEM(MASTER) REPLACE
CALENDAR SYSTEM
INVOKE 'SS.CAI.ESP.PROD.APPLLIB(CW325X01)'
WOBTRIG FILE_TRIGGER Agent(PESP042) -
FileName('/datawhse/inbound/BIOLABPIVOT_*_BCBST_*.txt') CREATE -
State(Monitored) Status('Monitored for CREATE')
This will create a new generation of the application every time the event "sees" a new file. So if you get 5 files, for example, it will create 5 generations. I suggest that you add a WAIT to the app so that each generation waits on the previous. Additionally, there are two built-in variables that are very useful. One is ESPWTFILE and the other is ESPFTFILE. The first variable resolves to the absolute path and file name, the second resolves only the file name. You can pass this as an argument to your job, and have it process a specific file. The result is that Gen 1 will process file 1, Gen 2 processes File 2, etc. The mainframe / dataset equivalent to this would be ESPRTDSN.
2. The second approach would be to schedule a an app. This app would contain 3 objects: A file dependency, a job that it releases, and a task that re-triggers the event. This has the same effect as a event level sensor, but gives a little more control, especially if the code that the job runs is written to process all of the files at once, and you don't want a generation per file. This looks something like this:
INVOKE SS.CAI.ESP.PROD.APPLLIB($SYMLIB)
INVOKE SS.CAI.ESP.PROD.APPLLIB($DEFAULT)
FILE_TRIGGER VW369_GRAPH
AGENT PESP042
FILENAME '/datawhse/inbound/bcbstn-connection-feed-*+
.txt.gz ' CREATE
RUN DAILY
RELEASE ADD(VW369X01)
ENDJOB
AIX_JOB VW369X01
RELDELAY 15
RUN TODAY
NOTWITH VW369X01
SCRIPTNAME /datawhse/prod/scripts/vw369x01.ksh
AGENT PESP034
USER dsadm
ARGS %USER1
RESOURCE ADD(1,CBDW.AVAILABILITY)
ENVAR JOBNAME=%ESPAPJOB
ENDJOB
JOB START TASK SELFCOMPLETING
RUN DAILY
ESPNOMSG TRIGGER VW369X01 ADD
AFTER ADD(VW369X01)
ENDJOB
Noticed there is a release delay (RELDEAY) in the job, so it waits 15 minutes after the file dependency is "tripped" before it kicks off. This is to ensure that all of the files have compmletley transferred into the folder before processing begins.
Hope this helps!