Nikhil_Karel_9136

Work flow dependencies and coordination

Discussion created by Nikhil_Karel_9136 on Jun 21, 2017
Latest reply on Sep 29, 2017 by Pete Wirfs
Hi all,
Let me jump into the question right away. We have two different workflows running in different client numbers and clusters. The data generated as output of WF1, WF2,WF3,WF4.....WF60 and so on is needed as input to run the WF100. For which we cross verifying the WF1,WF1, WF2,WF3,WF4....WF60  are running or done and then move the data from 1 cluster to another for which we have a shell command. If we move the data when the tasks are running then there is 100% chance that the data copied is inaccurate/incomplete and input data to WF100 is not reliable and output of  WF100 is inaccurate. To address this issue we are currently going through each status of  WF1,WF1, WF2,WF3,WF4......WF60 is complete or running and then do a data transfer, this manual effort is taking up a lot of time. So is there a way I can wrap this problem up ?

Is there way for WF100 to know that the WF1,WF1, WF2,WF3,WF4....WF60 are running or done as a precondition check? If done then launch the data move-shell command so that the data is copied from 1 cluster to another, the WF100 has the input data needed to perform the task designated to it.



Thanks,
Nikhil

Outcomes