Automic Workload Automation

Hello,

I am guessing the answer is no to this question but is there anyway of speeding up the workflow generation?

We have a generic solution that is activated via a job with various promptsets attached.  Selecting options from these prompts will then set up the required variables for the workflow.  

Once selected and the submit button is pressed it takes approx 5 mins for the workflow to go from generating to active.  The workflow does contain a fair amount sub workflows and each job has an INCLUDE that evaluates a run/no-run decision.

We gave a demo and someone asked the question, if all the same options have been selected from the promptsets why does it still take the same time to become active?  Can it not use the previous runs objects/data?

Just thought i would ask

Cheers,

Dan.

Is Generate at Runtime selected for all the sub workflows and jobs within the sub workflows?

Hi Dan

A generic question about a generic Workflow results in a generic answer :-P

My first attempt would be checking generate @ runtime as well, if possible, as Michael suggested.

If you encounter slowliness and latency in general on your system a hardware upgrade (or more WPs) would be good.

If its just the generation ot this one and only Workflow, you could try tuning your scripting (if possible) avoiding/reducing heavy use of PREP_PROCESS_XYZ, SQL(i) VARAs, etc.

@ Can it not use the previous runs objects/data?

Very generic answer- The processing amout is almost the same - the job has to check a VARA/PRPT/... if it should run or not. I do not think it will make a big difference, if its using a previous run information or a definintion from somewhere.

cheers, Wolfgang

Thanks for both the replies ... and i know ... lots of generics in the question

Cheers,

Dan

Hello,

Just an update/more questions on this really ..

Generate at activation time is selected for all the jobs

Each job has the same include call to decide if the job needs to run against the environment selected.  The include does a GET_VAR to determine this.

I am guessing this logic in every job is why it is taking a while to generate the full master workflow.

If I changed every job to generate at execution would I just be moving this time taken further down the chain?  I am thinking this code needs to execute at some point so whether that is at activation or execution that time taken will be the same?

Also, sorry ..... normally the workflow takes about 5 mins from activation to start ... on the odd occasion it takes over 10 minutes ... any idea where a good place to start would be in terms of finding any potential bottle necks?

How much impact should simultaneous activation's of the same workflow have on this kind of performance?

Cheers again ..

Dan

Hi!

I am guessing this logic in every job is why it is taking a while to generate the full master workflow.

=> possibly yes but without knowning the whole monster-structure its hard to tell :-)

Usually a couple of get_vars do not produce that heavy system workload...

If I changed every job to generate at execution would I just be moving this time taken further down the chain?  I am thinking this code needs to execute at some point so whether that is at activation or execution that time taken will be the same?

=> Yes exactly, the generation time will not be reduced but shifted. Thats a common step to "reduce" the actvation time at the start of bigger workflows. Activation of the single tasks is not perceived that long.

Also, sorry ..... normally the workflow takes about 5 mins from activation to start ... on the odd occasion it takes over 10 minutes ... any idea where a good place to start would be in terms of finding any potential bottle necks?

=> You could put a JOBI in your generic JOBI containing a put_var &LOGVARA# with Object name, timestamp parent etc.

then you can check where is the longest activation time of your subtasks

cheers, Wolfgang

If these scripts you are running are redundant, then you might consider moving some of the expensive script logic out of the jobs and into the workflow.  The workflow process script only executes once at the beginning (rather than for each task), and it could store results in workflow level variables that could be interrogated by each task.

Particularly if you could change it to read an external data source once instead of 20 times.  Reducing external I/O would be the biggest bang for the buck.

Hello Peter,

The script/logic that is running in every job is basically just a lookup (GET_VAR) to an environment specific variable object. e.g. VARA_RUN_JOB_#ENVNAME#  If a record exists in that environment variable for the job name then the job will not run. 

! Do not run the job if deeemed not neccessary
:IF &STOP_MESSAGE# <> ""
: STOP NOMSG,50,"&STOP_MESSAGE#"
:ENDIF

Am not sure how this can be done anywhere but in the pre-process of the job itself? Please shout up if there is a more efficient way we can achieve the same thing

Just for info here are the timings for the comparison:

So the overall 'press execute' to end time is faster when generate at runtime but workflow execution time is faster when generate at activation (as you would expect).  We have an additional time check job in the workflow as one of the first jobs anyway so in theory we can leave the jobs as 'generate at activation' and then start the workflow on the schedule 15-20 mins earlier than we actually want it to start.

Also for info when I executed the same workflow 10 times in parallel the Worklfow Generation time jumped from 15 minutes to 30 minutes.  This will need looking into.  Does anyone else have a drop off rate similar to this?

Cheers,

Dan

The scripting you've shown is not costly to run.

Your primary process script has to be generated before the task can execute too.  Do you have something expensive going on in that script?

Hello Peter,

There is nothing particularly clever/intensive from what i can see going on in any of the workflows/jobs (apart from the logic above).  There is one master workflow with probably another 15 sub workflows containing approx 600 jobs in total across the whole lot so not sure if the time taken is expected or not really.

Cheers,

Dan

Wow, 600.  

Each process script is dynamically compiled.  I'm not sure you can expect any better performance.

Could you break it into parts and use external predecessors between them, so they can fire up in a more parallel fashion?

Maybe someone else with a 600 task workflow could speak about their performance numbers?

Hello,

Yeah I did not think that was a massive amount but maybe it is.

Most of those jobs are contained in one workflow that is called 3 times from the main workflow.  Each one runs against a separate database instance.

I only raised this point as we are migrating from another scheduling tool where all the user has to do is press go on a queue and the first job starts immediately.  I am sure they will get used to the generation wait ... will give them chance to get a brew .. they should be grateful really

Dan