greda13

Product Announcement - Improvements to health of large adhoc batch jobs

Discussion created by greda13 Employee on Feb 12, 2018
Latest reply on Mar 9, 2018 by JohnStreeter

In an effort to increase the health of large batch jobs hitting our adhoc endpoint, we have made changes to queue large numbers of requests using that endpoint. This change will reduce the number of active adhoc requests in our system by a single user to 16. If more requests are are part of a larger batch, they will be placed in a queue.

 

Initial tests of the approach have proven successful. Of 137,896 initial jobs, 10,188 were queued. P95 measurements against an initial baseline suggested only a minimal loss of performance of 5 seconds. 100% these jobs completed successfully. Ultimately users making large adhoc requests to our systems should see less jobs failing and less jobs needing to restart for more data faster.

Outcomes