AnsweredAssumed Answered

Anyone else getting severe outliers during Load Tests?

Question asked by ipt_rha on Sep 7, 2015

Hi all,


We have noticed that, when running load tests, we get severe, isolated outliers. They present themselves in the pattern seen in the image attached below.



In this instance, we are running a test case of 16 steps consisting mainly of HTTP, Web Service and FTP calls. The long wait time appears to occur during a web service call, which takes much more time than usual (see image attached below).



The Web Service in this example is very stable and has no performance issues. Also, this outlier pattern occurs with many different backend systems, even if we run a test case against virtualized backends.


Details on the infrastructure:

Version: CA LISA 7.5.2 on Linux

DB: OracleDB (with the famous HST_STP_TEST_IX Index added)



VM1: 1 Registry, 1 Simulator. 6 cores, 16GB Ram

VM2: 1 Coordinator, 1 Simulator. 6 cores, 16GB Ram

VM3: 1 Simulator. 3 cores, 4GB Ram

log-level set to WARN


Test case is run with:

- Default Report Generator

- no PF

- Record Properties Set/Referenced

- Record Performance Metrics

- 50% Think Time (Think Time per Step: 0.5-1s)

- 100 instances



Has anyone else seen this behavior? Is there a way to mitigate this?