We have noticed that, when running load tests, we get severe, isolated outliers. They present themselves in the pattern seen in the image attached below.
In this instance, we are running a test case of 16 steps consisting mainly of HTTP, Web Service and FTP calls. The long wait time appears to occur during a web service call, which takes much more time than usual (see image attached below).
The Web Service in this example is very stable and has no performance issues. Also, this outlier pattern occurs with many different backend systems, even if we run a test case against virtualized backends.
Details on the infrastructure:
Version: CA LISA 7.5.2 on Linux
DB: OracleDB (with the famous HST_STP_TEST_IX Index added)
VM1: 1 Registry, 1 Simulator. 6 cores, 16GB Ram
VM2: 1 Coordinator, 1 Simulator. 6 cores, 16GB Ram
VM3: 1 Simulator. 3 cores, 4GB Ram
log-level set to WARN
Test case is run with:
- Default Report Generator
- no PF
- Record Properties Set/Referenced
- Record Performance Metrics
- 50% Think Time (Think Time per Step: 0.5-1s)
- 100 instances
Has anyone else seen this behavior? Is there a way to mitigate this?