Service Virtualization

Expand all | Collapse all

CA DevTest Throttle # of connections

Anon Anon

Anon AnonJun 02, 2016 11:28 AM

  • 1.  CA DevTest Throttle # of connections

    Posted May 26, 2016 12:18 PM

    Hi All,

    I am trying to implement below scenario with CA DevTest tool. Is it possible with CA DevTest tool ?

    • A CA DevTest HTTP Service is written that throttles connections if more than 10 per second connections are invoked on it
    • The client application is exercised in a manner that it does not call the CA DevTest stub at the rate of more than 10 (N) invocations per second
    • Validate that the application is not throttled and if throttled then respond with HTTP 429 error code on 11 (N) invocation onward.

    Please help. Thanks.



  • 2.  Re: CA DevTest Throttle # of connections

    Posted May 26, 2016 02:03 PM

    I have been wrong before, but I do not believe it is possible to accomplish all of these requirements in DevTest without writing some type of custom handler.  I'd definitely be curious to know if anyone has successfully done something like this before.

     

    - If you run the functional VSE (not L&P), DevTest will throttle the HTTP request / response throughput to approximately 10 TPS; however, the 11th transaction simply waits until one of the former transactions completes before sending its response.  If one assumes that each incoming transaction had a response time of 100 ms, the functional VSE would throttle to around 10 TPS (i.e., 10 x 100ms = 1000 ms).

     

    - I am unaware of a parameter on a given request that indicates the number of concurrent transactions and current TPS rate that is occurring within the VSM.  It seems that implementing this type of requirement via filters on the LISTEN Step is probably too late to trap this requirement.  You would probably have to back up into the HTTP Transport level.  Then, you need a customization that adds something to the request to indicate to the service that this particular transaction needs to send the HTTP 429 response vs the 'programmed' response.

     

    - I wonder if this requirement could be implemented using a load balancer such as F5.  Maybe there is a way to limit the connections so that the 11th txn gets a fault containing the 429 response.



  • 3.  Re: CA DevTest Throttle # of connections

    Posted May 31, 2016 03:08 PM

    Thanks Joel. I am still struggling with the implementation. I am thinking of custom Rate Limiter algorithm in Groovy or Java API hot deployment to achieve throttle number of transactions. Even I am looking for any variable which gives me the number of concurrent transactions and current TPS rate that is occurring within the VSM.



  • 4.  Re: CA DevTest Throttle # of connections

    Posted May 31, 2016 03:51 PM

    I do not believe there are any OOTB properties that you can access from a VSM that will help you determine concurrent transactions or TPS rates.  Think of the VSM as an instance (or worker thread) that is tied to a specific request.

     

    Perhaps your Groovy or Java API, could implement enough logic to create some form of regulator.  And, you would likely need to implement this customization on both the Listen and Responder steps so your transaction counts are managed (incremented / decremented). 

     

    However, my limited understanding is that DevTest uses a different architectural approach to throttling the functional VSE.  An internal response queueing mechanism uses a combination of Think Scale, Think Time Percentage, and a response queue to manage responses.  If true and if the response is actually sent using a background thread, the VSM's Responder step could theoretically fire allowing the thread to be available to the service the next transaction before the actual response is placed on the wire.  If this happens, your self implemented regulator script becomes suspect as to whether or not it is accomplishing your requirement.  



  • 5.  Re: CA DevTest Throttle # of connections

    Posted Jun 01, 2016 03:31 PM

    Yes Joel. I agree. As I believe, VSE maintains own thread pool and all virtual services deployed on it uses common thread pool to process incoming traffic. Basically VSE component is responsible for thread management. So its too late to identify the transaction per second count in VSM. Am I correct ? Even I tried to implement customized regulator to track traffic load in VSM but so far no luck. 



  • 6.  Re: CA DevTest Throttle # of connections

    Posted Jun 01, 2016 07:46 PM

    I am not sure internal thread mechanism in devtest.. but sometime back we have implemented similar thing in oracle middleware platform.. where we had number of threads issue so we listen request on one thread and respond back on another thread.. also there are different thread settings like minimum thread service and maximum thread service.. not sure how u can achieve in DevTest...

     

    is this 10 TPS applicable to performance VSE as well ? or only with functional VSE..



  • 7.  Re: CA DevTest Throttle # of connections

    Posted Jun 01, 2016 09:17 PM

    TPS is only throttled for functional VSE.  Performance VSE disables this feature.



  • 8.  Re: CA DevTest Throttle # of connections

    Posted Jun 02, 2016 09:32 AM

    Thanks Joel.. what is TPS limit on performance VSE ? ..

     

    Will that applicable for all protocols ? like HTTPS , MQ etc.,



  • 9.  Re: CA DevTest Throttle # of connections

    Posted Jun 02, 2016 04:33 PM

    There is no internal limit on TPS for a performance VSE from a software perspective as far as I know. 

    Off the top of my head, the major factors that contribute to overall TPS are:

    - Customer License (are there any limitations imposed by a given customer license agreement)

    - Server Size  (# of CPUs, # Cores per CPU, GHz rate of CPUs)

    - How much of the CPU is dedicated to VSE vs processing other services

    - # of VSEs available (is the service on a single VSE or running in multiple VSEs behind a load balancer)

    - Memory (RAM available on server, RAM available to VSE)

    - # of services running on the VSE

    - Protocol

        The most common protocols I see load tested are JMS and HTTP

    - Payload size (request / response payload sizes can impact throughput)\

    - Amount of customizations such as XPaths, filters, and/or additional steps, database I/O steps, custom Java or Groovy code, etc. that have been added into a VSM

    - Network capacity and latency (how quickly can the request / response get on and off the wire)

    - And, then there are the various tunings that one applies..  Some of which are sever based such as ulimit.  And, others such as setting logging levels to WARN, using an Enterprise DBMS rather than the Derby database, setting the capacity of a given service based on response times, and the list goes on.



  • 10.  Re: CA DevTest Throttle # of connections

    Posted Jun 02, 2016 07:54 PM

    Thanks Joel, very helpful.. we are facing this performance issue in MQ call.. we have MQ VS setup for one of the performance testing of soap call.. where soap is expecting response in 5 sec else request gets timeout in web service.. when load is minimal on these VS like 2 vuser n enough pacing time then VS will work fine whereas we are expected to put more load on our web service that is like 10vuser at a time and it runs for one hour, then we see some failures , also response is not quicker .. as result we are not getting accurate result as like live systems.. I have tuned vs model as much as possible and make sure logging is minimal n no extra steps are added in vs model but still response time is not great when load is more and simultaneous..So now I understood this may be thread issue.. if we are able to handle that it might help us to improve the performance..

     

    apart from these .. you have mentioned about CPU , Memory etc., will this also also make sense if VSE is functional VSE ?



  • 11.  Re: CA DevTest Throttle # of connections

    Posted Jun 02, 2016 02:36 PM

    Hi Joel, Just wanted to check, Is it possible to get a hook to the internal VSM worker thread where I can get a number of thread count and based on that I can calculate TPS. Another way is to write a daemon thread at VSM level which will be active after every second and get a hold of internal request counter. or Is there any way to set a scheduler at VSM level to achieve this ?



  • 12.  Re: CA DevTest Throttle # of connections

    Posted Jun 02, 2016 04:33 PM

    No hooks that I know of.



  • 13.  Re: CA DevTest Throttle # of connections

    Posted Jun 02, 2016 05:03 PM

    Hi Joel. I have implemented a groovy code using TimerTask and able to get the count of transactions executed per second. what I observed is for functional VSE the TPS limit is 10. Am i correct ? Is there any way to increase this TPS limit ?



  • 14.  Re: CA DevTest Throttle # of connections

    Posted Jun 02, 2016 07:43 PM

    You will not be able to increase the limit on a functional VSE.  You would have to switch to a performance VSE to "unlock" the internal TPS governor.



  • 15.  Re: CA DevTest Throttle # of connections

    Posted Jun 19, 2016 03:30 PM

    J_NeSmith

    had one more question on this topic..  we had situation where API hits VSM in Load testing mode.. here API load distribution is 20 concurrent Vuser hitting API at a time and same tests repeat for one hour.. as you mentioned functional server only handles 10 TPS. 11th trxn has to wait till 1st trxn completes , and 12th trxn has to wait till 2nd trxn to complete and so on.. so this will add 100ms extra waiting time for 11th trxn and 200ms extra waiting time for 12th trxn and so on.. as a result I always get different avg resp time in API when load testing is completed provided number of Vuser I use.. if use less like 5Vuser it looks fine.. any suggestion how can we improve these cases .. apart from VSE license type which we are aware already.. thanks..



  • 16.  Re: CA DevTest Throttle # of connections

    Posted Jun 20, 2016 09:03 AM

    There is not really a way to squeeze out more than approximately 10 TPS performance when using a single functional VSE as that would be tantamount to tricking a functional VSE into thinking it was a performance VSE.

    I suppose you could set up multiple VSEs (different endpoints) and deploy the same service on each.  Then, let either your API or a load balancer distribute the calls to the multiple functional VSEs / service.

     

    Couple of things to consider:

     

    -          State is not managed across VSEs:  If your transactions are stateful, some mechanism needs to pin subsequent calls from the same instance of the API to the same virtual service.

     

    -          Load balancing does not necessarily double throughput: Adding a load balancer does not necessarily mean that if one VSE can achieve 100 TPS, a second VSE behind the load balancer will increase throughput to 200+ TPS

     

    Joel



  • 17.  Re: CA DevTest Throttle # of connections

    Posted Jun 02, 2016 07:59 PM

    AVSDevTest thanks for sharing the details.. would you be to able share here or send me the details on timer task please..



  • 18.  Re: CA DevTest Throttle # of connections

    Posted Jun 03, 2016 12:12 PM

    Hi DevTestUser,

     

    I have added "Execute Script (JSR-223)" step in VSM and here is the groovy code I have written to get TPS details with counter mechanism. I am still working on some features to manager timer task thread properly. You can modify the code as per your need. FYI: With this code timer will be running all the time in VSE. You explicitly have to close the VSE to stop the timer.

     

    -------------------------------------------------------------------------------------

     

    import static Constants.*

    import com.itko.lisa.vse.stateful.model.Response;

    import com.itko.lisa.vse.stateful.model.TransientResponse;

    import com.itko.util.Parameter;

    import com.itko.util.ParameterList;

    import java.util.timer.*

    import java.util.TimerTask;

    import com.itko.lisa.VSE;

    import com.itko.lisa.test.TestExec;

     

    class TimerTaskThread extends TimerTask {

         public void run() {

              if(Constants.numberOfTransactions > 0) {

                   VSE.info(Constants.testExecObj, "TimerTask : Transactions in Last 1 Second : ", "" + Constants.numberOfTransactions);    

              }

              Constants.numberOfTransactions = 0;

         } 

    }

     

    class Constants {

         static final maxTransPerSec = 5; // maximum number of connections

         static numberOfTransactions = 0; // counter to maintain processed transactions

         static int delay = 0   // delay for 0 sec. 

         static int period = 1000  // repeat every 1 sec. 

         static TestExec testExecObj = null; // testExec object

         static Timer timer = new Timer().scheduleAtFixedRate(new TimerTaskThread(), delay, period); // timer

    }

     

    Constants.testExecObj = testExec;

     

    synchronized(this) {

         Constants.numberOfTransactions = Constants.numberOfTransactions + 1;

    }

     

    if(Constants.numberOfTransactions > Constants.maxTransPerSec) {

         //Build response object

    }

    -------------------------------------------------------------------------------------

     

    Thanks.



  • 19.  Re: CA DevTest Throttle # of connections

    Posted Jun 19, 2016 03:16 PM

    Hi AVSDevTest I have tried to capture TPS using the above code shared , instead VSE.info I have logged to server console (because easy for me to verify) and I ran test from load testing tool with more than 20 Vuser hitting VSM directly on every second. Once execution started and when I looked at the server console (where I logged) shows only count as '1' . and also this count shows only for first hit , any transactions after first transaction will not show what is the TPS count.. sorry if I am wrong here..please let me know if this is not right way to capture.. also are you successfully able to handle using demon thread and share the load when more than 10 TPS in VSM.. thanks for your time.. 



  • 20.  Re: CA DevTest Throttle # of connections

    Posted Jun 22, 2016 07:49 AM

    AVSDevTest  hi aniket , could you please check my above issue and would you be able to suggest anything please



  • 21.  Re: CA DevTest Throttle # of connections
    Best Answer

    Posted Jun 22, 2016 02:54 PM
      |   view attached

    Hi DevTestUser, hope this will be helpful. Please let me know if you still have any issue.

     

    Here is my VSM -

    Here is Groovy Rate Limiter code - PFA groovy_rate_limiter.txt

     

    Project Config Properties -

    • MAX_TRANSACTIONS_LIMIT
    • TRANSACTION_PERIOD_IN_SEC

    Attachment(s)

    zip
    groovy_rate_limiter.txt.zip   1021 B 1 version


  • 22.  Re: CA DevTest Throttle # of connections

    Posted Jun 22, 2016 07:39 PM

    thanks AVSDevTest .. it is more clear now.. i will use in my model and let you know if hit any issue.. so it will  tell the client that it has reached thread limit now..



  • 23.  Re: CA DevTest Throttle # of connections

    Posted Jun 03, 2016 09:31 AM

    it may not be 10 TPS always.. some time I have seen 13 TPS have been processed by VSE.



  • 24.  Re: CA DevTest Throttle # of connections

    Posted Jun 02, 2016 11:28 AM

    Functional VSE only.