Service Virtualization

  • 1.  Processing more than 200,000 records in VSM of MQ service

    Posted Oct 17, 2017 06:26 AM

    Hi ,

     

    I am working on a project which requires a service to search a SystemID of a employee from a List of 200,000 IDs on the basis of employee number in the requst:

    Request: employee Number

    Response: employee Number, SystemID

     

    Service is provided with the records of 200,000 employees and service has to look for EmpNumber from request and search this EmpNumber in the record and reply with corresponding SystemID.

     

    What I am trying :

    1- Store the record in Excel rows

    2- Read the EmpNumber from request and call subprocess

    3- Sub-process Iterate the dataset Excel records(mode: Local) one row at a time till match found.

    4- Return the SystemID to vsi for Publish.

     

     

    Above is working fine with few 100 records, but it getting very much slower in responding when records are above/around 1000.

     

    Please suggest some workaround or other method to reduce this time. The slowness is much beyond the SLA.

     

     

    Thanks,

    Dinesh Kumar



  • 2.  Re: Processing more than 200,000 records in VSM of MQ service

    Posted Oct 17, 2017 10:47 AM

    Hi Dinesh,

                     Why have not you tried storing the records in a database?

                                             Thanks,

    Vish



  • 3.  Re: Processing more than 200,000 records in VSM of MQ service

    Broadcom Employee
    Posted Oct 17, 2017 12:02 PM

    Hello Dinesh,

     

                  Since out of box, we do not support data driven vs for mq at this time, you need to build a custom implementation for this. One scenario you can try is to read the 200k records from an external source like a DB and load it into a map (shared model map). This load is a one time activity and once it is done, for all requests, you can just query the map to get data much quicker with low overhead. Hope this helps. Anytime the data changes, you need to redeploy the service, which will once again read the DB and refresh the data in the map with latest data. 

     

    Thanks,

    Surya.



  • 4.  Re: Processing more than 200,000 records in VSM of MQ service

    Posted Oct 23, 2017 06:37 AM

    Hi Surya,

     

    Thanks for suggesting the approach of SahredModeMap.

     

    I tried using the approach of ShareModelMap but  I am unable to load the data at once, it is storing max 50 records at a time, I am not much aware of limitations and properties of It. Can you please guide me on below queries:

    1- Does it have some upper limit of storing data on VSE?

    2- Any alternate of inserting records in SharedModelMap instead of service call.

    3- How can I ensure one time insertion of data into SharedModelMap.

     

    I am able to insert 100+ records at time in Workstation SharedModelMap, but same is not working on VSE using service call, please guide.

     

     

    Thanks,

    Dinesh Kumar



  • 5.  Re: Processing more than 200,000 records in VSM of MQ service

    Posted Oct 23, 2017 06:43 AM

    Hi Vish,

    my current Dev installation doesn't have external Database configured, we are in process of upgrading the DevTest once external database is approved.

     

    Thanks,

    Dinesh Kumar



  • 6.  Re: Processing more than 200,000 records in VSM of MQ service

    Broadcom Employee
    Posted Oct 17, 2017 02:05 PM

    Hi Dinesh,

     

    Is having 200,000 records even necessary?

     

    A virtual service is built to enable the workflow of predefined test cases.  Unless you have 200,000 independent test cases, you can solve this with a traditional virtual service image.  Predefine which employee IDs you will use, and add them (and only them) to the VSI.

     

    If this is for performance testing, the same goal can be achieved using magic strings and matching criteria.

     

    This will save you from the complexity of building (and supporting!) a customized service model.

     

    --Mike



  • 7.  Re: Processing more than 200,000 records in VSM of MQ service

    Posted Oct 23, 2017 06:40 AM

    Hi Mike,

     

    Yes Dev team wants to test the performance of the app server which validates the EmpID and SystemID.

    So they are providing us with the actual unique EmpIDs and SystemIDs.

     

    Please suggest a way to tackle this.

     

     

    Thanks,

    Dinesh Kumar