Hi,
If I have understood your question correctly you will have 5,000 or more records on the queue awaiting processing at a time.
If this is really the case, then the things you can do are increase capacity. ensure that you have VSE performance enabled and make sure that your process has enough memory. The processing will then be limited by the complexity of the virtual service, the size of the requests, system IO constraints and processor resource.
To optimise the virtual service, ensure that you are performing the minimum amount of work (reduce number of arguments etc). Also ensure that your logging is set to WARN or ERROR.
If you are running out of system resources after this you will probably need to scale out with additional VSEs on additional hardware.