Hi,
We recently implemented a solution for a customer which deals with a similar problem as your question. On a high-level there is a similarity. Not sure if what we did will work for your use case. Marcy is right to ask for more details about your requirements.
It is a recent implementation, I do not yet have good visual material to present the solution, so I'll try to describe it as simple as possible.
This particular "Virtual Service Validation" solution is a specific implementation of a Low-code API Test Generation framework that we have built.
The objective of the framework is to easily build a whole suite of test cases (.tst files) without having to manually adapt test steps, and filters, and assertions. The functionality of a test case (i.e. the steps, filters, assertions ...) is created based on a template test case. So, this means that the generated test case can be quite complex - meaning rich in functionality - if needed, but every test case has a similar structure, i.e. functionality, as it created from the same template. So, this is ok for API Tests where every testcase is similar, i.e. sending a request payload to the API, and then running some assertions on the response to validate it is as expected. So, this type of test we implemented via the "Template_APITest" template.
Something similar to your requirement was implemented via our "Template_VSValidation" template. The functionality of a generated VS Validation testcase is as follows:
- Pick up a request payload
- Send the request to the Live API
- Send the request to the Virtual Service API
- Filter sets of relevant information out of the Live Response
- Filter the same sets of relevant information out of the Virtual Response
- Assert that the filtered Live Response information is equal to the filtered Virtual Response information, if so test ends, otherwise test fails
The definition of such a test suite would be done completely by editing some files in a directory, it would look something like this:
The -req.xml files would contain a request to be sent to the API, it would look something like this:
The -val.xml files would contain the definition of the filters and assertions as they would be defined/required by the "Template_APITest" template, it would look something like this:
After creating a test definition like above for every test you want to generate, we would run a batch file which eventually would use testrunner to execute another test - i.e. CreateTestV3 - which would write out a .tst file for every pair of -req.xml + -val.xml file.
The generated testcase would look like:
The actual API calls are done inside a subprocess (doesn't have to be but here that is how the template is built). I selected the "Filter Live Response Payload" step, you can see it has 2 filters created (those are the ones defined in the "LiveCallFilters" Filters element of the -val.xml file. You can see the 2 assertions created in the final "Validate Filtered Data Equal".
So, the above is the long answer, the short answer to your question is 'yes, this is achievable'.
Cheers,
Danny