Test the Correct System

Blog Post created by Mike_Gavaghan Employee on Oct 24, 2017

The best virtual services are simple to build, easy to maintain and usable for testing. To get them to that point, I strongly recommend meeting the following preconditions:


  • Sufficient analysis of test cases
  • Identification of test data
  • For performance testing, determination of the target load.


Test data is an important part of the picture, and the best practice for building test cases is to build them so they validate the system under test and not the dependent system!. If your test cases are validating the virtualized dependent system, you're merely testing your virtual service!


Testing the virtual service adds unneeded complexity and may even create unnecessary work for your testing team.


One obvious exception, here, are test cases created to verify that your virtual services is behaving as intended.  These tests, however, hit the virtual service directly and through the system under test.


Remember, there is a contract between the service provider and the client. Your virtual service is an idealized version of the real service. If the real service misbehaves, it should be caught during system testing of the real service. If not, it will be caught during integration testing.


Building a virtual service to support test cases that are testing the behavior of the dependent service is unneeded work. At worst, the passing test cases will provide a false sense of security.


A Better Approach

Focus on getting the responses necessary to allow the system under test to complete its test workflow. The responses should capture the behavior of the dependent system as it flows through state changes. It is not necessary for the virtual service to actually track those changes. Your system testing goal is to verify that the system under test properly responds to the data it receives – regardless of how that data was obtained.