Skip navigation
All Places > DevTest Community > Blog > 2016 > April
2016

Recently I did a webcast on how CA Service Virtualization could be used for API performance testing. At the end of the event we received a number of great questions. So many that we ran out of time and could not get to them all.

 

With that, I’ve gone ahead and answered the questions from the event in this blog.  For those of you who missed the event and may want to watch you can view it at: http://bit.ly/1NaZmve.

Questions in the order answered below include:

 

  • Question 1: Is service virtualization the same thing as application virtualization?

  • Question 2: What is the difference between mocking a service, stubbing a service and service virtualization?

  • Question 3: What type of testing should service virtualization be used for: System Tests (ST), System Integration Tests (SIT), or User Acceptance Tests (UAT)?

  • Question 4: Typically external services have some sort of inherent wait time associated with network travel time; can we simulate this since the solution will most likely be in the same network as our System Under Test (SUT)?

  • Question 5: Can we replicate network latency/bandwidth limitation scenarios using service virtualization?

  • Question 6: If my actual application is not available in one environment, can I virtualize that application and make it available in the required environment?

  • Question 7: We have many developers that test against our externally facing APIs can we replace those with a virtual service on our end? Will the developer notice any difference between the real service and the virtual one?

  • Question 8: How does service virtualization factor into 3rd party testing?

    Question 9: What third party services can you emulate with your virtual services: Facebook, Twitter etc.?

  • Question 10: Is there a difference in the way service virtualization handles a virtualized service vs. a real service?

    Question 11: What are the requirements for an application to create a virtual service? Does an application ever need to be rearchitected in order to create a virtual service?

  • Question 12: How do you map the speed of the virtualized system to compare it against the performance of the real system?

    Question 13:  Can you create fluke error and edge conditions for performance testing with service virtualization?

  • Question 14: Is service virtualization beneficial for an IT ecosystem where all the services are available in all the environments 24X7, and there are no third party services?

  • Question 15: Is CA Service Virtualization supported in any public/private cloud servers?

  • Question 16: How can I try CA Service Virtualization to test Microservices?

  • Question 17: Can we virtualize RDBMS databases using service virtualization?

  • Question 18: What are some of the future enhancements planned in CA Service Virtualization?

 

Question 1: Is service virtualization the same thing as application virtualization?
Answer:
Service Virtualization does simulate the back-end of an application, so at a certain level they are the same. One thing to keep in mind is we are mocking, stubbing specific testing, and data scenarios, but we are not actually performing the actual business logic. Service virtualization does cool things like magic strings, magic dates and we have the ability to give dynamic responses. It's not the business logic of the backend system that we are testing.  We are testing the interactions between different layers of the application. But it’s good enough. It’s more than good enough. The first time I got introduced to the solution I was working with a client in the field and we were showing them the capability. After we were done I remember the developer coming up to me and saying something must have been wrong because they had put in a transaction that the virtual service had not seen yet and we had to explain that we can give dynamic responses.

 

Question 2: What is the difference between mocking a service, stubbing a service and service virtualization?
Answer:
Stubs provide pre-canned answers in support of a test and are deterministic at the time you write them.  Generally, mocks allow a data scenario or set of expectations to be initialized at run time. Both of these require development resources to write and generally are closely coupled with the actual code. Virtual services enable you to accomplish both mocking and stubbing.  In contrast, virtual services are created by recording actual services, by contract, or by being provided with sample requests and responses. They are inherently more dynamic and don’t require a developer to write them. Virtual services also already come with a deployment framework making creating them and consuming them a much more seamless operation.

 

Question 3: What type of testing should service virtualization be used for: System Tests (ST), System Integration Tests (SIT), or User Acceptance Tests (UAT)?
Answer:
Integration testing is usually where most clients start using service virtualization. System testing comes in many times during prototyping. The best practice for UAT testing is to use production systems. We do have some clients that simulate some third party calls during UAT. Regardless, the nice thing is with service virtualization most of the defects have been found before you get to UAT because you’ve already run comprehensive system, integration and performance tests.

 

Question 4: Typically external services have some sort of inherent wait time associated with network travel time; can we simulate this since the solution will most likely be in the same network as our System Under Test (SUT)?
Answer:
This is a very common requirement for any performance test because a virtual service can respond instantaneously.  To address this we have created what we call “think time”. It exists at both the service and transaction level. When we record a service we observe the think time, so if we see the response takes 15 milliseconds, then that is how long the virtual service will also take to respond. From there we can then scale times up or down. For example, we can slow things down for the first five minutes while we’re waiting for all the classes to preload. If we also know the performance decreases by 1/3 we can simulate that kind of start time, or we can create a spike for garbage collection every 10 or 15 minutes. We not only have the ability to control response times for the test run, but also at the transaction level, and at the component level. This becomes especially important when working in agile environments with multiple teams working on multiple pieces. In addition, if you are a CA APM customer and you’ve been collecting your performance data from production you can feed that into CA Service Virtualization and make you test system perform just like the real thing.

 

Question 5: Can cwe replicate network latency/bandwidth limitation scenarios using service virtualization?

Answer: Yes. Using think time you can introduce delays and specific timing to the responses to simulate network latency. The think time can be randomly selected from a range. 

 

 

Question 6: If my actual application is not available in one environment, can I virtualize that application and make it available in the required environment?
Answer:
Yes. That is much of what service virtualization does. A good example of this is how we virtualize third party services. Third party services are not even within your network, there not available. We can still capture the needed information using code contracts like WSDLS, or we can use Request and Response pairs. Service virtualization is all about removing obstacles, virtualizing what is not available. Virtual services can be deployed in various Virtual Service Environments (VSEs) so developers and testers have everything they need when they need it. 

 

Question 7: We have many developers that test against our externally facing APIs can we replace those with a virtual service on our end? Will the developer notice any difference between the real service and the virtual one?

Answer: Testing against external APIs is a bread and butter use case for CA Service Virtualization.  External APIs are a common constraint and SV focuses on eliminating constraints.  In general when used for testing, developers don’t need to know that they are using a virtual service vs. the real thing. However, virtual services don’t replicate the full business logic of a service. There are edge cases where it might be noticed.  Virtual services are generally also much easier to provide developers access to and for them to consume. This helps eliminate reduce wait times and test windows and enables developers to be more productive.

 

Question 8: How does service virtualization factor into 3rd party testing?
Answer:
The first challenge with 3rd party testing is connectivity. How do you get access to the 3rd party service? The second challenge is it not always free to test the 3rd party service. Even though it may only be fractions of a penny when it comes to performance and regression tests that are being run in an agile development cycle costs can add up quick.

 

Question 9: What third party services can you emulate with your virtual services: Facebook, Twitter etc.?

Answer: We focus on protocols.  Virtual Services have 2 layers of protocols.  Transport protocols handle things on the wire and support TCP, HTTP, MQ, JMS, etc.  On top of that, you have Data protocols that interpret the message formats such as SOAP, REST, JSON etc.  You can mix and match those as needed to support a third party API.  Most third party API’s will conform to one or more of these standards and can be virtualized out of the box. 

 

Question 10: Is there a difference in the way service virtualization handles a virtualized service vs. a real service?
Answer:
We’ve had that capability to switch between real and virtualized services built into the product for a while. You can set-up all your connections to go thru the virtual service, and then the virtual service can send it to a live service. Within a test environment, you can tradeoff between a virtual service and a live service if you have reasons for doing so. The ability to switch between live and virtual services also plays into what we call image validation. Sometimes live services change over times. It is important to periodically look at the comparison of what you see when you run a test against a virtual service vs. a live service to see what has changed over time.  This way we you know how services evolve over time. Our API Management product can also help you accomplish this. All of our virtualized services can be built to know where the live services sit.

 

Question 11: What are the requirements for an application to create a virtual service? Does an application ever need to be rearchitected in order to create a virtual service?
Answer:
The requirements are having two components that talk over a wire. Service virtualization has the ability to follow the conversation at a Packet Capture (PCAP) level. We can do straight network captures at an HTTP level or try to use whatever else you have available. The product is powerful in how it records and captures and should not require you to rearchitect an application in order to create a virtual service. We’re agnostic when it comes to how an application is coded. If there are two components that talk over a wire somewhere, we should be able to create a virtual service. We also have another technology we leverage called Opaque Data Processing (ODP). This can be used to capture a legacy protocol, like the one that is 15 years old, no one ever touches, it’s going over the wire but you don’t know what it is. With ODP, we can learn it well enough to give you a simulated response, one that will be good enough for you to test with.

 

Question 12: How do you map the speed of the virtualized system to compare it against the performance of the real system?
Answer:
One scenario is if you use CA Application Performance Manager (APM) you can capture it that way. If you’re pulling the data from another tool and then exporting it, we can set up the mappings based on the API call themselves. Bottom line, we do have a way for you to build out the mappings but may vary based on the tool you’re using. 

 

Question 13:  Can you create fluke error and edge conditions for performance testing with service virtualization?
Answer:
You can do this with service virtualization a few ways. For example, with service virtualization you have absolute control of the data scenarios. You can serve negative cases. This is especially useful when running performance tests. Using CA Continuous Application Insight, you can actually capture a bad transaction and then recreate it within service virtualization. By observing the chaos you can then recreate it more efficiently than tasking testers to recreate.

 

Question 14: Is service virtualization beneficial for an IT ecosystem where all the services are available in all the environments 24X7 and there are no third party services?
Answer:
Yes, there are still benefits of using service virtualization in highly available environments. Error conditions are one example. If you have all live systems, error conditions can present a challenge. Another area to look at would be race conditions. It’s very difficult to speed up or slow down systems in production so that you can see how the system under test would react. An additional advantage of CA Service Virtualization is related to maintenance costs. Virtual service environments are less costly than real environments. In fact, a recent study from Forrester showed them to be a lot less expensive. And then there is of course the issue of testing systems that don’t yet exist.

 

Question 15: Is CA Service Virtualization supported in any public/private cloud servers?
Answer:
Yes. CA Service Virtualization runs on the Microsoft Azure platform today. The Azure offering is designed for developers who want to work on a small project or just try out service virtualization. We have other cloud offerings in production. We also have some use cases with Docker and containers. We do see many customers moving to hybrid clouds, where they provide virtual services to users via a cloud architecture.

 

Question 16: How can I try CA Service Virtualization to test Microservices ?

Answer: The easiest way is to try CA Service Virtualization is by signing up for the Azure 30 day free trial. With the Azure instance everything is installed and ready to go.

 

Question 17: Can we virtualize RDBMS databases using service virtualization?

Answer: We can virtualize databases using JDBC virtualization.  However, that is an edge case.  CA also has a dedicated Test Data Management solution that is usually a better fit than databases. Particularly if you are interested in the performance of the database itself or testing data changes, CA Test Data Management will better solve those use cases.

 

 

Question 18: What are some of the future enhancements planned in CA Service Virtualization?

Answer: For the next major release, we are working on a full migration to the web based portal to provide a single unified experience for our users.  In addition, we are working on seamless integration with CA Test Data Management and CA API Management to provide customers with even better coverage, integrating data scenarios and security with the virtual services. We’ve also started work on the DevTest Catalog which enables organizations to share the virtual services more easily across cross-functional teams and throughout the SDLC.

We're gearing up over here in CA-land to get out a new release in the next few months. I wanted to share with you one key tidbit on what we're changing in this next release.

 

We're getting rid of the DevTest Server Console! Yea, this guy running on port 1505 is going bye-bye and we're replacing it with interfaces in the DevTest Portal!

 

Screen Shot 2016-04-28 at 12.07.13 PM.png

 

The new UI in DevTest portal will include:

  • Virtual Service Environment Monitor - This replaces the "VSE" in Server Console. It's actually available today in DevTest 8.5, 9.0 and 9.1 if you want to take a look and start using it. It's quite nice and much easier to find the virtual service you are looking for quickly.
  • Reporting - Another area that is already in the DevTest Portal, reporting was introduced in 9.0 and enhanced in 9.1. This is a much easier way to view your test results. More details on all the reports available today are here:
  • Continuous Validation Service (CVS) - This feature is another one that was introduced in 9.1 so you can check it out now.
  • Server Health Monitor - This was the main screen in Server Console where you could monitor your VSEs, Simulators and Coordinator servers through a graph. Now, in the new DevTest Portal, you'll have a quick chart to monitor and track server uptime, CPU usage, Memory Consumption and Server information for each of these assets.
  • Access Control Settings - This functionality will be moved from the DevTest Console over to the DevTest Portal in the next release. It includes setting Roles, Users, Resource Groups and Resources for the folks that will be using the DevTest tool.
  • Set Password - This is a new feature allowing the end-user to change their password within the DevTest Portal UI. This feature is only available for those who are NOT using the LDAP integration. For those using LDAP integration, password management is directly through LDAP.

 

So, what are your thoughts on this change? Check out some of these screenshots:

Screen Shot 2016-04-28 at 12.59.00 PM.png

Screen Shot 2016-04-28 at 1.09.00 PM.png

What You May Have Missed is a new blog series I'll be starting to help you catch up on all the content that comes into the DevTest Community. I know you're busy, and you can't catch everything that comes into the community, so this blog will help you catch up on anything you may have missed in the past 2 weeks!

 

Upcoming Events

Office Hours for DevTest: A Live Online Chat (May 2016) - May 24, 2016 at 9am ET

 

Webcast Replays

Webcast Replay: CA DevTest Webcast - What’s New in 9.1?

Webcast Recap: DevTest Architecture Best Practices

And check out this related blog post that covers Architecture Best Practices! -->  DevTest Architecture Best Practice - Overview

 

Product Announcements

End of Life Announcement for DevTest IBM Rational Quality Manager Integration

 

Noteworthy Content

How To: Insert BYOL License and Start Service Virtualization

Are you using the CA CAI extension for Google Chrome?

Tech Tips: DevTest 9.1.0 Silent Install not working for install that is connecting to an existing 9.1.0 DevTest Enterprise Dashboard

Share Your Expertise: CA World '16 Call for Speakers is Now Open

Congratulations to the 2016 CA Champions from the DevTest Community

 

New Tech Tips/KB Articles

Tutorial - How to Create a Custom Data Protocol Handler (Tech Tip)

Tutorial - How to Create a Custom Transport Protocol Handler (Tech Tip)

 

New Ideas

Jenkins-Plugin that uses Lisa Invoke API

Create Baseline Test From Existing Virtual Service

Push DevTest Docker Images to Docker Hub

I want to hold zoom setting of the DevTest console reporting page

DevTest Portal Integration with source control

Log file analysis in Portal

 

Recently Answered Questions

Proper way to define an MQ Cluster connection using new connection asset

How to Fetch Value from Response Header when using REST Step ( LISA 7.5.1)

Attempting to execute a script as part of an automation exercise - encountering RetrofitError: 500 Server Error

Service Virtualization 9.x Implementation Proven Professional Exam

Recording an API, HTTP protocol.

REST Service Virtualisation

Access Property Defined In VSM From VSI

Thank you to everyone who joined this webcast on April 20, 2016 which covered best practices for CA DevTest architecture.

 

Below are all the relevant webcast materials:

mel.pngHappy Thursday, everyone! I'd like to take this opportunity to introduce you to your new community manager for the DevTest Community -- Melanie_Giuliani , who will be taking the reins for all of our Continuous Delivery communities. You may have already noticed her picture on the left side of the overview page.

 

Mel has been with the team for more than two years and has been managing The Water Cooler community for most of that time. She is bright, energetic, knows the community platform inside and out and is beyond excited to take on one of our company's move vibrant communities.

 

Please join me in welcoming her. Expect to see her name a lot and mine a lot less as I focus on our Mainframe and CA World '18 communities. You're in great hands and she's going to be great for this community.

Swiss_Alps.jpgOnce a month we'll shine our virtual spotlight on one of the CA DevTest community's members. It's just a fun, easy way for us to get to know one another a little bit beyond our work-related interactions here on the site. This month, we're featuring, Joel Nesmith Services Architect, CA Technologies.

profile-image-display.png

 

Joel took this picture of the Swiss Alps while on an SV Engagement.

 

Location:  Cypress, TX (northwest of Houston)

Current Gig:  I am a Services Architect so I engage with new or existing DevTest Solutions customers every 8 – 12 weeks.

One word that describes how you work:  Diligent

How do you use CA Communities and any suggestions for others?  I’m unsure if there is a single person who understands all of the nuances of the CA DevTest product due to the breadth and depth of its capabilities.  The Communities are one of the first places I visit when I have a question about one of our products or a technique.  Everyone in the Community is helpful because they want you to be successful.  This is what I appreciate the most.  In terms of suggestions for others, I would say that one should never feel their question, idea, or contribution is unimportant or that it adds little value.  Get on here and go for it!  Someone will always try to help you find a solution.

Mac or PC?   I have a PC, but I have used a Mac on SV engagements.

Mobile Device? iPhone 5 – yeah I know, I need to upgrade.

Apps/software/tools you can’t live without? Waze & Google Maps / Candy Crush (I am on planes a lot) / Email

Besides your phone and computer, what's your favorite gadget? Not really a gadget, but my Harley Davidson

What’s your daily work space like? I spend the majority of my time at customer sites so work space varies.  I work from a large home office when not traveling.

How do you balance life/work?   I probably don’t do as good job of balancing as I should.  But, I try to complete my “honey do’s” as efficiently as possible and take trips to get away and relax.

Best advice you can give and you have received? The best advice I can give is that it is OK to make mistakes.  You just need to learn from them so they do not happen again.  My dad used to quote Yogi Berra.  So, I would say the best advice I have received is, ‘You can observe a lot by watching.’  Think about it for a while.

What everyday thing are you better at than anyone else?  Describing situations using Southern euphemisms and hyperbole.  Just ask our EMEA-based CA colleagues.  Those poor guys & gals put up with my expressions for about 8 months (or ‘a month of Sundays’) while we worked together on an international SV engagement.

image015.jpeg

 

 

DevTest

 

April 7, 2016

 

To:         CA DevTest Customers

From:     The CA Technologies DevTest Product Team

Subject: End of Life Announcement for DevTest IBM Rational Quality Manager Integration

 

CA Technologies is continually working to improve our software and services to best meet the needs of our customers. In order to focus our resources on the most up to date technology and continue to innovate on CA DevTest, effective May 7, 2016 we will be ending DevTest support for IBM Rational Quality Manager.

 

Your success is very important to us, and we look forward to continuing our successful partnership with you.

 

Connect with CA Technologies 

 

 

 

 

 

 

more

ca.comabout uscontact usnewseventscareerslegalprivacy |

CA, One CA Plaza, Islandia, NY 11749
Copyright © 2016 CA. All rights reserved. All trademarks, trade names, service marks and logos referenced herein belong to their respective companies.

 

 

CFS-header.jpg

Share your experience. Be the teacher. Enhance your resume.

 

CA World is the best opportunity to learn more about CA software and partner solutions that fuel digital transformations. And we want you to be a part of it.

 

The Call for Speakers is now open, and we look forward to receiving your amazing session ideas specifically stories about transformative projects, compelling use cases, and best practices that aim to educate and empower our customers in succeeding in the application economy.

 

We're looking for ideas covering CA Service Virtualization among other solution areas -- learn more and submit your topic today.

 

Make sure that you hear about all of the latest CA World '16 news by following the CA World '18 Community