Skip navigation
All Places > DevTest Community > Blog > Authors Arif_Muhammad

DevTest Community

9 Posts authored by: Arif_Muhammad Employee
Arif_Muhammad

DevTest 8.0.2 is GA

Posted by Arif_Muhammad Employee Apr 13, 2015

We are pleased to announce that the DevTest 8.0.2 including CA Service Virtualization, CA Application Test and CA Continuous Application Insight 8.0.2 is generally available (GA) as of April 13th, 2015.  This release includes both fixes to reported issues as well as enhancements to all 3 core products.

 

Enhancements:

This service pack release contains fixes for various reported defects in addition to the following product enhancements:

  • CA Service Virtualization
    • One click clear for Service Virtualization recorder
    • Copy and paste specific transactions and signatures in Service Virtualization recorder
    • Ability to switch performance vs. functional VSE mode
    • Create Virtual Service Image from Swagger 2.0 specification
  • CA Application Test
    • Code-less test execution event handler
    • Enhanced JSON Ensure Result Equals assertion to allow ignore nodes from comparison
    • Support to load data files dynamically
    • Enhanced Test execution experience via DevTest portal by allowing the user to choose specific configuration and staging doc.
  • CA Application Test for Mobile
    • Support for Xcode 6.2 and iOS 8.2
    • Easier way to change device target for all steps in the test
    • Ability to populate text fields from Datasets in Voyger while crawling through the app.
  • CA Continuous Application Insight (fmr CA Path Finder)
    • Ability to export captured Request/Response pairs
    • Ability to choose assertions for REST stateful baseline tests
    • Added the consolidate-by-name option on the command line utility
    • Allow user to set transaction list filter by starting and ending bookends
    • Ability to create baseline test from Document Transaction window
    • Japanese and French localized
  • General Enhancements
    • Ability to create projects using DevTest Portal
    • Enhanced browsing through artifacts in the repository using "All Resources" page on DevTest Portal.
    • Ability to customize portlet names and user preferences on DevTest portal as well re-set to default.
    • DB2 Support for Enterprise Dashboard

 

Documentation:

The full release notes are available here: https://wiki.ca.com/display/DTS802/Release+Information

 


Download:

The release is posted and available for download on our support site. Instructions to download are here - Download DevTest Solutions Installers - DevTest Solutions - 8.0.2 - CA Wiki

 

Our first community call to go over these enhancements is scheduled for Tuesday, April 14th at 11am ET/ 8am PT. Subsequent calls will be posted to the community soon.

Arif_Muhammad

DevTest 8.0 & Licensing

Posted by Arif_Muhammad Employee Dec 19, 2014

To provide maximum flexibility as well as control to our customers, in 8.0 we made following main changes from licensing perspective

 

  • Removed all license enforcements: In previous releases, our DevTest solution components e.g. Workstations, Registries etc. were individually licensed and needed to talk to licensing server for license enforcement. We felt that this restriction was hindering product adoption and utilization, so we removed all those license enforcements. Starting 8.0 none of our components need to talk to the licensing server or need a license file except Enterprise Dashboard which is further discussed below.
  • Enforced license audit data collection: As we transition from license enforcement model to honor based compliance model, we wanted to give the administrators the ability to have insight into the usage of the tools and components so that it help monitor compliance. So we made Enterprise Dashboard a mandatory component along with the need for Enterprise Dashboard to be activated via a license key file. The idea here is that only authorized administrators (having the key file) should be able to install Enterprise Dashboard so they have the full knowledge and control of Enterprise Dashboard instances within the organization and run audit reports off of. When Enterprise Dashboard is in place any number of registries could be installed just by pointing to an active Enterprise Dashboard, by anyone within the organization without the need of license file. Since all DevTest 8.0 components will be connected to the Enterprise Dashboard directly or indirectly, they will be sending usage metrics over to the Enterprise Dashboard. Those usage metrics are used by the administrators to help stay compliant and catch any unintended out of compliance situations. Here is diagram that shows how DevTest Components interact to push usage data to Enterprise Dashboard

Screen Shot 2014-12-18 at 1.28.41 PM.png

  • Report on New licensing metrics along with metrics used in old licensing model via Enterprise Dashboard: We understand that it will take time for customer contracts to move to newer licensing metrics. So we have both old licensing metrics as well as newer one being collected and reported via the Enterprise dashboard. What that means is that you can upgrade to DevTest 8.0 while you are still on old licensing contract and use the same tools to stay compliant. This will also give you a good understanding of utilization by newer metrics while you are on old contract, so that when contract renewal or change time comes you can make your choices based on the real data.

 

  • Giving control to Administrators to stay in compliance:  Next question is how can administrators be in control to keep organization going out of compliance. This is the only area where we are helping only from newer licensing metrics perspective. Since newer licensing is all user based (instead of component or transaction count based) we tied the control via ACL. The Access Control List gives the administrators to allow/restrict users to access different DevTest capabilities. Within our DevTest Server Console, we added information to allow administrators easily translate user roles into effective license. Please see below screenshot of DevTest Server console where effective license user type is shown based on selected permission list for each role.

Screen Shot 2014-12-18 at 2.32.59 PM.png

When a user with that particular role, logs in using any of the DevTets interfaces including DevTest portal, Workstation, command line utilities e.g. TestRunner etc. based on the role s/he is assigned, the effective license type is used and marked as an active user of that type for the time period s/he was active. This information about user being active is sent over to Enterprise Dashboard which collects information about all the active users and generate data on overlapping user sessions across different license types. This data is used to report maximum number of concurrent users for each user type in a given period which can be compared against the contract to asses compliance. Administrator also can see more detailed report of user sessions that can help them optimize the license usage.

 

Note: Indirect Run-time users of DevTest Services are not currently included in audit reporting. These are the users who are mainly manual testers not directly logging into DevTest portal or Registries. Those indirect users are testing user interfaces going against the Virtualized services or back-end servers monitored by CA Application Insight Agents.  

 

Please feel free to provide your feedback on these changes and anything else we can do to make your life easier in terms of managing and administrating the solution licensing.

"On the 11th day of DevTest the CA team brought to me ELEVEN Selenium UI Test Scripts


TEN JSON Assertions and Filters,

NINE API tests running

EIGHT transactions documented

SEVEN well calibrated agents,

SIX agents, protocols, categories intercepted,

FIVE generated assets,

FOUR opaque data types processes,

THREE new execution modes,

TWO RR pairs added,

and a Server-Side Recorder in a DevTest tree!"


Welcome to our 11th day of DevTest 8.0!  Today we are looking at the new CA Application Test capability around Selenium  powered UI test case maintenance and management


In 8.0 we enhanced Selenium based Web UI test case creation and management by 3 aspects.

Making target URL and Form Data variable: During selenium script import process we now automatically generate variable for target URL to make it easier to run the same test against different targets. We also added the capability to define properties for the form data so that you can use the script against different input data. Here is a quick video to demonstrate this new capability.

 

 

Bring it back to Se Builder for editing: We added a Selenium plugin that will allow you to import the Selenium based CA Application Test cases back into Selenium Builder so that you can use the recorder capabilities of Se Builder to add/modify UI steps. Here is a quick video to demonstrate that capability.

 

 

 


Configure Selenium WebDriver using DesiredCapabilities: We also added the ability to configure the WebDriver based on the desired capability configuration you can provide. You can place Desired Capability conf file anywhere in your project and point to it using "selenium.WebDriver.DesiredCapabilities.filePath" property. Here is an example of the configuration

          Screen Shot 2014-12-10 at 3.26.46 PM.png

      Here is an example of the Desired Capability configuration file. We do provide a copy in our samples. This allows you to configure behavior of the WebDriver including

      setting timeout values etc.

       Screen Shot 2014-12-10 at 3.27.24 PM.png

 

--------------------------------------------------------------------------------------

TThe 12 Days of DevTest Blog SeriesT

TWELVE Mobile Tests Generated

ELEVEN Selenium UI Test Scripts

TEN JSON assertions and filters

NINE API tests running

EIGHT Transactions Documented

SEVEN well calibrated agents

SIX Agents, protocols, categories intercepted

FIVE Generated assets

FOUR Opaque data types processed

THREE new execution modes

TWO RR pairs added

and A Server-Side Recorder in a DevTest tree!"

--------------------------------------------------------------------------------------


"On the 10th day of DevTest the CA team brought to me 10 JSON Assertions and Filters,


NINE API tests running

EIGHT transactions documented

SEVEN well calibrated agents,

SIX agents, protocols, categories intercepted,

FIVE generated assets,

FOUR opaque data types processes,

THREE new execution modes,

TWO RR pairs added,

and a Server-Side Recorder in a DevTest tree!"


Welcome to our 10th day of DevTest 8.0!  Today we are looking at the new CA Application Test capability around JSON based REST API Testing


Prior to 8.0 if you need to parse JSON formatted response from REST APIs you have very limited choices. Either you will have to convert the JSON into XML and use XPath (does not always work) or you will have to parse the response yourself. In both cases some form of coding was necessary. In 8.0 we are making JSON formatted data first class citizen. Using new set of Assertions and Filters you will be able to directly work on JSON data using JSON Path.

 

Quick look at JSONPath:

 

Like XPath JSON Path allows expression based query and filtration of JSON data. In the expression top level or root of JSON structured data is referred by $ sign while nodes are separated by dot (.) notation. For illustration purpose I will use following modified form of data from tutorial.

{      "id": 2,

        "name": "An ice sculpture",

        "price": 12.50,

        "tags": ["cold", "ice"],

        "dimensions": {

            "length": 7.0,

            "width": 12.0 },

        "warehouseLocation": {

            "latitude": -78.75

        }

}

Using the dot-notation and $ as the root you could refer to length as $.dimensions.length. This expression will return 7.0. Array notation can be used to retrieve specific element of an array. In above example tags is an array and if I want to get 1st tag, I could use the expression $.tags[0] which will return "cold".  You can get more info on creating JSONPath expressions from http://goessner.net/articles/JsonPath/.

 

Using JSON Path based filters to query data:

 

    New JSON Path filter will allow you to add a filter as shown below.

Screen Shot 2014-10-07 at 6.12.55 PM.png

The JSON Path Filter needs 4 attributes to be filled out. Basically input data in the form of CA Application Test property, JSON Path expression and two output property names to hold the result and result count. Following is an example of getting the names of all the objects with length less than 7.

Screen Shot 2014-10-07 at 6.11.49 PM.png

After the filter execution is done the name(s) will be saved in property called allObj1 while count can be found in property totalObj1. Please note that the return value might be an array or a single value depending upon the query result.

 

JSON Path based Assertions:

    We added 3 main assertions to make it easier to validate the JSON based responses, as shown below.

Screen Shot 2014-10-07 at 6.12.13 PM.png

  1st thing you would want to validate is the structure if the response conforms to the expectation. For which we added JSON Schema based validation assertion. The assertion takes Schema and the payload as input and validate if the payload is conforming the schema. Here is an example.

Screen Shot 2014-10-07 at 6.10.39 PM.png

For design time assistance we are also proving additional details in case the data is not conforming to the schema. Here is an example of the design time assistance where schema requires property name2 be there but payload is sending name instead.

Screen Shot 2014-10-09 at 10.26.50 AM.png

For data value validation we allow both equal and contain assertions. Equal assertion matches the result of JSON Path for exact value, as shown below we want to make sure that object with id=3 has length 3.1.

 

Screen Shot 2014-10-07 at 6.10.59 PM.png

 

 

Contain assertion gives bit more flexibility for array based data. Following we are validating that an object with id=3 exists in the payload we are validating.

Screen Shot 2014-10-07 at 6.10.28 PM.png

 

We believe these filters and assertions should help tremendously in testing REST based APIs. Do you have JSON based APIs? Looking forward to getting your feedback.

 

--------------------------------------------------------------------------------------

TThe 12 Days of DevTest Blog SeriesT

TWELVE Mobile Tests Generated

ELEVEN Selenium UI Test Scripts

TEN JSON assertions and filters

NINE API tests running

EIGHT Transactions Documented

SEVEN well calibrated agents

SIX Agents, protocols, categories intercepted

FIVE Generated assets

FOUR Opaque data types processed

THREE new execution modes

TWO RR pairs added

and A Server-Side Recorder in a DevTest tree!"

--------------------------------------------------------------------------------------


"On the 9th day of DevTest the CA team brought to me 9 API Tests Running,


EIGHT transactions documented

SEVEN well calibrated agents,

SIX agents, protocols, categories intercepted,

FIVE generated assets,

FOUR opaque data types processes,

THREE new execution modes,

TWO RR pairs added,

and a Server-Side Recorder in a DevTest tree!"



Welcome to our 9th day of DevTest 8.0!  Today we are looking at the new CA Application Test collaboration capabilities that allows team members to execute, monitor and analyze test results.

 

One of the new and shiny toy you are going to see in 8.0 is our  DevTest Portal. The goal of the portal is to makes it easier for your team members to collaborate. Many of you asked ways to share standard automated tests that can be initiated with click of a button without the need of running workstation. Further the team members should have an easy to analyze view of the test results so they can see what exactly was exercised by the test and if it failed what was the reason of the failure. I believe we have made a good progress on this front. In this blog I will walk you through some of the capabilities.

 

For illustration I will be using following test case "Validate Car Inventory Look up" which uses combination of REST and Selenium steps to validate inventory API (using new cool json based filter) as well as verify that the CARS Web application, presents right data to the user as a result of VIN Look up.  Please note the naming of the steps, assertions etc.

Screen Shot 2014-10-10 at 5.09.39 PM.pngAfter creating the test I published it into repository called "ResHub" (Short of resource hub). This is basically a file system based repository of standard DevTest content you would want to share among the team members.  After logging into the new portal the user is taken to a dashboard as shown below.

 

Screen Shot 2014-10-10 at 4.53.11 PM.png

        Screen Shot 2014-10-10 at 4.54.38 PM.png

 

 

 

From left hand panel user can navigate into different DevTest portal capabilities, he has access to. To explore what tests are available the user will click on Manage menu item where he is presented with available projects in the ResHub as shown above. As you see Demo80 has 2 automated tests and one test suite which can be executed using the right click "Run" context menu. Once the test is initiated user is taken into monitor page where the status of all the tests in different state is shown.

 

Screen Shot 2014-10-10 at 4.55.22 PM.png

 

As you see the summary section shows the count of tests in particular state and the list provides the high level detail of those tests. You can drill into a particular test and see what exactly the test did. By clicking CarsInventoy test I started before, I see following test execution detail page.

 

Screen Shot 2014-10-10 at 5.58.36 PM.png

 

Most exciting thing about this view is that, you not only see the steps executed but also see filters, assertions, data-sets and companion in their execution sequence. Which makes it very easy for even someone who is new to the Application Test see what the test did. For advanced user who wants to analyze further expanding the activity will provide lot more detail. For example below you can see not only the request and response of the REST call but events and properties for further analysis of the test result.

 

Screen Shot 2014-10-10 at 5.59.03 PM.png

 

Following is one of the selenium step which verifies the VIN number in VIN lookup response page.

 

Screen Shot 2014-10-10 at 6.00.10 PM.png

 

The goal here was to make it easier to execute and review functional test results. Do you think this will help your teams to be more effective in achieving better collaboration?

 

--------------------------------------------------------------------------------------

TThe 12 Days of DevTest Blog SeriesT

TWELVE Mobile Tests Generated

ELEVEN Selenium UI Test Scripts

TEN JSON assertions and filters

NINE API tests running

EIGHT Transactions Documented

SEVEN well calibrated agents

SIX Agents, protocols, categories intercepted

FIVE Generated assets

FOUR Opaque data types processed

THREE new execution modes

TWO RR pairs added

and A Server-Side Recorder in a DevTest tree!"

--------------------------------------------------------------------------------------


Many agile teams have started incorporating Behavior Driven Development into their practices. BDD has a systematic framework to stream-line the stories into Scenarios and forces teams to discuss and put in writing any of the acceptance criteria. Here is an example of the scenario and criteria

 

       Scenario: Money deposit activity for a given account

Given an account A with balance $100

When 50$ are deposited

New balance goes to $150

And When $50 is withdrawn

New balance goes back to $100

 

 

The formal acceptance criterion combined with Test Driven Development is helping bring emphasis on the importance of unit testing (remember the testing pyramid I discussed in my blog) as a key quality strategy. But there are many cases where unit testing at very focused code area alone is not enough specially where you want to test a full business workflow kind of scenario that spans across multiple application components and interfaces. That’s where CA Application Test Invoke and Verify framework become handy.

 

The question is can I take the Scenario and make it CA Application Test in such away that the test execution and results are easy to read and relate to the scenario test is validating. Yes you can, the trick is documenting your test case properly. Here is an example implementation of above scenario

test.jpg

Please note the names used for step as well as the assertions to improve the readability of the test and make it as close to the actual Scenario as possible.

 

Now if I run the test the trick is the results of this test should be presented in a fashion that you wont need to be CA Application Test expert to read the test results i.e. more something like following

result.jpg

 

That’s an interesting challenge. As part of CA Application Test 8.0 release our development team has been spending some time thinking about this. You could join us at CA world where we can show you some of that work.

 

Although above was a very simplistic example but you can easily extrapolate and apply it to implement a complete business workflow by using CA Application Test's Invoke/Verify framework across heterogeneous technologies. I will be interested in hearing your thoughts. Have you experimented BDD in your team? What are your experiences?

In my previous blog I discussed how API based federated design coupled with Service Visualization can help achieve First day test automation. In this blog I will go little further on automated testing of APIs and enabling end to end business process validation.

 

As you are witnessing today’s enterprise applications are increasingly being built using internal and external APIs. In many cases behind those APIs there can be several interconnected applications, services and databases. Although the end user is just interacting with the presentation layer, a simple end user click can generates a chain reaction of calls to different services encompassing web services, EJBs, legacy apps and databases. While there are many tools out there which will allow the tester to call each of the API individually, the end to end validation of the business process will require orchestration not only across API calls but across different technologies.

 

Business Process Testing is not only testing end to end processes and the complex systems that make up a business process — it requires collaboration between the system designers and implementers. Business analysts who understand the flow of information have the ability to augment and tie Test Cases together gaining end to end testing of a process. Basically Business Process Testing redefines the need for a strong collaborative testing platform.

 

CA Application Test provides a rich feature set to allow collaboration of testing assets and data between developers and analysts along with a powerful framework to Invoke and Verify requests across different services and technologies (as shown in the figure).

imageedit_1_6468712214.gif

These Invoke and Verify steps can be further stitched together to mimic full business process using a visual workflow style test designer. As the requests are being placed verification of those requests can be done using the powerful assertion framework. Since  CA Application Test support's verity of protocols, applications, technologies and interfaces it basically becomes your de-facto client to all underlying components, taking you out of the business of coding, creating and testing throwaway clients. While you are building a test case in CA Application Test, you can simulate a browser dialogue and verify calls against a web service or database, then jump right into staging that test, then jump back and modify your test case while your test is being staged without recompiling or modifying a single script.

When you are creating an automated end to end test you are also trying to validating how the system under test (SUT) will behave with different data. For successful test automation maintaining right set of data used to invoke and verify the API/interface calls is very important. CA Application Test also provides a framework to capture, manage and pass the data to automated tests. Powered by highly scalable architecture CA Application Test's test staging and invocation framework allows on demand as well as scheduled execution of test cases for regression testing as well performance and load testing.

 

Are your teams utilizing CA Application to its full potential?  Please share your experiences on validating end-to-end process using CA Application Test!

Do you have a need to validate your web based UI across multiple browsers? With LISA 7.5.1 we have

added the capability to drive execution of Selenium-based Web UI test scripts. One of the coolest feature of

Selenium WebDriver is the ability to run the same test script across multiple browsers. I created this short video

to demonstrate execution of Selenium-based tests not only across different browsers, but to show how to

leverage cloud-based testing platforms like Sauce Labs from within CA Application Test framework for your

Web UI testing needs.

 

I remember the good old days, when development teams used to have the luxury of planning 9-16 month releases. We would take our time to plan the release from soup to nuts. Generally developers would take a good amount of time to build the software while testing was considered to be throwing over the wall event. In majority of the cases, testing phase (stabilization) used to take much longer than anticipated. Fast forward 10 years, and now we can no longer afford that luxury.  With the changing realities in this application economy, we need to not only be faster but also be much higher in quality-- slipping dates is not as much of an option, and our customers are not forgiving on quality.

 

Agile methodology has promise, but being able to deliver working software at the end of each iteration is easier said then done. QA efforts still can be a bottleneck: how can you test something which does not exist yet? Traditionally, QA had been focused on UI testing to verify software functionality. However, in the beginning of the project, UI either is non-existent or very fluid making it almost impossible to do any sort of test automation a reality and it goes on the back burner. For agile teams, the “Aha!” moment came when in his book, “Succeeding with Agile”, Mike Cohn introduced the testing pyramid concept.

p3.gif

Cohn argued that for test automation, we have been targeting wrong place. We will have a better luck in automating if we focus on Unit Testing and services layers to validate the application functionality.


Let’s see if this testing services layer approach for automation can also help us achieve our First Day Test Automation goal. I will illustrate it through an over simplified project example below.

 

Requirement: We need to build this awesome “Calculator” which can Add, Subtract, Multiply and Divide 2 numbers. I would want my end users to interact with the calculator via a web app (which supports IE, FF and Safari), as well as through native Android and iOS Apps. Also partners are interested in consuming our “Calculator as a Service” using an API. I also want my end users to be able to see which functions of the calculator are more popular.

 

Design: My fellow architect takes this requirement and comes up with the architecture below.

p4.gif

At the release planning meeting, we started drilling down on the stories and high-level architecture. The architect explains he is using the DB to store the API usage statistics. We ask the team to estimate the work. The API developer says it’s going to take him approximately 20 points to get to the state where he has the basic service up, and he may potentially finish Add and Subtract in sprint 1. The UI and Mobile developers are sitting there thinking, “I can start basic application stuff…but finalized interfaces can only be achieved after the services layer is in place.”

My QA reviews the architecture and acknowledges that with CA Application Test he can invoke and verify each layer individually as well as combinations of multiple layers (WebUI, Mobile, APIs or JDBC). “But wait a minute. I also need these components available before I can start building my automated test cases!” At that point, a light bulb goes off in QA engineer’s head and he suggests that let’s quickly agree upon the API interfaces and create a virtualized calculator service using CA Service Virtualization which can be used by me but also by UI/Mobile developers. 

That did it! The architect creates  the virtualized service and hands it over to the engineering teams. By using this virtualized service, we not only achieved parallel development of both the services and presentation layers, but also enabled QA to start creating automated test cases right from sprint 1.

Do you strive for First Day Test Automation in your organization? What are the tips and tricks you use to achieve this goal?