Skip navigation
All Places > DevTest Community > Blog > Authors Joel NeSmith

DevTest Community

6 Posts authored by: Joel NeSmith

Have you ever noticed that it’s the high-performing teams that ask the best questions of themselves? They challenge themselves, think ahead and consistently work to validate team direction and priorities by asking the right questions—frequently. 


For service virtualization (SV), perhaps the most important questions teams can ask are, “What does our organization want our SV solution to look like?” and “What changes should we pursue at key milestones along our SV journey?” Teams that don’t ask these questions early and frequently in their journey and that don’t consider various scenarios will lag behind teams and their organizations that pursue SV progress by openly debating these questions.


Roles and Responsibilities

In a previous post, I pointed to the need to establish a maturity model. Within this model lies the development of roles, templates, processes and procedures that drive the scale organizations desire. Organizations that recognize the importance of establishing rules for delivering SV capability adopt SV more quickly with better alignment to desired business outcomes. These organizations define strategies and establish processes and procedures that address questions such as:


  • Who owns a given service?
  • How is the service federated and supported?
  • What criteria determine whether a service is managed through the service catalog?
  • What is the governance and support model that best suits the way we deliver SV projects?
  • How are skills, infrastructure, knowledge, maintenance and support managed and by whom?



Most organizations create some form of SV federation, but few examine the extent to which they should federate versus manage centrally. Fewer still create a single factory concept that delivers all aspects of SV to other departments. Establishing a federated model is important because it combines strength and scale with the flexibility of autonomy by creating natural divisions of work that maximize organizational investment. Using the list of functions below, think about your organizations and whether federating these functions would provide new benefits:



One approach to defining the most appropriate federation model is to align the functions listed above into a RACI model that maps to the existing organization. For example, skill development and self-learning might be the responsibility of an education department, whereas SV project and program management activities align with project management office functions. The RACI model drives the conversation about role identification and results in development of a potential execution model depicting the best way to support the SV-related activities.


Communities of Practice

More organizations are beginning to understand the value of implementing an online community of practice (aka learning network, domain or practice area) for sharing ideas and improving performance. People need a forum where members can ask questions, exchange ideas, learn from others, and mine SV intelligence to solve issues specific to the organization’s use of SV. In a community that’s limited to your organization, you and your colleagues can share and reuse proprietary information, governance processes and procedures, and sample SV patterns. The community of practice is an integral element of federation and self-learning.


As always, questions and comments are welcome.


Learn more about how CA Services helps customers with their Service Virtualization implementations.


Prior series posts:  Maximizing Your DevTest Investment on Your Way to SV MaturityMaking the Most of Your Service Virtualization Assets; Change Management: A Key Element of Your SV Strategy; SV Requires Transformation at Scale to Drive Maturity; On the Path to Service Virtualization Maturity: Measure Outcomes, Not Output

I mentioned earlier in this blog series that not many organizations are able to avoid “the adoption chasm.” Even after initial service virtualization (SV) success, organizational progress on the SV journey can stall or regress as organizations undertake efforts to create scale and operationalize. As successes occur, organizations inevitably add governance and process-oriented tasks. High-performing teams are often asked to document more thoroughly. Status reports and deadlines receive more scrutiny. Teams are expected to follow a new set of good practices, and post-its listing tasks proliferate on Kanban boards. The list of expectations and commitments grows, which translates to longer task lists for team members. Leaders tend to generate reports and measure project success based on completion of outputs.

Please understand: I am not discounting the importance of identifying, managing and reporting on SV project commitments. I am proposing that we avoid quantifying SV value in these terms.

Keep Your Teams Focused on Outcomes

Distinguishing between outputs and outcomes may seem like semantics to some and be nebulous to others. A colleague of mine uses this analogy to distinguish the two:

If you wanted to improve your fitness (an outcome), you might purchase fitness gear, join a health club and schedule workouts (all outputs), but these outputs alone would not improve your fitness. On the other hand, if you engage a personal trainer, he or she would ask clarifying questions to define “fitness.” Do you want to improve strength and stamina? Do you want to get closer to your target body mass index? Are you interested in improving heart health and/or reducing cholesterol levels? Answers to these questions help the trainer design a regimen that helps you achieve your desired outcome—improving your fitness. Similarly, the trainer measures your achievements based on your desired outcome, not the number of stations you visit in each workout.

We must understand and communicate the business outcome(s) that SV helps us achieve. Organizations that successfully identify and measure SV’s production against business outcomes will have greater success than those reporting on outputs. We increase the level of stakeholder buy-in when we communicate how SV is helping the business achieve its desired outcomes. The graphic further illustrates outputs versus outcomes:

Look to Metrics for Help

Measuring outcomes can be difficult because it requires connecting SV with business-relevant data. IT teams should rely on their business counterparts to identify simple, trackable metrics that gauge value. However, the metrics must be more impactful than simply the number of virtual services delivered or the number of times a virtual service is used.

Key concepts to think about when defining metrics are:

  • Do not substantiate the value of SV only once. Do it for every SV project.
  • Express metrics as they relate to outcomes. You may have to answer the question “Why are we doing this?” many times to identify the outcome the business is seeking.
  • Use metrics to incentivize and/or create healthy competition among development teams.
  • As part of your decision-making process, measure the potential value of a virtualization against the cost of building the virtualization; SV may be prohibitively expensive in some circumstances.

Develop SV Project Selection Criteria

Selecting the wrong projects can affect team morale and diminish SV ROI. Organizations can avoid many problems by establishing clear SV project selection criteria and adjusting them as needed. Consider these criteria:

  • A well-articulated business problem or challenge
  • Stakeholder sponsorship and commitment
  • The existence of before-and-after metrics for developing value statements
  • A good understanding of complexity and data dependencies
  • Availability of skills and project team willingess to support SV

SV adoption and transformation are personally significant for people involved in these projects. The above tactics can help teams focus on delivering long-term success.

My next blog will peek into federation and communities of practice. Your comments and questions are always welcome.


Maximizing Your DevTest Investment

Making the Most of Your Service Virtualization Assets

SV Requires Transformation at Scale

Change Management: A Key Element in Your SV Strategy

To make genuine, transformative progress through Service Virtualization (SV), organizations should adopt a 360-degree view that connects four critical elements: the technical SV solution, people, process and measurable business outcomes. By explicitly focusing on these elements and how they intersect, teams can develop effective tactics to improve SV maturity, increase scale and derive greater value from their SV investment. Additionally, when teams map the value of SV to business outcomes, they connect SV to business priorities, help validate and demonstrate SV contributions and establish measures for ongoing improvement.

Start with People—Your Most Important Asset

Organizational and individual resistance to change can impede the best plans and greatest opportunities for SV adoption. Organizations can beat this inertia by helping people answer the question: “Why change?” To be convincing, the answers must connect to business outcomes and their value. Some tactics to employ include:

  • Showing how SV aligns to executive priorities
  • Educating line-of-business and IT leaders on ways to communicate SV’s importance and relevance in achieving business outcomes
  • Developing an accountable core team of knowledge experts and deploying them on campaigns that support the use of SV on high-impact projects
  • Identifying processes and methods that impede change within the organization—and eliminating them
  • Allocating resource time to integrate SV into application lifecycle methods and processes
  • Communicating business outcomes facilitated by SV via lunch-and-learn sessions, internal demos and communications.

Process, Process, Process to Drive Scale

The absence of process-related artifacts, templates, and metrics that report value can inhibit organizational success. By creating reusable patterns and templates, SV can be operationalized at scale, thus delivering the greatest value and exposing new opportunities for SV adoption. Formalizing the following activities helps in developing scale:

  • SV discovery processes
  • SV project backlogs and pipelines
  • SV support processes
  • SV skills, learning and self-enablement processes
  • SV asset management and support processes
  • SV utilization metrics, socialization and consumer advocacy processes

Be Driven by Outcomes, Not Outputs

Organizations often find themselves trapped in the paradigm of socializing the value of SV by expressing value in terms of output, not outcomes. Whenever possible, don’t use an output metric to describe an outcome. Consider the impact of project value statements for the same project communicated by the PM in two different ways:

  • Example A: “The team had a virtual service with 900+ million hits and other virtual services with 500+ million hits.”
  • Example B: “We used SV to build better applications faster by ensuring that customer-facing applications could handle expected transaction volumes without experiencing outages and by providing a platform that enables daily performance tests rather than monthly or quarterly tests. Our approach resulted in reducing volume-related outages by 80%, completing performance testing 97% sooner in the application lifecycle and reducing infrastructure costs by $100K per quarter.


In example A, the numbers are measures of output; they don’t quantify or qualify how SV contributed to achieving a business outcome. We can only assume that the results had an impact on the business.


In example B, I can hear executives asking, “How did you do that? Can we replicate these outcomes?” It’s easy to feel the pride and excitement of everyone who contributed to these outcomes and to wonder if the focus on business outcomes motivated them to raise the bar for SV scale and value even higher!


I welcome your comments and questions below.


Links to previous posts:

Maximizing Your DevTest Investment on Your Way to SV Maturity

Making the Most of Your Service Virtualization Assets

Change Management: A Key Element of Your SV Strategy

To sustain progress and achieve substantial business value, organizations should aggressively integrate Service 

Virtualization (SV) into their software development lifecycle (SDLC). In fact, this may be your most important move in driving meaningful change that leads to increasing IT value.


Although this type of change is critical, it can also be overwhelming for many organizations, given the pressures on IT regarding time, budgets, competing priorities, staff availability, etc. Fortunately, metrics demonstrate the value of SV, and these metrics can be convincing when presented to business leaders.


Given the availability of metrics, why do many organizations miss the opportunity to leverage SV to drive change - specifically to transform processes?


Perhaps the reason is that change requires change.


Change is perpetual in IT, which may explain why the industry isn’t short on change management methodologies and practices. It seems that most standards bodies assume that process improvement is a technical challenge rather than a social and cultural challenge. Another perspective is that automation will compel needed change.


A change management function must focus on developing principles, practices and processes that enable organizations to improve software development outcomes by paying attention to the cultural and social aspects of change. Some key questions addressed in change management are:


  • How are data-rich SV metrics used to obtain buy-in at all levels of the organization?
  • How do organizational process changes related to SV mitigate IT risk?
  • Which SV-related change initiatives bring about the greatest organizational value?
  • What process improvement metrics aid in evaluating the success of change?


SV activities and maturity strategies such as business process alignment, staffing and organizational alignment, and initiative identification and measurement intersect with change management. Organizations benefit by identifying a skilled practitioner who understands these intersection points and defines and facilitates the necessary cultural and social changes. Change management activities that focus on the following questions can help:


  • How does change management drive executive and stakeholder understanding of value?
  • How does SV alignment with executive initiatives increase business value and outcomes?
  • What assessment approaches best measure the impact of behavioral changes?


Focusing on these areas helps teams develop a change management ethos that can be socialized across stakeholders at all levels and connects directly to desired business outcomes for SV. One of the best places to start is by identifying a talented SV practitioner or team who can step beyond the technology to link process and cultural/social changes to benefits the organization will experience due to change. In doing so, the SV conversation and its value is elevated to the enterprise. When organizations elevate the conversation, the discussion shifts from a focus on technical outputs to SV’s impact on desired business outcomes.


Stay tuned. The next blog examines people and process changes that help drive the value of SV. In the meantime, your comments and questions are always welcome. 


Previous posts:

Maximizing Your DevTest Investment

Making the Most of Your Service Virtualization Assets

When IT organizations don’t integrate SV into their culture and IT decision-making processes, they under-utilize the potential of service virtualization (SV). When SV is integrated in a holistic manner, SV maturity increases and the organization begins to develop the muscle and flexibility necessary to deliver the requisite scale, speed, velocity, and value that business leaders expect.


In his book, Digitally Remastered: Building Software into Your Business DNA, Otto Berkes identifies IT’s optimization of continuous development and delivery capabilities as important ingredients for building a Modern Software Factory. He notes that robust Agile and DevOps capabilities are essential to driving speed of innovation and responsiveness to deliver real customer value. In the Modern Software Factory era, IT teams—with support from business stakeholders—must:

  • Foster environments where continuous improvement, led or influenced by IT, is always top of mind
  • Create and operate software, and focus on its business value as a core capability—or better—a competitive differentiator
  • Operate in the mindset of developing muscle and flexibility through software
  • Figuratively put software at the center of the business.

The Importance of Defining an SV Maturity Model

Organizations that develop a robust SV maturity model or roadmap and integrate SV into their software development lifecycle (SDLC) are generally more successful. These organizations target low-effort/high-value virtualizations, demonstrate business value using SV, create standardized processes and methods for delivering SV, integrate SV into the SDLC, and find ways to federate SV delivery to move faster and generate greater business value. On the other hand, organizations that under-invest in continuous improvement of people and SV processes miss the mark and undermine the value-generating ability of their SV assets.


Airplane pilots carefully plan flights by incorporating frequent checkpoints to gauge progress and enable course correction. Similarly, organizations should to plan the milestones and activities necessary to continuously improve SV maturity, monitor progress, and correct course as needed. CA’s approach to adoption and maturity establishes a model with customizable sequenced activities that help organizations create scale. This crawl®walk®run approach separates activities into a logical focus areas.


Early-stage activities in the adoption model drive business value, increase knowledge of SV, and create reference implementations of SV artifacts. Middle-stage activities increase utilization and streamline SV delivery; many of these activities focus on people and process. Final-stage activities optimize and deliver SV at scale in the application lifecycle ecosystem. This graphic illustrates some of the activities and focus areas in the maturity model:



SV Integration into the SDLC is Critical

When presented with the model above, customers sometimes ask which activities are most important for developing a maturity model. While they’re all important, failing to integrate SV with SDLC processes and people will impede maturity. Integrating SV into the SDLC enables organizations to identify the points in the lifecycle processes where SV generates maximum value. This enables the constituent teams (business and product owners, architects, developers, managers, quality assurance, etc.) to understand where SV is expected to provide value. Without this understanding, teams are less effective in driving business outcomes.


A related issue is that many organizations have the initial SV conversation in the lifecycle’s development phase—too late to allow time for SV to generate positive impact. SV should be discussed when service needs are initially identified. At the latest, SV should be discussed during the analysis phase, when the service contract is being created or reviewed by an architect.


With the advent of Agile, test-driven, and behavior-driven development methodologies, organizations are striving to shift left. As they do, alignment between SV and the SDLC becomes even more critical. SV touch points must shift as developers and quality assurance personnel work side by side to iteratively deliver features.


While creating a roadmap to maturity and aligning SV with the SDLC are critical, they aren’t the only elements an organization must address to build SV muscle and flexibility. In a future blog, we’ll focus on change management as a key element of SV strategy.


I welcome your comments and questions below.


Related posts:

Maximize Your DevTest Investment

In his highly-regarded book, “Crossing the Chasm”, Geoffrey Moore presents a model for broad commercial adoption of new technologies. Moore’s macro-view of the technology adoption lifecycle and the chasm created by new or disruptive technologies has been a guide to many in the IT industry.  Moore’s crossing-the-chasm concept can be loosely applied at a micro-level to customers hoping to drive adoption of technologies such as CA DevTest©. While few (or no?) customers avoid Moore’s adoption chasm altogether, many use fast-track techniques that reduce time, cost, effort and frustration. This series of blogs will share insights and best practices from customers and CA Services – focusing on CA Service Virtualization (SV).


Consider the graphic:

Generally, CA Service Virtualization projects start well and make genuine progress in a short timeframe. Successes in this initial phase cause expectations to track upward. However, when CA Services disengages, some customer teams begin to slide down a slippery slope and find themselves in “the adoption chasm.” Some of the challenges that manifest are:

  • Early successes have caused adopters to prematurely declare victory or shift critical attention away from further developing adoption strategies.
  • Unjustified hope that superior technology results in strong adoption
  • Setbacks after the CA Services engagement ends cause projects to be de-resourced or de-prioritized.
  • Organizations postpone SV training, integration and business process changes
  • Good practices are not engrained
  • Effort and budget expended on implementation and early deployment leaves insufficient resources for high value integrations, solution architecture improvements, business process transformation
  • Metrics describing business outcomes are not developed and reported
  • Operations teams are tasked with both running an ‘adequate’ implementation and discovering, recommending and applying improvements to the deployment


These challenges, and more, result in higher long term costs and greater near-term frustration within the organization.  When one or more of these challenges slows the pace of adoption, a gap, as seen in the following graphics, develops between expectations and delivered value.

Organizations that successfully address these challenges do so by implementing tactics, good practices, and strategies that shorten the trip through the chasm (or avoid the chasm altogether) and produce measurable business outcomes.   


I have observed that through our own transformation and hundreds of successful SV engagements, CA has accrued and developed insights and strategies that help organizations build robust and mature SV capabilities. So in this series of blogs, we’ll share some of these maturity strategies. The blogs will highlight key activities for developing SV maturity and building the scale necessary to consistently deliver the right business outcomes. The series will include topics such as:

  • Making the Most of Your Service Virtualization Assets
  • Change Management: A Key Element of Your Service Virtualization Strategy
  • What Type of Adoption Models Support SV Expansion
  • SV Requires Transformation to Drive Maturity
  • Common Pitfalls on the Path to SV Maturity
  • What Do Federation and Communities of Practice Look Like


Stay tuned for more very soon.  In the meantime, I welcome your comments and questions below.