You can also take a look at the post following the below link -
What does quality project management mean? What does it look like? Can all project management be measured using the same metrics? What might those metrics be?
These are the kinds of questions that ramble around in my mind when trying to wrap my head around understanding the real issues related to measuring the quality of the way a project is managed. Here are some thoughts on the subject--if nothing else, a checklist that might help you with the designing of meaningful metrics for measuring the quality of your organization’s project management process.
As in most explorations, it seems logical to begin with baseline definitions; in this case, the definition of quality and the definition or project management. This might seem a bit remedial to some but, it is important to establish definitional foundation to work from in order to establish a rational framework for the discussion.
What is Quality?
According to the Merriam-Webster, quality is defined as “how good or bad something is” and “a high level of value or excellence”. Perhaps this is where the difficulty arises, as terms like “good”, “bad”, “value” and “excellence” are subjective. Thus, quality must have a context; something to be contrasted against.
For example, would an investment yielding 6% be a quality investment? The answer is obviously “maybe” or “it depends”. A 6% yielding investment might represent an excellent return if comparable (risk level, term, etc.) investments were yielding 3%. Inversely, that same investment might represent a poor value if comparable investments yielded 12%. It is the same 6%, but viewed through two different alternatives.
Is it good or bad to make a product that will last 10 years? If the products will be obsolete in three years and the organization releases new products on that cycle, the cost and the corresponding price related to creating a product that will function for 10 years might be bad--and be uncompetitive as well. The same would be true if the product replacement cycle was 20 years and the product began to fail after 10 years. So defining “good” definitely has an impact as to how an organization defines quality. This is why things like workmanship and precision are not sufficient to define the way an organization or its customers define quality. It is only through the lens of expectation and context to an alternative that the concept of quality can truly take shape.
To be an effective measure, quality needs to have a quantitative context--something that the process or the outcome can be measured against. This context needs to be established in advance so that all striving to achieve goodness, high value or excellence can monitor how well they are doing. Often, these measures are dynamic and change, making the achievement of high quality a moving target; thus the old TQM (Total Quality Management) adage: “Quality is a race without a finish line.” Perhaps this is why the TQM movement failed so badly. After all, who wants to play a game you can’t win?
What is Project Management?
PMI defines Project Management as follows:
“The application of knowledge, skills and techniques to execute projects effectively and efficiently. It’s a strategic competency for organizations, enabling them to tie project results to business goals--and thus, better compete in their markets.”
Once again, subjective terms find their way into the definition of project management--namely “effectively” and efficiently”. Like quality, these terms need context to be meaningful. How do we know that something is efficient or effective? We only can quantify them in terms of an established baseline or alternatives (as compared to what?).
Based on PMI's definition, project management is a process and not an end result. Perhaps this is why efficiently and effectively managing a project does not necessarily guarantee a successful outcome (e.g., the operation was a success but the patient died.). Thus, the second part of the definition that binds the process to the outcome of having projects achieving business goals--specifically those that allow businesses to better compete in their markets. While this additional part of the definition seems very narrow and confining, I believe it is PMI’s way of tying the process to the outcome achieved. Thus, in addition to how projects are…
…project management would have to include how projects are chosen, prioritized and evaluated in terms of overall process and results achieved. These additional elements extend project management beyond the boundaries of any one project to encompass the entire project realm of an organization.
What is Quality Project Management?
Now that a set of baseline definitions have been established, the logical next step would be to develop a framework for measuring the level of quality a project management process has or will yield. For the sake of this article, this framework will take the form of a template. The template is presented in a matrix that breaks projects into the following process categories (with simple and abbreviated definitions):
- Viability and Alignment Assessment – the process of determining how well the proposed project supports the goals and objectives of the organization and its overall feasibility and potential for success.
- Prioritization – the process of determining when the project should be resourced funded and started in context to the overall portfolio of project assets.
- Planning – the process of developing the actual project plan including workplans, communication plans, project team, etc. needed to monitor, track and oversee project progress and continued viability.
- Initialization – the process of starting the project including obtaining formal authorization to commence, funding, activating the workplan into the project management tracking system, kickoff, etc.
- Conduct – the process of performing all the activities and tasks needed to complete the project in accordance with the project plan and subsequent changes to scope.
- Closure, Turn-over & Deployment – the process of completing the project and moving the finished effort into production/operation/use.
- Process Evaluation – the process of reviewing the previous processes to determine how well they were accomplished in terms of efficiency, effectiveness, compliance to protocols and governance guidelines, project management standards, and alignment with the organization’s culture and values, etc.
- Results Monitoring – the process of measuring the outcome of the project once deployed to determine if the goals and objectives of the project (the value it was intended to deliver) were actually achieved.
Each of these process categories is then viewed across the following three areas:
- Effectiveness – how well the process achieved its intended result
- Efficiency – how timely and cost effectively the process was performed
- Compliance – how well the process conformed to established protocols, guidelines and standards
This provides for at least 24 data points upon which the quality of the project management process can be evaluated. While not exhaustive, this checklist provides a good place to start; a place to build from. For each data point, you can identify the attributes appropriate for assessing the quality of the project management process used to complete the associated project process. Each of these attributes should be quantified to make compilation of the data possible. Quantification could take the form of a scoring against target ranges or a binary/pass fail grade, etc. For example, part of the efficiency score for the process of conducting the project might be as follows (again, very simplified):
How well did the project achieve completion by the planned completion date?
- Completed early – 10 points
- Completed on time – 5 points
- Completed late – 2 points
- Failed to complete – 0 points
Here is a version of the template I designed in Excel. Your version will most likely vary as you tailor it to your specific needs.
Begin filling out the matrix by creating the criteria you want to use for evaluating each process’s quality for each of the three areas. You might find it easier to frame these criteria in the form of a question followed by potential answers; each answer having an associated score. Next, weight each question in terms of how important each criterion is in context to the other criteria. For instance, what is more important to assessing the quality of the project management process: budget compliance or on-time compliance?
These types of tradeoffs reflect how the organization values what matters most. For example, is the organization risk adverse or does it encourage risk? If the organization is risk adverse, then a process that was risk active would be bad, not good. The opposite could equally apply. Remember, part of defining quality is developing a definition of “good”. By weighting each of the criteria, you can develop an overall quality score for how well a given project was managed.
One last thing to consider when developing your matrix is to identify how and when the data that feeds the answers will be gathered. Will the data be gathered during the course of the process, or at the end? Will there be feedback that flows to the project manager as to how well they are doing in terms of achieving high quality, or will this only be known after the evaluation process is completed (after project turnover)? My bias is to provide feedback as much as possible throughout the project so that course corrections can be rapidly implemented where needed.
How are does your organization measure the quality of the project management process? What has worked, what hasn’t and what lessons have you learned over the years? Your ideas and observations are valued at ProjectManagement.com.
- Five Project Management Performance Metrics key to Successful Project Execution – Operational Excellence by Suchitra Mishra
- Quality Metrics Template
- Manage Quality and Metrics
- Project Metrics by Dr. R. Ocker
- Metrics in Project Management by Crystal Lee, PMP
- Measuring and Managing Project Quality - If you can’t measure it, you can’t manage it by Stacy Goff
- What is Project Management? by PMI