A reader over at PM Hut asks, "How [can I] calculate delivery compliance in an engineering/ Software(IT) project?"
From a Project Management viewpoint, I had some difficulty in understanding the term "delivery compliance", so I've taken the liberty of thinking of this as scope verification -- how we determine that what was delivered meets the project requirements. There is then a second part to this inquiry -- how is it measured?
This can be a complex question or issue. We all know from the 70s and the work of Gerald Weinberg (The Psychology of Computer Programming) that testing software does not prove the absence of bugs. Where scope statements require high reliability then, the measurement becomes uncertain. Complex and large projects also create challenges for verification and measurement.
I believe I can put some simple structure around this complex question. During the early years of my software and software project management career, I worked with engineering/software development teams at GE Information Services that were challenged to:
- develop systems that worked with high reliability - 99.999%
- meet the demand that memory and cpu utilization vary no more than 1% from release to release (since clients were billed on this basis)
- provide high client satisfaction -- an "A" average was required on client report cards
This was "RASM" (Reliability, Availability, Scalability, Maintainability) before the acronym was coined! Here's how this was achieved:
Verification
Verification required a robust process to maintain the chain of accountability for the requirements. This process included:
- written SMART requirements (specific, measurable, achievable, realistic, time bound)
- diligent requirements reviews (indirectly more of Weinberg's work)
- diligent design reviews
- coding standards
- peer reviews of all code
- management sign off on all changes
- developer created unit test cases which contributed to future regression tests
- test cases which went through a similar process to the software development process.
Testing was also a critical part of the process. In my opinion, testing these days has become too reliant on use cases. Use cases make fine exemplars or clarifications of requirements, but are poor test cases -- users rarely do what's expected of them 100% of the time. Testing needs to be a good mix of black box, white box, and glass box testing to ensure you are capturing as many issues as possible. The requirements need to become your checklist. Further, in addition to looking at the checklist an item at a time, you need to step back and make sure they meet the big picture (here's a nice role for use cases, as well as additional testing to verify the interaction of component).
Critical components of testing in the environment included:
- a strong, multi- and cross-platform configuration management system to manage builds and their versions
- ability to easily integrate developer unit tests into other test beds through defined test frameworks
- automated testing to drive high test coverage in shorter periods of time
- a measurable regression test bed (ensure no unexpected changes in CPU/memory or functionality)
- load testing (how many simultaneous users could use the system with no database or other system deadlocks)
- stress testing (what happened when the expected limits of any system aspect were exceeded).
Metrics
Now that we've had a look at the intended delivery outcomes and the process to achieve them, let's take a look at the key metrics which were in place and how they were used to assure delivery:
1) Regular reporting of number of open tickets and issues by priority, software system, client, and age
2) Regular reporting on system and application availability. Goal was 99.999% (scheduled down time was considered meeting goal, unscheduled was not)
3) Regular client survey results, requiring a specific average (usually something like A- or better). Clients were proactively called on a regular basis by client services to gather the information.
To further reinforce the compliance of delivery, these three metrics were generally tied to financial compensation and overall employee performance reviews. Everyone in the organization carried a responsibility to help meet the metrics. (Note: See my earlier article Time, Deliverables, or Outcomes? for more on the perspective of the triangular relationship of quality, availability, and high client satisfaction.)
It is also interesting to note that we found client satisfaction directly linked to number of open tickets. It wasn't enough to just close the high priority issues. Initially there was an attempt to ignore lower level issues. As the number of these issues grew, client satisfaction dropped. Once out of compliance, we initiated a period in which only low level tickets were addressed. Client satisfaction increased within two quarters and we then adopted a more balanced approach than just closing high priority, high impact tickets.
After these key metrics, there were other regular reports available including:
- measurement of time on projects
- measurement of time on issues and other non-project work
- measurements of system and application usage on display monitors throughout all facilities, including metrics such as up time, availability time, number of simultaneous users, transaction rates, etc.
Conclusion
Delivery compliance is a complex topic. Its not about a simple metric or measurement. I've attempted to put together a picture of a framework for a development process which drives an organization toward effective, high quality delivery. The process was supported by three key, simple metrics which kept everyone aligned to the highest priorities and values of the company. Additional metrics helped build team work and rally employees around milestones (e.g. the day we reached 10,000 simultaneous users on our private telecommunications network and mainframe processors).
Showing posts with label scope management. Show all posts
Showing posts with label scope management. Show all posts
Friday, February 19, 2010
Saturday, September 26, 2009
If Only We Had Known
By: Susan Peterson, MBA PMP
Copyright 2008, Susan Peterson, All Rights Reserved
There are at least three separate groups of analysts who are assessing the current global economic situation.
• One group is convinced that they “saw this coming a long time ago.”
• The second group admits to being “blindsided.”
• The third group includes those who may have seen something dire coming but didn’t think that there would be a monumental catastrophe.
Project managers know that they must always be in the first group in order to make adequate contingency plans, to mitigate problems, or to avoid catastrophes completely. This month’s column is the first in a series that will focus on indicators that can alert project managers to take preventive actions or to make contingency plans in order to avert disasters.
“Who needs goals?”
Some people think that project goals are actually defined by the final deliverable or target milestone. If the project purpose is to install a new software package, some project stakeholders will perceive that the project goal is successful software implementation. However, the goal of the project is its target accomplishment. In this example as in many others, the project goals need to be determined before the final deliverable (solution) is defined. If the organization actually wants to improve its processes (a potential project goal), there may be many final deliverables other than software implementation that would be better solutions. Specifying a solution in advance of or in place of project goal definition and agreement often leads to projects that are declared failures. Project managers must emphasize the necessity of determining goals in order to provide focus and direction for the project. Specifying the solution without identifying the problem to be addressed is akin to providing an answer without first asking a question. The process involved in goal setting also serves to involve stakeholders from the beginning and to resolve any areas of significant disagreement early in the project activities.
“We don’t know where we’re going, but we have to get there quickly.”
Planning is another classic project activity that can be perceived as a waste of time. “We used to plan, but no one followed the plan” or “How can we plan what we don’t know?” are common reasons given to justify the avoidance of project plan development. While there may have been past bad experiences with planning, no project proceeds smoothly unless there is an effective plan that is followed. Involving all stakeholders in the planning process can be effective in overcoming resistance to planning. This involvement builds an appreciation among stakeholders of the types of activities and related human resources that will be necessary to accomplish the project goals. It can also aid understanding across diverse stakeholders of the responsibilities of the entire team.
“It’s not wise to take only what you’re given.”
Even in the best of times project managers may find themselves in intense competition with each other for money, human resources, and capital assets that are often in scarce supply in organizations. In the interest of the enterprise perspective rather than a single project, it’s not expedient to declare a “winner take all” contest to identify the one project that receives the most and the best of everything. However, project managers need to work with each other throughout their projects to determine what is needed, when it is needed, and what degree of flexibility is appropriate.
Using this practice can result in a more coordinated allocation across current and future projects. It promotes cooperation that results in more effective execution of all projects and reduces or eliminates the time-consuming competition that can occur among projects. The practice can also uncover that the organization is attempting to accomplish too many concurrent projects and that it needs to defer or re-plan one or more efforts.
Project managers know that they have to remain alert and responsive to indicators of potential problems.
Susan Peterson, M.B.A., PMP, is a consultant who manages diverse programs and projects in both the private and public sectors for individual organizations and consortia. She also conducts enterprise assessments of project portfolio management practices. An overview of her program and project specialties is available at http://www.pmi-sd.org; select “Resources” then “Consultants”. She teaches the Project Management Simulation capstone course in the University of California, San Diego, Project Management certificate program and is a member of the curriculum committee. She can be contacted at susanada@aol.com.
Copyright 2008, Susan Peterson, All Rights Reserved
There are at least three separate groups of analysts who are assessing the current global economic situation.
• One group is convinced that they “saw this coming a long time ago.”
• The second group admits to being “blindsided.”
• The third group includes those who may have seen something dire coming but didn’t think that there would be a monumental catastrophe.
Project managers know that they must always be in the first group in order to make adequate contingency plans, to mitigate problems, or to avoid catastrophes completely. This month’s column is the first in a series that will focus on indicators that can alert project managers to take preventive actions or to make contingency plans in order to avert disasters.
“Who needs goals?”
Some people think that project goals are actually defined by the final deliverable or target milestone. If the project purpose is to install a new software package, some project stakeholders will perceive that the project goal is successful software implementation. However, the goal of the project is its target accomplishment. In this example as in many others, the project goals need to be determined before the final deliverable (solution) is defined. If the organization actually wants to improve its processes (a potential project goal), there may be many final deliverables other than software implementation that would be better solutions. Specifying a solution in advance of or in place of project goal definition and agreement often leads to projects that are declared failures. Project managers must emphasize the necessity of determining goals in order to provide focus and direction for the project. Specifying the solution without identifying the problem to be addressed is akin to providing an answer without first asking a question. The process involved in goal setting also serves to involve stakeholders from the beginning and to resolve any areas of significant disagreement early in the project activities.
“We don’t know where we’re going, but we have to get there quickly.”
Planning is another classic project activity that can be perceived as a waste of time. “We used to plan, but no one followed the plan” or “How can we plan what we don’t know?” are common reasons given to justify the avoidance of project plan development. While there may have been past bad experiences with planning, no project proceeds smoothly unless there is an effective plan that is followed. Involving all stakeholders in the planning process can be effective in overcoming resistance to planning. This involvement builds an appreciation among stakeholders of the types of activities and related human resources that will be necessary to accomplish the project goals. It can also aid understanding across diverse stakeholders of the responsibilities of the entire team.
“It’s not wise to take only what you’re given.”
Even in the best of times project managers may find themselves in intense competition with each other for money, human resources, and capital assets that are often in scarce supply in organizations. In the interest of the enterprise perspective rather than a single project, it’s not expedient to declare a “winner take all” contest to identify the one project that receives the most and the best of everything. However, project managers need to work with each other throughout their projects to determine what is needed, when it is needed, and what degree of flexibility is appropriate.
Using this practice can result in a more coordinated allocation across current and future projects. It promotes cooperation that results in more effective execution of all projects and reduces or eliminates the time-consuming competition that can occur among projects. The practice can also uncover that the organization is attempting to accomplish too many concurrent projects and that it needs to defer or re-plan one or more efforts.
Project managers know that they have to remain alert and responsive to indicators of potential problems.
Susan Peterson, M.B.A., PMP, is a consultant who manages diverse programs and projects in both the private and public sectors for individual organizations and consortia. She also conducts enterprise assessments of project portfolio management practices. An overview of her program and project specialties is available at http://www.pmi-sd.org; select “Resources” then “Consultants”. She teaches the Project Management Simulation capstone course in the University of California, San Diego, Project Management certificate program and is a member of the curriculum committee. She can be contacted at susanada@aol.com.
Monday, October 15, 2007
PMP Challenge Question - 10/15
Just a reminder this is that last of three PMP Challenge Questions for eligibility for the PMP Exam Prep book including PMP(r) Exam Success Series from Crosswind Project Management. This includes their "Brain Dump" review card and CD Exam Simulator.
For a lucky runner up, I have a Prepare for the PMP(r) Exam: Formulas and Important Facts to Remember card from Value Based Project Management.
As a reminder, to be eligible for these prizes, you must: a) correctly answer 2 of the last 3 PMP challenge questions (that would include 9/15, 10/1, and 10/15) b) email your responses to sdcapmp@aol.com before 12:01am 10/30/2007 c) have your name drawn from the eligible emails received. Winners will be notified by email and will need to supply a mailing address for prizes.
So here is today's question (actually a multi-part question):
1) List 7 benefits of a WBS. Be original. We all know it lists tasks, resources, and estimates. What are the real benefits of having a WBS?
2) Explain how you know when you have broken down the tasks of a WBS far enough.
Good luck! All the answers will be posted on 11/1 with the next PMP Challenge Question!
For a lucky runner up, I have a Prepare for the PMP(r) Exam: Formulas and Important Facts to Remember card from Value Based Project Management.
As a reminder, to be eligible for these prizes, you must: a) correctly answer 2 of the last 3 PMP challenge questions (that would include 9/15, 10/1, and 10/15) b) email your responses to sdcapmp@aol.com before 12:01am 10/30/2007 c) have your name drawn from the eligible emails received. Winners will be notified by email and will need to supply a mailing address for prizes.
So here is today's question (actually a multi-part question):
1) List 7 benefits of a WBS. Be original. We all know it lists tasks, resources, and estimates. What are the real benefits of having a WBS?
2) Explain how you know when you have broken down the tasks of a WBS far enough.
Good luck! All the answers will be posted on 11/1 with the next PMP Challenge Question!
Subscribe to:
Posts (Atom)