A recent post by @task’s CTO Nate Bowler, Software Engineering Management, Bug Week, and Giving Away Minivans, generated some discussion regarding Deming’s aversion to incentives and how @task applies incentives to the software development process. I think everyone can agree with Nate when he suggests that "creating meaningful performance metrics for a software engineering team is difficult."
I don’t plan on going into all 14 of Deming’s Key Principles here, but there’s some pretty good background on Deming and his list of key principles on Wikipedia for those unfamiliar with them. Suffice it to say, he has been credited with a significant contribution to helping Japanese manufacturing improve the quality of their products and their reputation in the world.
Of Deming’s list of 14, I think the following apply to the current discussion:
#11a—"Eliminate work standards (quotas) on the factory floor. Substitute leadership." In the sense that arbitrary quotas and artificially created deadlines hurt morale and inhibit workforce productivity, I agree with Deming. Does that mean that organizations should abandon metrics altogether? I don’t think so. Athletes successfully use benchmarks and performance objectives to improve performance—the key is to keep the definitions of success consistent. Oh, there is no substitute for effective leadership.
#11b—"Eliminate management by objective. Eliminate management by numbers, numerical goals. Substitute leadership." In a world without stakeholders, executive boards, and shareholders, objectives might not matter—unfortunately, for most organizations, managing and executing to objectives is important. Once again, the key here is to manage those objectives that make sense and not create arbitrary objectives that do nothing but force people to run around in circles.
Fortunately, Deming’s philosophy offers a number of valuable manufacturing work management best practices that can be embraced by any organization engaged in project based work. However, because I am in no way a purist, I tend to embrace those ideas that are likely to contribute positively to the project management process, and ignore those that might not. After all, my reading of Deming leads me to believe that he was not directly addressing the project management process, but rather the manufacturing process—similar, but not the same.
Have I missed the point? Feel free to weigh in with your opinion.
I can agree with your point “I think everyone can agree with Nate when he suggests that “creating meaningful performance metrics for a software engineering team is difficult.”" and therefore I wonder why you want to measure performance of a team? Why not instead measure the performance of the system as a whole? About the point #10: “but an empowered workforce, with the objective of zero defects, is the way to fix a broken system.” I do not agree with you cause I feel that the workforce does not have a method to reach that goal. Empowering in this particular matter is illusion: Nobody can sign off a objective of zero defects as there number of existing (!= found) defects is unknowable and are results of the system as a whole. “A goal without a method is just nonsense.” I agree with you about empowering the people to remove any special causes detected by statistical methods – but that is totally different thing than management by objective. About points 11a and 11b: I guess you have not read Deming’s Out of the Crisis as it explains the points in greater detail and offers lots of examples. Deming is not against metrics (he just points out why they are often flawed thanks to common and special causes, variation, system properties and such) nor he think that you do need. Same goes for objectives, numerical goals etc. I must admit that I felt that these points were rather purist and I still am on the journey to figure out if ranking, rating and incentives are truly as bad as Deming points out (I am currently discussing this with a researcher of Finnish education system). In the end of the day I happen to be a stakeholder, shareholder, part of executive board so I feel this stuff is important
)
Marko, I appreciate your contribution to the discussion. I believe we share some common areas of agreement. Establishing meaningful metrics for software engineering is a challenge. As long as the metrics are not arbitrary, I believe they have value. The challenge may be to ensure that the metrics stakeholders require are really relevant to whether or not a project is successful.
A few years ago, I wrote about all 14 points and how I felt they applied to project management at the time. http://pmstudent.com/point-1-deming-in-project-management/ Perhaps my views on some of these have changed a bit, but for the most part I think they can be very applicable. -Josh http://learn.pmStudent.com
I remember those posts Josh. I agree, for the most part I think they are still applicable myself. Thanks for contributing to the blog.
I attended an interesting PM seminar a few weeks back that touched on some of these topics. The most interesting takeaway for me was this: measure one level ABOVE where you expect change is required. For example, if you think one of your store managers is weak (e.g. low sales), group him with the other four stores in the area and reward all four manager based on their COMBINED sales. This simplifies the measurement, creates camaraderie rather than competition, and fosters trust — call it empowerment? — by leaving the particulars of “how” to those in the best position to decide, organically. So! For on my @task implementations, we’re all about the “Big Picture” tab — a bar graph of the overall system, reporting at a level higher than any one person can affect on their own, but Out There for all to see and contribute to, to everyone’s success. Anyone else buy this theory?
Deming’s point number 10 is true in theory and in practice. What is an enpowered workforce if not one in which systematic issues are being attended to? To say that the theory is different is quite frankly a weasely way of saying that point #10 is wrong. Furthermore empowerment is a vague notion that adds nothing without exact statement of how a group manager should empower his group members. For example, if issues identified are further upstream or across groups, what then. My experience is managers with little imagination hiding behind processes and gimmicks and constantly changing their processes so the focus is no longer on their previously failed methods. That touches on 11b. Managers sadly substitute multiple ornate metric tools for any sort of courage to pursue a vision. Rather the metric results become the vision. The goal becomes the scores in the metric and everyone pursues the score at all cost.