It seems to be a law of software development that things always take longer than we expect. So we expect them to take longer and make appropriate allowances and they still seem to take longer. Even if we carefully account for unrealistic expectations as Tom DeMarco so testily observed in Why Does Software Cost So Much?2 we are regularly and unpleasantly surprised when we overrun still more. And often the person doing the work is astonished that: it takes longer than it was thought, planned, and committed to; even in the middle of the work, while acknowledging it has taken longer than expected to this point, it will still take longer to finish than will be predicted right now; and we continue to repeat this behavior.
This scares project managers. Due to the rather opaque work done in the business of software it is difficult for a project manager to know what progress has been made without asking those doing the work how they are doing. The code is written, but is it the right code? Is there enough of it? The test was run, but was it the right test? Was the result what we expected? If it was not what are the consequences in terms of progress? If we unearthed a defect does that put us further ahead (one less defect to deal with) or further behind (yet another thing gone wrong)?
Zeno’s Paradox of Progress
When a project manager talks to a designer, programmer, or tester and tries to get a sense of how “complete” the assigned task is, the normal reply is “about 90%.” Asking the same person the same question the following week often elicits the same reply. Sometimes for weeks, with equal amounts of rising embarrassment and certainty that this time the estimate is good, the programmer or tester will affirm that he or she is really, truly, 90% complete. If the embarrassment gets too high, the engineer might employ the “Zeno’s Paradox”a approach of halving the distance to the goal each time. So 90% becomes 95%, then 97.5% then 98.75%. We may laugh at this, but perhaps it contains a germ of truth.
A programmer describing progress as “90% complete” is not usually intentionally lying, but clearly the implied 10% remaining time to complete is not correct. So why do people say this and why do project managers continue to operate on this assumption? There are several reasons.
Second Order Ignorance (2OI)
Systems development is primarily a knowledge discovery activity. Even in coding, the thing that takes the most time is figuring out what we don’t know. If systems development were a matter of transferring what we already knew into code, we would build systems as fast as we can type. And for some aspects of development, such as testing, discovering what we don’t know represents almost all the effort.
We may be quite aware of some of the things we don’t know—this is lack of knowledge or First Order Ignorance (1OI).1 But sometimes there are things about the system we don’t know we don’t know, which means we have Second Order Ignorance (2OI). Building what is already known is easy, resolving clearly defined questions is a bit more challenging, but discovering what we are not aware of is by far the most difficult and time consuming task. So asking the question “…how complete are you?” or its corollary “…how much more work is there left to do?” is the equivalent of asking “…how much do you not yet know?” There is no easy way for someone to answer that question. Most people extrapolate from their past experience and what they have learned on the task so far to figure how much they might have left to learn. But they don’t actually know. Then a certain amount of optimism, a modicum of professional pride, and perhaps a dash of hubris and the calculation yields the ubiquitous “90%.” I have observed that years of experience have a way of tempering one’s optimism and experienced engineers do tend to be more realistic about what they don’t know yet. But there is another reason why we overestimate progress.
Linear Equation. Project managers get confused because they associate percentage complete with percentage of resources used. They assume a linear relationship between these two variables (see Figure 1). So when the quantity produced = 90%, the resources used must be = 90%. But in development this is not so.
Rayleigh Curve. Research has determined that the rate at which we build things in systems development can be described by a Rayleigh Curve3,4 (see Figure 2). This curve shows the amount of work that can be done varies with time. It rises from zero to a peak, perhaps a third of the way into the activity, and then drops off back to zero. The cumulative version of this curve is shown in Figure 3.
Non-Linear. If we compare Figure 3 with Figure 1 we notice the cumulative curve relationship of percentage of product complete and percentage of resource used (time in this case) is not a straight line. Some percentage on one axis does not mean the same percentage on the other. In fact, in Figure 3, when the product is 90% complete, the activity is only about halfway through its total time. This is partly where the miscalculation of progress comes from. If developers or project managers, consciously or unconsciously, apply a mental model that is linear to something that behaves non-linearly it is not surprising that they incorrectly assess progress.
Three Stages
This model also makes intuitive sense. In Figure 3, the curve has three distinct stages and they apply as much to painting a room as they might to building software.
Preparation. The environment must be prepared before work can commence. For software this means setting up servers and version control systems, acquiring tools, planning and estimating, sourcing staff and other essential activities that take time but might not actually create much in the way of software product. When painting a room, we have to choose the color scheme, measure, tape up the woodwork, and buy the paint, all before we can start putting it on the wall.
Production is the steepest slope where the maximum rate of measurable work occurs—the most code is written, the most paint is applied.
Proving is the final long tail of the process. In software it is completing the exception code (the mainline code was done in production), testing it to prove it works and fixing it when it does not. This always seems to take longer than it should partly because we invariably find out things we were not expecting—which is where the Second Order Ignorance comes in. In painting, this is the detail work, the tricky corners and, of course, the cleanup—which also always seems to take longer than it should.
Mathematical Solution
There are many reasons why people overreport progress: optimism, pride, pressure to show results, inexperience, poor resource allocation, even how they are compensated, and they all play a part in overstating development and underestimating the remaining work.
But realizing that we cannot calculate exactly how far we have left to go because we do not know exactly where we are going and simply using the correct mathematical model of progress would certainly help.
Join the Discussion (0)
Become a Member or Sign In to Post a Comment