Sign In

Communications of the ACM

The business of software

Practical Application of Theoretical Estimation


View as: Print Mobile App ACM Digital Library Full Text (PDF) In the Digital Edition Share: Send by email Share on reddit Share on StumbleUpon Share on Hacker News Share on Tweeter Share on Facebook
3-D geometric pattern

Credit: Tiemen Rapati

One of the most popular and successful approaches to estimating software projects is the Putnam model. Developed by Larry Putnam, Sr., in the 1970s as the Software Lifecycle Model (SLIM®)3, this model shares with other models a reciprocal exponent relationship of effort/cost to development time. That is, effort/cost is given by a formula of the type: Effort or Cost ~ Timen (see Figure 1). The value of n varies between models. In the SLIM model it is 4. This reciprocal fourth-power relationship predicts an enormous increase in cost and effort (and the associated project headcount) when a project's development duration is compressed substantially. It also shows that there are great savings to be gained if a project's delivery schedule can be relaxed.

I suggested a qualitative explanation for this in a previous column1the enormous increase in effort with very short schedules is due to the distribution of much of the work on the project into a wasteful activity I called "optional chaos." When projects have relaxed schedules, they don't have much of this valueless overhead so a much higher percentage of their work goes into actually creating value in the product.

Back to Top

Theory Goes on Forever

The theoretical curve could extend out to infinite time. In practice, the SLIM-Estimate tool (see http://www.qsm.com) that employs this mathematics has constraints built into it and will not allow infeasible solutions that are too short/high cost or too long/low cost. Furthermore, the application of any theory must always be subject to real-world considerations: if a short-cycle solution requires more staff than the company can get in that time it is de facto infeasible irrespective of any model or calculation. And if a long duration project exceeds its market window, it might be cheap, but no one will buy the result. In between these unattractive options, the time-effort curve very usefully demonstrates the trade-off between the cost of a project and how long it will take to complete the project.

Back to Top

Regression Models

Other estimation models that have been created using different conceptual approaches or from empirical data support this reciprocal exponent model. However, several of them, past a certain point in project duration, show either a flattening out (no further decrease in cost) or predict a modest increase in cost and effort as the project timeline extends further.a At the other, compressed, end of the timeline, increases in project headcount and effort past a certain point have been anecdotally thought to increase the project schedule. To paraphrase Fred Brooks2: adding people to a project can make it take longer (Brooks was talking about late projects, but we can extend to projects that will be late). It seems as if, beyond certain points at each end, the observed project behavior curls away from the conceptual curve, as illustrated in Figure 2. Along the middle of the curve (which is where most projects will be and should be committed), the theoretical and practical curves are congruent and projects tend to behave as predicted. And this is what QSM has found in over three decades of tracking such projects. At the extreme ends, projects may not behave as they should.

Back to Top

Outliers: Compressed and Relaxed

In any multivariable stochastic system, such as a collection of software projects, there are always outliers. Projects that we try to do very quickly or very slowly are by their nature, different from the norm. At the compressed end, it is likely the curve curls over simply due to the overwhelming preponderance of the "optional chaos" noise. In this case, all of the additional work on the project is being absorbed by the effort involved in managing the sheer numbers of peopleindeed additional work and time is needed just to manage the noise. It is analogous to a multitasking processor being overloaded where all the machine cycles are going into switching between tasks and none into the tasks themselves. Adding more people to try to further reduce the schedule simply pushes the schedule out further (as well as costing a lot more).

At the "relaxed" end of the timeline, the project cost rises the further out we try to position the product delivery. The reasons for this are usually in the marketplace and external to the project, though an extended timeframe might increase the likelihood of turnover of staff or change in technology, which can add to cost. As the delivery time moves further out, there is a higher probability that the underlying reasons for the project will change. When the project tries to play catchup to the changing requirements, they find they have to rework more of the product thereby increasing costs.

Back to Top

Three Curves

A typical approach of any theory, when confronted by non-standard behavior, is to try to subdivide the observation into discrete components that are more understandable and predictable. In this case there seems to be three things in operation (see Figure 3):

  • Normal: The "normal" behavior of a projectthis is well-predicted by the Putnam model as years of experience and thousands of projects have shown.
  • Compression: The "Compression Curve" (shown in blue in Figure 3) is probably a log function of time, possibly something like: Effort ~ log(Time)/Time. Given the probable underlying organizational causes of this behavior, it is likely to be related to the stress boundaries beyond which a project team cannot function "normally."
  • Relaxation: This curve (shown in orange in Figure 3) is generated by underlying changes in the market requirements and feasibility of the product being built as time goes by. It could be viewed as a mirror image of the Compression Curve, but it probably has some different behaviors. It is shown here as an exponential curve and might have a function something like: Effort ~ Time*exp(Time).

The overall time-effort behavior curve would then be the sum of these three curves.

Back to Top

Practicing the Theory

Systems development is an entirely human activity and a cooperative one at that. Therefore it tends to be somewhat erratic; people themselves not being entirely predictable. While the relaxation curve might be more predictable, some research has tried to measure it, and a few estimation models have attempted to characterize it, I suspect the behavior is quite situational. That is, when and how quickly the costs start rising depends very much on the specific product and the underlying market forces that determine the product's value. I am not sure how generalizable and predictable this behavior might be.

The compression curve is probably even more unpredictable. It is related to the stress experienced on projects as they get to their limits of capability; it is probably also related to the limits of cooperative learning, but that is a subject for another day. The more predictable aspects of this behavior are already captured in the Putnam Modelindeed this is why the effort and cost rise so precipitately as schedules get shorter. But at some point the system breaks down, the noise overwhelms the signal and the chaos overwhelms the work. One of the characteristics of systems that approach their operational limits is that their behavior becomes locally unpredictablewe know the system will break, but we may not be able to foretell where or when.

On the relaxation side, normal schedule constraints should take care of most of the challenges. Any reasonable feasibility proposal for a system development should incorporate a market window limit, beyond which we expect the value of the system will start degrading and the cost of rework will start going up. So we should not aim to commit to development that goes past this point anyway.

We probably shouldn't worry too much about this unpredictability on compression too. Massively over-staffing is really expensive, it requires a lot of people, it generates a lot of defects, and it doesn't even work to reduce schedule like we want it to. Simply knowing this means we should never attempt to reduce a development schedule past the point where the system breaks down. Trying to predict the project's behavior beyond that point is not very useful and we've had plenty of warning that it's going to be way too expensive long before we leave the "theoretical" part of the curve.

So to summarize, we should never allow a project to plan too long a delivery schedule, and we should definitely never try to push a project to completion too soon, since both extremes are ineffective and really expensive.

The problem is, that is what we should do. In theory.

Back to Top

References

1. Armour, P. G. Real work, necessary friction, optional chaos. Commun. ACM 47, 6 (June 2004), 1518.

2. Brooks, F. P., Jr. The Mythical Man Month. Addison-Wesley, NY, 1975, 25.

3. Putnam, L. H. Measures for Excellence. Prentice-Hall, NY, 1992.

Back to Top

Author

Phillip G. Armour (armour@corvusintl.com) is a senior consultant at Corvus International Inc., Deer Park, IL.

Back to Top

Footnotes

a. This was brought to my attention by Charles Symons in correspondence on estimation models. Specifically the COCOMO I and II models, and the SEER and Price-S models show a flattening or increase of project cost past a certain point in time.

DOI: http://doi.acm.org/10.1145/1953122.1953135

Back to Top

Figures

F1Figure 1. Relationship of effort/cost to development time.

F2Figure 2. Project behavior curling away from the conceptual curve.

F3Figure 3. Three curves showing three things in operation.

Back to top


Copyright held by author.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2011 ACM, Inc.


Comments


CACM Administrator

The following letter was published in the Letters to the Editor in the August 2011 CACM (http://cacm.acm.org/magazines/2011/8/114929).
--CACM Administrator

I commend Phillip G. Armour's Viewpoint "Practical Application of Theoretical Estimation" (June 2011), as I'm always on the lookout for ideas concerning software estimation, even as I ponder my own eternal mantra: "Estimates are always wrong."

I agree with Armour but think he missed an opportunity in his section labeled "Practicing the Theory" to emphasize how agile methods avoid the extremes of compression and relaxation. Relaxation is avoided by breaking traditionally slow-to-deliver projects into small agile pieces, each easily delivered within the related market window. Working with these pieces also serves to avoid compression, since the same number of people can deliver the smaller agile pieces more quickly.

Armour also did say this is all theoretical and that even under the guise of agility companies regularly try to ramp up too many agile pieces too quickly.

Geoffrey A. Lowney
Issaquah WA


Phillip Armour

Geoffrey, you are entirely correct. Estimates are always "wrong." In fact I have a bit of an allergic reaction to the phrase "accurate estimate" and I've pointed out in my column that it's a straightforward oxymoron. Estimates are neither accurate nor correct, but they can be *useful*

A "good" estimate (and there's an article that describes exactly what that is) informs the decision-making process *at the time of estimating* and provides a more optimal solution than would have been chosen without doing the estimate.

You're also right about Agile. The careful workflow management that goes with good Agile practices greatly improves the prediction and control of resources. But estimation in Agile is more about workflow management than it is about macro resource allocation. It's hard to scale up true Agile to very large complex coordinated systems. But then. of course, it's hard to build very large complex coordinated systems no matter what you use.

Just recently I was able to use the (very good) Story Points metrics obtained on quite a large project to calibrate an entire organization in a most useful way. It also showed that this company's regression away from Agile and back to a massive and monolithic paper-based process was costing the company a lot of money. That's a useful estimation activity.

Thanks for the feedback


Displaying all 2 comments