Communications of the ACM
Estimation Is Not Evil
Credit: Alicia Kubista / Andrij Borys Associates
According to Ron Jeffries, estimation—as it is usually practiced—is "evil."a After making allowance for the hyperbole in the title (which does encourage people to read one's article) Jeffries makes the case that agile teams are often "excessively concerned" with estimating the work they need to do. More correctly, these are the "undistinguished" agile teams that may have achieved some improvement in performance due the adoption of agile, but have not fully achieved their potential, whatever that is. And estimating their work is getting in the way of this progress. But is this true?
The record of the business of software living up its promises, or promises made on its behalf, is generally considered to be poor. Software development has been accused of being too slow, too error-prone, and too costly. Though as Tom DeMarco observed in Why Does Software Cost So Much?2 we might reasonably ask: compared to what? If software were truly too expensive wouldn't market forces have replaced it with something else?
Comments
CACM Administrator
The following letter was published in the Letters to the Editor in the March 2014 CACM (http://cacm.acm.org/magazines/2014/3/172518).
--CACM Administrator
I agree with much of Phillip G. Armour's Viewpoint "Estimation Is Not Evil" (Jan. 2014), but there were several important aspects of real-world software development environments he did not mention and which I have encountered in my own 30 years in software development. For example, I agree with his complimentary characterization of the Waterfall method as not being overly rigid or unable to handle changing requirements and encounters with unknowns. I also suspect a good part of the criticism of the Waterfall method comes from people who never actually worked in software development in the method's heyday. I also agree with Armour's praise for the Agile method's focus on generating the most value in a constrained period of time. The Agile method is great for keeping customers involved in a project, though many (in my experience) neither want nor have the time to be too closely involved; they just want the software finished and to provide the functionality they had in mind.
I also agree with Armour's description of the calculated risk-covering reserve as being part of the overall commitment. In managing software development (mostly in small companies producing custom applications for outside customers), I have found the final price and schedule are often agreed to in the sales process prior to any real-world estimation by the developers. Alternatively, even when developers do have input, I have heard upper management say, "We acknowledge your concerns but will not budget for them," even though it is a rare software project that has nothing go wrong or does not take longer than expected. I have also heard, "We acknowledge your concerns but cannot sell the project at a price including budgeting for risks."
In the end, the software team takes the risk and is targeted if the non-risk-reserve-bearing commitment is not made. A software developer might think a workable solution would be to get the customer to buy into the risk-reserve idea, assuming, if the risk does not materialize, the customer gets the product at the lower price. In practice, however, this does not seem to occur very often. Customers know once they agree to a higher potential price, they will almost certainly pay that price. This scenario could occur for any number of reasons, but one I did not always appreciate was that once a customer agrees to the "potentially" higher amount, vendor management inevitably solidifies its own budgets based on achieving the full amount, sometimes exerting internal pressure on the development team to do so.
I understand Armour was primarily addressing the Agile method's relationship with estimation, but I was hoping he would have offered some advice that would give software developers a workable blueprint for successful software development. Even with Agile and estimations we do not seem to be there yet.
Richard H. Veith
Port Murray, NJ
------------------------------------------------
AUTHOR'S RESPONSE
Veith raises an interesting point about negotiating the cost of risk with customers. In a few instances, and only where the customer-vendor relationship is on a truly honest basis, I have seen the customer engage in a useful dialogue about who should bear the risk and how much risk is appropriate. Uncertainty and risk are intrinsic to software development, and a mature process would acknowledge and optimize it. We are getting better but are not there yet.
Phillip G. Armour
Deer Park, IL
Displaying 1 comment