June 28, 2006
One of the central themes in McConnell's Software Estimation: Demystifying the Black Art is the ominously named Cone of Uncertainty. The cone defines statistically predictible levels of project estimate uncertainty at each stage of the project.
The cone has several ramifications, the most important of which is that early project estimates will always be wildly inaccurate:
As you can see from the graph, estimates created very early in the project are subject to a high degree of error. Estimates created at initial concept time can be inaccurate by a factor of 4x on the high side, or 4x on the low side.
That means the total estimate range is a staggering 16x at the time of initial concept! And believe it or not, that's a best case scenario:
An important-- and difficult-- concept is that The Cone of Uncertainty represents the best-case accuracy that is possible to have in software estimates at different points in a project. It is easily possible to do worse. It isn't possible to be more accurate; it's only possible to be more lucky.
Furthermore, the cone doesn't narrow itself. If you don't actively work to reduce the variability of your project*, the cone of uncertainty quickly becomes a cloud of uncertainty. When will the software be done? Who knows. That's one reason for the long, dismal history of software project failure.
You may also be familiar with this software proverb:
The first ninety percent of the task takes ninety percent of the time, and the last ten percent takes the other ninety percent.
Getting trapped in the Cone of Uncertainty is a classic software project mistake. You think you're 99 percent done for months. It's a cruel failure of perspective that's endemic to the profession of software development.
It reminds me of the popular Mystery Spot tourist attraction in nearby Santa Cruz.
I've been there. The Mystery Spot is a tacky tourist attraction, sure, but it's fun. And even if you're a complete skeptic and card-carrying Mensa member, the Ames Room illusion is incredibly convincing when you, and everyone around you, is standing in it.
Are you really 99 percent done? Or is your entire project stuck in the early, distorted part of the Cone of Uncertainty where you only look 99 percent done? Sometimes it's awfully difficult to tell the difference.
* This is something the book goes into great detail on, naturally.
Posted by Jeff Atwood
When I was a freshman in Engineering at UCSB, our professor for Engineering 1 told us about the e to pi rule. The idea is that whenever the engineer says how long it will take or how much it will cost, multiply that by something between e and pi to get a more accurate estimate.
I love the reference to the "cloud uncertainty" because i was thinking about the same thing...
I've been stuck at 99% of my current project for the last two weeks.
But today, it will be finished, right?
having been involved in various contracting (real, physical building things type) companies, mixed in with this software stuff, what remains true is that real engineers keep track of their experience/history. Such that, at time t=1, they have a set of defined tasks they didn't have at time t=0, for which they have realized resource requirements. Thus, over time, they accumulate a set of tasks/resource requirements. From this set, call them widgets, they can figure out, with increasing accuracy, what a project (assembled from these known widgets) will cost.
I have not yet, in 30 years, seen a software organization that did this. The universal reason given: "we never do the same thing twice, so collecting performance data is a waste of time". Thus, all estimates are pulled kicking and screaming from the sphincter of MBA project managers who think Excel macros are programs.
call my cynical.
Personally, I am 99% done with all of the project tasks that were due yesterday. Of course, there are a handful of small outstanding issues left to deal with for each of those tasks, and while I was at it I found a ton of _extra_ tasks nobody ever thought of (do you think I should have my PM add those to the list?)... but hey, I'm on schedule man! ;-)
I have been 100% done with my current project...a dozen times over the last 3 years. The customer is stil waiting for that next "must have" feature before "rolling it out to the entire organization". The biggest problem is, of course, for every feature they see, they think of 3 other things they want. "Cross this line you die. Ok. Cross this line you die. Ok...."
Think about it this way: if we didn't firmly believe that we're 99% done at any given day, we wouldn't have the courage to go on for the next five years until the project is done!
Though I'm a developer, I can see it, occasionally from the manager's standpoint. Do you really want your developers saying: "Since this project isn't exactly like any other we've done, we don't know how long it will take or what resources we will need. Just pay us forever. Can't promise we'll ever ship a product."? There has to some kind of a committment from the developers, ideally based on some experience. Just because human estimates of the future aren't precise doesn't mean we shouldn't plan and commit. Hollywood produces movies and, years later, the "director's cut" comes out (which, of course, is "done right").
Hey, I once wrote an After Effects plugin that uses that illusion to create 3D objects from a 2D photo or piece of movie footage. You could set up 2D layers in 3D space and project from another 2D layer onto that one. It was fun to set up a bunch of randomly placed 2D layers in front of some random photo, then move the camera around and look at the distortions from different angles. And it turns out there are real artists who paint stuff like this on sidewalks. Check this out:
Regarding what barry said, you know damn well that the reason developers don't want to give an estimate is because an "estimate" suddenly turns into a deadline because managers often treat them as a promise rather than the guess that they are. While it has occassionally bitten me, I luckily have never lost a job over such a situation, but I know others who have. If that happened to me, then you bet I'd be hesitant to ever give an estimate based on past experience again.
If there's one useful contribution that the Agile folks have made (I think there are actually loads, but stick to this one for now) it's the Planning Game in XP, which derives from Scrum.
I've worked on projects with year-plus timescales where we knew that our carefully-concocted "estimates" were just saying "it'll take a long time and cost a lot of money".
I've been trying to reduce scope and increase iteration count ever since. Even with my profound optimism it's hard to get too much wrong when you're only having to think a week or so ahead.
Buggy Fun Bunny (and everyone): Steve McConnell's book is basically about how to pick "widgets" for estimating software projects. Also, he discusses both agile and traditional approaches.
It's a really good book!
I just finished a project phase that spent 6 weeks at 99% done. The relief is indescribable. How about the unimaginably vast swamp of uncertainty?