I'm currently reading Steve McConnell's new book, Software Estimation: Demystifying the Black Art. The section on individual expert judgment provided one simple reason why my estimates are often so horribly wrong:
If you ask a developer to estimate a set of features, the developer will often come back with an estimate that looks like this:
Feature Estimated Days Alpha 1.5 Bravo 1.5 Charlie 2.0 Delta 0.5 Echo 0.5 Foxtrot 0.25 Golf 2.0 Hotel 1.0 India 0.75 Juliet 1.25 Total 11.25 If you then ask the same developer to reestimate each feature's best case and worst case, the developer will often return with estimates similar to these:
Feature Best Case (days) Worst Case (days) Alpha 1.25 2.0 Bravo 1.5 2.5 Charlie 2.0 3.0 Delta 0.75 2.0 Echo 0.5 1.25 Foxtrot 0.25 0.5 Golf 1.5 2.5 Hotel 1.0 1.5 India 0.5 1.0 Juliet 1.25 2.0 Total 10.5 18.25 When you compare the original single-point estimates to the Best Case and Worst Case estimates, you see that the 11.25 total of the single-point estimates is much closer to the Best Case estimate of 10.5 days than to the Worst Case total of 18.25 days.
You'll also notice that both the Best Case and Worst Case estimates are higher than the original single-point estimate. Thinking through the worst case result can sometimes expose additional work that must be done even in the best case, which can raise the nominal estimate. In thinking through the worst case, I like to ask developers how long the task would take if everything went wrong. People's worst case estimates are often optimistic worst cases rather than true worst cases.
It's an eye-opening exercise. And I'm ashamed to report that I've always used single-point estimates when estimating my work. This is the starting point for many project scheduling disasters, something McConnell refers to as a Collusion of Optimists:
Considering that optimism is a near-universal fact of human nature, software estimates are often undermined by what I think of as a Collusion of Optimists. Developers present estimates that are optimistic. Executives like the optimistic estimates because they imply that desirable business targets are achievable. Managers like the estimates because they imply that they can support upper management's objectives. And so the software project is off and running with no one ever taking a critical look at whether the estimates were well founded in the first place
While it's impossible to give a perfect estimate, it's a good idea to start with the worst case scenario and extrapolate backwards to the best case.
I am absolutely TERRIBLE at estimating the time it takes for project development. I was hoping experience would change that. If seasoned veterans still have issues (Jeff?), is there any hope for me?
jdkludge on June 14, 2006 7:05 AM"You'll also notice that both the Best Case and Worst Case estimates are higher than the original single-point estimate."
Huh? It seems to me that 10.5 is less than 11.25.
Anthony Mills on June 14, 2006 7:13 AM"Huh? It seems to me that 10.5 is less than 11.25."
Delta rose from 0.5 days to a whooping 0.75 days... Which I agree doesn't sound all that impressive in the whole context.
Yaron on June 14, 2006 7:30 AMHere's a suggestion to use when estimating *someone else's* time:
Estimate = (1*Best case estimate)+(2*Realistic estimate)+(4*Worst case estimate) / 7
It weighs the estimate towards the worst case, but takes the best case and expected estimate into account.
As for your own, if you've been in the game for long enough, you should know them (hmm... could those archived timesheets have some value after all?). Of course, this may mean that you know that your own estimates sucks (even if its through no fault of your own). In that case, start with your own formula and tweak it 'til it works.
Oh, and good luck :)
PS, why is b-l-o-g-s-p-o-t objectionable content?!
Opticmystic on June 14, 2006 8:22 AMYeah, I like Steve McConnell, but the given data doesn't really back up his sweeping statement there. Three items were estimated to take less time in the best case, one was estimated to take more time in the best case, and six were estimated the same. So I think it supports the statement that single-point estimates tend to be best-case estimates, but I don't think it supports the statement that thinking about the worst case /always/ makes one gloomier about the best case.
Anthony Mills on June 14, 2006 8:25 AMHow I estimate the time it will take for a project :
Estimate = [very first estimation] * 2 * 3
FlorentG on June 14, 2006 8:43 AMAt my old consulting firm our project managers would double our estimates.
I try to give high-end estimates with a degree of certainty attached. "I'll be able to complete this in 100 hrs with 95% certainty."
Haacked on June 14, 2006 8:55 AMOne of the most important lessons my first programming mentor taught me is that estimation is harder than developing. He had me estimate a project, then worked with me to understand just how optimistic I'd been.
As with your previous post on maintenance work, I believe that junior developers should have to do estimates early in their programming careers. There are two possible feedback mechanisms - work with them to see where they've gone wrong, or secretly pad it for them and then review their estimate with them after the job's completed.
Jon Galloway on June 14, 2006 9:16 AMI read the book too. I liked the diversity of approaches he covers and how he suggests using at least two methods. If they converge, you have more confidence in those numbers; if not, you should investigate why and address it.
My favorite quotes:
page 14: "Accuracy of +/- 5% won't do you much good if the project's underlying assumptions change by 100%."
page 18: "Avoid using artificially narrow ranges. Be sure the ranges you use in your estimates don't misrepresent your confidence in your estimates."
page 36: "The only way to reduce the variability in the estimate is to reduce the variability in the project."
page 132: "Do not address estimation uncertainty by biasing the estimate. Address unceartainty by expressing the estimate in uncertain terms."
Yes, I took notes.
Joe Grossberg on June 14, 2006 9:24 AM> Huh? It seems to me that 10.5 is less than 11.25.
I think McConnell is referring to the fact that the best case got "bester" and the worst case got "worster". That's the goal--better estimates.
> PS, why is b-l-o-g-s-p-o-t objectionable content?!
Because I get 3-4 new spam *.blogsp0t trackbacks EVERY SINGLE DAY. Cleaning it up is very tedious. I apologize to anyone who is incovenienced, but until blogsp0t (a *GOOGLE* company) cleans up their spam act, I have no choice..
Jeff Atwood on June 14, 2006 9:28 AMAs a program manager, my estimation method would be to take the "best case scenario" and multiply it by three. This is my within-team estimate that I will use to set the team working to.
I will take the worst-case scenario and multiply it by eight. This is what I will tell management. Then, if it looks like the team will be done early, *I* look like a hero to management.
Izzy on June 14, 2006 1:37 PMJohanna Rothman has a lot to say about project estimation. This podcast is a pretty good starting point:
http://www.itconversations.com/shows/detail392.html
as is her blog:
http://www.jrothman.com/weblog/blogger.html
Ugh, I can't believe how many commenters are repeating the "take the estimate and multiply it by 'X'" approach. That's exactly what the book says *not* to do.
Joe Grossberg on June 14, 2006 7:22 PM"Huh? It seems to me that 10.5 is less than 11.25."
I was also puzzled by this, so I looked it up when I got home. The edition I have actually reads "If you examine the estimate for Feature 4, you'll also notice that both the Best Case and the Worst Case estimates [...]"
'Feature 4' corresponds to 'Delta' above, which clearly does increase...
ploer on June 14, 2006 10:45 PMIf you start with a guess, factoring in another guess is not materially improving your situation.
Matching planned work with historical measurements would perhaps be a beginning strategy. Surely McConnell has something more to say on such *real* estimation.
hxa7241 on June 15, 2006 7:03 AMorange
orange on June 15, 2006 11:06 AMTouché.
Jeff Atwood on June 15, 2006 12:01 PMFunny, as I'm reading this book as well. Estimations are not a developers strong skill, I've found (including myself).
We're also going through Code Complete as a group at work which is interesting (this will be my second time reading it).
Brian Swiger on June 15, 2006 8:23 PMActually I think three point estimation is even better than 2 point estimation. We have a tutorial on this on our page (its a quite long read as it contains a little stories and some explanations in the beginning):
http://www.visionarytools.com/decision-making/3-point-estimating.htm
Daniel Lehmann on June 16, 2006 2:03 AMI always use:
(best case + 4 * best guess + worst case) / 6
This is surprisingly accurate, and I think it is a standard formula (part of Pert?)
Brad on June 16, 2006 6:22 AMLate to the game here, but I'm surprised that estimation is still primarily based on guessing. How possible would it be to develop metrics based on historical data and use them along the lines of "The last time we had to create a feature/product/method/whatever like this, it took time." Assuming one could come up with an apples and apples comparison, of course.
I'm sure that most people would say "Well, that was last time, but THIS time we're going to ..." and fantasize about some super-improved process that's going to make everything go way better this time around. Historical data suggests, however, that this won't be the case ...
mike on June 16, 2006 11:47 AMWhatever
Joe on June 19, 2006 4:54 AMI agree estimation is a tough problem, and I also like Steve McConnell's work.
Obviously, if you (or your company) has previous experience in a similar project, this is the best way to go. In any case, I've found the most important thing to do is to break down estimates as best you can. Keep breaking them down until no tasks have estimates greater than 3 days. I find this forces you to think about the project in detail, and avoid big vague estimations.
I also find it useful just to estimate raw development time (with some self-testing and/or unit testing, depending on what you do). Then add a risk percentage (ideally proportional to how risky the project is, and/or how much variance is in your raw estimates). Finally, add a further percentage for (integration/system) testing, documentation, optimization, sickness/holidays.
So far, this has worked well for me, but it requires familiarity with the project domain and the programmers.
(And I like Steve's advice of gradual refinement. In practice though, it's a hard sell to management.)
Ben on June 19, 2006 7:31 AMestimation is done due to uncertainty
the only way to improve estimate is to reduce uncertainty
clarify requirements
define scope
small project length
I think the best one-word key ingredient to accurate estimation is experience. Sure a lot of the other factors affect how well you're able to pinpoint an estimate, but having the ability to recognize the scenario you're in and apply lessons learned to the entire issue is the best way to really get more accurate over time.
Retrospector on July 10, 2006 9:47 PMgood practis.
Adel on December 24, 2006 4:02 AMIf you start with a guess, factoring in another guess is not materially improving your situation.
Matching planned work with historical measurements would perhaps be a beginning strategy. Surely McConnell has something more to say on such *real* estimation.
Chris Tacker on January 29, 2007 10:56 AMIt is really helpful to be able to provide an estimate that is as truthful and realistic as possible, particularly for the developer(s) that will ultimately do the work. It has been my experience that when I've understood the problem domain and the requirements were not subject to considerable fluctuation, I could, given time to complete the estimation task, provide a reasonably accurate response to the level of effort and basic timeframe for development.
I have been in situations, however, where the requirements were an evolving beast and the problem domain sometimes was not completely known at estimation time. Despite this state, management/marketing were insistent on having estimations and that proved to be considerably more difficult. When the requirements are in considerable flux, the task of estimating software development degrades into an exercise of pulling numbers from one's nether region. When management then hand-waves at the number of unknowns and takes those "estimates" as the gospel truth, it puts the developer(s) behind the eight ball from square one and creates an environment where software and management are at odds with one another. Such an environment encourages short-cuts to meet unrealistic deadlines, often requires considerable overtime by software to meet milestones, breeds discontent in the ranks and likely results in a product of questionable quality.
Matthew Cuba on February 22, 2007 10:36 AM| Content (c) 2009 Jeff Atwood. Logo image used with permission of the author. (c) 1993 Steven C. McConnell. All Rights Reserved. |