programming and human factors
by Jeff Atwood
I've been unhappy with every single piece of software I've ever released. Partly because, like many software developers, I'm a perfectionist. And then, there are inevitably … problems:
The list goes on and on. Reasons for failure on a software project are legion.
At the end of the development cycle, you end up with software that is a pale shadow of the shining, glorious monument to software engineering that you envisioned when you started.
It's tempting, at this point, to throw in the towel -- to add more time to the schedule so you can get it right before shipping your software. Because, after all, real developers ship.
I'm here to tell you that this is a mistake.
Yes, you did a ton of things wrong on this project. But you also did a ton of things wrong that you don't know about yet. And there's no other way to find out what those things are until you ship this version and get it in front of users and customers. I think Donald Rumsfeld put it best:
As we know,
There are known knowns.
There are things we know we know.
We also know
There are known unknowns.
That is to say
We know there are some things
We do not know.
But there are also unknown unknowns,
The ones we don't know
We don't know.
In the face of the inevitable end-of-project blues -- rife with compromises and totally unsatisfying quick fixes and partial soutions -- you could hunker down and lick your wounds. You could regroup and spend a few extra months fixing up this version before releasing it. You might even feel good about yourself for making the hard call to get the engineering right before unleashing yet another buggy, incomplete chunk of software on the world.
Unfortunately, this is an even bigger mistake than shipping a flawed version.
Instead of spending three months fixing up this version in a sterile, isolated lab, you could be spending that same three month period listening to feedback from real live, honest-to-god, annoyingdedicated users of your software. Not the software as you imagined it, and the users as you imagined them, but as they exist in the real world. You can turn around and use that directed, real world feedback to not only fix all the sucky parts of version 1, but spend your whole development budget more efficiently, predicated on hard usage data from your users.
Now, I'm not saying you should release crap. Believe me, we're all perfectionists here. But the real world can be a cruel, unforgiving place for us perfectionists. It's saner to let go and realize that when your software crashes on the rocky shore of the real world, disappointment is inevitable … but fixable! What's important isn't so much the initial state of the software -- in fact, some say if you aren't embarrassed by v1.0 you didn't release it early enough -- but what you do after releasing the software.
The velocity and responsiveness of your team to user feedback will set the tone for your software, far more than any single release ever could. That's what you need to get good at. Not the platonic ideal of shipping mythical, perfect software, but being responsive to your users, to your customers, and demonstrating that through the act of continually improving and refining your software based on their feedback. So to the extent that you're optimizing for near-perfect software releases, you're optimizing for the wrong thing.
There's no question that, for whatever time budget you have, you will end up with better software by releasing as early as practically possible, and then spending the rest of your time iterating rapidly based on real world feedback.
So trust me on this one: even if version 1 sucks, ship it anyway.
In a previous post (I don't have the time or inclination to find it, at the moment), Jeff revealed how the the MS Excel team discovered a menu item that did absolutely nothing and had been in the application since v1, yet no-one had noticed. I would say that, in that situation, it was worth releasing early, rather than wasting the time on developing something that nobody used (although, maybe, the item should have been removed first).
Steven on December 7, 2009 7:38 AMWe always ship high quality software on time. It always has all of the features promised and very few (if any) "bugs".
If you are faced with this kind of decision on a regular basis then you have failed miserably at the software development process. Why do we keep tolerating this kind of practice? Why do so many people find it so hard to get it right the first time?
Matt on December 7, 2009 7:49 AMI agree to a point. While it's true that striving to release a perfect v1.0 is a fool's errand, it's also wrong to value time-to-market over creating something of which you're intensely proud.
Why release something a few months early when you can get those BUGS out and maybe refine a few features? Sure there will be bugs you didn't know about, and sure some features won't scale properly, etc. But spending a little extra time is something that I think users will notice and appreciate.
Not to mention, the more time you spend on version 1 and are dedicated to the follow-up (as you emphasize) the better your versions 2+ will be.
So yeah, don't spend an extra year working on 3 awesome features that you originally planned would be done by now, but don't release something that's clearly buggy when you could at least take care of most (if not all) the known bugs. You want your users finding those unknown unkowns, and the less known unknowns you have when you release; the more unkown unknowns you'll find.
Josh on December 7, 2009 10:30 AMYou don't get it people!
It is not about ship crap to the word, is about cutting down to a basic product. Errors will always come up no matter how much effort you put on. Unless you're NASA there is not much to do about it.
Forget about the perfect solution since version 1, that's the worse kind of crap, it is suicidal.
"The later any mistakes get found the more they cost": Absolute overrated, obsolete, and stupid idea. There are plenty of new technology there that embrace change rather than trying to avoid it, there's plenty of new approaches to do software (there's more that waterfall did you know?).
Too sad to see how many people missed the hole point of this excelent article!
Mexpolk on December 7, 2009 10:56 AMYes... ship it anyway...
That's what the guys did with KDE4, loosing half of their user base...
this blog post sucked but you have shipped it anyway
second on December 7, 2009 11:05 AMMy two cents:
*****************
* If you have no way of speaking to the customer before a particular date, simulate a customer -
(a) Demand a functionally competent tester who does not code
(b) Get a friend / colleague in the company who can help with a few hours of testing for this one project, then onwards, demand a tester from the first day citing that project as an example.
* If the customer team has even one person of integrity and has genuine interest in the quality of the software (for his own benefit or for any other reason), get a hour or two to show him the working parts - even in that time you will have got enough valuable feedback.
The key is to look for and find the cooperative/proactive user.
He could be the overworked nice guy in his company (he'll help out more than anyone else) or someone who values competence - yours or his own.
WARNING:
Make sure this guy is not the bean-counter at the customer's, or invovled in their office politics (there's a lot of that everywhere), or you are screwed beyond hope of repair.
The friendly user is often your lifeline.
JoeHuh on December 7, 2009 11:59 AM*********
FEATURE REQUEST:
*********
I want to tag comments 'helpful', 'vicious', 'troll' etc.
Ratings / stars / "Was Helpful" / etc are NOT as important.
One would think that professional developers have evolved into humans living in society, and have Netiquette or even basic courtesy to **discuss** things.
Seems not to be so.
In some ways, this is a bug report for many of you guys to fix.
The global user base of this blog - we, the readers, would like the **viciousness** bug to be fixed.
No. I'm not talking of making a Demolition-Man style San Angeles society of weak sheeple.
*********
@Mexpolk - if that was the point of the post, then why didn't it say that? I guess those nuances were lost in the RewriteAsSmackDown(blog) method.
"The later any mistakes get found the more they cost" - no sorry, this is not outdated, but is unfortunately quite correct even in today's web 3.5 world.
The newer methodologies (Agile) try to minimize this axiom by always focusing on what's important and always trying to have a usable product that is verifiably correct. If you follow the rules, you're not likely to encounter this too often. Don't kid yourself - Agile doesn't ignore this.
I also feel like if you want a public beta, you should call it a public beta. If you don't feel like your v1.0 is ready for actual release, but you want feedback on it... that's precisely what a public beta is. But don't make people *pay* for your crappy, full-of-bugs initial release... The way I see it, people are generally pretty excited to be involved in a beta - they know it'll be buggy, but they also know their feedback will help make the final product better. Because, yes, it's definitely true that a product used by its intended users is more likely to have bugs or other issues raised, than one only used by a handful of developers and testers.
But nobody wants to pay for crap. Especially since, as far as I can tell, once a product has shipped, and people have paid for it, few companies are actually particularly motivated to fix complaints anymore, unless they're 100% crippling, and common.
neminem on December 8, 2009 1:57 AMMany time some error found during making software that's why makers are update interpretation. I think so.
Airstream on December 8, 2009 3:53 AM@second:
"this blog post sucked but you have shipped it anyway"
Priceless.
Captcha: 200-mile jilting.
Reminds me of state highway 1, Cambodia
"I'm a developer and my software is far too important to release with bugs. JEFF, YOU ARE WRONG!!"
OK, that wasn't a real quote, but it's a exaggeration of some of the comments I've read here.
So, great, wonderful, you've now identified the fact that you are talking about different kinds of software. Perhaps Jeff is talking about consumer focused software, the non-mission-critical software. You know, the stuff Jeff ALWAYS writes about.
Therefore, if you're so smart with your military-pin-the-bomb-on-the-third-world-targetting-device software, or your figures-must-be-correct-or-lotsa-people-gonna-lose-money software, why are you getting so angry about Jeff writing about a different kind of software with a different market and different types of users - most people who read this blog are aware of the difference. And those that aren't will learn in time. There's plenty of evidence that the most well-intended texts (for example the US constitution and various amendments) can be used mistakenly by the less enlightened.
Perhaps you should put your 'angry' back under your bonnet Bopeep, and perhaps write a researched response on your own blog, covering those important aspects of QA that YOU see as being important.
It just reeks of programmer classism in here. No pun intended.
Keep pumping out the goods, Jeff!
misteraidan on December 8, 2009 6:17 AMI have seen, (having read only about half the comments because There is an astounding quantity of them), just one objection to this advice.
"But Jeff, You can't release buggy software!".
If your primary reservation to shipping is that your software has a too long a defect backlog, then the advice was never for you to begin with. How has your development cycle allowed you to create such a huge mess of bugs in the first place? The advice you should be considering is something to the effect of "No new features until all known defects are resolved".
Better yet, you should ship. Your customers will hate it, and abort the failing project for you.
If you can't ship your product now, right now, then your problems are deeper than the shortage of features your customers expect.
(captcha "natal Department"... seems apropriate)
TokenMacGuy on December 8, 2009 6:49 AMI THINK THE BEST PROCESS IS FOLLOWED BY JOHN CARMACK AND BLIZZARD...
THEY SHIP SOFTWARE WHEN ITS READY..AND ITS WORKED QUITE WELL SO FAR..AND VERY PROFITABLE TOO...
SO I DON'T KNOW WHAT JEFF IS TALKING ABOUT...
This would seem to be a couter-example:
The game "Saboteur" was launched with a known bug that causes it to crash on Windows 7 or Vista computers with ATI cards.
http://games.slashdot.org/story/09/12/09/0542246/Saboteur-Launch-Plagued-By-Problems-With-ATI-Cards
Somehow I really don't think that was a good idea
Rick on December 8, 2009 11:01 AMI think you mean that you *underestimated* how long it would take the team to learn a new technology.
Jose on December 8, 2009 11:48 AMHey Jeff, could you add Agree and Disagree buttons to your posts? I'm curious how many readers fall on each side of the fence.
Rick on December 8, 2009 12:23 PMgood one. everyone should face it in IT field
muthu on December 9, 2009 3:03 AMYou know Jeff there is also something called Beta testing...
Cris on December 9, 2009 3:26 AMnice try, publisher of defect tracking software. ;)
ifatree on December 9, 2009 3:26 AMhmmm and you don't think people are getting tired of testing your shit application for you, or have a lower opinion of you for serving them crap. Sorry but this is going from one extreme to another.
Bob on December 9, 2009 4:04 AMI guess part of this is about what Derek Sivers is talking about here: http://www.codinghorror.com/blog/archives/001313.html
Let how the users use the product inform you about how to do it. But yeah, alpha testing for features, beta for polish and big bugs and release candidate for tiny errors.
ambrosen on December 9, 2009 5:25 AMSorry, that last link was http://sivers.org/walkways . My profuse apologies.
Let the users find out what works and doesn't, and how.
ambrosen on December 9, 2009 5:26 AMI agree with most of this just as long as the code is not buggy and/or unsafe. There are many embedded systems that can cause property damage or cause injury to people. That's not the kind of code you just want to be tossing out into the wild and fixing later. Of course, if you're just talking about missing features or enhancements to usability, then I'm fine with the approach.
Mike on December 9, 2009 8:52 AMI've come to the same conclusion. When I first started developing I was almost unable to get behind with releasing software that wasn't "perfect". Aside from all of the great points that were made in the article, I realized one key thing that allowed me to put my OCD aside. We build software to solve problems for our clients but we do it for one ultimate, self serving goal - to make money. If version 1 is usable but not perfect, put it out there and begin making some ROI. Putting it out there and getting 3 months of user feedback is a plus but the even bigger bonus is making money three months sooner from your clients. This may not be the case for everyone but in the financial software industry, the moment a piece of my software is managing money, I'm getting paid.
AnotherGoldenTenet on December 9, 2009 9:17 AMThat's a very long way to say what could have been said in one or two sentences...
patmanpato on December 9, 2009 10:07 AMFor the first time while reading your blog i have the feeling that you are completly wrong. Not only about your opinion, that is what is happening already. Barely any software is released today that doesnt require several patches to fix major bugs. I think its even wrong to write that here. You even reassure those who do this to go on with their bad quality.
The customer pays for a working software, handing him anything else than that is just not ok.
If software developers don`t have the time for enough testing it doesnt make it right to hand out a bad produkt. If you dont include the time for testing, it is your fault, you made an error in your project management, you have to correct it, the customers are supposed to use your software, they are not supposed to do your work for you by beeing abused as beta testers.
Imagine you would order a car and at the delivery date they present you a car without tires, telling you that they didnt had the time to assemble them.
You expect quality for other products you buy, why different rules for software?
If cars would be "released" with the same quality standards as software i would use my bicycle way more often.
booga booga wooga dee dee
whocares on December 10, 2009 3:47 AMI think this is nonsense, sorry.
Agile methodologies teach us to involve customers right from the start and to build only features that add business value.
A software product may be crappy from an architectural perspective and may need improvement there, but if you need to release a product to get customer feedback, you've never been building it right in the first place.
Roy Dictus on December 10, 2009 8:04 AMif you're so smart with your military-pin-the-bomb-on-the-third-world-targetting-device software, or your figures-must-be-correct-or-lotsa-people-gonna-lose-money software, why are you getting so angry about Jeff writing about a different kind of software with a different market and different types of users - most people who read this blog are aware of the difference. And those that aren't will learn in time. There's plenty of evidence that the most well-intended texts (for example the US constitution and various amendments) can be used mistakenly by the less enlightened.
laptop battery on December 10, 2009 9:31 AMthis sort of summarises all that is wrong in the world of slow and half-baked web, java, .net, and mobile phone software. Most of them useless for many reasons (if they even serve their intended purpose).
Most bugs are signs of deeper architectural problems, and the ones that aren't, aren't very serious, and usually get marked as low priority. To make matters worse, it usually only becomes clear after someone has tried and failed to fix the bug in a short timeframe.
Not finding and fixing them before releasing to the public is a very bad idea, since it increases technical debt tremendously.
In effect, the customer will be stuck with beta grade software until v2.5 because no sane project manager will want to touch these bugs that now have become a huge undertaking.
Nothing is perfect, just look in the mirror.
If God himself had wanted to launch a perfect 'Adam' then we would not exists.
Mac on December 10, 2009 12:20 PMIt's not the imperfections that are the problem.
It's the imperfections that cannot be tolerated.
Or, to put it another way:
If your nose is too big that is not really a problem for most people.
If you have pancreatic cancer that is a big problem for just about anyone.
It goes the same way for software. The little used feature not workking or cosmetic flaw can be tolerated. The major feature not working correctly or heaven forbid, a major flaw that corrupts the purpose of the software, cannot be tolerated.
I disagree,
I develop commercial software which has to be reliable for clients, we have set levels of allowable bugs. We have to have the features we agreed to develop in the final product and have to have a low number of bugs. If the software fails, our client will be fined vast sums of money and we will be sued to breach of contract.
There are no ifs or buts version one has to work first time around. That's is why we employ a team of testers and simulate entire systems in our office, to grantee it works.
That's the real world not the wonderful word of web development
Simon on February 6, 2010 11:22 PMI kinda equate the "sucks vs buggy" to Vista vs XP. XP sucked when it first came out - but it worked as well or better than the current 2000 patched machines.
Vista was buggy as hell
When a product sucks because it looks terrible but does the job as good or better than it's predecessor is one thing. A product that stinks like a cold bag of poo is another thing.
Don on February 6, 2010 11:22 PMGood article and in complete agreement. I'm currently working on a site that's going to be released "incomplete" for the very reason I want my customers to have a hand in the design.
Not sure where you get the notion of developers being perfectionists though... in my experience most developers are far from perfectionists - more in the realm of "mediocre" actually - but that's no different to the rest of the human race. That said, I don't see being a perfectionist as being a particularly useful trait for most situations either.
Jason Snelders on February 6, 2010 11:22 PMAnother alternative is to work with your customers and do technical previews with them before shipping your v1. That is what we're doing. Our close "test group" knows there are issues. They know what our tests look like, but they also know that their day to day use will uncover issues that we missed, and their users will find use cases that even their product managers missed.
The good thing doing it this way is that the customer tends to be more involved (they don't just drop the software thinking it is too buggy). You still get the real world feedback that you wanted. Your customers get to more strongly influence the software. And you can release a v1 version to the world feeling a bit more comfortable with its quality.
Ken Jackson on February 6, 2010 11:22 PM@Matt, I'm curious as to what kind of software you're building? I've never seen any company do what yours does. Do you have any links to your company/software? Your team has found the holy grail of SW Engineering.
Ken Jackson on February 6, 2010 11:22 PMOur software is also made up of our own assumptions that need to be quickly validated by our customers. The longer we delay shipping, the further down the wrong road we could go. Mark Twain put it best: "It ain't what you don't know that gets you into trouble. It's what you know for sure that just ain't so."
Pekpongpaet on February 13, 2010 8:46 PMSee you can't just make a rational, well argued point, you have to take extremes and shove it Oyunlar down someone's throat, which doesn't really work for this particular post (or any if you ask me).
I have to say, the writing was very cloudy. There were a few times of the "but this is wrong" rhetoric that Oyun I completely lost what the point was. Then I realized I don't care.
Bravo your web is very well done can you tell me more regularly with all the news of your web thank you in advance.
dialogue sexe , tarologie , tarot , Horoscope , voyance gratuite , voyance , Tarot Marseille
Thank you for taking the time to publish this information very useful!
Tarot Marseille
escort "Microsoft uses a lot of statistics how many bugs could eventually be in a software at what level and how much bugs a customer wouldn't take care before switching to something else."
istanbul escort Perhaps that could be called the peanut butter approach to release management?
eskort Other than that we all agree in principle that shipping schedules have to be realistic, and failing that, the software being shipped should reflect as close to that shipping schedule as possible.
"Microsoft uses a lot of statistics how many bugs could eventually be in a software at what level adnd how much bugs a customer wouldn't take care before switching to something else."
istanbul escort f
The comments to this entry are closed.
Subscribe in a reader
Subscribe via email
|
|
Traffic Stats |