programming and human factors
by Jeff Atwood
For as long as I've been a software developer and used bug tracking systems, we have struggled with the same fundamental problem in every single project we've worked on: how do you tell bugs from feature requests?
Sure, there are some obvious crashes that are clearly bugs. But that's maybe 10% of what you deal with on a daily basis, and the real killer showstopper bugs -- the ones that prevent normal usage of the system -- are eradicated quickly, lest the entire project fail. The rest of the entries in your bug tracking system, the vast majority, exist in an uncertain gray no-man's land. Did users report a bug? Not quite. Are users asking for a new or enhanced feature? Not quite. Well, which is it?
It's an insoluble problem. Furthermore, I think most bug tracking systems fail us because they make us ask the wrong questions. They force you to pick a side. Hatfields vs. McCoys. Coke vs. Pepsi. Bug vs. Feature Request. It's a painful and arbitrary decision, because most of the time, it's both. There's no difference between a bug and a feature request from the user's perspective. If you want to do something with an application (or website) and you can't do it because that feature isn't implemented -- how is that any different than not being able to do something due to an error message?
Consider an example: Visual Studio doesn't use the correct font when building Windows applications. Is this a bug or a feature request?
Personally, I consider this a bug. I guess Microsoft does too, at least in theory, because it's been in Microsoft's Connect bug tracking system for over four years now. When you build a Windows application, wouldn't you expect it to use the default font of the underlying operating system you're running it on, unless you've explicitly told it otherwise? Well, guess what happens when you create a new form in Visual Studio 2008 and instantiate a label control.
Party like it's 1996, folks, because you'll get MS Sans Serif, and you'll like it. That is the default for each new form. Never mind that every new application you build will look like -- let me put this as delicately as I can -- ass.
Here's a comparison of a label with the default font, versus one that was explicitly set to the default GUI font.
Judging by the applications I've used, most Windows developers couldn't care less about design. That's bad. What's even worse is learning that same design carelessness has shipped in the box with every copy of Visual Studio since 2002.
Of course, matters of design are so subjective. If only there were some definitive source we could refer to on the matter of proper Windows GUI font usage. Some sort of reference standard, as it were. Like, say, the top rules for Windows Vista User Experience from Microsoft:
There are 12 rules in total, but the rule I'm looking for is right at the top -- applications should use the system font.
The hilarity of this list is already sort of self evident, given that I've written an entire post bemoaning the general lack of fit and finish in Windows Vista. I couldn't help but laugh at rule number 12: Reserve time for "fit and finish"! Now there's a rule Microsoft should have taken to heart while developing Windows Vista. Understand this is all coming from a guy who likes Vista.
But I digress.
Despite the windows forms font behavior in Visual Studio 2008 contradicting rule number one of Microsoft's own design guidelines, this "bug" has gone unfixed for over four years. It has been silently reclassified as a "feature request" and effectively ignored. Nothing's broken, after all: using the wrong font hasn't caused any application crashes or lost productivity. On the other hand, imagine how many BigCorpCo apps have been built since then that violate Microsoft's own design rules for their platform. Either because the developers didn't realize that the app font didn't match the operating system, or because they didn't have the time to write the workaround code necessary to make it do the right thing.
Yes, this is a small thing. And I'm sure fixing it wouldn't result in selling an additional umpteen thousand Visual Studio licenses to BigCorpCo, which is why it hasn't happened yet.
But the question remains: is this a bug, or a feature request?
One of my favorite things about UserVoice -- which we use for Stack Overflow -- is the way it intentionally blurs the line between bugs and feature requests. Users never understand the difference anyway, and what's worse, developers tend to use that division as a wedge against users. Nudge things you don't want to do into that "feature request" bucket, and proceed to ignore them forever. Argue strongly and loudly enough that something reported as a "bug" clearly isn't, and you may not have to to do any work to fix it. Stop dividing the world into Bugs and Feature Requests, and both of these project pathologies go away.
I wish we could, as an industry, spend less time fighting tooth and nail over definitions, painstakingly placing feedback in the "bug" or "feature request" buckets -- and more time doing something constructive with our users' feedback.
So many comments, someone else has probably already said it, but I want to throw in my $0.02, the question is moot.
We use a product Called StarTeam and it has something called a 'Change Request'. Notice, I didn't say a 'Bug Tracker'. We don't have 'bugs' and 'features' we simply have 'Change Requests'. A Change Request has a Priority, and it is assigned to a release.
It doesn't matter what you call something, what matters is how important it is, and having a plan for when it will be released.
Using the example of Microsoft's font problem. The font problem still exists for exactly the 2 reason I listed. One, It's importance was never measured. Two, A plan was never made for when it would be implemented and released.
So if you are currently one of those silly zealots that waste my time in meetings arguing feature vs bug, just stop. It doesn't matter.
New Idea:
1 - Consider the cost to make the change
2 - Consider how many people will be happy about it
3 - Consider how many people will be unhappy about it
4 - Consider how many people will be upset if you don't do it.
Then assign it to a release. Even if that release is a year from now.
Customers want to feel valued. Being able to tell them You talked, we listened! Your change request will be included in version 8.7.4, which should be released in June . This makes them feel valued.
Even more important than the ChangeRequest is gaining customer confidence. Telling the customer you listened makes them feel valued. Telling the customer you have a plan gives them confidence. And delivering on that plan, that builds trust.
@Theo and @irkregent
I suggested in a previous post that the argument is moot, what matters is how important a change is, and in your post you argued that classification is the basis for your business model.
So who's right?
Well I think whats probably happening, were using 2 different terms for the same thing. I expect you probably fix for free anything that is really important to a lot of users, and charge for all the unique requests.
Kyle on November 21, 2008 10:58 AMI think that most enhancements or feature requests are really bugs...but not in the code. They are often bugs in requirements. If a user comes up with a whole new concept it's a new feature, but if they need something improved it's probably a bug in requirements.
Blizard on November 21, 2008 11:06 AMI'm a little OCD when it comes to users calling enhancements defects. For example, your specification states display date so I display the date. Now if the date I'm displaying is correct - don't enter a #$(*#( defect to make the date purple instead of black because you don't like black. That's a cosmetic issue, not a (*#$ defect!!!!
coward on November 21, 2008 11:56 AMGreat article as always :)
Jef Claes on November 21, 2008 12:02 PMInteresting. Basically, we should simply keep the users happy, no matter if it's a bug or a feature.
But perhaps this division doesn't come because of developers not wanting to write more code. Perhaps it comes also because the users should pay for new features, while bugs are free of charge. Am I off?
It seems clear to me that the differing opinions here are between developers that do contract project work and developers that create shrink-wrapped applications.
@Kyle:
New Idea:
1 - Consider the cost to make the change
2 - Consider how many people will be happy about it
3 - Consider how many people will be unhappy about it
4 - Consider how many people will be upset if you don't do it.
If you're developing a fixed-fee project and the change, while making the client very happy, is significant enough that it will result in a financial loss on the project, it will result in very unhappy developers/managers/owners/stockholders in the development house that took the project. So what is classified as a bug and what is classified as an out-of-scope feature request makes quite a bit of difference.
In the case of a shrink-wrapped application or service, where the product/service is sold many times, then your argument makes sense since the development costs are always borne by the house.
The font thing has been burning me up in the inside for all these years.
Mark Allanson on November 22, 2008 3:16 AMObsolete it may be, but the problem is bigger for VB6 devs than it is for Jeff using a late VS product. VB6 does use the old MS Sans Serif as its default.
Though he missed the boat re. MS Sans Serif vs. Microsoft Sans Serif, I still think it makes sense to adapt to the system font, which must be done at runtime.
Yes, since this means size as well as face you have potential control resizing and placement issues that can kill you too.
Da Bob on November 22, 2008 9:37 AMThe 'insoluble' aspect is simply that humans are the weak link. Either the client or the programmer or some intermediary got it wrong. For any project of a decent complexity, you cannot convince me that you can specify some document so perfectly that there will be no errors, either in implementation, omission, nor fickle users changing their minds.
Go ahead and take 10 years to write that document, in the meantime some of us have work to do.
@Kyle
If all my clients have the same request I still charge them. If only one of them has a bug I have to fix it for free.
Theo on November 23, 2008 5:37 AMThis is actually a bug. The question is what kind of bug. Well, it is a usability bug. The problem is that the user would rightfully expect the default to be the font that is SET as the default font by the user in the operating system. I would be surprised if many who have not started to expect otherwise because of experience, would expect otherwise.
Paul de Vrieze on November 23, 2008 7:29 AMTo the for-fee folks out there: I think it's fairly clear Jeff is thinking here only of non-contract work (shrink-wrapped isn't a good term because a huge body of work which fits this model is delivered on servers ... and who 'shrink-wraps' anything any more?). Visual Studio? Who thinks they operate on a contract fee-per-feature basis?
If they do, then someone in management at Microsoft needs to be shot. Well, I suppose someone in management at Microsoft needs to be shot anyway, but ...
One of the most obvious (to me) principles of agile development is that there is no artificial distinction between bugs and features. There are issues people have using the software we created. Some, primarily bugs but often enough missing features that we just didn't think of, keep them from seeing the way the software is SUPPOSED to work; these are a first priority because until those are fixed we have no way to judge the rest. The rest, be they bugs in the software or bugs in the analysis or bugs in what the customer said they wanted, need to get prioritized as backlog and worked from the top down.
Honestly, there's no reason why a traditional waterfall type of development house couldn't work the same way, erasing the false distinction between feature and bug (except where it actually HAS a purpose, such as in customer contract work), and just get things done. However, my experience with waterfall methodologies is that there's a lot more interest in accurately pointing the finger of blame that groups will spend a huge amount of time and effort bickering about where the bug was (and every feature IS a bug: something the user expected he software to do which it could not do) instead of just fixing it and shipping software. The rationale here is that no one will get better at their particular siloed jobs unless they get metrics about how well they've been doing in the form of bug counts; in reality, though, I've NEVER seen anyone improve any part of their workflow or quality of work as a result of such metrics.
A note about every feature is a bug: the necessary corollary to that is and not every bug will get fixed. Obviously, the bug that your MP3 player doesn't control their LPs isn't likely to get fixed, ever. At the same time, the bug that when you put xyz in a number field it silently makes it 0, while embarrassing, may well never get fixed because you have bigger fish to fry (or, you've already decided to scrap all number fields). There is this odd sense of programmer's shifgrethor about never producing bugs and bending over backwards to fix the bugs we do produce. We all produce bugs, or we wouldn't be producing software. Get over it. Fix the most important thing *to the customer* first, be that a result of your bug or a result of the customer telling you that the sky is green not blue.
Competition plays great role in this. I can choose between IE and Firefox and I select Firefox. Why? Because I like it more. I like it more, because it feels better. A company can make a choice between different software vendors. Usually there are bunch of things that affect the decision, but one issue is do the users like the software in general - does the software feel good or not. The management might go for the cheaper software, but then the company has to accept the bugs and the fact that the software doesn't feel as good as the other. Unless the cheaper software has less but better features, of course.
Anyway, if a software vendor can create a template or such for their software, then that helps them making the software feel better in general. The vendor has then a more robust base for the development.
I like more robust software architecture too. There are architectures that are easy to use and architectures that are difficult / complex to use. When I program some logic, I don't want to know anything about little things that should be taken care of by the template.
Silvercode on November 23, 2008 11:32 AMThe best way around the bug vs feature request argument that I've come across is to call them all Defects... if it's a bug its a defect in the way the software has been written, if its a missing feature then it's a defect in the functionality of the software - most people seem to understand this concept and it often takes the heat out of thing. Which doesn't mean that the blue screen of death isn't a bug! :-)
Steve Cohn on November 23, 2008 12:06 PM This happened once at one of project at my work. There was one ambiguous situation with the project.
The QA opened the situation as a bug, while the developer argued that as a feature. While the discussion lasted for 15 minutes, it was fun watching that. I think each has his own motives to brand it as a bug or feature.
The other occasion was there were lot of issues with some functionality with a product. So the project manager decided that the same issues which occur in N version is feature while if it happens N+1 version then it is a bug.
the requirements of the system and agreed to and signed off on prior to development AND a feature is something ELSE that they want added (or even taken away) from the agreed to/signed off features of the software application.
This is a rather old school way of thinking that conforms to the rigid waterfall model. It is an INCREDIBLY good designer/analyst who can drag out *all* of user's requirements at the start of a project and get everything perfect first time. In fact, so incredible that they don’t exit IMHO. That is not to say that we don’t do requirements gathering and write specifications. We do – it’s a crucial part of our quality procedure. But if you take the approach that every single enhancement the client asks for is returned with “NOT IN THE SPECIFICATION”, you’ll not be working for that client again! Sure, for the big things like “Ohh didn’t we mention the user interface has to be in French and German” but given that it is impossible to foresee everything, then you have to have a degree of flexibility. That said, the specification should define things like screen resolution and fonts if not as a constraint for the developers if for nothing else.
I read a great article somewhere that argued the case that software design carries on right until the product ships. Unfortunately, I can’t remember much more about the article. There is a lot of sense to that idea. The catch-22 situation though that all developers should be aware of is that clients *will* ask for up-front costs and you saying “But I can’t tell you that until the design is complete” just won’t wash. So one has to live in this weird world of best guesses and estimations.
The one reason I personally force myself and the team to do the requirements and specification is to force *them* to think about the entire system up front. The advantages of tackling the big problems at the start cannot be underestimated. It also gives the project leader a much better lists of tasks so at least the project can be tracked.
Cheers, Rob.
PS. We always try for the Staged Delivery model if at all possible whereby you expect to do another version of the program. This a) gives you repeat business with the client but also b) allows you to agree Yes, good idea but outside the scope of the first version. The clients are often very happy with this as a) the project more likely hits it's release and b) their requirements aren't getting lost.
Rob d'mouse on November 24, 2008 1:21 AMBasically, to many users, a bug is when the system works contrary to how they *expect* it to work.
That is still a bug but the bug wasn't in your code, it was in the requirements gathering phase. It was, however, a joint bug by both ends.
Rob.
Rob d'mouse on November 24, 2008 1:24 AMWhat you're saying makes sense if you're working for the company that you're designing software for. If you're a custom software development shop, then it's a different situation, and gets back to do you charge for fixing bugs?
Ahh that's an age old problem! We have tried the idea of one year warranty which has sometimes works. We basically agree to fix any obvious bugs for free. I feel, that as a developer, you should do this. It's expected in almost every other industry. But this is where Bug = Fault. Given that we've agreed that user's tend to define Bug = Anything that doesn't work they way I expect, then the discussions about what is a bug/feature request is tricky.
Rob.
Rob d'mouse on November 24, 2008 1:26 AMMicrosoft doesn't want to address the Visual studio issue and a lot of other issues too.
Nick Masao on November 24, 2008 2:35 AMThis Illinois license plate photo might make you smile...
http://www.flickr.com/photos/dratz/1045336659/
The title of the post remained me an indecent, that I experienced a developer who's lazy enough to look into the problem had finally said, the things which we reported is not a bug but a feature. The bug was silly and we could easily find out and fixed. :)
Sarath on November 24, 2008 5:58 AMI'm not sure if this was mentioned earlier as there was quite a bit of discussion and I didn't read it all.
I classify all changes as either a bug or a change request and the reason I do this (I suspect its the same for most developers) is that the customer is not going to pay you to fix a bug, but they do need to pay for feature requests.
Bugs are generally defined as a crash or error that prevents the application from executing properly. Beyond that if something was in the specifications and did not get implemented, it also falls into this category. Basically anything that I was supposed to do, any issue thats my fault as the developer is a bug.
Change Requests are anything that the user would like that does not fall into the current scope statement. Signatures are always required from both parties on these as there is an exchange of money for them.
Cody Rioux on November 24, 2008 10:46 AMMy reply here.
http://riprowan.com/Blog/tabid/36/EntryId/49/Coding-Horror-ibly.aspx
Rip Rowan on November 25, 2008 9:47 AM@Rip Rowan
Excellent reply!
Theo on November 26, 2008 2:49 AMSmall solution..
----------------------
Simply DON'T touch any font in the designer and in the form constructor do this ..
public Form1()
{
this.Font = SystemFonts.MessageBoxFont;
InitializeComponent();
}
Since all controls will inherit the font from the form
(If you didnt touch the font setting on the control)
Vista users will have Segoe UI and XP users will have Tahoma
when they run the application..
Works fine in standard apps.
Unless you're in the (perhaps) enviable position of having a single user, one user's feature enhancement request may turn into another user's bug report.
In the era of intertwined code and database, traditional bugs may be more fixable than other forms of unexpected behavior which do not cause crashes but do produce wrong answers. Difficult to test.
Most specs are not written by users, but by committees. Most users care nothing about specs, but how things work. Often developers don't meet users until the delivery phase (when bug/feature reports arise). I suspect the spec/committee/thou shall portion of the process is mostly at fault. If developers and users met/conferred/agreed at an early stage, everyone might be happier
How many times has that FBI case software been shitcanned? For $$$?
@Theo - Thanks.
I've expanded the reply here:
http://riprowan.com/Blog/tabid/36/EntryId/52/Bugs-Features-Defects-Who-Cares-I-Do.aspx
I've got my team using the generic concept of a product backlog (borrowed from Scrum).
If a change is needed to the software, whatever that change may be, it goes on the backlog. The product manager works with the team and a customer advocate (hopefully an actual customer) to determine the order in which the team will attack the backlog.
The whole defect vs. feature request problem just disappears!
(As an aside, Rip Rowan, classifying defects into different types rarely ends well in my experience, and doesn't add any value to the discussion. A 'bug' is something that bugs someone. Any other view of the world strikes me as a way to get out of listening to the customer, and probably to avoid doing some work to improve the user experience.)
Usability is a feature. Perhaps the only feature that really matters...
Chris Jaynes on December 10, 2008 3:03 AMAccording to Donald Gause and Gerald Weinberg in Exploring Requirements: Quality Before Design, a problem can be defined as the difference between things as perceived and things as desired. From this we can derive that both a feature request and a bug are problems, because this description fits for bugs and feature requests as well. It sheds some light on why some applications use the term issue tracking and not bug tracking. GoogleCode, for instance, then uses labels to organize the various issues.
When talking to the customer I assume everything is a bug unless it is:
a) So obviously a new feature that there could never be any doubt.
b) Called a feature by the customer.
c) Contractually specified and/or important (see fixed-fee discussion).
Of course internally we handle things differently - most medium/low severity bugs are just lumped into the feature list and prioritized as if they were features (or vice-versa if you prefer). The high/urgent severity items are fixed as soon as possible.
So really, I guess the gray area you refer to only matters in direct communication with the customer, otherwise the distinction doesn't serve much purpose on our team, at least in terms of actually getting fixed.
Brian B on December 16, 2008 5:56 AM... applications should use the system font.
How's cursor in vista? It's very annoying the old applications have windows default cursors within their library and often ignoring the system's theme cursors.
Not to mention the very-hard-to-replace drag-drop item and shortcut cursors. I notice these two fixed in Vista.
Saya on February 28, 2009 3:28 AM Who knows where to download XRumer 5.0 Palladium?
Help, please. All recommend this program to effectively advertise on the Internet, this is the best program!
Here's a comparison of a label with the default font, versus one that was explicitly set to the default GUI font.
*fails to see the difference*
Runaway Moon Docs on February 6, 2010 11:14 PMWhile I enjoy the irony of Microsoft violating its own standards, I think the gnome-screensaver more than makes up for it in the open source world. http://bugzilla.gnome.org/show_bug.cgi?id=316654 The ability to choose your settings is an important part of a screensaver application, and yet when gnome-screensaver is missing it, it is a feature request which is somehow... evil. This is an excellent example of the fuzziness being used against the user.
none on February 6, 2010 11:14 PMThis is interesting. When reading the responses, I'm trying to guess where the poster is in the spectrum from a short-term contract programmer to a developer in a software that is continuously being maintained or enhanced. I think I can guess who is who, but I guess I'll never be sure. :-)
Carleton on February 6, 2010 11:14 PMWell I AM a contractor. And I'm more then happy as such. And the reason that I need to have a clear distinction is to handle my budget. A BUG is something that is my fault and might come out of my original estimated budget. A FEATURE REQUEST is something new that the customer (or their users) want that wasn't in the original contract. There is a very important need for a clear distinction. Perhaps the feature should have been in there in the first place, but the money discussed did not include that. If the customer wants that build . . well it will cost them. If they want something fixed, well that's on my head.
Of course that introduces the common practice of my trying to push everything reasonable into the feature request pile, and the client attempting to shovel everything into the bug pile.
Jason on February 6, 2010 11:14 PMis there really a difference between MS Sans Serif and Segoe UI at size 8.25 in ClearType? Sure one or two pixels are different, but not so you could name the font. If you used OS X's font rendering there would be a difference :) I don't like Segoe anyway.
Bug or feature? Sure some users can't tell the difference, but if you have a featureset you can at least explain, that there are things the app is designed to do. If it doesn't do what you said it would: bug. If it's something it should do in the future: feature request.
John Ferguson on February 6, 2010 11:14 PMOh, and Aero Glass shouldn't be used. I like translucency as much as the next OS X user who can see through dialog sheets, but Aero glass does it soooo badly. ugh.
John Ferguson on February 6, 2010 11:14 PMSpeaking of ugly font issues, has anyone complained about your blog font? Your css is set up for font-size:90%;font-family:calibri,tahoma,arial,sans-serif; and it looks awful on both IE and Firefox in a fairly run-of-the-mill laptop with WinXP. I did a little testing and the culprit seems to be the calibri font - it looks fine when not scaled, but the 90% hurts the eyes.
sln
Shannon on February 6, 2010 11:14 PMThe comments to this entry are closed.
Subscribe in a reader
Subscribe via email
|
|
Traffic Stats |