*
programming and human factors
by Jeff Atwood
*
Who files better bugs? Users or developers? In How to Report Bugs Effectively, Simon Tatham notes that software developers, contrary to what you might think, file some of the worst bug reports:
It isn't only non-programmers who produce bad bug reports. Some of the worst bug reports I've ever seen come from programmers, and even from good programmers.I worked with another programmer once, who kept finding bugs in his own code and trying to fix them. Every so often he'd hit a bug he couldn't solve, and he'd call me over to help. "What's gone wrong?" I'd ask. He would reply by telling me his current opinion of what needed to be fixed.
This worked fine when his current opinion was right. It meant he'd already done half the work and we were able to finish the job together. It was efficient and useful.
But quite often he was wrong. We would work for some time trying to figure out why some particular part of the program was producing incorrect data, and eventually we would discover that it wasn't, that we'd been investigating a perfectly good piece of code for half an hour, and that the actual problem was somewhere else.
I'm sure he wouldn't do that to a doctor. "Doctor, I need a prescription for Hydroyoyodyne." People know not to say that to a doctor: you describe the symptoms, the actual discomforts and aches and pains and rashes and fevers, and you let the doctor do the diagnosis of what the problem is and what to do about it. Otherwise the doctor dismisses you as a hypochondriac or crackpot, and quite rightly so.
Programmers file bad bug reports for the same reason that programmers screw up usability tests: they mean well, but they're heavily prone to prescription instead of observation. In a usability test, most developers can't bear to see the user fail, and will actually intervene on the user's behalf. "Wait-- don't click there! Click here!"
This, of course, completely screws up the usability test.
Developers' heads are full of cool technical arcana, and that's precisely why they need to resist the urge to screw up their bug reports with inappropriate prescriptions. Cultivate your observation skills! Slapping together a hastily concocted call to action is easy; patiently observing and collecting evidence to make a compelling case is much more difficult.
Although I agree that developers tend to try to do at least a partial diagnosis when doing bug reports, I think they usually provide the right kind of data to determine what the problem is. It all depends on the situation of course.
I have seen user bug reports that look like this:
"I logged into the system and got an error. Guess I can't test tonight!"
Now no developer worth anything is going to produce an bug report like that. At the very least they'll give a description of what led up to the error, and what the error message actually was.
The best thing to do is to train testers to provide the relevant information. You don't really want developers being the ones who do a large portion of the testing anyways, since we use the app how it's supposed to be used, and don't try funky things that'll cause it to break.
Marty Thompson on December 6, 2005 5:32 PMJeff, you must really be looking to push my buttons today! :)
I totally disagree that developers give the "worst" bug reports. There are certainly some developers who aren't any better than the average user. And there are definitely others who will try to diagnose the problem before it is time. But I have to agree with Marty on this one. Users typically can't even tell you what they were doing when a problem occurred. Developers are usually familiar enough with the system to be able to pin down the exact part of the app (in technical terms) that they were exercising when the problem occurred.
Anyway, I'd take developer's bug reports over user's bug reports any day of the week.
matt on December 6, 2005 6:19 PM> I totally disagree that developers give the "worst" bug reports.
The actual text is "some of the worst". Well, at least in the body text. ;)
Not all developers are prone to this, of course, but it's definitely something I've seen a lot. I like to respond to unnecessary prescriptiveness with a non-denominational "Yes, but what's your goal?"
This helps people focus on what they want to accomplish -- what you usually want to know -- rather than how they think we should do it.
Jeff Atwood on December 6, 2005 7:37 PMI don't think he is describing a problem with developers in particular, I think he is describing a problem with anyone in any kind of profession. "I'm sure he wouldn't do that to a doctor." but a doctor might (ignoring the fact they can write their own prescriptions). Most people think they are knowledgeable enough to diagnose a problem in their field of expertise.
Erik on December 6, 2005 8:40 PMWhat I hate is that developers usually find ways around the bugs and then never report them.
Brian Russell on December 6, 2005 9:49 PMCar mechanics are also familiar with the situation where customers -- would-be mechanics -- tell them the problem rather than the symptoms.
mike on December 6, 2005 10:28 PM"What I hate is that developers usually find ways around the bugs and then never report them."
I'm bad about that. I'll put a break point at the problem code, correct it with the immed window, and then continue on. Sometimes I'll report it, but other times I just assume that the guy responsible for it knows it's a problem.
Marty Thompson on December 7, 2005 10:07 AMI've always maintained that programmers should be made to stand and watch their end-users working with the software. Their thumbs should be tied behind their backs, and they should be wearing gaffer tape over their mouths. As for bug reports, it's only a matter of professionalism. Even a programmer can write down the scenario to reproduce an issue.
Dominic Cronin on December 7, 2005 11:30 AM| Content (c) 2009 Jeff Atwood. Logo image used with permission of the author. (c) 1993 Steven C. McConnell. All Rights Reserved. |
Subscribe in a reader
Subscribe via email