The Ultimate Unit Test Failure

February 14, 2008

We programmers are obsessive by nature. But I often get frustrated with the depth of our obsession over things like code coverage. Unit testing and code coverage are good things. But perfectly executed code coverage doesn't mean users will use your program. Or that it's even worth using in the first place. When users can't figure out how to use your app, when users pass over your app in favor of something easier or simpler to use, that's the ultimate unit test failure. That's the problem you should be trying to solve.

I want to run up to my fellow programmers and physically shake them: think bigger!

A perfect example of thinking bigger is Alan Cooper's Interaction 08 keynote, An Insurgency of Quality.

Alan Cooper's Insurgency Talk video

There's a transcript of his keynote available, or you can view a video of his keynote video with the slides in place.

Alan is a well known interaction designer , and the author of several classic books in the field, such as About Face, and a few others that are on my recommended reading list. In the Q&A after the talk, he had this to say:

"We are not very important because we don't cut code." (A boo and hiss from the audience.) In the world of high-technology, if you cut code, you control things. It's the power to destroy the spice, it's the power to control the spice. It's not a fine kind of control: it's bruce-force kind of things. [Interaction designers are] largely marginalized. We're constantly asking for permission from the folks who shouldn't be in a position to grant permission. We should be working with business folks and marshalling the technology to meet the solutions to business problems.

But when it comes time to marshal the solution to the problems, we find ourselves slamming into this kind of Stay-Puft Marshmallow Man of software development.

Stay-Puft marshmallow man, from Ghostbusters

We don't need to change interaction design; we need to re-orient organizations to build things right. When we come to programmers and say, "Look at the people I've talked to; look at the personas I've created" and present them with research, programmers understand that, and that's how we will influence.

It pains me to hear that Cooper considers most programmers twenty-story marshmallow barriers to good interaction design. Please don't be one of those programmers. Learn about the science of interaction design. Pick up a copy of Don't Make Me Think as an introduction if you haven't already. There's a reason this book is at the top of my recommended reading list. Keep your unit testing and code coverage in perspective -- the ultimate unit test is whether or not users want to use your application. All the other tests you write are totally irrelevant until you can get that one to pass.

Posted by Jeff Atwood
77 Comments

unit testing is used for checking code quality: to see if waht you've written is well enough written. Does'nt tell you what to write. Does'nt tell you if the code is useful. Does'nt tell you anything about your users, like the compiler that does not tell you how many copies of the software will be sold, and the version control system does'nt know if your icons and splashscreen are nice.

Jan on February 15, 2008 1:09 AM

I always thought it was "State Puff" marshmallow man. That always sounded more scary.

(Programmers can be tangential as well).

Jeff Davis on February 15, 2008 1:39 AM

Hey now Jeff,

This is a bad and confusing post. I don't understand why you are talking about this.

Kris Katto

katto on February 15, 2008 1:49 AM

sounds like Jeff is having a bad test day :)

eryn on February 15, 2008 2:14 AM

I generally find unit tests and code coverage a bit over rated. Code coverage for core classes in a system is vital. But unit tests for fringe components like UI classes, xml configurations, data...? give me a break.

That said unit tests and UI design/interaction are orthogonal concerns. The post itself is not completely valid. If it had been importance of UI interaction/design versus system architecture, then I would say it makes a lot of sense. Many projects/products fail due to over zealous architects wasting time and resource on 'cool' system internal designs and implementations, while completely ignoring the UI.

PV on February 15, 2008 2:20 AM

Breath through your mouth. No! Breath through your nose. Arrg your doing it wrong!!

Steve on February 15, 2008 2:24 AM

I dig where Jeff is coming from here, let me help him a bit :)

Engineering typically obsesses (and arguably rightly so) about the engineering effort. We focus on the building a robust, extensible system with lots of moving parts - ensuring the quality of said parts through significant engineering effort.

What "we" (the royal we - "the editorial") sometimes fail to do is understand that what and how we build should change when faced with expertise telling us what to build.

For example:

"We need to rework the reports part of our app now that we have this acceptance data telling us we got it wrong."

"But we won't be able to get the code coverage up to our required 75% by release time! Forget it!"

The "for engineering" part of that response fails to take into account is that code coverage on a crap system doesn't give you anything more than... code coverage.

Rob S. on February 15, 2008 2:34 AM

I'm just impressed that Cooper can throw out a Dune reference like that and then switch gears in mid-argument to Ghostbusters and have it still make sense. Slick.

Alex Chamberlain on February 15, 2008 3:10 AM

Agreed. The first thing that came to my mind was verification versus validation. While Engineers should be concerned with whether they are building the product right (verification), their overriding concern should be whether they are building the right product (validation).

I kind of think that one of the only things that separates one software company from another is mastery of a particular problem or user domain. Our company, Primordial, is always trying to master the military user domain. We keep a risk database for each of our projects to facilitate risk mitigation analysis, and the risk that "users won't find our software useful" is always near the top.

It is important for engineers to get their head above the water and see the bigger picture sometimes.

Kyle on February 15, 2008 3:40 AM

Unit testing should be separated from User Acceptance testing; that is, your Unit Test should never drive the acceptance of the user interface as by their nature, Unit Tests do not and should not have the entire execution "thread" for the process that the app is implementing.

Sometimes the developers have to get out of the way.

ActiveEngine Sensei on February 15, 2008 3:42 AM

The sad part is when the specs aren't coming from experts they are coming out of some managers head and you get no idea what the customers really want. Then its Dilbert time and the product is likely to be crappola even if it does have unit tests or not.

We may have all been marshmallow men at some stage but sometimes its justified, sometimes its not.

Tim Yen on February 15, 2008 4:14 AM

A gree with the sentiment that this post is confusingly titled and confusingly written.

the ultimate unit test is whether or not users want to use your application.

Absolutely not. That's not a unit test at all, since it is a test of *the entire application* not a discrete unit.

It's a User Acceptance Test; and, as you rightly point out, it's the most important one of all.

bignose on February 15, 2008 4:15 AM

"Unit Tests do not and should not have the entire execution "thread""

So very true. Unit tests are not an endpoint, just one tool in a suite of many required to ensure your product, well, works.

Where "works", of course, means "correctly does the tasks expected of it in a manner that makes people willing to use it."

Dan Fleet on February 15, 2008 5:11 AM

I thought the post was right on-target. Seems like some of the comments are by some of those "obsessive by nature" people who can't see the forest through the trees.

If you user interface sucks or is even remotely difficult to use or understand, you'd better hope you're the only kid on the block. Because users won't spend more than a few minutes with a piece of software that they can't navigate no matter how many unit tests you ran before you published it.

Craig Eddy on February 15, 2008 5:39 AM

Jeff, I'm sorry, but I just don't see how these two things are in any way exclusive. A [well-tested unusable application] and a [under-tested highly-usable application] are NOT two ends of a 1-dimensional spectrum.

Rajiv on February 15, 2008 5:51 AM

Too often, I find myself working on projects where the visual style and to some extent the GUI mechanics are viewed as something that can be determined "later", after the "engine" is written. When later comes, there is inevitably some major change that needs to be made that reaches several layers down into the system. It's like I'm being asked to remove raisins from cookies after the dough is mixed.

I strive to deal with this gracefully, but I have to ask: is it any wonder that the average engineer tries to defend their project against seemingly-whimsical changes imposed by some distant self-styled authority? Respect comes from *trust* which is not gained by dictatorial power (which appears to be the solution put forward by your quotes). Rather, it grows from experience of the designer's competence, which never happens in a development process where engineers are isolated from users by designers and management.

sapphirecat on February 15, 2008 6:07 AM

This post could be replaced with:
"Make your applications simple to use and easy to understand."

Jeff, You should unit test this post for usability.

Niyaz PK on February 15, 2008 6:10 AM

His book „Inmates are Running the Asylum” has changed my way of thinking about UI and interaction design.

Rimantas on February 15, 2008 6:11 AM

Hey Now Jeff,
Strange point, we should provide UAT?
Codeine Horror Flan,
Satto

Satto on February 15, 2008 6:14 AM

Hey Now Jeff,
Nice point, we should provide solutions.
Coding Horror Fan,
Catto

Catto on February 15, 2008 6:24 AM

I think the knee-jerk reaction is because it feels like unit testing is getting singled out. Some people are tired of defending unit tests as a useful thing. All you need is one article saying that unit tests are not important (we all SHOULD know they're not the most important) and we go back to write some code and throw it over the wall to QA.

I think Jeff could have listed tons of things that are less important than having a usable product. Separation of concerns, source control, backups, lots of monitors, a nifty keyboard, encryption, etc. They're all important, but don't bring money to the coffers quite as directly as having a good usable product. I, for one, have never been near a shop that obsessed about unit testing as much as say - which version control system to use.

SteveJ on February 15, 2008 6:34 AM

This is great for the programmer who doesn't have a spec sitting in front of them ... *sigh*

jminkler on February 15, 2008 6:45 AM

Hey, there's no harm here. It's a good point.

I think the point Alan Cooper's trying to make is that, essentially, those interaction designers are being marginalized when they're dealing primarily with the developers during implementation.

Success means you're thinking about this stuff before the product exists: think Jobs and Ives.

I have respect for agile methods, but it appears that the most well-*designed* products had the most effort put into their *design* before coding begins. Who'd have thunk?

In the majority of products, though, you're not going to have the level of detail in the spec that excellence demands (pity--but that's life). Because of this, it'll still be necessary for the designer to be extremely accessible to the developer for the myriad of tiny decisions that always come up.

Rich on February 15, 2008 6:54 AM

Unit testing tends to be black and white: f(3,4) == 5. It either passes or fails. Simple for a coder to understand, implement, and produce a report of results.

UAT / user interaction design tends to be more complicated, harder to put your finger on, given to personal preference and irrational debates. The kinds of troubles many programmers feel they already have enough of and not difficult to see why it's commonly avoided...

Brillo on February 15, 2008 6:55 AM

This is exactly why presentation should be separated from implementation. UIL with Motif is great for this, because in many cases you don't even have to recompile the application when GUI changes are made. Let the Software Engineers do their work on the implementation, and let the GUI and usability experts and "interaction designers" do their thing on the GUI.

T.E.D. on February 15, 2008 7:00 AM

I'd be interested in a poll to see how many developers have actually

* seen an interaction designer in the wild (do they really exist?)
* worked with an interaction designer on a project
* pooh-pooh-ed an interaction designer's input
* considered the interaction designer their new best friend

Steve Campbell on February 15, 2008 7:05 AM

I agree with the faction that says this post is off-target, in that User Interaction is generally not in the typical programmer's purview - most of us write the server code behind the app, and there is different group responsible for the UI.

That said, I point to the management as the primary problem - they decide to skimp on UI design, tell the UI developers to slap up some screens, and think that the app is done (This applies to commercial software, mainly - OSS guys, developers are the mgmt).

Pete on February 15, 2008 7:06 AM

You said it Jeff, this is the WHOLE point.

If users don't use it then it's all for naught.

Mac on February 15, 2008 7:09 AM

I work in QA, but this looks like apples and oranges to me. Both kinds of testing are critical. UAT and Unit tests. To that I'd add the ones that most developers hate, integration testing via the GUI.

[Begin off topic rant]

Yeah, integration testing is messy, and difficult to understand, difficult to automate and high-maintenance if you don't know what you're doing (I'd say over 90% of companies don't). Moreover, the results are often difficult to untangle.

The only problem is, that, after human testing, most of our bugs are found after the code is integrated and tested with a GUI-based automated testing system. Oops!

Some developers think you can fix all bugs with unit tests and "integration" testing consisting of basic unit tests. Good luck with that. I don't know *any* experienced QA professionals who believe this. I certainly don't.

[End off topic rant]


ThatGuyInTheBack on February 15, 2008 7:37 AM

Jeff,

I should admit I myself sometimes "marshal the solution to the problems". I need to pay more respect to users/business analysts.

However I don't think this has too much to do with unit test or code coverage. Actually if I replaced "unit test" with "refactoring" or "code review" in your post, it would still make sense. Don't you think?

I think I got your idea. Just the keyword "unit test" may be too sensitive. :)

Kai Wang on February 15, 2008 7:45 AM

Hey Jeff,

You should check out Harold Thimbleby's book "Press On: Principles of Interaction Programming." He makes the case that programmers can play an important role in the usability of a product. He says that it's not enough to just polish the user-facing features -- for a truly usable product, you also need make sure you get the engineering right.

Patrick Dubroy on February 15, 2008 7:52 AM


Again, an apples oranges dichotomy; those are far too common on Jeff's blog and are starting to chip at his credibility.

Great interfaces will make the product easy to use, good testing will maintain the functional stability that supports that usability. Why these two ideas that are so different and involve different skills and personnel are even contrasted is just beyond me.

Please Jeff, if two ideas are not mutually exclusive, could you not draw them up into a mutually-exclusive conditional statement ? That's not only a basic tenet of programming, it's also a fundamental element of proper argumentation.

Louis on February 15, 2008 7:55 AM

I gave my wife your recommended reading list as part of my Christmas list and she was kind enough to get me "The Inmates are Running the Asylum." It was my first introduction to Alan Cooper and completely changed my perspective on how I do my job as a software engineer and de facto designer (but not a very good one). Thanks for the recommendation.

Wade Kaple on February 15, 2008 8:14 AM

I think it's a mistake to portray software engineers as resistant to good interaction design. We all want our hard work to get used. The trouble with interaction design is that it requires the organization as a whole, especially including upper management, to have some clue about good software engineering practices. Good interaction design is a higher order activity which depends on requirements and use cases, in other words it requires an organization which knows how to decide what it will build. Unfortunately, most organizations can't get past a juvenile emotional need for instant gratification, and since the time required for good interaction design defers that, it's seen as an obstacle rather than a tool. Production of code becomes the only emotionally justifiable activity among engineers whose management and culture care about little other than the ship date and can't understand why rushing makes things later. So don't blame the engineers. We just work here.

Pete on February 15, 2008 8:16 AM

love that book!

maybe UI and interaction designs are best taught with examples?

chakrit on February 15, 2008 8:20 AM

Actually, the guy you're quoting agrees with me. I should have read the transcript before posting. He writes:

"Then you enlist programmers: you tell them you will fight for quality if they implement your designs. You can say to them, 'I will go to management and fight to get you the resources to determine how much its going to cost and how you are going to accomplish these problems, if when you build it, you will follow my specifications.' I believe programmers have been waiting for someone to make this deal, and I’m convinced management won’t make this deal. You’re giving them what they want: the ability to be great craftsmen."

This is almost exactly right. My only quibble with it is that it's not a three-way deal. Engineering would be more than happy to implement good interaction design if only Management understood its value. The deal needs to be between Design and Management, and Engineering can chime in briefly if necessary with "what that Design guy said."

Pete on February 15, 2008 8:33 AM

What is an "unit test"? Seriously, we don't use them where I work and they are't used in web development. You should see the user interface for the application I'm working on. For example, the scheduling form has three rows of buttons labeled "S20","S16","A00" etc which are the codes for vehicles. The main window form is just a menu on the left and a huge image on the right with a bad aspect radio. No other information is ever displayed in this large screen area.

Robert S. Robbins on February 15, 2008 8:54 AM

I've never met an Interactive Designer, or wait... maybe all my managers are interactive designers. However I think they don't measure what clients want.

Now unit tests are great, you know exactly what percent pass and what fails.

When I start seeing actual numbers from an Interactive Designer, such as whar % of users can solve their test cases. Then we'll become great friends!

Anyways one of my hobbies is to imagine I know how to design useful stuff. So normally when I make some small internal app, I'll write a short guide and let someone around the office try it while I remain in silence and take notes of their actions. Normally by the third iteraction I can find a good balance, but unfortunately not enough to get good solid numbers. But the point of iterating is a strong point for an agile development.


boredguyathome on February 15, 2008 8:54 AM

Two or three comments noted that Unit Testing is no substitute for User Acceptance Testing and that both are equally important.

They missed the point entirely. Both are worthless if the interaction design sucks. (Any Lotus Notes user can testify to that - http://lotusnotessucks.4t.com/index.html)

And because Interaction Designers are so rare in the wild, every developer that develops any kind of user interface must strive to become a better interaction designer, knowing that they represent a conflict of interests. And every Business Analyst the designs user interfaces has all the more responcibility to excel at great interaction design.

Jeff Schwandt on February 15, 2008 8:55 AM

Could anyone get the video to play past about 2m20s? I tried multiple times but it just hung at that point every time.

Dave on February 15, 2008 9:13 AM

When I start seeing actual numbers from an Interactive Designer, such as what % of users can solve their test cases. Then we'll become great friends!

See that? Right there. The whole "It must have a number target that we can obsess-compulse about!"
That's the 50 foot marshmallow man talking.
Trying to put numbers on human competence is hard. Does mildly annoying count as half a virtually unusable, or a third?
Not to mention that the whole point of interactive design is to make sure that the user doesn't have to "solve" anything.

Tom on February 15, 2008 9:26 AM

You make it sound like low-level, internal testing is an activity mutually exclusive of high-level, external interaction design.

They both are important and one shouldn't be done to the exclusion of the other.

Would you buy a home that had a beautiful design, great layout and flow, tasteful treatments and decorating, but the internal framing, plumbing and wiring was shoddy and broken?

David Avraamides on February 15, 2008 9:26 AM

I'm with the folks that think this is a pretty poor post. It looks like Jeff may have submitted a first draft with the ideas sketched out but not fully thought through.

Neil on February 15, 2008 9:32 AM

Perhaps we should ensure that we begin writing the user help material whilst the interaction design is still in progress, and bring the document authors more fully into the design process?

Whilst I realize the sad reality that the documentation will rarely be read, it can be an easy way to spot an awkward design when a specific section of a user guide becomes overly complex.

Phil Fearon on February 15, 2008 9:54 AM

What? Posts on "focus", "password reveal", and now this?

It's nice weather in the Bay Area, go for a walk, ride the merry-go-round at Tilden!

Steve on February 15, 2008 9:56 AM

Ooops, merry go round is closed for renovation. :-(

http://www.ebparks.org/news/12272007

Steve on February 15, 2008 10:11 AM

Actually, Jeff's point is further proven by many of the comments bitching about the quality of the post, whether it's accurate, or what the effing semantics are.

Come now.

The point is that most developers miss the forest for the trees. We focus most of our thoughts and discussions on the intricate things we (rightly) do to ensure technical quality, and very few on the core issue of quality user experiences.

He's not saying don't unit test, pinwads. He's saying do that, but all of us probably should be more focused on improving how we build software that users love than we should on writing code that all greens.

We're collective decent or at least interested in one and (trust me) we don't even have a clue about the other.

Grant on February 15, 2008 10:16 AM

Jeff is right on the ball here, and I'm disappointed (but not surprised) that so many people are not merely disagreeing but attacking him for presenting the point.

You can create a meticulously tested, 100% correct application that is nonetheless a piece of junk that nobody would want to use. It happens all the time. More insidiously, you can create a highly-tested and reliable application which solves an important need and is heavily used but does it in a way that makes it frustrating and painful to use. It does not cut it as a rebuttal to claim that "most programmers" only write the server portion of the code, or that the UI designers will magically make all the user interface code appear, no doubt with the help of the good UI fairies. It takes programmers to write it.

I'd like to recommend a book which does an even better job at getting the message across. When I was hunting through books on interaction design this past fall, the best one I found was a href="http://www.amazon.com/Sketching-User-Experiences-Interactive-Technologies/dp/0123740371/"iSketching User Experiences/i/a by Bill Buxton. Rather than simply preaching at you about how UI is important, it preaches and ithen/i shows you what the role of prototyping or "sketches" are in design, shows you why it's not adequate to do just one sketch or prototype (virtually guaranteed to have everybody it's shown to grunt reluctantly "yeah, I guess that would work") and then gives you a bunch of techniques you can use to help work up UI design sketches, with or without programming. If you're only going to buy one book on UI design, I'd recommend that one. If you're only going to buy two, I'd recommend that plus a href="http://www.amazon.com/GUI-Bloopers-2-0-Interactive-Technologies/dp/0123706432/"iGUI Bloopers 2.0/i/a to remind you to stay clear of the "obvious" mistakes that virtually every programmer and every company make over and over. (If you're going to buy 3, add iDon't Make Me Think/i or a href="http://www.amazon.com/About-Face-Essentials-Interaction-Design/dp/0470084111/"iAbout Face 3/i/a.)

Clifton on February 15, 2008 11:10 AM

There are developers and developers, designers and designers.

I have bumped into interaction designers that created dark-gray with brow text UI that you could not read on Windows. Because they designed the UI on Photoshop for Mac. But a Mac is not a PC, and Photoshop has 5 smoothing options of text rendering, none of them matching the OS ones.
Or asking for fonts that don't exist on a typical machine, and don't support any foreign language.
As user, I have UI font preferences as a global OS setting. If one designer wants to force a certain font on me with owner-draw stuff, I hate it. And no quick keys (most designers are mouse types).

As developer I am ready to implement 100% of the design, as long as the guy is really UI designer, not just a poster-creator-moved-to-ui-design-because-this-is-where-the-money-is.

Mihai on February 15, 2008 12:04 PM

Aren't you mixing here two absolutely different things: 1) the market potential of particular software, i.e. WHY something should be done; 2) code quality assurance of particular product, i.e. HOW something should be done?
IMHO, Test-driven development is a nice way how to (at least try to) ensure the 2), it has nothing to do with the 1).

Martin on February 15, 2008 12:42 PM

java beats c++...

hahahahaaa...

LOL on February 16, 2008 3:07 AM

We’re constantly asking for permission from the folks who shouldn’t be in a position to grant permission. We should be working with business folks and marshaling the technology to meet the solutions to business problems.

The simplest solution to the above is to *start* cutting code. And why shouldn't developers be in the position to grant permission? The statement contradicts Alan's earlier sentiment about interaction designer being a facilitator, not authority figure.

And why do developers need to communicate through intermediaries with their stakeholders? This will ensure developers drift toward irrelevancy - just look at Microsoft being taken over by sales all across units - it's no longer a developer company.

And lastly - yes developers should learn about user interface design and pass the most important test (not unit test - apple and oranges) - user acceptance. Honestly - computers are much harder than all these soft skills; if one can master computers, one can master UI designs and can manage the communication (although it can be vexing).

The forest here is - pick up another skill besides working with computers.

foobar on February 16, 2008 3:12 AM

Wow, that's one big fat bald guy selling snake oil. Love the hand gestures.

Err ... what? Oh yes. Code coverage -- completely pointless, unless it's done in an academic environment with some stupid language like Z. You want that to work? Try a functional language. (I don't program in a functional language, partly because nobody asks, but mostly because it makes my head hurt.) Doesn't work unless you can guarantee no side-effects.

Unit tests, yes, yummy. We used to do those, back in the days. Then we did integration tests. Then we did acceptance tests. Now we have Professional Testers. These people are Good. On the whole, they don't have a clue how the system works, and they're not very useful when it comes to explaining what went wrong. And they couldn't rewrite the code if their lives depended upon it.

But that does save me, and countless other thousands of programmers out here, from having to write requirements, test specs, tests, and automated scripts built off scraping last night's SVN update. For that, I thank these people.

I never wanted to be a know-it-all, anyway.


Puurvert on February 16, 2008 5:13 AM

"unit testing is used for checking code quality."

No, it's not, Jan. Unit testing is (with luck, spit, and the wind behind you) designed to show that the code works. The quality is entirely besides the point. That's let until later; or, hopefully (and rarely) earlier.

That's why we call them Unit Tests.

Do you see my point, here?

If not, do you look good in stockings, high heels, and suspenders?

Puurvert on February 16, 2008 5:21 AM

"We're constantly asking for permission from the folks who shouldn't be in a position to grant permission."

Sorry, but that's crap. An interface designer shouldn't be able to add interface elements at random any more than a salesman should be able to sell products we don't provide (which our salesmen do regularly.) It's counter-productive at best when you have to go back and recode things for new interface changes, and then send out non-tested code? What, the user experience is somehow better when none of the features work?

Frankly, if the software has gotten that far and the interface is that broken the design was bad in the first place.

Telos on February 16, 2008 7:41 AM

Agree with the spirit of the post, if not the actual practice. I take interaction designers more seriously if they have some practical knowledge of the underlying technology. I'm primarily a web developer, so when I ask an "Interaction Designer" if what they're asking for is an HTML Select element, I expect more than a blank stare back at me.

Oscar on February 16, 2008 10:01 AM

Many of the comments (above) regarding Jeff's thought's make Alan's point, perfectly.

John Crockett on February 17, 2008 4:23 AM


Interaction design/UAT is nothing at all to do with unit testing. They are completely orthogonal practices.

Whether the users want to use your app is not any sort of unit test, let alone the ultimate one.

Jim Cooper on February 17, 2008 4:27 AM

@Steve Campbell,

Yes, I've seen a user interaction person "in the wild." We call them UX (for User eXperience.) I'm team lead on a new project and met with UX and documentation to introduce them to the project -- on Monday or Tuesday I'll get estimates from them and will go to my management to ensure that we have budget. Then someone from UX will be assigned to my team.

Other than some cheesy prototypes to prove a concept, no code has been written on this project yet.

ArtK on February 17, 2008 4:53 AM

The negative camp is complaining about how the point of view is presented.

When I read the byline, I thought this might be about something exciting. A shuttle disaster where we were presented with a cautionary tale about something related to Unit Tests? A coding horror perhaps!??!

Yet, Unit Testing has NOTHING to do with this post, aside from being used as an inflammatory statement used to draw people in while some proselytizing goes on and we're all told to go hug the next UI designer we come across and say sorry. Shame on all the commenter’s who got sucked in to saying anything about UAT or Unit Testing.

* Don't think about how people use your software. It's not in your job spec.
* Design complicated things that people can't use without formal training.
* Make your software incredibly complex so you can brag about it.

If I was jaded and cynical, I would think that the posting style is meant to provoke people commenting just to increate site traffic.

FlatTop on February 18, 2008 1:03 AM

simon kim said

"I can't understand what this post say.
somebody tell me what is problem about unit test???"

There is no problem with unit tests. Unit tests should never have been mentioned. Jeff is saying (again) that programmers are not very good at user interface design and tend to ignore it, and get on with coding (which they are supposed to be better at, or at least more comfortable with). This is not exactly a surprise :-)

Like any sort of design, UI/application design is a skill most people are pretty bad at. End users/customers are no better than programmers at this, but at least if they design their UI and hate it they only have themselves to blame :-)

But however valid this point, it has absolutely nothing to do with unit testing.

Jim Cooper on February 18, 2008 6:48 AM


Steve Campbell said:

I'd be interested in a poll to see how many developers have actually

* seen an interaction designer in the wild (do they really exist?)

Yep, seen one (well, a team of them since this was the BBC and they never use one person when a whole team will do). Not sure how qualified any of them where. Judge for yourself:

http://www.bbc.co.uk/iplayer/


* worked with an interaction designer on a project

The above project, in a remote sort of way. I didn't work on the UI

* pooh-pooh-ed an interaction designer's input

No, just laughed when I saw the pink on black colour scheme.

* considered the interaction designer their new best friend

No, although they seemed nice enough people

Jim Cooper on February 18, 2008 7:09 AM

Thanks Jeff

Many commenters are equating interaction design with (user) interface design. Please do read up and see how interaction design is looking at a much larger picture

Asokan Pichai on February 18, 2008 8:15 AM

Puurvert writes:

Wow, that's one big fat bald guy selling snake oil.

That big fat bald guy happens to be someone who's forgotten more about interaction design than you'll ever know...

Dave on February 18, 2008 9:33 AM

At first blush, I thought that the comparison of unit testing to the experience of working with an app was misguided. It didn't seem like the best way to sell the author's thoughts on the matter.

Reading through all of the comments, however, I see how wrong I was. He understands his readers and the mind of the engineer focused on coding a solution that compiles and passes a unit test.

I am reminded of a recent conversation with one of my colleagues. He had just returned from a round table discussion about the transition to a "Live Web". Almost immediately, the conversation went into the definition of "live"... 10sec delay, 1 sec, no delay. Everyone was focused on definition and platforms and such. The conversation had to be brought around to the user on the end end of the line. If there's no user, then who cares.

It is critically important that everyone involved in the creation of a product understand the users' needs. Interaction designers are often the champions of users and therefore must have deeper penetration into engineering ranks. If you are an engineer, then please take a designer out to lunch and become friends. You will both be better in your individual disciplines for it.

Read up on IxD: http://www.ixda.org/about_interaction.php

Dan Harrelson on February 18, 2008 9:34 AM

Guidelines of Select New Furniture
The following are basic guidelines to help you select new furniture pieces:

Finish—Look for a consistent finish, color, and grain in wood surfaces as well as being smooth to the touch. Ask the retailer what the finish will stand up to. Will it endure heat and water spills? It should. Poorer quality furniture may be stained to camouflage imperfections in wood color or grain, whereas a better piece is more likely to have a clear finish that enables you to see the grain of the wood.

Dressers, Chests, and Nightstands—Make sure that the drawers in these pieces feel substantial, slide easily with drawer stops, and fit evenly without gaps or misalignment. If you see staples or nails, this is most likely a sign of lesser quality. Higher-end pieces have solid wood drawer sides and dove-tailed joints as well as dust panels between each drawer.

Structure—Don't be afraid to ask about the construction of the piece of furniture you like. Most wood furniture is built using "floating construction," which means that it allows for expansion and contraction of wood panels in dry or humid weather throughout the seasons. If you pick up the piece, you shouldn't hear a lot of noise due to racking because the frame should be solid. Where the sides and top come together, make sure that it's a tightly assembled with no gaps or glue visible. The back of the piece shouldn't bulge or be out of square, nor should there be unevenness or poor fit in any of the other areas of the piece.

If you know what to look for, you can get better quality furniture for less money. By shopping around and broadening your options, you can find plenty of places like antique stores, wood furniture store, and thrift stores, for instance, www.fuzi-wood-furniture.com, that have good stuff at lower prices. The key to getting your money's worth is shopping carefully to meet your decorating goals. Sometimes it's better to get a a really superior piece, especially if you love it, than settle for a lower quality piece. And sometimes getting a practical, mid-range piece is exactly right for your needs. Once you recognize value, you'll be able to shop wisely and create the dcor of your dreams.

Tom on February 18, 2008 12:00 PM

I'm reading Cooper's "About Face" right now. It's definitely worth a read. The scarey thing about it is that a lot of what he says about the need for interaction design is head-slappingly obvious but I still don't see much evidence of it around me. Even in the things that I produce!

When users apologise to me that they aren't very good at using our application I tell them that the fault is entirely ours for making it too complicated for them to use. They rarely disagree with that statement.

I guess the problem might be education in as much as interaction design is regarded as a 'lesser science' to the traditional programming courses. Been a while since I was at University though, so perhaps I'm wide of the mark.

Steve on February 18, 2008 12:14 PM

Steve Campbell said:

I'd be interested in a poll to see how many developers have actually
seen an interaction designer in the wild (do they really exist)?

No, never seen one. I've heard tales, seen plaster casts of footprints, and have occasionally been down wind of a foul odor, but have never sighted one...

Gunther on February 19, 2008 6:43 AM

Absolutely! A good programmer should not only bend himself on good coding and testing but also consider the benefits you can bring to the users.

Catherine Sea on February 19, 2008 11:03 AM

Off-topic, with apologies: Mr. Atwood, your posts are always good and thought-provoking, and evidently popular in light of all the comments.

However, I am dismayed by the number of inappropriate comments. Don't get me wrong--most of them are fine. But those that aren't are either barely literate, are comment spam (furniture?), or descend into senseless personal attacks (the person who picked on Mr. Cooper's appearance).

I know it's a lot of work, but please consider some editorial effort for your blog's comments. Screen each comment, and don't post anything that isn't relevant and doesn't add something to the discussion. Ban spam and commenters who post personal attacks. Your blog should be an intelligent and civilized place.

John on February 27, 2008 6:07 AM

Puurvert writes:
Wow, that's one big fat bald guy selling snake oil.

Mr. Atwood, why do you allow comments that pick on a person's appearance? How very Junior High. Edit, please, and make this place civil.

John on February 28, 2008 5:38 AM

Ok, folks, let's talk about metaphor and hyperbole, shall we?

"Metaphor (n.) - a figure of speech in which a term or phrase is applied to something to which it is not literally applicable in order to suggest a resemblance."

"Hyperbole (n.) - an extravagant statement or figure of speech not intended to be taken literally, as 'to wait an eternity.'" (Both definitions are from Dictionary.com.)

Jeff is applying both constructs to this post.

His metaphor is that a user choosing not to use an application is a kind of unit test. If users pass over an application because it's UI design is terrible, that's a unit test failure. Essentially, users "unit test," too, but they do so by not using our applications.

His hyperbole is that this form of unit testing trumps all other forms of unit testing. That is, if we can't get users to even USE our applications, then all our other unit tests are inconsequential. This is hyperbole because it's obviously not true. Read through some of Jeff's other posts you'll see that he's a unit testing proponent.

The other point that Jeff makes is that we engineers like to ENGINEER (and by implication we don't like that mushy UI design stuff) and we're a controlling lot. If we ARE actually unit testing, we engineers like to make sure that we get good test coverage, that we've tested everything the best that we can. However, there's one unit test we tend not to cover and that's making sure our UI design makes user WANT to use our apps. Obsessing over unit testing and test coverage when our users won't use our applications is a waste of time (more hyperbole).

I hope this brings some clarity to the posts about how unit testing and UI design aren't related. They aren't, much. But that's not the point Jeff's is trying to make. He's telling us that usability is more important than almost everything else -- maybe even unit testing.

Tom on March 25, 2008 1:47 PM

Unit testing is no substitute for UAT - but that doesn't mean one is more important than another. Both are important.

Anon on February 6, 2010 10:20 PM

Why am I not surprised that most of the replies so far are knee-jerk reactions explaining how design quality has nothing to do with unit tests and unit tests are super-awesome?

You're confirming his point, guys. Tons of programmers obsess mindlessly over unit tests as if it actually is (or ever could be) a confirmation of product quality. Oh, sorry, "code quality".

Do some interaction testing. Watch how users work. Think about how a screen or an application actually needs to be architected, not how you want it to be architected based on the code you've already written and/or created tests for. And for god's sake, throwing it in UAT for users to sign off on is NOT a substitute. UAT is just a stopgap to prevent something from going into production if it's wrong or not ready; it's not and never will be a true quality indicator or useful feedback mechanism, and you're fooling yourselves if you believe otherwise.

Aaron G on February 6, 2010 10:20 PM

I couldn't agree more. I actually posted something very similar to this the other day: http://www.mcdonaldland.info/2008/02/14/the-quadruple-edged-sword/

Software practices are only one part of a whole that determines the success of a given piece of software.

Jason McDonald on February 6, 2010 10:20 PM

I can't understand what this post say.
somebody tell me what is problem about unit test???

simon kim on February 6, 2010 10:20 PM

To Jim Cooper

Thank you..
I understand meaning of this post with your reply..
i've got point of this post..
thank you...

^^!

simon kim on February 6, 2010 10:20 PM

The comments to this entry are closed.