I <3 Steve McConnell*
Coding Horror
programming and human factors
by Jeff Atwood

January 26, 2006

Google is the Help Menu

Jensen Harris recently cited some Microsoft Office usability research which produced a rather counter-intuitive result:

One of the most interesting epiphanies I've had over the last few years seems on the surface like a paradox: "help" in Office is mostly used by experts and enthusiasts.

How can this be? I think my biased assumption was that experts know how to use the software already and eager novices would be poring over the documentation trying to learn how to be more effective using it.

Yet, in usability tests we see it again and again: novices and intermediates click around and experiment, experts try to reason things out and look them up in help.

Dispelling myths based on actual usability data. I love it. Jensen offers a few potential explanations for this phenomenon:

  • Only experts know the right "magic words" to search for.
  • Users need a broader scope of help: not a recipe, but a community college course in cooking.
  • Help requires a context switch-- it's difficult to look at help side-by-side with the software.

While there's definitely merit to these observations, Jensen misses the biggest one. You need only read through a few of the many post comments to see what it is:

It's been my experience that Office help often doesn't include anything useful, so I don't use it. For example, I was using Access and somehow became aware of the DoCmd object. It seemed like it would do what I want, but there is no help for it. How could this object with 100 different features have no online help? Luckily Google came to the rescue, pointing me directly to the microsoft.com page telling me all about it. Why wasn't the included in the help originally?

I once found an Excel function in the help that required I install one of the add-in tools to use it, but the help didn't mention this add-in at all. So I did a google search and found out what I needed to do to use this function.

I have to agree that the online help in Office is not the best. It is a great product, but when I need help I usually give up after examining the first 50 hits and the move directly to Google.

Regarding relevance, I would suggest adopting Google's algorithms for ranking. A Google search for "Excel operators" yields what I was looking for ("About calculation operators") as the second hit from Microsoft, with another relevant document (differences between 1-2-3 and Excel operators) being first. Excel 2002's help gives rankings 4 and 14 respectively, with most of the other hits having nothing whatsoever to do with operators.

I practically never use online help. However, I literally live and die by Google Groups. I am almost never the only person to encounter a problem or misunderstand a feature and Google Groups usually proves a much faster route to success.

I have stopped using help in Office and just rely on Google. The replies are more relevant and better sorted, and it's faster. Even if the answer turns out to reside on the Office Assistance site, it's easier to find it there with Google.

It seems to me that the best "help file" for your software is no help file at all. It's a GoogleMSN Search query.

Local help simply can't compete with internet search. I'll go even further-- if you are building local help files for your application, you're wasting your time. And more importantly, you are wasting your users' time.

Smart developers will stop wasting time on useless local help and build community around their product on the internet. One of my favorite examples of this is http://www.regular-expressions.info. It's the top hit for a Google search on the term "regular expressions". The author, Jan Goyvaerts, is also the author of RegexBuddy, a fantastic regular expression tool. The tool is great, but when I need help with my regular expressions, I don't bother with the local help file. I just type what I'm looking for into MSN Search and bring the full weight of billions of dollars of search optimization to bear on the entire internet. If I happen to be offline, Jan has thoughtfully provided both PDF and CHM versions of the content on regular-expressions.info.

Local help seems rather quaint in comparison.

This kind of online community can easily be linked from the UI, too. One of the few generally helpful "help" features is contextual point-and-help -- eg, clicking on a "what the heck does this do?" button. But even those should spawn generic search engine queries to be truly useful.

If you really want to help your users, set up a wiki for your product. Set up forums for them to participate in. Become an impartial nexus for all information related to your product, whether your company wrote it or not. And most importantly, have the guts to give your users control over these sites. You may have written it, but they have to live with it.

Posted by Jeff Atwood    View blog reactions

 

« Visual Design Patterns VSLive! 2006 »

 

Comments

That's very nicely put - knowing the right "magic words" to ask...

Often finding the solution to a problem - in help or via search - requires knowing what's wrong. One of the greatest GSCP (<a href="http://weblogs.asp.net/fmarguerie/archive/2006/01/18/435837.aspx)">http://weblogs.asp.net/fmarguerie/archive/2006/01/18/435837.aspx)</a> skills is the selection of keywords.

Even experts can't know the answer to everything. They just need to know how to find the answer quickly.

Jon Galloway on January 27, 2006 10:21 AM

"If you are building local help files for your application, you're wasting your time."

Rubbish. The people who were frustrated with the Office help did try to use it -- whacking F1 is as simple as it gets, after all -- it's just that the help system of recent Office editions sucks! That's a specific problem with Office, not a general problem with local help files.

As far as I'm concerned: if your application has no local help file I'll assume you're a lazy bum who wants to offload documentation onto your users, and that means no sale from me.

Chris Nahr on January 27, 2006 10:45 AM

As a counterexample, look at <a href="http://www.freetextbox.com/">FreeTextBox</a>. They've done everything you suggest: forum, wiki, etc. And it's utter disaster. You'd think that FTB would inspire some community but the documentation is beyond piss-poor.

Bill Brown on January 27, 2006 11:56 AM

> That's a specific problem with Office, not a general problem with local help files

Really? For every app I use, I have found the "Google Help" to be far, far better than the built-in help. The only time I find integrated help useful is for ultra-specific "what does this radio button mean" type of questions.

Of course, the best solution of all is to make your app so intuitive and discoverable that nobody needs help in the first place.

> You'd think that FTB would inspire some community but the documentation is beyond piss-poor

Interesting. Can you cite examples? The last time I used FTB, I found lots of online examples for anything I wanted to do. In contrast, we used a competing product on a MS project recently, and I really struggled with it because there's virtually nothing available on it online.

Jeff Atwood on January 27, 2006 12:12 PM

QT3 represent.

Help in revent versions of Office sucks -- let's all agree on that. The best part about it is how the table of contents comes up in the side pane, and clicking on the items brings up a seperate window, which can then be hidden under the Word window. Great stuff.

I think the major problem is still the context switch. I don't think non-expert users can deal with that. Have Word and IE open at the same time? Can you do that?

However, Visual Studio 2005, which I've only played with briefly, appears to be exactly what you're talking about -- the help system is online by default, has links to Microsoft forums, and all that.

XPav on January 27, 2006 12:37 PM

So I bought this software that lets me connect to the internet, but I can't seem to get it to work. Luckily the help files are online. Doh!

I still think some combination of local help and community is useful. In general, people don't write about how to do the easy things as much as they do about the hard things. I sometimes find the answer in help files faster than in Google. Other times, I don't.

I really think you need to provide some help files AND build a community. Documentation as whole is almost always piss poor. So why not focus on both?

Haacked on January 27, 2006 01:08 PM

When office moved their help online, it added a two second delay to every refresh. It drives me crazy.

Remote help is for big questsion, local, snappy, fast help for looking up the name of a function or a key shortcut.

TV on January 27, 2006 01:51 PM

> When office moved their help online, it added a two second delay to every refresh.

That's true, but on the other hand it's WAYYY faster to search MSDN ( http://msdn.microsoft.com/ ) through Google (or MSN Search) than it is to use MSDN's built-in search functions.

Jeff Atwood on January 27, 2006 03:21 PM

Here's an amusing anecdote about MSDN help:

http://blogs.msdn.com/vstsue/archive/2005/05/24/421546.aspx

"Once you find a topic you like, see what’s around it. We did a usability study once and found that almost no one uses the table of contents to find what they’re looking for in the docs. Unfortunately, this is one of the BEST ways to find things."

They also recommend using an external search engine to search MSDN rather than the integrated search. AND they recommend asking experts and looking at blogs.

They pretty much recommend everything except using MSDN. Which goes to show just how deep the rot is. And in a sense, it's like pagerank: can you really trust the parent company's content to be objective about itself? Not really. The best sources of help are sometimes the best BECAUSE they're external sources. Internal help just has too much baggage.

Jeff Atwood on January 27, 2006 03:33 PM

Jeff -

Yes. YES! This is so true. My favorite development tool, Delphi, could really benefit from a help system that let the community get involved.

Nick Hodges on January 27, 2006 04:22 PM

Well, since you asked, here's the Wiki front page:

<a href="http://wiki.freetextbox.com/default.aspx/FreeTextBoxWiki.HomePage">http://wiki.freetextbox.com/default.aspx/FreeTextBoxWiki.HomePage</a>;

Here's the FTB_API page, which one would think would tell one about how to access the API and what sorts of things might be there:

<a href="http://wiki.freetextbox.com/default.aspx/FreeTextBoxWiki.HomePage">http://wiki.freetextbox.com/default.aspx/FreeTextBoxWiki.HomePage</a>;

Here's the "Major FreeTextBox JavaScript Functions":

<a href="http://wiki.freetextbox.com/default.aspx/FreeTextBoxWiki.MajorFreeTextBoxJavaScriptFunctions">http://wiki.freetextbox.com/default.aspx/FreeTextBoxWiki.MajorFreeTextBoxJavaScriptFunctions</a>;

And they are definitely only a subset of the major ones if you look through the source, which we have the ability to do because we paid for it. Woe to anyone who doesn't shell out the dough.

And this page about creating custom toolbar items tells you oh-so-helpfully how to create a button that inserts the phrase "FreeTextBox is great!!" into your editor. Try to do anything more complex than that and it's to the forums.

The forums, however, are by far the worst part of the FreeTextBox community. Do a search for something pretty common, like say setting a default font for the design-view body:

<a href="http://www.freetextbox.com/forums/Search/SearchResults.aspx?q=body%20font%20style&f=&u=">http://www.freetextbox.com/forums/Search/SearchResults.aspx?q=body%20font%20style&f=&u=</a>;

The developer consistently suggests that the solution is to create a special CSS file and then link it to the FTB variable designModeCss, even though the Javascript suggests that designModeBodyTagCssClass or even designModeBodyStyle would be more appropriate.

Nowhere is it ever answered about how to change the dropdown lists for font and size to something other than the browser defaults. Every time, the same suggestion to use designModeCss is given.

Am I bellyaching? Maybe, but I see this same sort of pattern repeated whenever I use third-party products. I can't remember the last time I saw adequate community-produced documentation. I guess it's better than nothing, but that's not saying much.

Sadly, there's almost nothing available outside of freetextbox.com about the product.

Bill Brown on January 27, 2006 04:53 PM

I don't know if I agree. I think if you would have picked Visual Studio, or Emacs, or the Windows command shell utilities (findstr, for, etc.) you would have found the online help is very good and often easier to use then searching the web.

Office is a ripe target because its help sucks. But to use it as an example to generalize across all software is a bit extreme.

The people who tend to read this blog are techy and comfortable googling for help. They are good at figuring out the right search terms and wading through the search hits and homing in on the relevant ones. But I'd guess we also tend to be looking for the kinds of information you won't find in help files (like figuring out what registry keys Excel uses). And we also tend to just use the UI to figure things out rather than online help (like seeing what file formats you can save an Excel file as by looking at the File Save dialog).

But to say that writing a good help file is a waste of time is just wrong. That help file can be posted on the web as well as packaged with your app. Then it can serve your users whether they look for help in the app or online.

Your point about building a community is valid - but I'd say the way to go about it is setup a message board (like phpBB) and then make the commitment to answer questions to get the ball rolling. This doesn't obviate writing a good help file.

David Avraamides on January 27, 2006 11:53 PM

The pitiful support for FTB is the reason why I took it out and started using FCKEditor. Of course, the damned programmers for that product don't seem to know what a pop up blocker is, so the image gallery doesn't work unless you enable popups for your site.

I even tried fixing it myself, but the code for the image gallery is so obtuse (whoever worked on that part didn't have a clue about what they were doing) I'd have to rewrite it from scratch.

foobar on January 29, 2006 12:20 AM

Alan Cooper, and probably lots of others, have pointed out that applications and help systems should be targeted towards the "intermittent power user" because that is generally the largest category of users. Beginners might learn better with a super detailed help system and tutorials and experts do well with "keyword searches" but the fact remains that people quickly move out of "beginner mode" and are frustrated with too-helpful interfaces. And few people will spend enough time with an application to become an expert.

So this finding that help systems are generally most useful to beginners and expert users might explain why there is such a large community of "helpful intermittent power users" for most apps. The largest group of users has no use for the help system!

By the way, just before I read your post I wrote this on why MSDN help is the way it is: <a href="http://micahdylan.com/archive/2006/01/29/WhateverhappenedtoMicrosoftBob.aspx">http://micahdylan.com/archive/2006/01/29/WhateverhappenedtoMicrosoftBob.aspx</a>;

Micah Dylan on January 29, 2006 03:23 PM

> Beginners might learn better with a super detailed help system

How so? According to Jensen's usability studies, beginners don't use help in Office, eg, "Help is for experts".

> you would have found the online help is very good and often easier to use then searching the web.

Oh yes, MSDN help is truly wondrous..

http://www.codinghorror.com/blog/archives/000451.html

9 times out of 10 the Google results are better. The MSDN page *could* be the top hit-- but it shouldn't be unless it truly deserves the top position.

That's online help vs. local help in a nutshell.

Jeff Atwood on January 30, 2006 01:14 AM

Thanks for the FCKeditor tip.

We were trying to use FreeTextbox, but $199 USD for the source code, I don't think so, and the documentation: Horrible.


James Wilson on March 8, 2006 06:02 PM

>> Beginners might learn better with a super detailed help system

> How so? According to Jensen's usability studies, beginners don't use help in Office, eg, "Help is for experts".

So what? That's for *Microsoft Office*. It's a case study for *Microsoft Office*, not a broad study of user interaction with help systems across software in general. Just because many help systems suck, that doesn't mean that this study explains all our frustrations and 'proves' us right.

FWIW, most programs I use have extensive and useful help, and it's rarely worth the time to check google for help and weed out all the irrelevant results. (Sometimes it is, but with good documentation, that's the exception rather than the rule)

Mork on March 2, 2007 02:33 PM

Visual Studio has excellent documentations in MSDN
considering I don't have a internet connection at home I find
google useless there. Lispworks doc is also great.
The question is they you shouldn't just write some reference , rather
put some tutorials in.
Products that come without documentation are just demo crup
even experts need to learn from somewhere, if you don't make any docs development team is the only source, beside trial and error.

cheers
bobi

slobodan on March 26, 2007 07:59 AM

--
I often say "a very smart person will say 'I don't know' and ask for help far more often than a very dumb person." And I think it applies here as well: The "experts" read the manual and consult the help files far more often than the beginners!
--

http://weblogs.sqlteam.com/jeffs/archive/2007/04/05/F1-.-Studies-have-shown-this-key-is-the-least.aspx

Jeff Atwood on April 9, 2007 07:52 PM

Jeff,

You win:

<a href="http://www.google.com/microsoft">http://www.google.com/microsoft</a>;

don on January 23, 2008 03:49 PM







(hear it spoken)


(no HTML)




Content (c) 2008 Jeff Atwood. Logo image used with permission of the author. (c) 1993 Steven C. McConnell. All Rights Reserved.