Revisiting "The Fold"

After I posted my blog entry on Treating User Myopia I got a lot of advice. Some useful, some not so useful. But the one bit of advice I hadn't anticipated was that we were not making good use of the area "above the fold". This surprised me. Does the fold still matter?

The fold refers to the border at the bottom of the browser window at the user's default screen resolution. Like so:

Way back in the dark ages of 1996, it was commonly thought that users didn't know how to scroll a web page.

On the Web, the inverted pyramid becomes even more important since we know from several user studies that users don't scroll, so they will very frequently be left to read only the top part of an article.

Thus, it was critically important to cram in as much content in as possible above that fold, as anything below it was invisible to a huge number of users. They didn't know how to scroll, so they would never find it. Jacob Neilsen, renowned usability expert, is the author of the above quote. But he recanted his position in 2003:

In 1996, I said that "users don't scroll." This was true at the time: many, if not most, users only looked at the visible part of the page and rarely scrolled below the fold. The evolution of the Web has changed this conclusion. As users got more experience with scrolling pages, many of them started scrolling.

Scrolling is an example usability versus learnability. It was always my belief that users quickly learned to scroll, otherwise they were permanently crippled as web citizens. If you can't learn to scroll within an hour or so of using the web, you're going to have an awfully stunted experience -- so much so that you're probably better off not using it at all. In short, if you use the web, you know how to scroll, almost by definition. It is a fundamental skill.

Even today, people will cite the ancient, irrelevant rule of The Fold as if it's still law. In fact, I was just talking to a friend of mine who expressed his frustration at dealing with a middle manager who was using the "content must be above the fold" rule as a weapon, and demanding that all page content appear above the fold. It's terribly misguided.

Although thoroughly debunked, there are still some hidden dangers from the fold, and subtlety to how users react to it. As documented by a recent usability study on the fold, there are three specific pitfalls to watch out for:

1. Don't cram everything in above the fold. Users will explore and find your content -- as long as the page "looks" scrollable.
2. Watch out for stark, horizontal lines that happen to line up with the fold. This is the only factor that causes users to stop scrolling, because the page looks done and complete. Instead, have a small amount of content just visible, poking up above the fold. This encourages scrolling.
3. Avoid in-page scroll bars. The standard browser scrollbar is an indicator of the amount of content on the page that users learn to rely on. Placing <iframe> and other elements with scroll bars on the page can break this convention -- and may lead to users not scrolling.

These are excellent guidelines, backed by actual eye tracking and experimental results. You know, science! But how do they apply to me? First, I established where the fold actually was. Per Google Analytics, about 25% of our users are using screen resolutions where the page fold is at about 700 or 800 pixels of height. And remember, browsers have a lot of horizontal chrome that tends to squander that height -- toolbars, status bars, tabs, etcetera. The fold is probably much closer than you think it is.

Next, I looked at the advice I had been given regarding the top of the page. Sure enough, we had a bunch of irrelevant UI at the top that didn't really matter: things like redundant page titles, and two line title entry. We were wasting critical real estate at the top of the page! For the 25% of users who have a 700 or 800 pixel fold, items were pushed down far enough that they might not actually be visible. Worse still, the strong bottom border of the text entry area with the drag slider could possibly align with the page fold itself -- leading the user to believe that nothing is below there and failing to scroll.

It's not only a basic rule of writing, it's also a basic rule of the web: put the most important content at as close to the top of the page as you can. This isn't new advice, but it's so important that it never hurts to revisit it periodically in your own designs.

In treating user myopia, it's not enough to place important stuff directly in the user's eyepoint. You also need to ensure that you've placed the absolute most important stuff at the top of the page -- and haven't created any accidental barriers to scrolling, so they can find the rest of it. The fold is far less important than it used to be, but it isn't quite as mythical as Bigfoot and the Loch Ness Monster quite yet.

Treating User Myopia

I try not to talk too much about the trilogy here, because there's a whole other blog for that stuff. But some of the lessons I've learned in the last year while working on them really put into bold relief some of my earlier blog entries on usability and user behavior.

One entry in particular that I keep coming back to is Teaching Users to Read. That was specific to dialog boxes, which not only stop the proceedings with idiocy, but are their own delightful brand of user interface poison. Fortunately, you don't see dialogs in web apps much, but this sort of modal dialog lunacy is, sadly, becoming more popular in today's AJAX-y world of web 2.5. Those who can't learn from history are doomed to repeat it, I guess.

Having five more years of development experience under my belt, I no longer believe that classic Larson strip is specific to dialog boxes.

The plain fact is users will not read anything you put on the screen.

What we're doing with the trilogy is not exactly rocket surgery. At its core, we run Q&A websites. And the most basic operation of any Q&A website is … asking a question. Something any two year old child knows how to do.

When we launched superuser.com, that was our thirdfourth Q&A website. This one is for power users, and it's the broadest to date, topic-wise: anything dealing with computer software or hardware (that isn't gaming) is allowed.

We've been at this for over a year now, doing nothing but relentlessly polishing and improving our Q&A engine based on community feedback. We're not particularly good, but we do try very, very hard not to suck. I thought surely, surely we must have something as simple as the ask question form down by now.

How foolish I was.

Let's take a look at one recent superuser question. I'm presenting it here as it would have been seen by the user who asked the question, while they were entering it on the ask question form.

Immediately, there's a problem. The question formatting is completely wrong! It's one big jumble of text.

Our formatting rules aren't complicated. You can get a lot done with a bunch of simple paragraphs. We use Markdown, which offers basic formatting conventions that ape ASCII conventions. On top of that, we offer a real-time preview of how your question will look once submitted, directly under the question entry area. But none of that seemed to work for this particular asker, who, apparently, was totally satisfied with obviously broken formatting -- even though a few choice carriage returns would have worked wonders, and been immediately visible in the live preview.

Yes, yes, it inevitably gets whipped into shape through the collective efforts of our legions of community editors -- but that's not the point. It's best if the original asker gets the question formatted right to start with, and it is our job as UI designers to make that outcome as statistically likely as we can.

To that end, we've put a bunch of helpful tools on the ask question page to help users get the formatting right. As UI designers, here's how we see the ask question page:

We've provided a toolbar with a neon pink help button above the question body, and to the right of the question body, we've provided a handy formatting quick reference with a link to the full formatting reference (which opens in a tab / new window by default).

But none of that matters, because here's how the user sees the ask question page:

Or rather, here's everything the user doesn't see.

When I said users don't read anything you put on the screen, I was lying. Users do read. But users will only read the absolute minimum amount of text on the screen necessary to complete their task. I can't quite explain it, but this kind of user myopia is epidemic. It's the same problem, everywhere I turn.

How do we treat user myopia? How do we reach these users? The ask question page is already dangerously close to cluttered with helpful tips, but apparently these helpful buttons, links, and text are all but invisible to a large segment of the user population. Sure, you could argue that Super User tends to attract less sophisticated users, but I see the exact same problem with programmers on Stack Overflow. As new users, a significant percentage of them can't figure out how to format code, even though there's not only a toolbar button that does it for you, but help text on the right explicitly describing how to do it manually. (Just indent 4 spaces. Spoiler alert!)

More and more, I'm thinking we need to put the formatting help -- for new users only -- directly in their line of sight. That is, pre-populate the question entry area with some example formatting that is typical of the average question. Nothing complicated. But at least then it'd be in the one -- and apparently the only one -- place myopic users are willing to look. Right in front of their freakin' faces.

The next time you're designing a UI, consider user myopia. You might be surprised just how myopic your users can be. Think long and hard about placing things directly in front of them, where they are not just visible, but unavoidable. Otherwise they might not be seen at all.

The Interview With The Programmer

If the internet has perfected anything, it's the art of the crappy, phoned-in, half-assed email "interview". For all those who have bemoaned the often pathetic state of internet journalism, when it comes to interviews, you're largely correct. The purpose of most of these interviews is quick and dirty content filler with semi-famous folk spouting off whatever random thoughts they happen to have in their head at that exact moment. The Nixon Interviews, it ain't.

That's why I'm normally not a huge fan of interview books, because interviews take an enormous amount of time and an enormous amount of legitimate, skilled journalistic effort to get right. Almost nobody does.

Imagine my surprise when Coders at Work: Reflections on the Craft of Programming turns out to be that wonderfully rare intersection of uncommonly skilled interviewing and 15 of the most influential programmers to ever touch a keyboard.

Yes, this is the same book Joel recently recommended in his controversial Duct Tape Programmer entry, which is why I was all the more skeptical. But he's dead on. I could (and probably will, knowing me) fill a year worth of blog posts just with the thought provoking quotes and two-paragraph insights revealed in these interviews. It's astonishingly good. If, after reading what these brilliant programmers have to say, you aren't motivated to research some programming topic mentioned inside, pack it in, because you aren't even trying any more.

I also realized Coders at Work can potentially serve as a job interview filter. If the next programmer you interview can't identify at least one of the programmers interviewed in Coders at Work and tell you roughly what they're famous for …

… I'd say that's an immediate no-hire.

Incidentally, I saw the first Stack Overflow user reference on page 265, in the interview with Simon Peyton Jones, who mentions one Norman Ramsey. Hmm, I thought, that name sounds awfully familiar. And indeed it was!

I would be remiss if I did not mention that the author, Peter Seibel, was directly inspired by Susan Lammers' classic 1986 book Programmers at Work: Interviews With 19 Programmers Who Shaped the Computer Industry.

This is one of my absolute favorite musty old computer books for many of the same reasons. As sources of inspiration go, this one is particularly … er, inspired. Programmers at Work isn't just the archetypal programmer interview book -- it also holds up amazingly well for a book that is over twenty years old. It is a testament to the timelessness of not just code, but the art of coding, as exemplified by these 19 programmers. I believe Peter has legitimately crafted a modern remake that will be relevant for another twenty years. And I hope I don't have to tell you how extraordinarily rare that is among technical books.

(Some -- but not all from what I can tell -- key interviews from Programmers at Work were placed online last year by the author. So if you want to get a flavor of the book, check it out.)

Another notable recent collection of interviews is Masterminds of Programming: Conversations with the Creators of Major Programming Languages.

Although I definitely enjoyed this book, there's something about the focus on programming languages and interview style that didn't quite grab me as forcefully as Coders at Work did. Also, if we're going to do languages, I'd like to see a bit broader representation -- perhaps a Volume II with Smalltalk, Ada, Pascal and so on?

These books are a potent reminder that computers are mostly a reflection of the people using them. In the art of software development, studying code isn't enough: you have to study the people behind the software, too.

The State of Solid State Hard Drives

I've seen a lot of people play The Computer Performance Shell Game poorly. They overinvest in a fancy CPU, while pairing it with limited memory, a plain jane hard drive, or a generic video card. For most users, that fire-breathing quad-core CPU is sitting around twiddling its virtual thumbs most of the time. Computer performance is typically limited by the slowest part in your system. You'd get better overall performance by building a balanced system and removing bottlenecks.

One of those key bottlenecks -- and in my experience, the one typical users are most likely to underestimate -- is the hard drive. I've been a long time advocate of having a small, fast 10,000 RPM boot drive for your OS and key applications, and a larger, slower secondary drive for storage. The two drive approach is a smart strategy, regardless of your budget.

Hard drives may not be particularly sexy bits of hardware, but we're on the verge of a major transition point in storage hardware -- from physical, magnetic platters to solid state memory. I was an early solid state (SSD) drive adopter with my last laptop purchase, and it was a profound disappointment. Those first and second generation SSD drives turned out to be slower than their magentic equivalents, despite the eager promises of vendors. On top of that, they were incredibly expensive, and of limited capacity. Running Windows Vista on an early 32 gigabyte SSD was an exercise in pain and frustration on so many levels. What's not to love? A lot.

I eventually sold that SSD and replaced it with a traditional hard drive. I had grown deeply disillusioned with SSD drives. That is, until I read Linus Torvalds' report on the Intel X-25 SSD:

I can't recall the last time that a new tech toy I got made such a dramatic difference in performance and just plain usability of a machine of mine.

The whole thing just rocks. Everything performs well. You can put that disk in a machine, and suddenly you almost don't even need to care whether things were in your page cache or not. Firefox starts up pretty much as snappily in the cold-cache case as it does hot-cache. You can do package installation and big untars, and you don't even notice it, because your desktop doesn't get laggy or anything.

So here's the deal: right now, don't buy any other SSD than the Intel ones, because as far as I can tell, all the other ones are pretty much inferior to the much cheaper traditional disks, unless you never do any writes at all (and turn off 'atime', for that matter).

At that moment, SSD drives came of age. And by that I mean, they began to justify their hefty price premium at last. But that was almost a year ago, and even the Intel drives -- as great as they were -- had some teething problems. Not to mention that price and capacity were still ongoing concerns.

But when Intel introduced the second generation, mainstream X25-M drives, that's when I knew SSDs were poised to go mainstream. Now, those drives are still anything but cheap, at $289 for 80 GB and $609 for 160 GB. But they offer more performance than the original X25-E that Linus reviewed, at about half the price, with hardware fixes to address the fragmentation issue that plagued the original model.

Intel was the only game in town for about a year, but fortunately for us consumers, the competition finally caught up. The new Indilinx controller models, such as this Crucial 128 GB SSD, are just as fast as the X25-M. And, best of all, they're cheaper, while also offering a not-insubstantial bump to 128 GB of storage!

I picked this model up for $325 plus tax and shipping. And, frankly, I was blown away by the performance difference compared to the 300 GB Velociraptor I had in my system before. That drive is not exactly chopped liver; it's incredibly fast by magnetic platter drive standards. But it's beyond slow next to the latest SSDs. See for yourself:

This is just an excerpt, browse the reviews for more detail, but I was astonished how often this drive (based on the Indilinx Barefoot controller) topped the charts. Suffice it to say, the performance increase is not subtle. All those little pauses while your system pulls some chunk of data off the hard drive? They simply cease to exist.

How much do I like this drive? I like it so very much I bought one for every member of the Stack Overflow team, as a small gesture of thanks for enduring new feature crunch mode. I couldn't sell my old Velociraptor on eBay fast enough.

In my humble opinion, $200 - $300 for a SSD is easily the most cost effective performance increase you can buy for a computer of anything remotely resembling recent vintage. Whether you prefer the 80 GB X25-M SSD or the 128 GB Crucial SSD, it's money well invested for people like us who are obsessive about how their computer performs.

Trust me, you will feel the performance difference of a modern SSD in day to day computing. That's far more than I can say for most of today's CPU and memory upgrades. The transition from magnetic storage to solid state storage is nothing less than a breakthrough. It's already transformative; I can only imagine how fast, cheap, and large these drives are going to be in a few years. So, if you've ever wondered what performance would be like if everything was in RAM all the time -- well, we just got one giant step closer to that.

The Xanadu Dream

Links are the fundamental building blocks of the web. And every time I click on one, I can't help recalling the odd visionary who came up with the original idea of clickable links in text, aka hypertext, in 1963 -- Ted Nelson.

Ted Nelson is, shall we say, a character. He has gone on record many times with the four maxims that guide his life. He isn't shy about sharing them with anyone he meets:

1. most people are fools
2. most authority is malignant
3. God does not exist
4. everything is wrong

And that, in a nutshell, is pretty much everything you need to know about Ted Nelson. He is the archetypal borderline autistic, non-conformist, free-thinking technologist. Any resemblance between Ted and your average programmer is, I'm sure, completely coincidental. That's why his story is so fascinating to me. It hits close to home.

Like most programmers, Ted's reach often exceeded his grasp. Ted's vision of hypertext was far more grandiose than the motley assortment of links that is today's web. His vision was (and is) a bit of computer history lore that has become the stuff of legend: Project Xanadu.

Xanadu, a global hypertext publishing system, is the longest-running vaporware story in the history of the computer industry. It has been in development for more than 30 years. This long gestation period may not put it in the same category as the Great Wall of China, which was under construction for most of the 16th century and still failed to foil invaders, but, given the relative youth of commercial computing, Xanadu has set a record of futility that will be difficult for other companies to surpass. The fact that Nelson has had only since about 1960 to build his reputation as the king of unsuccessful software development makes Xanadu interesting for another reason: the project's failure (or, viewed more optimistically, its long-delayed success) coincides almost exactly with the birth of hacker culture. Xanadu's manic and highly publicized swerves from triumph to bankruptcy show a side of hackerdom that is as important, perhaps, as tales of billion-dollar companies born in garages.

Among people who consider themselves insiders, Nelson's Xanadu is sometimes treated as a joke, but this is superficial. Nelson's writing and presentations inspired some of the most visionary computer programmers, managers, and executives - including Autodesk Inc. founder John Walker - to pour millions of dollars and years of effort into the project. Xanadu was meant to be a universal library, a worldwide hypertext publishing tool, a system to resolve copyright disputes, and a meritocratic forum for discussion and debate. By putting all information within reach of all people, Xanadu was meant to eliminate scientific ignorance and cure political misunderstandings. And, on the very hackerish assumption that global catastrophes are caused by ignorance, stupidity, and communication failures, Xanadu was supposed to save the world.

The above text is excerpted from the definitive 1995 Wired article on Project Xanadu, which is still as electrifying to read today as it was then. The hubris and sheer scale of the Xanadu dream are at turns both inspiring and desperately, hopelessly out of touch.

Xanadu has 17 rules defining its behavior. As you're reading through this list, ask yourself how many of these rules are satisfied by the current world wide web, even in part:

1. Every Xanadu server is uniquely and securely identified.
2. Every Xanadu server can be operated independently or in a network.
3. Every user is uniquely and securely identified.
4. Every user can search, retrieve, create and store documents.
5. Every document can consist of any number of parts each of which may be of any data type.
6. Every document can contain links of any type including virtual copies ("transclusions") to any other document in the system accessible to its owner.
7. Links are visible and can be followed from all endpoints.
8. Permission to link to a document is explicitly granted by the act of publication.
9. Every document can contain a royalty mechanism at any desired degree of granularity to ensure payment on any portion accessed, including virtual copies ("transclusions") of all or part of the document.
10. Every document is uniquely and securely identified.
11. Every document can have secure access controls.
12. Every document can be rapidly searched, stored and retrieved without user knowledge of where it is physically stored.
13. Every document is automatically moved to physical storage appropriate to its frequency of access from any given location.
14. Every document is automatically stored redundantly to maintain availability even in case of a disaster.
15. Every Xanadu service provider can charge their users at any rate they choose for the storage, retrieval and publishing of documents.
16. Every transaction is secure and auditable only by the parties to that transaction.
17. The Xanadu client-server communication protocol is an openly published standard. Third-party software development and integration is encouraged.

It is instructive, then, to consider the primary ways in which the modern web is functionally broken:

• Link rot. The odds of a hyperlink working are inversely proportional to the age of that hyperlink. Old links frequently break, because the server hosting the content disappears for any number of reasons over time. I've gotten to the point where I dread clicking on links from old web pages, because the per-click success rate is so abysmally low.
• Every website has unique usernames and passwords. There is almost no reliable centralized form of identity on the internet, and those few that do exist are often poisoned by explicit commercial affiliation, such as Facebook Connect and Microsoft Passport. This is why curated anonymous, lightweight participation dominates the net -- best illustrated by Wikipedia.
• No redundancy. If content is driven offline by temporary high traffic levels or, worse, catastrophic data loss, there may not be any way to recover that content. I know that Digg has services which auto-mirror highly rated links because Digg traffic can be so toxic to the destination links. (Ironically, all duggmirror.com links redirect to amazon.com now, which illustrates how ephemeral all this stuff tends to be.) I suppose if you're lucky the wayback machine will eventually pick up a historical copy of the content, or the Google cache will hold a copy for some unknown amount of time while the site is offline. You'd probably have better odds praying for missing content to reappear.

Let's put aside the more ambitious parts of Project Xanadu for the moment. The current world wide web does basically one thing: simple, stupid, mindless hyperlinks. But even that alone was enough to build a functional and useful internet for the world. And Google was able to build a zillion dollar algorithm out of discovering the relationship between those dumb hyperlinks.

All that, when the most fundamental building block of the web, the hyperlink, barely works at all. Hyperlinks are fraught with peril and pitfalls even under the best of conditions. The current state of hyperlinking is almost literally the stupidest thing we could build that works. Frankly, the current system sucks beyond belief, as Ted himself notes:

HTML is precisely what we were trying to prevent -- ever-breaking links, links going outward only, quotes you can't follow to their origins, no version management, no rights management.

The next time you're about to embark on a grandiose, glorious software Xanadu Dream of your own, take Dare's advice.

The bottom line is that a lot of the time it's OK to create a solution that solves 80% of the problem. Always remember that shipping is a feature.

Consider the reality of what's actually possible, what people can understand, and what us all too human programmers can practically implement. It might not be the Xanadu you dreamed of -- heck, it might even suck -- but it'll at least have a fighting chance of existing in reality rather than fantasy.