Are You An Expert?

February 18, 2009

I think I have a problem with authority. Starting with my own.

It troubles me greatly to hear that people see me as an expert or an authority, and not a fellow amateur.

If I've learned anything in my career, it is that approaching software development as an expert, as someone who has already discovered everything there is to know about a given topic, is the one surest way to fail.

Experts are, if anything, more suspect than the amateurs, because they're less honest. You should question everything I write here, in the same way you question everything you've ever read online -- or anywhere else for that matter. Your own research and data should trump any claims you read from anyone, no matter how much of an authority or expert you, I, Google, or the general community at large may believe them to be.

Have you ever worked with software developers who thought of themselves as experts, with almost universally painful results? I certainly have. You might say I've developed an anti-expert bias. Apparently, so has Wikipedia; a section titled warnings to expert editors explains:

1. Experts can identify themselves on their user page and list whatever credentials and experience they wish to publicly divulge. It is difficult to maintain a claim of expertise while being anonymous. In practice, there is no advantage (and considerable disadvantage) in divulging one's expertise in this way.

4. Experts do not have any other privileges in resolving edit conflicts in their favor: in a content dispute between a (supposed) expert and a non-expert, it is not permissible for the expert to "pull rank" and declare victory. In short, "Because I say so" is never an acceptable justification for a claim in Wikipedia, regardless of expertise. Likewise, expert contributions are not protected from subsequent revisions from non-experts, nor is there any mechanism to do so. Ideally, if not always in practice, it is the quality of the edits that counts.

6. There is a strong undercurrent of anti-expert bias in Wikipedia. Thus, if you become recognized as an expert you will be held to higher standards of conduct than non-experts.

Let's stop for a moment to savor the paradox of a free and open encyclopedia written by people who view the contributions of experts with healthy skepticism. How could that possibly work?

I'd argue that's the only way it could work -- when all contributions are viewed critically, regardless of source. This is a radical inversion of power. But a radical inversion of power is exactly what is required. There are only a handful of experts, but untold million amateurs. And the contributions of these amateurs is absolutely essential when you're trying to generate a website that contains a page for.. well, everything. The world is a fractal place, filled with infinite detail. Nobody knows this better than software developers. The programmers in the trenches, spending every day struggling with the details, are the people who often have the most local knowledge about narrow programming topics. There just aren't enough experts to go around.

So what does it mean to be an expert, then, when expertise is perceived as impractical at best, and a liability at worst? In a recent Google talk, James Bach presented the quintessential postmodern image of an expert performing -- Steve McQueen in The Towering Inferno:

[turns to fire commissioner] What do we got here, Kappy?
Fire started, 81st floor, storage room. It's bad. Smoke's so thick, we can't tell how far it's spread.
Exhaust system?
Should've reversed automatically. It must be a motor burnout.
Sprinklers?
They're not working on 81.
Why not?
I don't know.

[turns to architect] Jim? Give us a quick refresher on your standpipe system.
Floors have 3 and 1.5 inch outlets.
GPM?
Fifteen hundred from ground to 68, and 1,000 from 68 to 100, and 500 from there to the roof.
Are these elevators programmed for emergencies?
Yes.
What floor are your plans on?
79. My office.
That's two floors below the fire. It'll be our Forward Command. Men, take up the equipment. I want to see all floor plans, 81 through 85.
Gotcha.
[turns to security chief] Give me a list of your tenants.
Don't worry, we're moving them out now.
Not live-ins. Businesses.
We lucked out. Most of them haven't moved in yet. Those that have are off at night.
I want to know who they are, not where.
What's that got to do with anything? Who they are?
Any wool or silk manufacturers? In a fire, wool and silk give off cyanide gas. Any sporting good manufacturers, like table-tennis balls? They give off toxic gases. Now do you want me to keep going?
One tenant list, coming up.
[turns to crew leader] What do we got?
Elevator bank, central core. Service elevators here. Air conditioning ducts, 6 inches.
Pipe alleys here?
One, two, three, four, five.
Have you got any construction on 81? Anything that can blow up, like gasoline, fabric cleaner?
I don't think so.

What does this tell us? I mean, other than .. Steve McQueen is a badass? Being an expert isn't telling other people what you know. It's understanding what questions to ask, and flexibly applying your knowledge to the specific situation at hand. Being an expert means providing sensible, highly contextual direction.

What I love about James Bach's presentation is how he spends the entire first half of it questioning and deconstructing everything -- his field, his expertise, his own reputation and credentials, even! And then, only then, he cautiously, slowly builds it back up through a process of continual learning.

Level 0: I overcame obliviousness
I now realize there is something here to learn.

Level 1: I overcame intimidation
I feel I can learn this subject or skill. I know enough about it so that I am not intimidated by people who know more than me.

Level 2: I overcame incoherence
I no longer feel that I'm pretending or hand-waving. I feel reasonably competent to discuss or practice. What I say sounds like what I think I know.

Level 3: I overcame competence.
Now I feel productively self-critical, rather than complacently good enough. I want to take risks, invent, teach, and push myself. I want to be with other enthusiastic students.

Insight like this is why Mr. Bach is my favorite Buccaneer-Scholar. He leaves us with this bit of advice to New Experts:

• Practice, practice, practice!
• Don't confuse experience with expertise.
• Don't trust folklore -- but learn it anyway.
• Take nothing on faith. Own your methodology.
• Drive your own education -- no one else will.
• Reputation = Money. Build and protect your reputation.
• Relentlessly gather resources, materials, and tools.
• Establish your standards and ethics.
• Avoid certifications that trivialize the craft.
• Associate with demanding colleagues.
• Write, speak, and always tell the truth as you see it.

Of course, Mr. Bach is talking about testing here, but I believe his advice applies equally well to developing expertise in programming, or anything else you might do in a professional capacity. It starts with questioning everything, most of all yourself.

So if you want to be an expert in practice rather than in name only, take a page from Steve McQueen's book. Don't be the guy telling everyone what to do. Be the guy asking all the questions.