August 3, 2005
Joel has a lengthy entry in which he asks, does having the "best programmers" really matter?
This is something I've talked about before: extreme skill disparity is unique to the profession of software development. The odds of working with a genius or a jackass on any given job are about fifty-fifty.
What's worse is that there's no correlation between experience and skill, either. I've worked with amazingly talented interns and 20 year vets who produced terrible code. Joel provides additional data to support this hypothesis:
The data I rely upon comes from Professor Stanley Eisenstat at Yale. Each year he teaches a programming-intensive course, CS 323, where a large proportion of the work consists of about ten programming assignments. The assignments are very serious for a college class: implement a Unix command-line shell, implement a ZLW file compressor, etc.
There was so much griping among the students about how much work was required for this class that Professor Eisenstat started asking the students to report back on how much time they spent on each assignment. He has collected this data carefully for several years.
I spent some time crunching the data; it's the only data sets I know of where we have dozens of students working on identical assignments using the same technology at the same time. It's pretty darn controlled, as experiments go.
Here's a representative sample for a single CS 323 assignment, showing time spent versus grade/score:
There is absolutely no correlation whatsoever between time spent on the problem and the resulting score for these CS students. It's not much of a stretch to extend this to time spent in the profession of software development.
The rest of Joel's post veers dangerously close to being self-serving-- look at my awesome company and the smart employees I hired! but at least he's finally acknowledging that he is really talking about a very narrow niche:
It's not just a matter of "10 times more productive." It's that the "average productive" developer never hits the high notes that make great software.
Sadly, this doesn't really apply in non-product software development. Internal, in-house software is rarely important enough to justify hiring rock stars. Nobody hires Dolly Parton to sing at weddings. That's why the most satisfying careers, if you're a software developer, are at actual software companies, not doing IT for some bank.
Since Joel's commercial shrink-wrap software development is a tiny percentage of the overall IT market, does this mean that having the best programmers doesn't matter most of the time?
Sadly, I believe the answer is yes.
Posted by Jeff Atwood
I actually think this might not be unique to software. For example, it might apply to writing. :-)
Having the best programmers matters when having the best code matters. Sadly, the best code doesn't usually win; instead, having the best business idea wins. And sometimes having the best marketing.
Business thinking and marketing are two other fields where the distribution of skill is non-linear, maybe as much so as programming. So you have to have the best business people, the best marketing people, AND the best programmers to win.
As a developer who's been in the game for more than 20 years, and who has been doing business app development for a little more than half of it, I think there is a lot of good work being done in the non commercial setting. I've done my share of "real programming" and a lot of the time was spent battling with marketing on features and less than realistic time frames to deliver the next great version. In the 7 or so years that I did commercial software, I met some very smart folks. I also met some tremendous knuckleheads. The same goes in the business app development world. I agree with your initial point: it's about 50 - 50. And as someone who (by absolute choice) has been doing development for a bank for 3 years, I think I've got a very satisfying career. Plus rock solid stability excellent hours. I'm not maintaining old COBOL junk, but working on pretty cutting edge stuff (SWT Eclipse RCP)...so as usual, I agree with your POV and find Joel's to be...well...typical for Joel.
So if your business plan includes having great software,
Only *software* companies have this business plan, though. And what percent of the overall software development job market is that?
Most companies have business plans like "sell lots of widgets at really high profit margins". Software is just a distant side-effect for these companies.
Jeff, I want to ask you something about that CS 323 assignment you talked about.
So say you look at all who got above 55 in score. How many of those students are A students, B student, C students. Are the ones who spend 30, 40 and 50 hrs on the project B or C students and because they work hard they get the A grade.
I personally know A students from both ends of the spectrum. The one who works really hard, takes a lot more time and does a comparable work to the one who's really smart, has been coding since he was 10 and can do the same work assinged to the first in 1/2 the time.
While Joel's point is kinda funky, I can concede that there is some truth in that there are some people who are really smart and who can code in 1/2 the time that a person who may also be good takes to do.
It is really an art to find those guys who are just exceptional.
One of these exceptional guys I know made a claim to me that if he was given a small team of 5 people like him, he could do a project in 3 months that a team of 10-20 would take 1 year to do. I flatly called him on his bluff pointing out that most teams are not made of idiots (mostly). So i agree with you but I can also see the other side of the fence. It certainly helps to have really smart people but they only give a smaller advantage over normal teams in long term sizable projects.
Do I make any sense or am I just rambling again? :-)
What is this "sadly" stuff I see in this post (and one of the comments)?
Why software development? Do we do it to produce elegant structures and/or the most efficient algorithms - or do we have a business problem to solve?
For the most part, we don't develop code for own jollies - we do it to produce something. Well, the value of that something is, and should be, how nicely we produce it. If you make a really elegant piece of code for something no one wants, what makes you think that it's sad?
If the effect of your elegance and brilliance is that you enable that business idea to get to market, that's super. Ditto if your efficiency lets you beat your competition to market consistently. But, it's a tool - just like marketing activities are tools. The nub of it all , without which all the other stuff falls apart, is the idea.
Now, can you bring a great idea to market with bad software and still be successful? Define "bad". If the performance and functionality are sufficient to buy you enough time and/or funding to get V2.0 out, it's not "bad" - it's just only "good enough" to accomplish the short-term goals and may reflect the trade-offs necessary to get to market. If we waited until all the bugs were out, and all the features we envisioned were in, we'd never get products out the door before the money ran out.
And, to make one more point, what percentage of real-world software development requires a high level of skill? I submit to you that, unless you're trying to develop something bleeding-edge, you really don't - you require a basic level of competence, but not brilliance.
I submit to you that, unless you're trying to develop something bleeding-edge, you really don't - you require a basic level of competence, but not brilliance
This opens the door to a lot of offshoring. When all you need is rote, mediocre "by the numbers" performance, then you can get that at 1/3rd the rate by hiring from India, or some other developing nation with a decent educational system. There's no reason to pay Stanford/Yale/etc graduates large salaries that justify their $60k++ education.
One of these exceptional guys I know made a claim to me that if he was given a small team of 5 people like him, he could do a project in 3 months that a team of 10-20 would take 1 year to do. I flatly called him on his bluff pointing out that most teams are not made of idiots (mostly).
I dunno-- small is generally better, a lot better, if you can maintain quality and commitment:
Small Is The New Big:
I'll go you one step further and claim that product quality is not the determining factor for shrink-wrapped software as well. Hell, it's not a determining factor for non-software products either. That is, a company can have a great product and fail (business-wise) and a company can have a mediocre product and succeed (business-wise) (and the other two combinations work as well). The old saying about better mousetraps is just plain wrong.
That's not to say that having a great product won't help your company. It will. It just won't be the "make or break" factor.
The good observation in Joel's article is that a mob of mediocre programmers (no matter how many) simply can't create great software. Only great programmers can create great software.
So if your business plan includes having great software, then you need to go to the effort to get those great programmers.