Gee, I Wish I Had Spent More Time Alone With My Computer

I was recently reminded of this Dani Berry quote:

One of the great pioneers of computer and online gaming, Dani Berry died in 1998. He was born 'Dan Bunten,' but underwent what she always referred to as her "pronoun change" in the early 90s. Some of her aphorisms are still frequently quoted by game developers, including "No one ever said on their deathbed, 'Gee, I wish I had spent more time alone with my computer.'"

Strangely enough, Dani is just one of a number of transgendered game designers. But you may remember Dan Bunten as the mind behind 1983's M.U.L.E., revered by many as among the finest computer games ever published.

M.U.L.E. was designed for 4-player multiplayer, a rarity in games of that era:

M.U.L.E. is widely considered considered a groundbreaking classic in multiplayer computer gaming today, but it was widely underestimated at its time. Compared to popular arcade games of the eighties like Arkanoid, Frantic Freddie or The Last Ninja, M.U.L.E. had simple graphics and did not depend on joystick skills. Like chess, it required thinking skills and strategy to win, especially when playing against other human players. M.U.L.E. was somewhat complex to learn, at a time when electronic entertainment was new and simple.

Dani also went on to design Modem Wars, which was arguably the first commercially released network multiplayer game:

Basing the design around head-to-head modem play gave Modem Wars an unparalleled fun factor, but it is ultimately what caused the game to fail. Consumer modems in 1988 topped out at 1200 baud and were expensive peripherals. The networking technology employed by Modem Wars ensured that the low bandwidth wasn't a problem, but the relatively low installed base of modems was. The fact that you had to scare up a friend who also had a modem (and a mouse, as most PCs in those days were controlled via the keyboard) didn't help matters.

Programming has been described as "a private love letter between the developer and the hardware", but the explosive growth of the internet makes me wonder if any developer can truly be alone while using a computer today. It's hard to imagine using a computer-- any computer-- without being immersed in IM, blogs, message boards, email, skype, and a dozen other "always on" communication services.

Dani Bunten spent a significant part of her career writing computer games that encouraged social behavior. It's unfortunate that she didn't live to see the internet fully expose the computer as the social tool she so clearly wanted it to be.

Google Hardware circa 1999

The always entertaining Dan linked to something I hadn't seen before-- an archived page of the Google server hardware circa 1999:

Here it is in all its, uh, glory. I don't know what's more impressive, the two dual Pentium II 300 servers, or the 90gb SCSI drive array made from Duplo blocks. Here's a summary of all the systems listed on that page:

• 2-proc Pentium II 300mhz, 512mb, five 9gb drives
• 2-proc Pentium II 300mhz, 512mb, four 9gb drives
• 4-proc PPC 604 333mhz, 512mb, eight 9gb drives
• 2-proc UltraSparc II 200mhz, 256mb, three 9gb drives, six 4gb drives
• Disk expansion, eight 9gb drives
• Disk expansion, ten 9gb drives

That's a total of:

• 1792 megabytes of memory
• 366 gigabytes of disk storage
• 2933 megahertz in 10 CPUs

Humble beginnings, indeed. I'm trying to remember the first time I used Google, and I think it had to be sometime in 2000. This motley bunch of hardware grew to own the internet in a little more than five years. That's incredible.

Troubleshooting .NET performance using Peanut Butter

Here's some excellent, concise advice on troubleshooting performance in managed code. It all starts with peanut butter, naturally:

My last entry was some generic advice about how to do a good performance investigation. I think actually it's too generic to be really useful -- in fact I think it fails my Peanut Butter Sandwich Test. I review a lot of documents and sometimes they say things that are so obvious as to be uninteresting. The little quip I have for this situation is, "Yes what you are saying is true of [the system] but it's also true of peanut butter sandwiches." Consider a snippet like this one, "Use a cache where it provides benefits," and compare with, "Use a peanut butter sandwich where it provides benefits." Both seem to work... that's a bad sign.

As promised, Rico then provides a prescriptive list of windows performance monitor counters with comments indicating what the values should look like in a healthy app. He also linked to another blog post with a bit more detail specific to .NET memory performance counters which is also worth reading through.

Incompetence Considered Harmful

A research paper from two psychologists at Cornell offers an interesting insight:

For example, consider the ability to write grammatical English. The skills that enable one to construct a grammatical sentence are the same skills necessary to recognize a grammatical sentence, and thus are the same skills necessary to determine if a grammatical mistake has been made. In short, the same knowledge that underlies the ability to produce correct judgment is also the knowledge that underlies the ability to recognize correct judgment. To lack the former is to be deficient in the latter.

We focus on the metacognitive skills of the incompetent to explain, in part, the fact that people seem to be so imperfect in appraising themselves and their abilities. Perhaps the best illustration of this tendency is the "above-average effect," or the tendency of the average person to believe he or she is above average, a result that defies the logic of descriptive statistics.

According to the data presented in this paper, the least competent people are the ones most likely to erroneously think they are competent:

Across 4 studies, the authors found that participants scoring in the bottom quartile on tests of humor, grammar, and logic grossly overestimated their test performance and ability. Although their test scores put them in the 12th percentile, they estimated themselves to be in the 62nd. Several analyses linked this miscalibration to deficits in metacognitive skill, or the capacity to distinguish accuracy from error. Paradoxically, improving the skills of participants, and thus increasing their metacognitive competence, helped them recognize the limitations of their abilities.

That is a paradox indeed, but our profession is rife with exactly this kind of paradox. It has parallels in several areas of software development:

1. Wicked Problems. You can't understand the problem you're trying to solve until you've partially solved it.
2. Iterative development. Users can't fully express what they want you to build until you build a version of the software for them to experience.
3. Extreme skill disparities. The worst software developers are profoundly bad; the best software developers are absurdly good.

According to this paper, it's extremely likely that the authors of The Daily WTF code snippets remain blissfully unaware of the pain they are inflicting on themselves, and everyone else. As I've said before, our biggest challenge is disseminating best practices to other developers. Making fun of incompetence is amusing, but until these developers' skills are bootstrapped to a moderate level, they're going to keep pounding out more and more WTFs. And that's no laughing matter.

John Dvorak, blogging O.G.

Like Steve Broback, I spent many of my formative years in computing reading John Dvorak's magazine column.

I started enthusiastically reading John Dvorak's columns back in 1984, at my first job selling IBM PCs and Mac 128k computers from a storefront in Seattle. I have always enjoyed his candor and attitude despite the fact that he has been so wrong, so many times. I still have the 1984 column where he derides the Macintosh mouse as being like a "joystick" and how it tries to make computing like "a game".

It's true-- John Dvorak was the archetypal cranky blogger, way before blogging was even a glint in Dave Winer's eye. But as Steve wryly notes with a graph of Google search results, Dvorak now plays second fiddle to the very bloggers he derides in his latest column:

The influential bloggers should be defined here. These are people whom you've never heard of, but whom other influential A-list utopianist bloggers all know. I reckon there are about 500 of them. He (or she) influences other like-minded bloggers, creating a groupthink form of critical mass, just like atomic fission, as they bounce off each other with repetitive cross-links: trackback links, self-congratulatory links, confirmations, and praise-for-their-genius links. BOOM! You get a formidable explosion -- an A-bomb of groupthink. You could get radiation sickness if you happen to be in the area. Except for Wired online and a few media bloggers, nobody is in the area, so nobody outside the groupthink community really cares about any of this. These explosions are generally self-contained and harmless to the environment.

One thing is for sure: all those damn blogging kids need to get the hell off Dvorak's lawn. It's fascinating how the web can cause these amazing inversions of power. A guy like Dvorak who "has been pounding the keyboard since the day the World Wide Web came online, and was one of the first and most prolific contributors of ongoing content to the Web" won't even rate the first page of your search results.

(May 2007: Steve posted an interesting update elaborating on the inversion of influence between traditional print media and bloggers.)

VM Server Hosting

My friend Josh Carlisle was kind enough to host this website during my move to California. Josh set me up with a Microsoft Virtual Server slice of Windows 2003 Standard on his Xeon 2.8 server. I'm currently running a WIMP (Windows, IIS, MySql, Perl) configuration which I was able to set up remotely without issue.

Although everything is generally running quite well, and the commit charge is well under 256mb in Task Manager, I am disappointed with VM performance ..again. Intel's Xeon 2.8ghz is basically just a rebranded Pentium 4 2.8ghz, but that's still way more performance than I need. Unfortunately, under actual use, it performs more like a 1.4ghz Pentium 4-- the older version with only 512kb L2 cache! HTTP post operations that used to take under a second take multiple seconds; installs that used to be a minute long take upwards of five minutes, etcetera.

VMs are great for convenience, but the performance cost is quite a bit higher than I expected it to be-- on both client and server. Even if you aren't emulating the x86 processor, the cost of emulating the motherboard hardware is clearly substantial. Particularly for disk and video. I found this list of Virtual Server performance tips, although it's not very server specific-- it's basically the same advice I've seen for Virtual PC. No silver bullet there; get the fastest disks you can afford, dedicate them to VMs, and make sure you have enough memory. Virtual PC guy also has some interesting tips for remote desktop-ing into a virtual server.

Success through Failure

I found this Will Wright quote, from a roundtable at last week's E3, rather interesting:

Will Wright said he's learned the most from games that seemed appealing on paper, but were failures in the marketplace. "I actually ask people when hiring how many failures they've worked on," he said, "and I'm actually more likely to hire someone based on how many failures they've experienced. I think it's the best learning system."

As a developer, the likelihood that you're working on a project that will fail is high. Every failure should be considered a rich opportunity for learning what doesn't work, and why. As Thomas Edison once said:

I remember thinking, rather bitterly at the time, about the story of Thomas Edison's early attempts to come up with the right material for a lightbulb. He had tried a thousand different elements and all had failed. A colleague asked him if he felt his time had been wasted, since he had discovered nothing. "Hardly," Edison is said to have retorted briskly. "I have discovered a thousand things that don't work."

In fact, the difference between success and failure can ultimately hinge on how you handle failure-- as illustrated in this New Yorker article about predicting the success or failure of surgeons:

Charles Bosk, a sociologist at the University of Pennsylvania, once conducted a set of interviews with young doctors who had either resigned or been fired from neurosurgery-training programs, in an effort to figure out what separated the unsuccessful surgeons from their successful counterparts.

He concluded that, far more than technical skills or intelligence, what was necessary for success was the sort of attitude that Quest has -- a practical-minded obsession with the possibility and the consequences of failure. "When I interviewed the surgeons who were fired, I used to leave the interview shaking," Bosk said. "I would hear these horrible stories about what they did wrong, but the thing was that they didn't know that what they did was wrong. In my interviewing, I began to develop what I thought was an indicator of whether someone was going to be a good surgeon or not. It was a couple of simple questions: Have you ever made a mistake? And, if so, what was your worst mistake? The people who said, 'Gee, I haven't really had one,' or, 'I've had a couple of bad outcomes but they were due to things outside my control' -- invariably those were the worst candidates. And the residents who said, 'I make mistakes all the time. There was this horrible thing that happened just yesterday and here's what it was.' They were the best. They had the ability to rethink everything that they'd done and imagine how they might have done it differently."

This should always be a key interview question when you're hiring. Software development is difficult in the best of conditions. You should always be failing some of the time, and learning from those failures in an honest way. Otherwise, you're cheating yourself out of the best professional development opportunities.

On Managed Code Performance, Again

Managed code may be fat and slow, but it fares surprisingly well in Rico's C# port of Raymond Chen's C++ Chinese/English dictionary reader:

Sure, the C++ version eventually outperforms the managed code by a factor of 2x, but what's interesting to me-- and what this graph makes very clear-- is that the point of diminishing returns has set in well before that happens. As Rico notes:

So am I ashamed by my crushing defeat? Hardly. The managed code achieved a very good result for hardly any effort. To defeat the managed version, Raymond had to:

• Write his own file/io stuff
• Write his own string class
• Write his own allocator
• Write his own international mapping

Of course he used available lower level libraries to do this, but that's still a lot of work. Can you call what's left an STL program? I don't think so, I think he kept the std::vector class which ultimately was never a problem and he kept the find function. Pretty much everything else is gone.

So, yup, you can definitely beat the CLR. I think Raymond can make his program go even faster.

It's a pyrrhic victory once you divide the execution time by the development time of a top Microsoft C++ coder*. Now, for certain applications at the very tip of the development pyramid, this tradeoff may still make sense. But that list of apps gets shorter and shorter with every passing day.

* Raymond Chen, who has "fixed more Windows bugs than you've had hot dinners"

Bridges, Software Engineering, and God

Based on the number of times I've seen the comparison come up in my career, you might think that bridge building and software development were related in some way:

[..] my Dad, who is a "real" engineer, is out visiting for a few days. We got talking tonight about the essence of real engineering and attempted to understand if software development is approaching the level of say mechanical or chemical engineering in terms of maturity of the field. (Brad Abrams)

[..] building software is an immature engineering discipline, which most notably shows in our lack of ability to make true black boxes. "Classical" engineering, like building bridges, dams, and other structures, has mastered the art of specifying components to such a degree that they can be described with only a few parameters. In the art of software engineering, we do not have this down yet. (Kees Leune)

"Our standards have inappropriately been lowered by our daily experience," said Ken Jacobs, Oracle's vice president of product strategy. "We have to bring software engineering the kind of maturity we have in building bridges and buildings. We don't expect buildings to fall down every day."

I find these discussions extremely frustrating, because I don't think bridge building has anything in common with software development.* It's a specious comparison. Software development is only like bridge building if you're building a bridge on the planet Jupiter, out of newly invented materials, using construction equipment that didn't exist five years ago.

Traditional bridge-building engineering disciplines are based on God's rules-- physics. Rules that have been absolute and unchanging for the last million years. Software "engineering", however, is based on whatever some random bunch of guys thought was a good idea in the early 1980's. We don't have the luxury of working within a known universe. God didn't invent x86. That makes the comparison with traditional engineering disciplines tenuous at best. More than half of everything I know will be obsolete in ten years; can any civil engineer say that?

In "Computer Science" is Not Science and "Software Engineering" is Not Engineering, B. Jacobs proposes that software is more like math:

So, if physical engineering is applied science, but software design does not follow the same pattern, then what is software design? Perhaps it is math. Math is not inherently bound to the physical world. Some contentiously argue that it is bound because it may not necessarily be valid in hypothetical or real alternative universe(s) that have rules stranger than we can envision, but for practical purposes we can generally consider it independent of the known laws of physics, nature, biology, etc.

The most useful thing about math is that it can create nearly boundless models. These models may reflect the known laws of nature, or laws that the mathematician makes up. Math has the magical property of being able to create alternative universes with alternative realities. The only rule is that these models must have an internal consistency: they can't contradict their own rules. (Well, maybe they can, but they are generally much less useful if they do, like a program that always crashes.)

Software is a lot like math, and perhaps software is math according to some definitions. The fact that we can use software to create alternative realities is manifested in the gaming world. Games provide entertainment by creating virtual realities to reflect actual reality to varying degrees but bend reality in hopefully interesting ways. A popular example is The Sims, a game that simulates social interaction in society, not just physical movements found in typical "action" games.

The hypothesis that software is mathematics is certainly more credible. But like Rory, I'm not even convinced that math and software are all that similar:

When I was growing up, I remember hearing people say things like, "If you like computer programming, then you'll love math." I always thought that these people were absolutely nuts. While there is something intrinsically similar about certain types of math and computer programming, the two are different in many more ways than they are similar.

With math, and I'm not talking about the crazy number-theory math philosophy "Do numbers really exist?" side of things, but with the applied stuff, there are correct answers. You're either correct or you're incorrect.

With coding, the best you can hope for is to do something well. With so many different ways to effect a single outcome, it's up to some very right-brained sensibilities to determine if you've met your goal, as there isn't anybody (except [another more experienced developer]) who can tell you if you're right or not.

If you ignore your right brain, and I'm talking generally about abstraction and aesthetics, then you can slap some code together that might work, but it also might be one hell of a maintenance nightmare. If you focus only on the right brain, you might have something that works, but is so utterly inefficient and personalized that you're the only person on Earth who could make sense of the code and maintain it.

Unlike math, software can't be objectively, formally verified to be correct. Or even good, for that matter.

Software development is unquestionably a profession, but I don't think we can learn as much from the fields of mathematics or traditional engineering as is so often assumed. However, we do have a lot to learn from ourselves. Disseminating best practices to other developers is our biggest challenge, not adapting processes from unrelated industries. I recommend reading through this recent interview with Steve McConnell for his thoughts on how much the field of software development has advanced in the last 10 years-- and how to keep advancing.

* However, it is fun to build bridges in software!

Blogging about Blogging

I've avoided the incestuous nature of blogging about blogging until now, but the topic does come up occasionally. Not everyone is a believer in the utility of blogs; I was a skeptic only two years ago, and Michael Brundage went out of his way late last year to point out that his web site is not a blog. What makes a blog worth reading? I think Rory nailed it with his simple list of qualifications:

• you have to want to write
• you have to believe you have something to say
• you have to have an interesting way of saying it

This is excellent advice, and it cuts to the heart of the question-- you should write blog entries because you are compelled to. If writing a blog entry feels like work to you, or if if you're worried about satisfying anyone other than yourself, then you'll have a difficult time maintaining a blog.

Blogs are interesting because they are honest windows into other people's interests and passions. As it turns out, the world is full of fascinating, extremely smart people. The opportunity to learn what motivates, interests and excites them-- professionally or personally-- is invaluable. And often in a purely practical sense. I've found an answer to a Google query in a blog entry more than once.

I will add two riders to Rory's excellent guidelines:

• you have to be a decent (not great, but decent) writer
  
  I'm not the greatest writer, but I know bad writing when I see it. The deck is stacked heavily against you if you can't meet the basic grammar, spelling, and style rules of readable English. You have to be 10/10 in the other areas to overcome truly bad writing. Unfortunately, writing is a hard skill to develop. People literally spend lifetimes becoming better writers. However, the more writing you do-- and the more input you solicit on your writing-- the better you'll get at it. I also find that people who read a lot tend to be better writers. The next best thing to actually writing is reading a lot of good writers. One of my favorite pieces is Martin Luther King Jr.'s Letter from a Birmingham Jail, which I re-read every year. It's amazing on so many levels.
  
  
• you have to enable blog comments
  
  A blog without comments is not a blog. Period. If there's no two-way communication-- if readers of your blog can't politely point out that you're full of crap-- then whatever you're writing may be great, but it isn't a blog. Without the social dialog of feedback, you're merely publishing-- and publishing is something newspapers have been doing for hundreds of years.