The One Trillion Dollar Development Pyramid

Kit George is the program manager for the .NET Base Class Library team. Kit recently posted an entry on the BCL blog describing a solution to a customer problem:

We recently got asked this question by a customer: "In C#, how do I ensure that a string entered into a text box is of the format: letter,number,letter,number,letter,number ?"

The first answer seems to be pretty straightforward: use RegEx! Regular Expressions are a pretty powerful mechanism for matching strings, and seem the obvious choice. However, you've always got to remeber that RegEx, while powerful, is also a pretty hefty mechanism for String matching. When you're looking for complex strings it's often a good choice (since writing the code yourself can be unbelevably tricky), but when what you're looking for is pretty simple (as in this case), then doing your own matching shouldn't be too tough, and is going to perform a lot more solidly.

Kit goes on to illustrate that, indeed, writing a simple per-character string check is faster than a regular expression. There are some caveats with his example, but ultimately it works out to be at least 3 times faster. And performance is incredibly important.. when you're writing base class libraries designed to be used by millions of developers. Therein lies the problem.

The techniques that make sense for the relative handful of developers writing the .NET framework rarely make sense for the vast legions of developers who are building on top of the framework. The guys at the top of the kernel-OS-framework pyramid don't have much in common with the lower (and order-of-magnitude larger!) levels of the coding pyramid. I occasionally run into developers who point to the Linux source code as a model for development, and my question to them is always the same: since when are we writing an operating system? Everyone likes to think that they're working on something fantastically complicated that will be used by millions of developers, but reality is.. somewhat less exciting.

We're all participants, willing or not, in the One Trillion Dollar Development Pyramid.

1. Dozens. The developers working on your kernel should be Dave Cutler-esque geniuses. There aren't many guys playing the game at this level. Copying UNIX over and over is laudable enough, but developing an entire OS architecture from scratch is the ultimate in hardcore development.
2. Hundreds. Those kernel developers support several hundred OS developers, who should be the best of the best, your Raymond Chens, if you will. They create the OS infrastructure that makes everything else possible.
3. Thousands. Those OS developers support a thousand framework developers. Highly talented, handpicked developers building abstract APIs that enable huge productivity gains.
4. Millions. The framework supports millions of developers of wildly disparate skill levels: everything from rank beginner to near-genius. And they're pounding out trillions of lines of code on every imaginable kind of application.

The development techniques used on each level of the pyramid may not have a whole lot in common. Can you imagine IIS 7.0 written in managed C#? The personalities and skills required at each level are quite different as well. Can you imagine working on GenericBusinessApp 3.7 with Dave Cutler? "I've got chunks of guys like you in my stool!"

The real risk here is delusions of grandeur: thinking that you're working at a higher level of the pyramid than you actually are, and choosing techniques that don't make sense for your project. Jon Galloway has an insightful rebuttal of Joel Spolsky's recommendation that every college graduate learn low-level C programming which illustrates this conceit:

Modern programming languages run on top of frameworks. .NET apps use the .NET framework, Java uses J2EE (et. al.), and arguably web apps run on top of a browser / communication communication that constitutes an application framework. The list could go on (GTK, XUL, web service, Flash). Most good frameworks are standards based, and all of them host your solutions so you only solve new problems.

C code, by and large, is not about frameworks. At its best, it uses some libraries and links to some API's. C gives you a toolset that can solve just about any problem, but requires that you solve each and every problem every time. Frameworks were created to obviate solving the same problems in each program you write. Software development has made a steady progression from code to libraries to components to frameworks. Thankfully, you don't need to retrace this path just as you don't need to experience the dark ages to live in the post-renaissance world.

To be productive as a programmer these days, you either need to be learning to get the most out of existing frameworks or helping to build tomorrow's frameworks. To learn to be a productive programmer, you need to learn how to work with frameworks.

Kit George has the privilege of working on a team producing code that will be used by millions of developers. It's his job to make the base classes as performant as possible, even at the cost of readability. We're writing GenericBusinessApp 3.7, which will have a few hundred users at most. Users who are far more concerned with leaving at 5pm every day than they are with the incomparable thrill of using yet another version of GenericBusinessApp. A regex that uses one line of code to validate "letter,number,letter,number,letter,number" may not be faster in processor time, but it's certainly faster in development time-- and for us, that wins every time.

Managing with Trust

Marco Dorantes recently linked to a great article by Watts Humphrey, who worked on IBM's OS/360 project: Why Big Software Projects Fail. Watts opens with an analysis of software project completion data from 2001:

Figure 2 shows another cut of the Standish data by project size. When looked at this way, half of the smallest projects succeeded, while none of the largest projects did. Since large projects still do not succeed even with all of the project management improvements of the last several years, one begins to wonder if large-scale software projects are inherently unmanageable.

There's a strong correlation between project size and likelihood of failure. I'm sure that comes as no surprise; it's a lot easier to build a doghouse in your back yard than it is to build the Brooklyn Bridge. What is surpising is the "radical" management solution he proposes for these large projects: trust.

This question gets to the root of the problem with autocratic management methods: trust. If you trust and empower your software and other high-technology professionals to manage themselves, they will do extraordinary work. However, it cannot be blind trust. You must ensure that they know how to manage their own work, and you must monitor their work to ensure that they do it properly. The proper monitoring attitude is not to be distrustful, but instead, to show interest in their work. If you do not trust your people, you will not get their whole-hearted effort and you will not capitalize on the enormous creative potential of cohesive and motivated teamwork. It takes a leap of faith to trust your people, but the results are worth the risk.

If you don't delegate some measure of trust to your teammates, can you even call it a team? Watts also notes that trusting your team is not a substitute for managing them. Trust shouldn't imply a free pass through the "how ya doin'?" school of feel-good non-management. That's what Paul Vick is complaining about in his defense of the Microsoft Shipit award:

As for the rest of [Joel Spolsky's] article slagging the idea of performance reviews, I can only fall back on Churchill's immortal quote: "Democracy is the worst form of government except for all those others that have been tried." There's no question that performance reviews can have terrible effects, but what's the alternative? Give everyone a pat on the head, say "nice work" and send them off to a nap with some warm milk and cookies? This isn't to say that there aren't better or worse ways to do performance reviews, but it seems cheap to dispatch them without suggesting some alternative.

And he's right. In order to manage a project, you have to objectively measure what your teammates are doing-- a delicate balancing act that DeMarco and Lister call measuring with your eyes closed:

In his 1982 book Out of the Crisis, W. Edwards Deming set forth his now widely followed "Fourteen Points." Hidden among them, almost as an afterthought, is point 12B:

Remove barriers that rob people in management and in engineering of their right to pride of workmanship. This means [among other things] abolishment of the annual or merit rating and of management by objectives.

Even people who think of themselves as Deming-ites have trouble with this one. They are left gasping, What the hell are we supposed to do instead? Deming's point is that MBO and its ilk are managerial copouts. By using simplistic extrinsic motivators to goad performance, managers excuse themselves from harder matters such as investment, direct personal motivation, thoughtful team-formation, staff retention, and ongoing analysis and redesign of work procedures. Our point here is somewhat more limited: Any action that rewards team members differentially is likely to foster competition. Managers need to take steps to decrease or counteract this effect.

Measuring with Your Eyes Closed: In order to make measurement deliver on its potential, management has to be perceptive and secure enough to cut itself out of the loop. Data collected on individual performance has to be used only to benefit that individual as an exercise in self-assessment. Only sanitized averages should be made available to the boss. If this is violated and the data is used for promotion or punitive action, the entire data collection scheme will come to an abrupt halt. Individuals are inclined to do exactly what the manager would to improve themselves, so managers don't really need individual data in order to benefit from it.

If this sounds difficult, well, that's because it is. Managing people is unbelievably difficult. Getting code to compile and pass all your unit tests? Piece of cake. Getting your team to work together? That's another matter entirely. Joel Spolsky commented on Paul's post, elaborating on his position:

The Shipit stupidity replaced a genuine form of employees being recognized for shipping a product (being given a copy of the shrinkwrapped box) with a ersatz form of recognition which made it pretty clear that management didn't even know that employees were already motivated for shipping software. And it's a classic case of gold-starism. It was universally derided by the hard core old-school developers that make Microsoft what it is today.

Joel's problem with the Shipit awards was exactly the pitfall that DeMarco and Lister described. Managerial trust relationships take investment and work; facile shortcuts like the Shipit award undermine this relationship. Even if you're only building a doghouse, avoid taking these shortcuts.

iPod Hacking via Modem

It's the coolest hack in years -- The Sound of iPod:

I got an iPod for christmas. The ipodlinux project was one of the main reasons for my choice and so I started exploring the iPod as far as I was able to. I patched the bootloader and got some basic code to run but there was no way to access any hardware other than the two CPUs yet. To get the LCD, Clickwheel and the harddisk working we needed to reverse engineer the bootloader in the flashrom. But to do that we first had to find a way to get that code. Seems quite impossible without any knowlegde about the IO-Hardware but I found a solution...

His solution? To output the BIOS using sound, old-school modem style! Vitanuova elaborates:

Thanks to Mako, I heard about a remarkable piece of reverse engineering. A reverse engineer (Nils Schneider) wanted to study the firmware of the Apple iPod in order to figure out how to write software that runs on iPods. But he experienced a chicken-and-egg problem: after learning how to write simple programs to run on an iPod, he found that he couldn't figure out how to use the iPod's I/O hardware (in order to extract a copy of the firmwire) without studying the firmwire first to see how Apple does I/O. At the same time, he couldn't study the firmware without first extracting a copy of it.

His ingenious solution was to use someone else's technique for making the iPod squawk and squeak, in order to write a program that output the firmware as a series of sounds (which could then be recorded using a microphone, and analyzed using software on a PC in order to convert them back into a digital representation of the firmware). In effect, he turned the iPod and microphone system into an acoustic modem, and wrote his own modulation scheme for representing data as sound. He wasn't using the iPod's headphone jack; he was making the iPod itself squeak and squawk, using a piezoelectric element somewhere inside the iPod. To protect against background noise, he had to put the iPod and the microphone together inside a padded box, and let them sit for eight hours.

Totally badass. The last hack this clever was the incredible trojan switcheroo the DirecTV guys pulled on the card hackers in 2001:

Last Sunday night, at 8:30 pm est, DirecTV fired their new gun. One week before the Super Bowl, DirecTV launched a series of attacks against the hackers of their product. DirecTV sent programmatic code in the stream, using their new dynamic code ally, that hunted down hacked smart cards and destroyed them. The IRC DirecTV channels overflowed with thousands of people who had lost the ability to watch their stolen TV. The hacking community by and large lost not only their ability to watch TV, but the cards themselves were likely permanently destroyed. Some estimate that in one evening, 100,000 smart cards were destroyed, removing 98% of the hacking communities' ability to steal their signal. To add a little pizzazz to the operation, DirecTV personally "signed" the anti-hacker attack. The first 8 computer bytes of all hacked cards were rewritten to read "GAME OVER".

Truly funny, and masterfully done. A great read.

Remotely Managing Remote Desktop

Some of my coworkers have an annoying habit of remoting into our Win2k servers and never logging out. They also like to do this in pairs, which means nobody else can remote into the machines due to Microsoft's default two-user administrative mode Terminal Services limit. Yeah, I could rclient in, or use remote MMC snap-ins, but sometimes it's just faster to manipulate the GUI via Remote Desktop.

There's a tool to remotely manage remote desktop connections in Win2k, but I couldn't find any equivalent in XP. A little searching turned up Microsoft's Windows Server 2003 Administration Tools Pack which provides server management tools that allow administrators to remotely manage Windows 2000 Servers & Windows Server 2003 family servers. And indeed it does! The kit installs the following tools, which appear under the Start, Programs, Administrative Tools menu:

• Active Directory Domains and Trusts
• Active Directory Management
• Active Directory Sites and Services
• Active Directory Users and Computers
• Authorization Manager
• Cluster Administrator
• Connection Manager Administration Kit
• DHCP
• Distributed File System
• DNS
• IP Address Management
• Network Load Balancing Manager
• Public Key Management
• Remote Desktops
• Remote Storage
• Telephony
• Terminal Server Licensing
• Terminal Services Manager
• UDDI Services
• WINS

All this for the low, low price of nothing. The only part I care about is the Terminal Services Manager, which lets me terminate idle remote desktop sessions from target servers. Take that coworkers!

Pentium-M Home Theater PC

I recently built a new, much lower wattage home theater PC using the Pentium-M processor. The P-M was, until very recently, a mobile-only part. And that's why it's ideal for HTPC duties-- it offers very high levels of performance at an astonishingly modest power draw. For example, per these system power consumption measurements at Tech-Report:

Pentium 4 640 @3.2ghz
125w idle, 211w load

Pentium-M @ 2.4ghz (overclocked)
92w idle, 107w load

So the P-M system consumes 35% less power at idle and 97% less power at load-- while offering comparable performance. The Pentium-M is much more efficient per clock, so it doesn't need to run at exorbitant clock rates. There's a direct correlation between power consumption and heat/noise. The cooler a system runs, the quieter it will be. And quiet is what you want for a HTPC.

In my bench testing of the Pentium-M 1.6ghz, the large, fanless Alpha heatsink never exceeded 50c under Prime95 torture test load, even when left running overnight. And that's totally passive, with virtually zero directed airflow! Try that with a Pentium 4 and you'll create your own personal China Syndrome. Because the P-M runs so cool, I was able to get away with using only two fans in the final system-- the power supply fan, and a dramatically undervolted 60mm exhaust fan:

See the three hard drives in there? Two are mounted under the DVD-R. Here's a breakdown of all the parts.

Core system

• AOpen i855GMEm mobo
• 1.6ghz Pentium-M "Dothan" (2mb L2 cache)
• Alpha PAL8952 P4 heatsink
• Zalman northbridge cooler
• 512mb DDR

Storage

• Samsung Spinpoint 160gb (3)
• Generic 4x DVD-R

Case

• Logisys acrylic HTPC case
• Sparkle 180w MicroATX power supply

Expansion cards

• Radeon 9600 128mb AGP, passively cooled with Zalman ZM-17
• Generic PCI 802.11 b/g adapter
• Hauppauge PVR-350
• AOpen SPDIF bracket

The AOpen mobo offers voltage and bus speed controls; I undervolted the P-M from ~1.3v to ~1.1v, and overclocked it to 1.7ghz. This system, as measured by my trusty Kill-a-Watt, consumes 72w at windows desktop and 81w in Prime95 torture test. Now, that's power draw from the wall; due to inefficiencies in all power supplies, some of that power is immediately lost as conversion heat inside the PSU. A typical power supply is about 70 percent efficient, so the actual power consumption inside the PC is closer to 50w idle / 57w load. Pretty amazing for a fully loaded 1.7ghz PC that performs like a Pentium 4 2.6ghz.

When building a quiet PC, pay close attention to the hard drives and video card you use:

1. Modern video cards are the second largest consumer of power after the CPU; I recommend the Radeon 9600 which is an outstanding blend of respectable DX9 performance and ultra-low power consumption. They are trivial to cool passively. Can you get a faster card? Sure, at double or triple the power budget. Do be careful to avoid the hotter running 9600XT models; 9600Pros are fine but will require a passive heatsink retrofit.
2. Hard drives are the second loudest item in your system after the fans. Unfortunately, very few hard drive manufacturers make low noise a priority, although that is slowly changing. It helps to decouple your hard drives; they are spinning significant chunks of metal at 7,200rpm. I used sorbothane in my case to soft mount the drives and damp vibration. That definitely helps, but it can't silence a fundamentally noisy drive. Buy a quiet hard drive or you'll regret it. I've had excellent results with the Samsung Spinpoints, which are a favorite of silent PC enthusiasts.

Why Is Forever

In Revenge of the Right Brain, Daniel Pink sees a future where being technologically savvy isn't enough :

Few issues today spark more controversy than outsourcing. Those squadrons of white-collar workers in India, the Philippines, and China are scaring the bejesus out of software jockeys across North America and Europe. According to Forrester Research, 1 in 9 jobs in the US information technology industry will move overseas by 2010. And it's not just tech work. Visit India's office parks and you'll see chartered accountants preparing American tax returns, lawyers researching American lawsuits, and radiologists reading CAT scans for US hospitals.

The reality behind the alarm is this: Outsourcing to Asia is overhyped in the short term, but underhyped in the long term. We're not all going to lose our jobs tomorrow. (The total number of jobs lost to offshoring so far represents less than 1 percent of the US labor force.) But as the cost of communicating with the other side of the globe falls essentially to zero, as India becomes (by 2010) the country with the most English speakers in the world, and as developing nations continue to mint millions of extremely capable knowledge workers, the professional lives of people in the West will change dramatically. If number crunching, chart reading, and code writing can be done for a lot less overseas and delivered to clients instantly via fiber-optic cable, that's where the work will go.

But these gusts of comparative advantage are blowing away only certain kinds of white-collar jobs - those that can be reduced to a set of rules, routines, and instructions. That's why narrow left-brain work such as basic computer coding, accounting, legal research, and financial analysis is migrating across the oceans. But that's also why plenty of opportunities remain for people and companies doing less routine work - programmers who can design entire systems, accountants who serve as life planners, and bankers expert less in the intricacies of Excel than in the art of the deal. Now that foreigners can do left-brain work cheaper, we in the US must do right-brain work better.

I can't even remember the last time I spent an entire day doing uninterrupted, solitary programming as I did in the early to mid 90s. I'm in a constant stream of communication, via the internet, intranet, telephone, and even face-to-face meetings with actual human beings. Has software development already become a right brain activity?

I hope the next big thing is a consolidation of what we have learned over the last forty years. I hope the next big thing is the realization that software is not about hours worked, but about care. I hope the next big thing is the gradual understanding that developing good software is not about man-hours and raw labor; but about creativity, inventiveness, and a drive for elegance and beauty. I hope the next big thing is a change in attitude from big vanilla software groups, to small energetic teams. I hope the next big thing is the growth of professionalism and craftsmanship, and the realization that these are the attributes, not documented process or raw manpower, that will make our industry productive, accurate, and respected.

I've suspected for some time that this is already true. But maybe it's just wishful thinking from a software guy who is completely mediocre at math and far better than he needs to be at English. Regardless, the trend is clear. Memorizing the answers to difficult technical questions won't save our jobs. We should be learning how to ask questions instead of answering them. Asking "why" is a more valuable-- and more right-brained-- skillset than merely knowing "how". How lasts about five years, but why is forever.

You Gotta Own It

One of the frustrations I've experienced with offshoring projects is the diminished sense of ownership. We're still responsible for the software put in front of the end users, and yet we're not allowed to put our hands on the code. Instead, we draw UML diagrams, we enter tickets, we send emails back and forth. In short, we do everything except the only work that actually produces a software product: writing code.

As a result, my interest in these projects has dipped precipitously close to zero. Why should I care? I'm a middleman. I have no ownership stake in the project. And neither do the offshore developers, except in terms of fulfilling their contractual obligations. What happens when nobody owns a project? Well, that's why lots of internal software sucks pretty badly. Renters don't take pride in their homes. Only homeowners do.

In their defense, sometimes you make the conscious choice not to own something. Attempting to own everything is another kind of failure mode. However, if you do want great software, you have to let the developers own what they're building. The developers are inevitably the ones who have the most control over the success or failure of the project. Creating an environment where your developers have no emotional attachment to the project they're working on is a recipe for mediocre software-- and job disillusionment.

The notion of "egoless programming" is dangerous and wrongheaded. It's even listed in Robert Glass's Facts and Fallacies of Software Engineering as Fallacy #3: Programming can and should be egoless:

The first best-selling software engineering book was The Psychology of Computer Programming (1971). There was a peculiar idea among the many excellent ideas in that book-- the idea that programming should be egoless. Programmers, the author said, should not invest their ego in the product they were building. Too many programmers, the author said, get so ego-invested in their product that they lose all objectivity. They see error reports as personal attacks. They see review sessions as threats. They see any questioning of their accomplishments as counterproductive.

What's the alternative to an ego-invested programmer? A team player. The team player sees the software product as a team effort and a team achievement. Error reports and reviews and questions become team inputs to help improve the product, not threatening attacks to derail progress.

At first glance, it's hard to argue with the notion of the egoless programmer. Certainly the defensive programmer, caught up in his or her ego, is not open to the changes that he or she must inevitably make to improve the product. But, after further thought, this notion begins to unravel. It's easy to advocate egoless programming, but it's difficult to find find people who can-- or even should-- divorce their ego from their work. Consider the notion of an egoless manager. That idea, of course, is preposterous! The ego of the typical manager is the driving force that makes him or her effective. We can no more divorce ego from the average programmer than we can from the average manager. We can, however, strive to keep that ego under control.

Smart organizations cultivate this sense of ego-driven ownership with with an implied trust: they go out of their way to hire smart people, and then they get out of the way. Microsoft, by all accounts, does this extremely well:

"How's it going, Joel?" he asked. "I hear you've been having some issues with the App Architecture group."

"Oh no!" I said. "Nothing I can't handle."

"Say no more," he said, "I understand." He left. By the next day the rumor had gotten back to me: the App Architecture group was disbanded. Not only that, but each member of the group was sent to a different department at Microsoft, as far apart as possible. I never heard from them again.

I was blown away, of course. At Microsoft, if you're the Program Manager working on the Excel macro strategy, even if you've been at the company for less than six months, it doesn't matter - you are the GOD of the Excel macro strategy, and nobody, not even employee number 6, is allowed to get in your way. Period.

This sends a really strong message. For one, it makes everyone that much more conscientious about their jobs. They can't hide behind the idea that "management approved their spec," since management really didn't look too closely at their spec. All management did was hire smart people and gave them something to do. For another, it makes for an extremely nice place to work. Who doesn't want to be king of their own domain? Software, by its nature, is very easy to divide into smaller and smaller components, so it's always possible to divide up responsibility among people and let people own an area. This is probably THE reason why software people love working at Microsoft.

This is something Peopleware calls the "Open Kimono" approach to management:

This Open Kimono attitude is the exact opposite of defensive management. You take no steps to defend yourself from the people you've put into positions of trust. And all the people under you are in positions of trust. A person you can't trust with any autonomy is of no use to you.

One of my first bosses was Jerry Wiener, who had run a development team for General Electric on the Dartmouth time-sharing project. He later formed a small high-technology company. At the time I came along, the company was about to enter into a contract that was larger than anything it had ever done before. The entire staff was assembled as our corporate lawyer handed Jerry the contract and told him to read it and sign on the last page. "I don't read contracts," Jerry said, and started to sign. "Oh, wait a minute," said the lawyer, "let me go over it one more time."

The lesson here is not that you should sign contracts without reading them (though that may not be a terrible rule in cases where you pay counsel to look out for your interests). If you've got the wrong counsel, you're in deep bananas anyway. Managers who are most proficient at getting the work done are likely to be way out of their depth in evaluating contracts for the work. Reading contracts may be little more than a conceit. Jerry had taken great pains to hire the best counsel he could find. He'd certainly looked over other instances of the man's work. This was not the time to be defensive; it was the time to make it clear to everyone that the boss was assuming and depending on competence around him. It's heady and a little frightening to know that the boss has put part of his or her reputation into the subordinates' hands. It brings out the best in everyone. The team has something meaningful to form around. They're not just getting a job done. They're making sure that the trust that's been placed in them is rewarded. It is this kind of Open Kimono management that gives teams their best chance to form.

If you're not working in an environment where ownership is encouraged, either wrest control from the powers that be, or start looking for another job. You gotta own it.

Is your PC boring?

The "beige box" PC industry has been slow to capitalize on Apple's proven track record of design. I may not be interested in the Mac Mini, but I sure hope it spurs Taiwanese manufacturers* to produce more interesting looking small form factor PCs designs. And some of that will, hopefully, spill over into desktop PCs as well. I've certainly never seen any Taiwanese manufactuer produce any cases as original as these new ASUS Vento models:

Kinda pricey at $149 via NewEgg, but you have to put that in perspective by viewing the vast wasteland of Taiwanese PC cases. Yes, you can get whatever PC case you want, as long as you want either hideous or boring. Don't make me bring up the Doggie case again.

You can read a review of the ASUS Vento 3600 at Extremetech, by the venerable Loyd Case. Loyd's been a fixture in the PC hardware community for ages.

* "Taiwanese manufacturers" is a synonym for "Dell's crack team of product designers".

Multiple LCDs

In multiple monitors and productivity, I proposed three LCD panels as the standard developer desktop configuration. The only thing holding us back was price, and the minor inconvenience of obtaining a second video card to drive the third monitor.

I recently upgraded my home system to match my work configuration. I purchased two of these Rosewill R910 19" panels, and they live up to all the positive user reviews: they're dirt cheap and offer excellent image quality.

I distinctly remember paying around $1000 for a 19" CRT in 1998, and another $900 for a 21" CRT in 2000. For about the same price, I can now get three times the display area, less total power consumption, and crisper image quality in 2D applications. Now that's progress! In my opinion, price is no longer a valid reason to choose a less-productive single monitor configuration.

Sure, you could buy a larger single panel, such as this 1920x1200 Dell model, priced at a reasonable $1,200. But is that really any match for the effective 3840x1024 you'd get with three 19" Rosewill panels for $200 less? Buying large LCDs rarely makes sense because of the exponential increase in price as the size goes up. There's always a price/performance sweet spot, and right now the sweet spot is unquestionably the 19" LCD panels. Another reason to avoid extremely high resolution single displays: Windows is trapped in a bitmapped world. The pitiful, wonky "Large Fonts" mode just isn't cutting it. Until Avalon arrives, with its perfectly scalable PDF-style vector display engine, I can't recommend suffering through traditional win32 apps on a 1920x1440 display.

Now, if you are making the leap to a third panel, some things to consider:

1. You'll need a PCI video card, something like this 128mb 5200fx PCI card. I recommend nVidia because of their superior driver support for multimonitor modes. Ideally you would have nVidia cards in both your AGP and PCI slots, so you only need to load one video driver. Running multiple video drivers from different manfuacturers can work, but can also be a total nightmare.
2. for LCDs, use DVI interfaces whenever possible. I wouldn't even consider buying a panel that lacked a DVI interface. I have the two Rosewill displays running side by side here, one on DVI and one on analog VGA-- nearly an ideal apples-to-apples comparison. The VGA connected panel certainly isn't chopped liver, but it lacks the perfect per-pixel digital crispness I've come to expect from DVI connected panels. It's not a dealbreaker, but always choose DVI if you want the best LCD experience. In particular, try to get dual DVI ports on your primary video card, because I have yet to see a PCI video card with dual DVI ports. This is something PCI Express will fix once it becomes more mainstream.
3. XP has mature support for multiple monitors, but it's not as good as it could be. I would be remiss if I didn't mention the outstanding UltraMon utility. This adds all the "missing" multiple monitor functionality-- it's essential.

I'm increasingly certain that, sometime in the next few years, two LCD panels will be a standard configuration for not just developers and power users, but a sizable percentage of mainstream Dell systems. Why? Well, the same reason that all CPUs will eventually be multi-core-- sometimes there's nowhere to go but sideways.

Improved Unhandled Exception behavior in .NET 2.0

I recently posted a question about console apps and AppDomain.CurrentDomain.UnhandledException -- specifically, why doesn't it work as described in the MSDN documentation? I even filed an official bug report on this. I guess it worked, because Microsoft's Jonathan Keljo was kind enough to explain this behavior in the comments:

Sorry for the confusion. This behavior is actually the design, though the design can be a little convoluted at times.

The first thing to understand is that the UnhandledException event is not an unhandled exception "handler". Registering for the event, contrary to what the documentation says :-(, does not cause unhandled exceptions to be handled. (Since then they wouldn't be unhandled, but I'll stop with the circular reasoning already...) The UnhandledException event simply notifies you that an exception has gone unhandled, in case you want to try to save state before your thread or application dies. FWIW, I have filed a bug to get the docs fixed.

Just to complicate things, in v1.0 and 1.1, an unhandled exception did not always mean that your application would die. If the unhandled exception occurred on anything other than the main thread or a thread that began its life in unmanaged code, the CLR ate the exception and allowed your app to keep going. This was generally evil, because what would often happen was, for example, that ThreadPool threads would silently die off, one by one, until your application wasn't actually doing any work. Figuring out the cause of this kind of failure was nearly impossible. This may be why Jeff thought it worked before...he just always saw crashes on non-main threads.

In v2.0, an unhandled exception on any thread will take down the application. We've found that it's tremendously easier to debug crashes than it is to debug hangs or the silent-stoppage-of-work problem described above.

BTW, on my 1.1 machine the example from MSDN does have the expected output; it's just that the second line doesn't show up until after you've attached a debugger (or not). In v2 we've flipped things around so that the UnhandledException event fires before the debugger attaches, which seems to be what most people expect.

Jonathan Keljo
CLR Exceptions PM

It's good to hear that unhandled exception behavior will be more coherent in .NET 2.0. I can't think of any reason I would want the debugger to attach (or, if no debugger is registered, the .NET crash dialog to appear) before our global unhandled exception event fires!

I knew that exceptions were only handled on the main thread in .NET 1.1; you have to add the handler to any new managed thread you spawn. However, I've also seen the behavior Jonathan describes here:

if the unhandled exception [ occurs in ] a thread that began its life in unmanaged code, the CLR eats the exception and allows your app to keep going

I remember spending an entire day working on winforms drag and drop problems that were a side effect of this unwanted exception behavior-- in retrospect, I guess it's because drag and drop occurs on an unmanaged COM thread. Anyway, through a painful process of elimination, we finally decided that our code was encountering some kind of error and silently returning from functions. I'm looking forward to not going through that in .NET 2.0!