Open Wireless and the Illusion of Security

Bruce Schneier is something of a legend in the computer security community. He's the author of the classic, oft-cited 1994 book Applied Cryptography, as well as several well-known cryptography algorithms.

The cheeky Norris-esque design above is a reference to the actor names commonly used in examples of shared secret key exchange.

What I find most interesting about Bruce, however, is that he has moved beyond treating computer security as a problem that can be solved with increasingly clever cryptography algorithms:

Schneier now denounces his early success as a naive, mathematical, and ivory tower view of what is inherently a people problem. In Applied Cryptography, he implies that correctly implemented algorithms and technology promise safety and secrecy, and that following security protocol ensures security, regardless of the behavior of others. Schneier now argues that the incontrovertible mathematical guarantees miss the point. As he describes in Secrets and Lies, a business which uses RSA encryption to protect its data without considering how the cryptographic keys are handled by employees on "complex, unstable, buggy" computers has failed to properly protect the information. An actual security solution that includes technology must also take into account the vagaries of hardware, software, networks, people, economics, and business.

This is the programming equivalent of realizing that Peopleware is ultimately a much more important book than The Art of Computer Programming. The shift in focus from algorithms to people is even more evident if you frequent Bruce's excellent blog, or read his newest books Practical Cryptography and Beyond Fear.

As much as I respect Bruce, I was surprised to read that he intentionally keeps his wireless network open.

Whenever I talk or write about my own security setup, the one thing that surprises people -- and attracts the most criticism -- is the fact that I run an open wireless network at home. There's no password. There's no encryption. Anyone with wireless capability who can see my network can use it to access the internet.

I've advocated WiFi encryption from the day I owned my first wireless router. As I encountered fewer and fewer open WiFi access points over the years, I viewed it as tangible progress. Reading Bruce's opinion is enough to make me question those long held beliefs.

It's a strange position for a respected computer security expert to advocate. But I think I get it. Security is a tough problem. If you take the option of mindlessly flipping a WPA or WEP switch off the table, you're now forced to think more critically about the security of not only your network, but also the fundamental security of the data on your computers. By advocating the radical idea that your wireless network should be intentionally kept open, Bruce is attempting to penetrate the veil of false algorithmic security.

I may understand and even applaud this effort, but I don't agree. Not because I'm worried about the security of my data, or any of the half-dozen other completely rational security arguments you could make against intentionally keeping an open wireless network. My concerns are more prosaic. I desperately want to protect the thin sliver of upstream bandwidth my provider allows me. Some major internet providers are also talking about monthly download caps, too. Bruce's position only makes sense if you have effectively unlimited bandwidth in both directions. Basically, I'm worried about the tragedy of the bandwidth commons. As much as I might like my neighbors, they can pay for their own private sliver of bandwidth, or knock on my door and ask to share if they really need it.

So, to me at least, enabling wireless security is my way of ensuring that I get every last byte of the bandwidth I paid for that month.

It's worth realizing, however, that wireless security is no panacea, even in this limited role. Given a sufficiently motivated attacker, every wireless network is crackable.

With that in mind, here are a few guidelines.

1. WEP = Worthless Encryption Protocol

   WEP, the original encryption protocol for wireless networks, is so fundamentally flawed and so deeply compromised it should arguably be removed from the firmware of every wireless router in the world. It's possible to crack WEP in under a minute on any vaguely modern laptop. If you choose WEP, you have effectively chosen to run an open wireless network. There's no difference.

2. WPA requires a very strong password

   The common "personal" (PSK) variant of WPA is quite vulnerable to brute force dictionary attacks. It only takes a trivial amount of wireless sniffing to obtain enough data to attack your WPA password offline -- which means an unlimited amount of computing power could potentially be marshalled against your password. While brute force attacks are still for dummies, most people are, statistically speaking, dummies. They rarely pick good passwords. If ever there was a time to take my advice on using long passphrases, this is it. Experts recommend you shoot for a 33 character passphrase.

In the end, perhaps wireless security is more of a deterrent than anything else, another element of defense in depth. It's important to consider the underlying message Bruce was sending: if you've enabled WEP, or WPA with anything less than a truly random passphrase of 33 characters, you don't have security.

You have the illusion of security.

And that is far more dangerous than no security at all.

Regular Expressions: Now You Have Two Problems

I love regular expressions. No, I'm not sure you understand: I really love regular expressions.

You may find it a little odd that a hack who grew up using a language with the ain't keyword would fall so head over heels in love with something as obtuse and arcane as regular expressions. I'm not sure how that works. But it does. Regular expressions rock.They should absolutely be a key part of every modern coder's toolkit.

If you've ever talked about regular expressions with another programmer, you've invariably heard this 1997 chestnut:

Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems.

The quote is from Jamie Zawinski, a world class hacker who I admire greatly. If he's telling us not to use regular expressions, should we even bother? Maybe, if you live and die by soundbites. But there's a bit more to the story than that, as evidenced by Jeffrey Friedl's exhaustive research on the Zawinski quote. Zawinski himself commented on it. Analyzing the full text of Jamie's posts in the original 1997 thread, we find the following:

Perl's nature encourages the use of regular expressions almost to the exclusion of all other techniques; they are far and away the most "obvious" (at least, to people who don't know any better) way to get from point A to point B.

The first quote is too glib to be taken seriously. But this, I completely agree with. Here's the point Jamie was trying to make: not that regular expressions are evil, per se, but that overuse of regular expressions is evil.

I couldn't agree more. Regular expressions are like a particularly spicy hot sauce -- to be used in moderation and with restraint only when appropriate. Should you try to solve every problem you encounter with a regular expression? Well, no. Then you'd be writing Perl, and I'm not sure you need those kind of headaches. If you drench your plate in hot sauce, you're going to be very, very sorry later.

In the same way that I can't imagine food without a dash of hot sauce now and then, I can't imagine programming without an occasional regular expression. It'd be a bland, unsatisfying experience.

But wait! Let me guess! The last time you had to read a regular expression and figure it out, your head nearly exploded! Why, it wasn't even code; it was just a bunch of unintelligible Q*Bert line noise!

Calm down. Take a deep breath. Relax.

Let me be very clear on this point: If you read an incredibly complex, impossible to decipher regular expression in your codebase, they did it wrong. If you write regular expressions that are difficult to read into your codebase, you are doing it wrong.

Look. Writing so that people can understand you is hard. I don't care if it's code, English, regular expressions, or Klingon. Whatever it is, I can show you an example of someone who has written something that is pretty much indistinguishable from gibberish in it. I can also show you something written in the very same medium that is so beautiful it will make your eyes water. So the argument that regular expressions are somehow fundamentally impossible to write or read, to me, holds no water. Like everything else, it just takes a modicum of skill.

Buck up, soldier. Even Ruby code is hard to read until you learn the symbols and keywords that make up the language. If you can learn to read code in whatever your language of choice is, you can absolutely handle reading a few regular expressions. It's just not that difficult. I won't bore you with a complete explanation of the dozen or so basic elements of regular expressions; Mike already covered this ground better than I can:

• The absolute bare minimum every programmer should know about regular expressions
• 5 regular expressions every web programmer should know
• Extreme regex-fu: what you need to know to become a regular expression pro

I'd like to illustrate with an actual example, a regular expression I recently wrote to strip out dangerous HTML from input. This is extracted from the SanitizeHtml routine I posted on RefactorMyCode.

var whitelist =
 @"</?p>|<br\s?/?>|</?b>|</?strong>|</?i>|</?em>|
  </?s>|</?strike>|</?blockquote>|</?sub>|</?super>|
  </?h(1|2|3)>|</?pre>|<hr\s?/?>|</?code>|</?ul>|
  </?ol>|</?li>|</a>|<a[^>]+>|<img[^>]+/?>";

What do you see here? The variable name whitelist is a strong hint. One thing I like about regular expressions is that they generally look like what they're matching. You see a list of HTML tags, right? Maybe with and without their closing tags?

Honestly, is this so hard to understand? To me it's perfectly readable. But we can do better. In most modern regex dialects, you can flip on a mode where whitespace is no longer significant. This frees you up to use whitespace and comments in your regular expression, like so.

var whitelist =
 @"</?p>|
   <br\s?/?>| (?# allow space at end)
   </?b>|
   </?strong>|
   </?i>|
   </?em>|
   </?s>|
   </?strike>|
   </?blockquote>|
   </?sub>|
   </?super>|
   </?h(1|2|3)>| (?# h1,h2,h3)
   </?pre>|
   <hr\s?/?>|  (?# allow space at end)
   </?code>|
   </?ul>|
   </?ol>|
   </?li>|
   </a>|
   <a[^>]+>|  (?# allow attribs)
   <img[^>]+/?> (?# allow attribs)
   ";

Do you understand it now? All I did was add a smattering of comments and a lot of whitespace. The same exact technique I would use on any code, really.

But how did I cook up this regular expression? How do I know it does what I think it does? How do I test it? Well, again, I do that the same way I do with all my other code: I use a tool. My tool of choice is RegexBuddy.

Now we get syntax highlighting and, more importantly, real time display of matches there at the bottom in our test data as we type. This is huge. If you're wondering why your IDE doesn't automatically do this for you with any regex strings it detects in your code, tell me about it. I've been wondering that very same thing for years.

RegexBuddy is far and away the best regex tool on the market in my estimation. Nothing else even comes close. But it does cost money. If you don't use software that costs money, there are plenty of alternatives out there. You wouldn't read or write code in notepad, right? Then why in the world would you attempt to read or write regular expressions that way? Before you complain how hard regular expressions are to deal with, get the right tools!

This trouble is worth it, because regular expressions are incredibly powerful and succinct. How powerful? I was able to write a no-nonsense, special purpose HTML sanitizer in about 25 lines of code and four regular expressions. Compare that with a general purpose HTML sanitizer which would take hundreds if not thousands of lines of procedural code to do the same thing.

I do have some tips for keeping your sanity while dealing with regular expressions, however:

1. Do not try to do everything in one uber-regex. I know you can do it that way, but you're not going to. It's not worth it. Break the operation down into several smaller, more understandable regular expressions, and apply each in turn. Nobody will be able to understand or debug that monster 20-line regex, but they might just have a fighting chance at understanding and debugging five mini regexes.

2. Use whitespace and comments. It isn't 1997 any more. A tiny ultra-condensed regex is no longer a virtue. Flip on the IgnorePatternWhitespace option, then use that whitespace to make your regex easier for us human beings to parse and understand. Comment liberally.

3. Get a regular expression tool. I don't stare at regular expressions and try to suss out their meaning through sheer force of will. Neither should you. It's a waste of time. I paste them into my regex tool of choice, RegexBuddy, which not only tells me what the regular expression does, but lets me step through it, and run it through some test data. All in real time as I type.

4. Regular expressions are not Parsers. Although you can do some amazing things with regular expressions, they are weak at balanced tag matching. Some regex variants have balanced matching, but it is clearly a hack -- and a nasty one. You can often make it kinda-sorta work, as I have in the sanitize routine. But no matter how clever your regex, don't delude yourself: it is in no way, shape or form a substitute for a real live parser.

If you're afraid of regular expressions, don't be. Start small. Used responsibly and with the right tooling they are big, powerful -- dare I say, spicy -- wins. If you make regular expressions a part of your toolkit, you'll be able to write less code that does more. It'll just.. taste batter.

You might enjoy them so much, in fact, that you completely forget about that "second problem".

Smart Enough Not To Build This Website

I may not be smart enough to join Mensa, but I am smart enough not to build websites like the American Mensa website.

Do you see the mistake? If so, can you explain why this is a mistake, and why you'd desperately want to avoid visiting websites that make this mistake?

(hat tip to Bob Kaufman for pointing this out)

Revisiting the XML Angle Bracket Tax

Occasionally I'll write about things that I find sort of mildly, vaguely thought provoking, and somehow that writing turns out to be ragingly controversial once posted here. Case in point, XML: The Angle Bracket Tax. I'm still encountering people online who almost literally hate my guts because I wrote that post. You'd think I kicked their dog, or made inappropriate romantic overtures toward their significant other.

Well, first of all, we are talking about XML the markup language, not XML the religion, right?

I hope so. I try not to get emotionally involved with the tools and technologies that I use, if I can avoid it. This doesn't mean I can't be enthusiastic or critical of those tools and technologies, but I'm not married to the stuff either way. Who needs all the emotional baggage?

Obviously I failed to communicate this before. I talked about this a little bit on Stack Overflow podcast #5 with Joel, where I tried to amplify and explain my position a little better.

I wasn't trying to present it as "Oh, XML is bad, let's all switch to this new markup language that all the cool guys are using". What I was trying to say is why don't we think about what we're doing? That's the general theme of a lot of the stuff in my blog. Can we just stop programming for a minute to think about what we're doing and not make a blind choice based on "Well this is what my tool does, so that's what I have to do"?

I think obviously there's pros and cons to each. I'm not saying that one is the right solution all the time. But I think, ironically, that is what is happening with XML. I think people are saying "It's always the right answer, because it can store anything, right? And all the stuff I use uses it, so it must be the right choice for everything." That bothers me a little. Maybe I'm just contrarian. Maybe I'm an iconoclast and I want to try different things and see different things, but I think actually understanding the alternatives helps you understand XML better, a little bit, too.

And I hope people reading my blog would not get the idea that it's about a knee-jerk reaction one way or the other. It's about understanding the tradeoffs and applying those tradeoffs to your particular situation. I think that is the absolute art of programming. It's understanding what you could do, and which one of those things fits your situation best. Versus what so many programmers do, which is "I've learned to use a hammer, and I'm gonna hammer everything." Ultimately, to me, it's about self-awareness.

By the way, I'd like to thank everyone who pitches in to make those Stack Overflow podcast transcriptions possible. It is because of your generously donated time that I am able to quote that audio here.

I don't post stuff to push people's buttons, I post it because I want programmers to think about their tools, their technologies, their methods.

If what I post here seems unnecessarily confrontational sometimes, a far smarter person than myself said it better than I can:

I blog to help others and also to learn. As it turns out both are aided by getting folks to actually read the stuff. Please pardon the necessary devices.

Please do pardon the necessary devices; I find that I often learn best through the smackdown learning model. That works for me. Maybe it doesn't work for you, and that's OK. There are millions of websites to choose from.

That said, I do actually have a problem with XML, or I wouldn't have written anything in the first place. I think there's a real issue here that is, for the most part, being completely ignored. XML fever may not be as debilitating as, say, Dengue fever, but it has side effects as well.

Consider Norman Walsh's Defending the Tax. Norman is an XML Standards Architect at Sun.

On the other hand, the difference between:

fruit=pear
vegetable=carrot
topping=wax

and

<doc>
<fruit>pear</fruit>
<vegetable>carrot</vegetable>
<topping>wax</topping>
</doc>

isn't really that large, is it? (Or maybe you think it is, de gustibus non est disputandum.)

The de gustibus dismissal means Norman considers it is a matter of taste, but it isn't. The difference is large. There is a very real mental cost to parsing even a few short lines of XML.

As a Visual Studio ecosystem programmer, XML is pervasive, in every nook and cranny of a project. Every time I look at my web.config XML file, there's a mental cost of me having to parse all these tags in the file. Here's this tag, which lines up with this tag. Here's this giant, verbose thing where only half of it actually matters.

Sure, it's a small effort. Insignificant, even. But what's the mental cost of that insignificant effort times the number of developers in the world, times the number of projects in the world?

I also posit that these minor headaches may be more significant than you realize. In Stumbling on Happiness, author Dan Gilbert makes a similar assertion.

His research found that people are bad at predicting their own future happiness. They tend to radically overestimate the positive or negative impact of large events in their lives -- losing your job, getting rich, getting divorced, having children. That's generally good; it means we have defense mechanisms in place to adapt and survive in our changing circumstances as human beings. But, we also tend to radically underestimate the impact of the dozens of small events in our lives throughout the day. Thus, small injustices don't trigger our defenses. The effect of that squeaky screen door, the neighbor's barking dog, the interrupting telephone call -- all of these may have far more profound cumulative impact on your day to day happiness than you realize.

It's a fascinating book, and I'm only paraphrasing the smallest part of it. I highly recommend reading it if this is at all interesting to you. It won't exactly unlock the secrets to happiness, I'm afraid, but you may gain a deeper understanding of why we tend to make the choices we do in our neverending pursuit of happiness.

I'm not trying to change the world overnight, but I wouldn't mind planting a few seeds of dissent in people's minds. This small stuff matters.

The next time you're trying to figure out an XML file, just think about it.

That's all I'm saying.

The Ultimate Code Kata

As I was paging through Steve Yegge's voluminous body of work recently, I was struck by a 2005 entry on practicing programming:

Contrary to what you might believe, merely doing your job every day doesn't qualify as real practice. Going to meetings isn't practicing your people skills, and replying to mail isn't practicing your typing. You have to set aside some time once in a while and do focused practice in order to get better at something.

I know a lot of great engineers -- that's one of the best perks of working at Amazon -- and if you watch them closely, you'll see that they practice constantly. As good as they are, they still practice. They have all sorts of ways of doing it, and this essay will cover a few of them.

The great engineers I know are as good as they are because they practice all the time. People in great physical shape only get that way by working out regularly, and they need to keep it up, or they get out of shape. The same goes for programming and engineering.

It's an important distinction. I may drive to work every day, but I'm far from a professional driver. Similarly, programming every day may not be enough to make you a professional programmer. So what can turn someone into a professional driver or programmer? What do you do to practice?

The answer lies in the Scientific American article The Expert Mind:

Ericsson argues that what matters is not experience per se but "effortful study," which entails continually tackling challenges that lie just beyond one's competence. That is why it is possible for enthusiasts to spend tens of thousands of hours playing chess or golf or a musical instrument without ever advancing beyond the amateur level and why a properly trained student can overtake them in a relatively short time. It is interesting to note that time spent playing chess, even in tournaments, appears to contribute less than such study to a player's progress; the main training value of such games is to point up weaknesses for future study.

Effortful study means constantly tackling problems at the very edge of your ability. Stuff you may have a high probability of failing at. Unless you're failing some of the time, you're probably not growing professionally. You have to seek out those challenges and push yourself beyond your comfort limit.

Those challenges can sometimes be found on the job, but they don't have to be. Separating the practicing from the profession is often referred to as code kata.

The concept of kata, a series of choreographed practice movements, is borrowed from the martial arts.

If you're looking for some examples of code kata -- ways to practice effortful study and hone your programming skills -- Steve's article has some excellent starting points. He calls them practice drills:

1. Write your resume. List all your relevant skills, then note the ones that will still be needed in 100 years. Give yourself a 1-10 rating in each skill.

2. Make a list of programmers who you admire. Try to include some you work with, since you'll be borrowing them for some drills. Make one or two notes about things they seem to do well -- things you wish you were better at.

3. Go to Wikipedia's entry for computer science, scroll down to the "Prominent pioneers in computer science" section, pick a person from the list, and read about them. Follow any links from there that you think look interesting.

4. Read through someone else's code for 20 minutes. For this drill, alternate between reading great code and reading bad code; they're both instructive. If you're not sure of the difference, ask a programmer you respect to show you examples of each. Show the code you read to someone else, and see what they think of it.

5. Make a list of your 10 favorite programming tools: the ones you feel you use the most, the ones you almost couldn't live without. Spend an hour reading the docs for one of the tools in your list, chosen at random. In that hour, try learn some new feature of the tool that you weren't aware of, or figure out some new way to use the tool.

6. Pick something you're good at that has nothing to do with programming. Think about how the professionals or great masters of that discipline do their practice. What can you learn from them that you can apply to programming?

7. Get a pile of resumes and a group of reviewers together in a room for an hour. Make sure each resume is looked at by at least 3 reviewers, who write their initials and a score (1-3). Discuss any resumes that had a wide discrepancy in scoring.

8. Listen in on a technical phone screen. Write up your feedback afterwards, cast your vote, and then talk about the screen with the screener to see if you both reached the same conclusions.

9. Conduct a technical interview with a candidate who's an expert in some field you don't know much about. Ask them to explain it to you from the ground up, assuming no prior knowledge of that field. Try hard to follow what they're saying, and ask questions as necessary.

10. Get yourself invited to someone else's technical interview. Listen and learn. Try to solve the interview questions in your head while the candidate works on them.

11. Find a buddy for trading practice questions. Ask each other programming questions, alternating weeks. Spend 10 or 15 minutes working on the problem, and 10 or 15 minutes discussing it (finished or not.)

12. When you hear any interview coding question that you haven't solved yourself, go back to your desk and mail the question to yourself as a reminder. Solve it sometime that week, using your favorite programming language.

What I like about Steve's list is that it's somewhat holistic. When some developers think "practice" they can't get beyond code puzzles. But to me, programming is more about people than code, so there's a limit to how much you can grow from solving every obscure programming coding interview problem on the planet.

I also like Peter Norvig's general recommendations for effortful study outlined in Teach Yourself Programming in Ten Years.

1. Talk to other programmers. Read other programs. This is more important than any book or training course.

2. Program! The best kind of learning is learning by doing.

3. Take programming classes at the college or graduate level.

4. Seek out and work on projects with teams of programmers. Find out what it means to be the best programmer on a project -- and the worst.

5. Work on projects after other programmers. Learn how to maintain code you didn't write. Learn how to write code so other people can effectively maintain it.

6. Learn different programming languages. Pick languages that have alternate worldviews and programming models unlike what you're used to.

7. Understand how the hardware affects what you do. Know how long it takes your computer to execute an instruction, fetch a word from memory (with and without a cache miss), transfer data over ethernet (or the internet), read consecutive words from disk, and seek to a new location on disk.

You can also glean some further inspiration from Pragmatic Dave's 21 Code Katas, or maybe you'd like to join a Coding Dojo in your area.

I don't have a long list of effortful study advice like Steve and Peter and Dave do. I'm far too impatient for that. In fact, there are only two movements in my book of code kata:

1. Write a blog. I started this blog in early 2004 as a form of effortful study. From those humble beginnings it has turned into the most significant thing I've ever done in my professional life. So you should write blogs, too. The people who can write and communicate effectively are, all too often, the only people who get heard. They get to set the terms of the debate.

2. Actively participate in a notable open source project or three. All the fancy blah blah blah talk is great, but are you a talker or a doer? This is critically important, because you will be judged by your actions, not your words. Try to leave a trail of public, concrete, useful things in your wake that you can point to and say: I helped build that.

When you can write brilliant code and brilliant prose explaining that code to the world -- well, I figure that's the ultimate code kata.

Department of Declaration Redundancy Department

I sometimes (often, actually) regress a few years mentally and forget to take advantage of new features afforded by the tools I'm using. In this case, we're using the latest and greatest version of C#, which offers implicitly typed local variables. While working on Stack Overflow, I was absolutely thrilled to be able to refactor this code:

StringBuilder sb = new StringBuilder(256);
UTF8Encoding e = new UTF8Encoding();
MD5CryptoServiceProvider md5 = new MD5CryptoServiceProvider();

Into this:

var sb = new StringBuilder(256);
var e = new UTF8Encoding();
var md5 = new MD5CryptoServiceProvider();

It's not dynamic typing, per se; C# is still very much a statically typed language. It's more of a compiler trick, a baby step toward a world of Static Typing Where Possible, and Dynamic Typing When Needed.

This may be a cheap parlor compiler trick, but it's a welcome one. While writing C# code, I sometimes felt like I had entered the Department of Redundancy Department.

Sure, there are times when failing to explicitly declare the type of an object can hurt the readability and maintainability of your code. But having the option to implicitly declare type can be a huge quality of life improvement for everyday coding, too.

There's always a tradeoff between verbosity and conciseness, but I have an awfully hard time defending the unnecessarily verbose way objects were typically declared in C# and Java.

BufferedReader br = new BufferedReader (new FileReader(name));

Who came up with this stuff?

Is there really any doubt what type of the variable br is? Does it help anyone, ever, to require another BufferedReader on the front of that line? This has bothered me for years, but it was an itch I just couldn't scratch. Until now.

If that makes sense to you, why not infer more fundamental data types, too?

var url = "http://tinyurl.com/5pfvvy";
var maxentries = 5;
var pi = 3.14159;
var n = new int[] {1, 2, 3};

I use implicit variable typing whenever and wherever it makes my code more concise. Anything that removes redundancy from our code should be aggressively pursued -- up to and including switching languages.

You might even say implicit variable typing is a gateway drug to more dynamically typed languages. And that's a good thing.

Coding For Violent Psychopaths

Today's rumination is not for the weak of heart. It's from the venerable C2 Wiki page Code For The Maintainer:

Always code as if the person who ends up maintaining your code is a violent psychopath who knows where you live.

Perhaps a little over the top, but maybe that shock to the system is what we need to get this important point across to our fellow developers.

If scare tactics don't work, hopefully you can develop a grudging respect for the noble art of maintenance programming over time. It may not be glamorous, but it's 99% of the coding work in this world.

Physics Based Games

I've always been fascinated by physics-based gameplay. Even going back to the primeval days of classic arcade gaming, I found vector-based games, with their vastly simplified 2D approximations of physics and motion, more compelling than their raster brethren. I'm thinking of games like Asteroids, Battlezone, and Lunar Lander.

Accurately simulating the physics of the real world has been the domain of supercomputers for decades. The simulation of even "simple" physical phenomena like fire, smoke, and water requires a staggering amount of math. Now that we almost have multicore supercomputers on every desktop, it's only natural that aspect of computing would trickle down to us.

This topic is particularly relevant in light of today's introduction of NVIDIA's newest video card, the GTX 280, which contains a whopping 1.4 billion transistors. That's a lot. For context and scale, here's a shot of the 280 GPU next to a modern Intel dual-core CPU.

I've talked about this before in CPU vs. GPU, but it bears repeating: some of the highest performing hardware in your PC lies on your video card. At least for a certain highly parallelizable set of tasks.

We were able to compress our test video (400 MB) in iPhone format (640*365) at maximum quality in 56.5 seconds on the 260 GTX and 49 seconds on the 280 GTX (15% faster). For comparison purposes, the iTunes H.264 encoder took eight minutes using the CPU (consuming more power overall but significantly less on peaks).

While one of the primary benefits of manycore CPUs is radically faster video encoding, let's put this in context -- compared to the newest, speediest quad core CPU, you can encode video ten times faster using a modern video card GPU. It's my hope that CUDA, Microsoft's Accelerator, and Apple's Grand Central/OpenCL will make this more accessible to a wide range of software developers.

All this physics horsepower, whether it's coming from yet another manycore x86 CPU, or a massively parallel GPU, is there for the taking. There are quite a few physics engines available to programmers:

• Havok
• Newton
• Open Dynamics Engine
• Actionscript Physics Engine
• Farseer Physics Engine
• NVIDIA PhysX
• Bullet Physics

There are no shortage of physics games and sandboxes to play with this stuff, too. Here are a few of my favorites.

Perhaps the most archetypal physics based game is Chronic Logic's Bridge Construction Set, the original version of which dates way back to 1999. I'm showing a picture of their fancy NVIDIA branded version below, but it's hardly about the graphics. This is pure physics simulation at its most entertaining. Who knew civil engineering could be so much fun? Highly recommended.

Oh, and small hint: after playing this game, you will learn to love the power and beauty of the simple triangle. You'll also marvel at the longer bridges you manage to drive across without plunging into the watery abyss underneath.

I've professed my love for The Incredible Machine and other Rube Goldberg devices before. The physics based game Armadillo Run is a modern iteration of same. Get the armadillo from point A to point B using whatever gizmos and gadgets you find in your sandbox -- rendered in glorious 3D with a full-blown 2D physics engine in the background.

The latest physics based game to generate a lot of buzz is Trials 2: Second Edition. I haven't had a chance to try it yet, but the gameplay movie is extremely impressive. Like Armadillo run, the action is all on a 2D plane, but the physics are impeccable.

I'm sure I've forgotten a few physics based games here; peruse this giant list of physics games to see if your favorite is already included.

See, physics can be fun -- and increasingly complex physics engines are an outstanding way to harness the massive computational horsepower that lies dormant in most modern PCs.

Don't Go Dark

Ben Collins-Sussman on programmer insecurity:

What do you do when somebody shows up to an open source project with a gigantic new feature that took months to write? Who has the time to review thousands of lines of code? What if there was a bad design decision made early in the process -- does it even make sense to point it out? Dropping code-bombs on communities is rarely good for the project: the team is either forced to reject it outright, or accept it and deal with a giant opaque blob that is hard to understand, change, or maintain. It moves the project decidedly in one direction without much discussion or consensus.

And yet over and over, I'm gathering stories that point to the fact that programmers do not want to write code out in the open. Programmers don't want their peers to see mistakes or failures. They want to work privately, in a cave, then spring "perfect" code on their community, as if no mistakes had ever been made.

I don't think it's hubris so much as fear of embarrassment. Rather than think of programming as an inherently social activity, most coders seem to treat it as an arena for personal heroics, and will do anything to protect that myth. They're fine with sharing code, as long as they present themselves as infallible, it seems. Maybe it's just human nature.

Ben's talking about open source development, but this anti-pattern exists in commercial software development, too. The very same phenomenon is documented in Jim McCarthy's 1995 book Dynamics of Software Development. It's presented as Rule #30: Don't go dark.

You have to manage the granularity of development tasks in such a way that you emerge with visible deliverables over short intervals. In our group, we argue back and forth over how big the intervals should be: five days, ten days, three weeks? In our world, three weeks is going dark.

I don't know what's appropriate for your world, but we want team members to have contracts with the other parts of the team so that they surface pretty often with visible components. When somebody surfaces and the deliverable isn't done, we know right away. We know that this week we slipped one day. That's worth knowing, much better than getting to the end of the project and observing, "Oh, we slipped six months!" At that point it's too late to even bother counting up how much you've slipped.

Rule #30 is directly followed by a related rule, Rule #31: Beware of a guy in a room.

Specialist developers who lock themselves away in a room, who go dark for long stretches, are anathema to shipping great software on time. No matter how brilliant a developer might be, don't give the developer a significant assignment unless he or she understands and buys into the type of development program you intend to run. The brilliant developer must be capable of performing on a team, making his work visible in modest increments and subjecting it to scrutiny as it matures. Some people find this intolerable, and although there is a role for people of this disposition in the software world, it is not as a part of a team devoted to shipping great software on time.

This is easier to deal with in the workplace, because you typically have some kind of (theoretically) rational project management in place, and everyone works under the same umbrella. It's effectively impossible to go dark if you're practicing any form of agile software development. For example, Ron Jeffries borrowed this concept from Jim McCarthy's book and codified it into extreme programming lore. Tasks are always sliced up so they fit into a single iteration, and you never let them spill over into multiple iterations. You'll always have something to show at the end of each iteration. You can't go dark without quitting the project or, perhaps, your job.

An open source project is a very different animal. It's a motley collection of widely distributed, loosely coupled volunteers. There's no project manager breathing down your neck, urging you to break your work into short, shareable increments. The risk of going dark is severe. The burden of proof falls on the individual developers, not only to make their work on the project visible in modest increments, but also to get over their code insecurity and share their in-progress code with other people working on the project. How do you expect your fellow coders to take you seriously if you aren't regularly showing them code? It's the only form of currency that matters on an open source project.

Don't go dark. Don't be that guy in the room. Hiding your code until it's "done" may feel safer, but it isn't. Sharing your ongoing code with your coworkers is scary, much less the world -- but it also results in feedback and communication that will improve your code and draw you closer to the project you're working on. And isn't that why we all write code in the first place?

ASCII Pronunciation Rules for Programmers

As programmers, we deal with a lot of unusual keyboard characters that typical users rarely need to type, much less think about:

$ # % {} * [] ~ & <>

Even the characters that are fairly regularly used in everyday writing -- such as the humble dash, parens, period, and question mark -- have radically different meaning in programming languages.

This is all well and good, but you'll eventually have to read code out loud to another developer for some reason. And then you're in an awkward position, indeed.

How do you pronounce these unusual ASCII characters?

We all do it, but we don't necessarily think much about the words we choose. I certainly hadn't thought much about this until yesterday, when I read the following comment left on Exploring Wide Finder:

A friend sent me a Java code fragment in which he looped through printing "Thank You!" a million times (it was a response to a professor who had extended the deadline on a paper). I responded with a single line of Ruby to do the same, and a single line of Lisp.

He wrote back: "Underscores, pipes, octothorpes, curly braces -- sheesh... I'll take a mild dose of verbosity if means I don't have to code something that looks like it's been zipped already!"

What the heck is an octothorpe? I know this as the pound key, but that turns out to be a US-centric word; most other cultures know it as the hash key.

I'm often surprised to hear what other programmers name their ASCII characters. Not that the words I personally use to identify my ASCII characters are any more correct, but there's far more variability than you'd expect considering the rigid, highly literal mindset of most programmers.

Perhaps that's why I was so excited to discover the ASCII entry in The New Hacker's Dictionary, which Phil Glockner turned me on to. It's a fairly exhaustive catalog of the common names, rare names, and occasionally downright weird names that programmers associate with the ASCII characters sprinkled throughout their code.

How many of these ASCII pronunciations do you recognize? Which ones are the "correct" ones in your shop?

If you're curious about the derivation of some of the odder names here, there are an extensive set of footnotes (and even more possible pronunciations) at the ascii-table.com pronunciation guide.

So the next time a programmer walks up to you and says, "oh, it's easy! Just type wax bang at hash buck grapes circumflex and splat wane", you'll know what they mean.

Maybe.