I <3 Steve McConnell*
Coding Horror
programming and human factors
by Jeff Atwood

July 12, 2008

Monkeypatching For Humans

Although I love strings, sometimes the String class can break your heart. For example, in C#, there is no String.Left() function. Fair enough; we can roll up our sleeves and write our own function lickety-split:

public static string Left(string s, int len)
{
    if (len == 0 || s.Length == 0)
        return "";
    else if (s.Length <= len)
        return s;
    else
        return s.Substring(0, len);
}

And call it like so:

var s = "Supercalifragilisticexpialidocious";
s = Left(s, 5);

Fairly painless, right?

But with the advent of C# 3.0, there's an even better way -- extension methods. With an extension method, we "extend" the String to add the missing function. The code is fairly similar; I'll highlight the changed parts in red.

public static string Left(this string s, int len)
{
    if (len == 0 || s.Length == 0)
        return "";
    else if (s.Length <= len)
        return s;
    else
        return s.Substring(0, len);
}

And now we can call it as if this very method existed on the String class as shipped:

var s = "Supercalifragilisticexpialidocious";
s = s.Left(5);

Pretty slick. It's difficult not to fall in love with extension methods, as they allow you to mold classes into exactly what you think they should be. This is fairly innocuous in C#, as extension methods only allow you to add new functionality to classes, not override, remove, or replace anything.

But imagine if you could.

Well, that's exactly how it is in other, more dynamic languages such as Javascript, Python, Perl, and Ruby. Something as prosaic as C# extensions is old hat to these folks. In those languages, you could redefine everything in the String class if you wanted to. This is commonly known in dynamic language circles as monkeypatching.

monkey patching

If the idea of monkeypatching scares you a little, it probably should. Can you imagine debugging code where the String class had subtly different behaviors from the String you've learned to use? Monkeypatching can be incredibly dangerous in the wrong hands, as Avdi Grimm notes:

Monkey patching is the new black [in the Ruby community]. It's what all the hip kids are doing. To the point that smart, experienced hackers reach for a monkey patch as their tool of first resort, even when a simpler, more traditional solution is possible.

I don't believe this situation to be sustainable. Where I work, we are already seeing subtle, difficult-to-debug problems crop up as the result of monkey patching in plugins. Patches interact in unpredictable, combinatoric ways. And by their nature, bugs caused by monkey patches are more difficult to track down than those introduced by more traditional classes and methods. As just one example: on one project, it was a known caveat that we could not rely on class inheritable attributes as provided by ActiveSupport. No one knew why. Every Model we wrote had to use awkward workarounds. Eventually we tracked it down in a plugin that generated admin consoles. It was overwriting Class.inherited(). It took us months to find this out.

This is just going to get worse if we don't do something about it. And the "something" is going to have to be a cultural shift, not a technical fix. I believe it is time for experienced Ruby programmers to wean ourselves off of monkey patching, and start demonstrating more robust techniques.

Try to imagine a world where every programmer you know is a wannabe language designer, bent on molding the language to their whims. When I close my eyes and imagine it, I have a vision of the apocalypse, a perfect, pitch-black storm of utterly incomprhensible, pathologically difficult to debug code.

I was just looking at random PHP plugin code the other day, and it was, frankly, crap. But that's because most code is crap. Including my own. It is, sadly, the statistical norm. That's why sites like The Daily WTF are guaranteed to have more material than they can possibly ever publish for the next millennia. (Note to self: invest in this website). I can only imagine what that PHP plugin code would have looked like, had its developer been granted the ability to redefine fundamental PHP keywords and classes at will. These are the sort of thoughts that drive me to drink Bawls. And that stuff is disgusting.

You might say that PHP, sans the fundamental dynamic language ability to monkeypatch, is just another crappy Blub language. But there's also a ton of incredibly useful PHP code out there. So it seems to me that the ability to monkeypatch doesn't stop people from producing a huge volume of useful code, even in a kind of.. horrible language. Some of it is even good!

While I acknowledge the power and utility of dynamic language monkeypatching, I know enough about programmers -- myself absolutely included -- to know the vast majority of us have absolutely no business whatsoever re-designing a programming language. There's a reason some of the most deeply respected computer scientists in the world end up as language designers.

Perhaps then, given the risks, monkeypatching should mean reaching for the meta-hammer as infrequently as humanly possible. This is a position that Avdi himself espouses in a followup comment:

I'm afraid a lot of people have missed the actual meat of my argument -- that dynamic extension of classes is currently overused in Ruby, in ways that are:

  • Needless - another technique (such as a mixin, or locally extending individual objects) would have worked as well or better.
  • Overcomplicated - the use of a monkey patch actually created more work for the author.
  • Fragile - the solution is tightly bound to third-party internals, reducing the usefulness of the plugin or gem because it is prone to breakage.
  • Excessively wide in scope - by hardcoding extensions to core classes, the author takes the choice to scope the change out of the plugin/gem user's hands, further limiting utility.

My point is that there are alternatives - often alternatives which are actually easier to implement and will make your plugin or gem more useful to the user.

While I enjoy the additive nature of C# extensions, even those are enough to make me a little nervous, as mild as they are. Full-blown dynamic language monkeypatching goes even further; it might even be the ultimate expression of programming power. Is there anything more pure and godlike than programming your own programming language?

But if wielding that power doesn't scare and humble you a little, too, then maybe you should leave the monkeypatching to the really smart monkeys.

[advertisement] Read the largest case study ever published about lightweight peer code review in Best Kept Secrets of Peer Code Review. Free book, free shipping.

Posted by Jeff Atwood    View blog reactions

 

« iTunes is Anti-Web Maybe Normalizing Isn't Normal »

 

Comments

I have the same feelings about this topic. Monkeypatching is really powerful and tempting, but on the other side it's a really dangerous technique. Simply spoken, power corrupts. The more power a programmer gets, the more he is responsible to use this power in the right way. I think there are very few use cases where monkeypatching should be preferred to conventional means. But sometimes its really useful, e.g. in the context of codegeneration (and especially for programs that are not too big).

Florian Potschka on July 13, 2008 05:32 AM

Oh it happened to me once in a huge JavaScript file, where I accidentally redefined "window"...it took me hours to track that!

david on July 13, 2008 05:32 AM

Your Left function blows up if the string is null, which is cool if you intended that, but I thought I might mention it to anyone copying your function. I wrote a similar function when I was copying a VB program into C#.

In order to change the function, you would put this as the first line:

if (s == null)
return null;

As to the matter at hand, I agree that extension methods are pretty scary in the wrong hands. It's definitely something to think about.

I have already had a talk with all the C# programmers where I work and we have agreed that we will only use them to add functionality that would commonly be associated with a class of that type generically, like your left method.

Anything specific to a class or a project should not be an extension method.

PRMan on July 13, 2008 05:52 AM

Where did you get that beautiful monkey image from?

Peter on July 13, 2008 05:58 AM

Regardless of the extension technique, be it monkeypatching, overloading, or runtime, binary self-modification, it's never the language's fault; it's the community's.

Ruby's is one of those few communities where you get to see both good and bad sides of monkeypatching. Application-level plugging ala Rails is good. Language-level plugging in Gems, no matter how well-documented, is usually bad.

But hey, if we're going to abuse the power no matter what, then we might as well do it with a big sinister smile, knowing that Joe Rookie can never take on a job debugging it.

Rami Kayyali on July 13, 2008 06:01 AM

Perhaps this could raise another question: is there benefit from using a dynamic feature of the language when an equivalent static feature exists?

"Static" or "compile time" usually mean that important information about a program's behaviour can be inferred from a program's text alone without having to actually run the program to determine what a line of code does.

This idea of "determining a program's behaviour from reading it" goes back to the days of "Goto statement considered harmful".

I admit that we need dynamic language features at various points in a program and that they certainly have their role in making code more expandable or even more understandable, but we need to keep in mind the "understandability" tax that we might sometimes need to pay when using them.

Mohamed Samy on July 13, 2008 06:01 AM

I don't think the C# language designers will succumb to the madness of allowing full monkey patching. It would be very difficult, if not impossible, to implement in the CL anyway.

By the way, bugs like the one with inherited() would immediately show up when stepping through the code with a debugger, right? Proper tools are so important... and most dynamic languages are gravely lacking proper tooling. This is partly because it's so difficult to implement things like code completion when nearly everything depends on the runtime behaviour of your program.

Thomas on July 13, 2008 06:03 AM

Is it possible to clearly identify whether a method has been monkeypatched?

In the C# example I guess by advent of its presence in the current class you'd know you were extending the class, but I wonder in multi-dev environments how a developer would know that another has added an extension method within the designer (instead of having to email or IM someone).

One other thing, OT - over the last couple of weeks I've noticed lots of "first" posts. I dont know if its a testament to the blog's increasing popularity, but its freakin' annoying.

Ordinary Geek on July 13, 2008 06:05 AM

actually there's no monkeypatching in Python

Alisic on July 13, 2008 06:11 AM

"One other thing, OT - over the last couple of weeks I've noticed lots of "first" posts. I dont know if its a testament to the blog's increasing popularity, but its freakin' annoying."

Hey I am tenth!

Too right about it being annoying!

rippo on July 13, 2008 06:21 AM

This reminds me of Aspect Oriented Programming. It faces the same difficulty. You have to be very careful with your ascpects because debugging is suddenly a whole lot scarier.

Stefan on July 13, 2008 06:31 AM

Unlike Ruby, you can't monkeypatch the builtin types in Python. (You can also protect your own classes from monkey patching if you want).

You can shadow builtins or patch your own classes / extensions if you want.

The actual term "monkey patching" comes from the Python community (from the Zope community so the legend goes - where it originated as 'guerilla patching'). Within the Python world the term is definitely pejorative...

Michael Foord on July 13, 2008 06:32 AM

@PRMan

That's a terrible idea. You want to know as soon as possible when you have a null. If you consistently write code like that, when the time comes that you truly need a non-null value it will blow up so far away from where the null was originally introduced it will be incredibly difficult to find it.

@Alisic

Sure there is.

class A:
def f(self):
return "I'm an A."

old_f = A.f
def new_f(self):
return old_f(self) + " or so he thinks."

a = A()
print a.f()
A.f = new_f
print a.f()

I apologize in advance if the formatting comes out wrong.

Logan Capaldo on July 13, 2008 06:33 AM

"no monkeypatching in Python" - Alisic

import __builtin__
__builtin__.len = lambda x: 9

then do "len(0)"

class Spam(object): pass

def speak(self): print "spam, spam, spam, spam!"

Spam.speak = speak

>>> Spam().speak()
spam, spam, spam, spam!

Andrew Dalke on July 13, 2008 06:37 AM

I have enough trouble debugging my own crappy code without trying to debug someone elses if they've changed some of the fundamentals of the language!

Sounds to me like its the kinda thing you'd strictly control if it was something else. like a drivers license. or license to kill :p.

In fact, I might apply for my monkeypatching license right now so I too can completely screw with other dev's minds.

Josh on July 13, 2008 06:44 AM

I do not fullt agree with your post, monkey patching is not as bad as you put it. If only we could control the scope of patch, it is no worse than any other coding idiom.

C# does a nice job controlling the scope: you must import the extension method to use it! This give the programmer complete control over the lexical (ie textual) scope of the language. I can add my String.Left, you can add your String.Left, both with different semantics and nothing fails.

Ruby and the like to not allow to control the scope, any patch applies to any code. If I patch something, it applies to all code ... this is the path that leads to the gates of madness! I agree on that. Alas, none of the well known scripting language does allow such scoping. Even research does, to my best knowledge, go into the wrong direction. There are a lot of people thinking about dynamically scoped extensions, rather than using lexical scope. Dynamic scope is even worse than no scope at all!

So I am still looking forward to the language with lexical-scoped monkeypatching ...

Adrian Kuhn on July 13, 2008 07:02 AM

It's not just the addition only nature of extension methods that makes them safe, it's the fact that they are simply a regular static method once compiled. You can certainly make your code very confusing by overusing them (which I have done) but you won't cause problems for anyone else.

Sometimes being able to modify a standard method easily would be helpful, but with all the trouble that can cause it's probably best that the solution is something that is obviously an ugly hack. Some things shouldn't be easy.

Tom Clarkson on July 13, 2008 07:08 AM

Even if C# allowed Monkey Patching, unless you were defining your Monkey Patching in a system namespace somehow, it would be easy to find by checking out what using directives you had to find the offending overwriter.

Extension methods in C# roll like that. You don't get them at all unless you opt in for them. You can't universally add .Left to string that works in code you don't control unless said code subscribes to it.

Aaron Erickson on July 13, 2008 07:20 AM

you can check for a null string and an empty string with String.IsNullOrEmpty. Saw this in some code somewhere and wondered how much time i had wasted typing s == null || s == ""

kevin on July 13, 2008 07:58 AM

> Well, that's exactly how it is in other, more dynamic languages such as Javascript, Python, Perl, and Ruby.

Unfortunetely, Python doesn't allow you to extend built-in classes. Such a shame!

MaS on July 13, 2008 08:04 AM

You really like to comment on things you don't fully understand ... now don't you? The quality of the blog is going downhill.

Check this out (python)

>>> a = "Hello"
>>> a.__len__ = lambda x: x
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'str' object attribute '__len__' is read-only

I guess it is not that simple to change the behavior of a builtin type. By the way this is not monkeypatching (even if it worked).

Toothy on July 13, 2008 08:12 AM

The fact that the langague gives you the tools to do this monkeypatching thingy doesn't mean that you should do it.

It's just there... as lots of other stuff.

There should be some kind of specification where you allow or disallow things to be done...

Nicolás Miyasato on July 13, 2008 08:13 AM

One of the smartest monkeys I know is Reg Braithwaite (http://weblog.raganwald.com/). Rather than argue one side or the other, Reg is working on making it possible to have your cake and eat it too, through localized language alterations which don't contaminate the entire codebase. His Ruby-rewriting project (see e.g. http://weblog.raganwald.com/2008/07/separating-concerns-of-what-to-do-from.html) bears watching.

Avdi on July 13, 2008 08:17 AM

Since you're extending the methods of a class to make it more usable, may I suggest choosing more usable method names?

Maybe it's just because I've been spared this convention, but "Left" means nothing to me. I had to read your code just to figure out what it was expected to do.

In the Objective C Cocoa framework, we have a couple methods to accomplish what your "Left" and "Right" would. They're called "substringFromIndex" and "substringToIndex". Judging from your use of a built-in "Substring()" ... it sounds like these better-named enhancements would also be at home in your C# project.

Daniel

Daniel Jalkut on July 13, 2008 08:33 AM

If you are the only person in the world, then many this is an OK idea.

The other exception would be to build a centralized library of monkey code, which had been laboriously commented.

Other than that, it's just asking for trouble...

steve on July 13, 2008 08:43 AM

MORE php trolling ? This is getting old...

Other than that, interesting article :)

manu on July 13, 2008 08:53 AM

Nevermind, I overreacted.

manu on July 13, 2008 08:55 AM

Here's something that will blow your mind if you think monkeypatching classes is weird enough - in Ruby, I can monkey patch *individual instances* of a class:

class Foo
def bar
puts "Baz"
end
end

irb(main):012:0> a = Foo.new; b = Foo.new
=> #<Foo:0x5a18ec>

irb(main):013:0> a.bar
Baz
=> nil

irb(main):014:0> def a.bar
irb(main):015:1> puts "Bamf"
irb(main):016:1> end
=> nil

irb(main):017:0> a.bar
Bamf
=> nil

irb(main):018:0> b.bar
Baz
=> nil

Paul Betts on July 13, 2008 09:06 AM

Yeah, there is nothing like good old write-only code.

lmbo on July 13, 2008 09:08 AM

And remember, kids, ALWAYS MOUNT A SCRATCH MONKEY!

(If you're too young to CTR, Google is your friend.)

Dave Aronson on July 13, 2008 09:09 AM

There are certainly pros and cons to monkeypatching, but you outlined them perfectly. Another great post!

Steve on July 13, 2008 09:16 AM

What about Java language? There's the "override" which makes it possible to do monkey patch as well, correct?

pallu on July 13, 2008 09:18 AM

The real problem with Left() (ie. the lack thereof) is that the substring function COMPLETELY UNNECESSARILY throws an exception if you pass a second argument longer than the string itself. WHY? (I guess the answer is arrogance amongst the original language devs, who came from a C background, and therefore did the "right" thing rather than the "useful" thing, when VB would always have done the later!)

Syd on July 13, 2008 09:46 AM

@Daniel Jalkut

The "left" and "right" conventions no doubt come from Microsoft BASIC, LEFT$ and RIGHT$. You are blessed, perhaps, not to have programed on a platform that Microsoft BASIC was a default language on. ;-) I'm learning Cocoa and when *I* see code like yours I am the one who gets puzzled, much as you are by the above convention, so I think your suggestion is probably not a good one for the typical C# programmer, despite what you may think, considering that many of them are familiar with some form of Microsoft BASIC and not with Objective-C and Cocoa.

Ploni Almoni on July 13, 2008 09:49 AM

Hey Now Jeff,
Monkeypatching, I didn't know of that term till after I read the post. You gotta love learning from reading Coding Horror.

Coding Horror Fan,
Catto

Catto on July 13, 2008 09:53 AM

Also called duck punching in languages with duck typing.

The idea being that if it walks like a duck and talks like a duck, it’s a duck, right? So if this duck is not giving you the noise that you want, you’ve got to just punch that duck until it returns what you expect.

Andrew Nesbitt on July 13, 2008 09:54 AM

Indeed, you can monkey-patch in Python, including monkey-patching individual instances. As far as I know, all of the metaprogramming techinques in Ruby have a nearly direct parallel in Python. (Except for the aformentioned restrictions on builtin types)

That "monkey-patching" is a more perjorative term to Pythonistas than to Rubyists has a lot to do with the culture surrounding the languages, obviously. But, I've noticed an interesting coincidence.

Black-magic metaprogramming in Python produces code with lots of __dict__'s and __class__'s, dunders and quoted identifiers in it. Its appearance just screams "I am rewiring my microwave and voiding the warranty," and is just generally quite visually distinct.

On the other hand, Ruby metaprogramming code just looks like any other Ruby code, unless you're paying particular attention to what methods it's calling. It doesn't "look" out of the ordinary.

I wouldn't be surprised if one didn't have something to do with the other :)

Kyle S on July 13, 2008 10:01 AM

I could see how this could be really, really bad- confusing the hell out of the other people you work with.

Nobody enjoys it when what you have worked with for the last 10 years changes, even the slightest. (see: vista)

Yeah, I'm no language designer. Think I'll take heed, and will stay away from thinking about doing stuff like this.

David McGraw on July 13, 2008 10:13 AM

This monkeypatching is old hat to Lisp and Smalltalk programmers too. All those other languages you mentioned (javascript, perl, ruby, python) are flawed in some way compared to Lisp and Smalltalk (slower VMs, disabled lambdas, ugly syntax, string-based eval/no-macros etc). They have a few bright spots, but for the most part, it is far better to use Lisp and Smalltalk as examples whenever meta-programming is mentioned.

To fully explore the possibilities of monkeypatching, you need to use a Smalltalk system like Squeak, or maybe Self, or maybe Io.

OMouse on July 13, 2008 10:20 AM

GvR is a respected computer scientist?

tc on July 13, 2008 10:29 AM

In C# you can only "override" inherited class members. That really doesn't screw up anything unless you are very creative. I suppose same goes for Java.

Palotasb on July 13, 2008 10:30 AM

Meh.

Summarized: powerful languages give you more ways to shoot yourself in the foot - be careful.

C++ let you redefine operators; misused, this could result in apparently simple code with complex and hard to see or debug consequences.

The C preprocessor allowed, via plain ol' macros, the construction of code that looked nothing like C.

Java was designed as a C-like language which would avoid these pitfalls by removing the capabilities entirely - programmers responded by building preprocessors and auxiliary languages that effectively added them back in.

Let's not forget the most obvious way of becoming a "language designer": actually writing an interpreter or compiler for a new language (usually a domain-specific language). This happens *all the time*.

And it happens because it's useful. *Really* useful. If you have a lot of code that pulls off the leftmost characters from a string, then that code becomes much more clear once your "Left()" method is introduced.

Don't run scared from your language. Learn to use it.

Shog9 on July 13, 2008 10:35 AM

I absolutely agree that problems caused by monkey patching are the result of cultural problems, not technical problems. Objective-C has had categories for over 20 years, and there hasn't been an apocalypse yet. Of course, the use of categories for overriding existing methods is strongly discouraged (http://developer.apple.com/documentation/Cocoa/Conceptual/ObjectiveC/Articles/chapter_4_section_3.html#//apple_ref/doc/uid/TP30001163-CH20-TPXREF141), so there's the cultural factor at work.

I find it discouraging that the hard-won lessons of older languages like Lisp, Smalltalk, and Objective-C seem to get lost in the buzz around newer dynamic languages. From an engineering standpoint, trendy != good. I think Ruby, Python, et al. are giving older dynamic languages a bad reputation.

Bruce Johnston on July 13, 2008 11:08 AM

Luckily I program only VB

Omar Abid on July 13, 2008 11:10 AM

Good examples of the evils of monkey-patching can, I think, be seen in Javascript -- probably the most-widespread programming language in the world today, since you have at least as many JS interpreters on your computer as you have web browsers.

Popular, widespread frameworks like Prototype change (or, used to -- don't know if they managed to fix it by now, but I don't think so) the innards of built-ins to the point of breaking OTHER frameworks (and much other JS code written for JS proper, without knowledge of the exact details of monkey-patching provided by this or that framework).

Just as theory predicts, it is thus shown to be true in practice that "independent" frameworks (and even modest snippets of code one is trying to reuse) cannot live together if one or more of them do use monkey-patching -- because monkey-patching's effects are covert, global and pervasive. Even the modest amount of monkey-patching allowed in Python (e.g. changing __builtins__, as in Andrew Dalke's example) would be deleterious IF popular modules used it -- but a feature that's not to be used should not be in the language (I've long maintained that builtins should be locked against runtime patching, and it should at least be possible to similarly lock other modules, at least, and ideally any other object too -- beyond ensuring against accidents, that would ease Python optimizations).


Alex

Alex Martelli on July 13, 2008 11:11 AM

"Is there anything more pure and godlike than programming your own programming language?"

I wouldn't say you'r creating your own language, rather your own framework.

But both cases are equally dangeruous :D


and btw, please give a lifetime ban to all lamers who write "first"-comments and similar crap.

ivovic on July 13, 2008 11:21 AM

Agreed. I think the love afair programmers have with dynamic languages (granted, monkey patching isn't a symptom of all dnymaic languages) is way overblown. You often end up with this overly clever solution that is hard to debug and harder to maintain. These crappy solutions are attractive to people who are more interested in hacking together some code then taking a disciplined software engineering approach.

Jim Greco on July 13, 2008 11:25 AM

Now imagine monkey patching in a programming language with true multiple class implementation inheritance _and_ dynamic multiple argument method dispatch... Common Lisp was fun!

Jason Trenouth on July 13, 2008 12:02 PM

Remember that recursive FindControl method you wrote? Wouldn't that be a good example of a partial method?

I guess this stuff isn't for breaking programming languages, but more as a tool to augment closed libraries with useful functionality - useful for the User of these libraries.

Use it sparingly.

Ah, and yes: Monkey patching is kinda bad - but sometimes you want to be able to hack into your language. Very seldom. But being able to really does make you trust it just that bit more.

Daren Thomas on July 13, 2008 12:18 PM

I'm an old developer. I've been at it for over 35 years. I've never been very impressed with the C syntax all over the place. When I was a young fellow in the 70's I thought that the language FORTH was where everything would go. Extensible languages were just so cool. In hindsight, though, Jeff, I now can see the apocalypse of which you speak. Still and all, it was cool being involved at a time when languages were still evolving.

Doc on July 13, 2008 12:23 PM

Oh Jeff... now even you're referring to all .NET features such as extension methods as "C#". You've obviously lost your self-claimed agnostic point of view.

"While I enjoy the additive nature of C# extensions..."

Ya know, you coulda just said "...nature of .NET extensions..."

Not to pick you know what off the wall, but if you're going to use a VB function such as Left as your example, at least give the lang a little love brotha... Pffft.

Erin on July 13, 2008 12:30 PM

What happened to your Spartan Programming ;)

You can leave out 'else' twice:

if(len == 0 || s.Length == 0)
return "";
if(s.Length &lt;= len)
return s;
return s.Substring(0, len);

Eelco on July 13, 2008 12:32 PM

Jeff,
I just wanted to make an observation about monkeypatching, Avdi Grimm's experience, your earlier posting on unit testing, and your earlier posting on subversion. I think that Advi could have saved himself a lot of grief if he had some way to do regression testing, i.e., if his code was under version control and he evaluated it at different versions until he found the first location where it broke to see what the change was and why it broke.

I'm assuming of course, that Advi had access to all of the code and that all of the code was under version control. If the problem child in the ruby control was not under his availability to inspect, well, then, in so many words... he was royally screwed.

After having to deal with other peoples' code I've become a *True Believer* in version control, first with subversion, and then with git. (I love being able to access subversion repositories using git-svn, a git client that checks out from subversion into my local git repository where I do all of my experimentation/development and then push out the history of changes back to the subversion repository with a single "git-svn dcommit"). If you can't track the changes in the code you're using, then life is soooo much harder when finding regression bugs.

Johnny on July 13, 2008 12:41 PM

Or how about:

public static string Left(string s, int len)
{
return s.Substring(0, len);
}

KeepItSimple on July 13, 2008 01:05 PM

+

(I'll leave it as an exercise for the reader to determine what I meant.)

orange on July 13, 2008 01:24 PM

Perl has this option of overriding methods for years.
(either on other object, or a built-in function)
and generally, it's not miss-used.
As said, it's a cultural this. This ability is marked in bold letters saying "if you really know what you are doing", and anyone how don't know, just don't do.

Shmuel Fomberg on July 13, 2008 02:22 PM

"Monkey patching"

Is that what the cool kids are calling it these days?

What exactly is your problem with php? Must you continuously suck at the Microsoft teat?

Nick Waters on July 13, 2008 02:31 PM

So... what should we reply to the sheeple who like to parrot the following:

"In C, you can shoot yourself in the foot.
In C++, you can blow your whole leg off."

How about:

"In Ruby, you can not only blow yourself up but take your entire team with you!"

W on July 13, 2008 02:44 PM

You missed Groovy in your list of Dynamic languages. http://groovy.codehaus.org/ It works seamlessly back and forth with Existing Java code.

I like Groovy's approach. It uses the methodMissing API to dynamically add methods to specific classes, so that if you wanted to implement String#left() you would simply write:
String.metaClass.left = { len ->
if (len == 0 || s.Length == 0)
return "";
else if (s.Length <= len)
return s;
else
return s.Substring(0, len);
}

then every single string would now have a left(int len) function on it :-)

Colin Harrington on July 13, 2008 02:47 PM

For .net extension methods, the current recommendations are to put them in a separate namespace so that there aren't any unexpected side-effects from other developers consuming your code (especially if you're writing a library / framework). I certainly don't want string.Left polluting my code just because I use some widget. But the widget writer is free to use any of that and I never have to see it.

Thinking back to C++, where there are a dozen different string implementations, it certainly is nice to just add 1 extension method that you need, rather than use / struggle around using an implementation that is missing that 1 method.

It's another thing done mainly for Linq but retaining nice side benefits outside of using linq. Typesafe macros on steroids I guess, but it's just syntactic sugar in the end. You're not altering the class in any way, nor can you (easily/unknowningly) affect already-written code.


blue on July 13, 2008 02:48 PM

I have the same thoughts on Aspect Oriented Programming. All sorts of goop get unexpectedly injected into all sorts of places.

Sometime people just want clever toys rather than having to just hack out good code.

And as you point out; most code is junk. Good programmers will write good code that uses AOP or Monkeypatching; they will have good reasons, it will be well documented and it wont break. Conversely a poor programmer is going to write junk no matter what tool they use; at least using MP they wont whine about having to use old fashioned tools

pm on July 13, 2008 02:57 PM

It's always great to see the scientists you link to... thanks.

Patrick on July 13, 2008 03:07 PM

It's worth mentioning that monkeypatching is an incredibly valuable technique for writing unit tests.

For example, to test how your code-under-test behaves when it is unexpectedly unable to open a file, your tests should not have to go sniffing around the file system, deleting or changing permission on critical files. This obviously would have huge potential for unintended side-effects if your test fails to clean up properly after itself, and hence would be a very fiddly test to write.

Instead, your test should simply patch the file 'open' method, replacing it with 'raisePlease()', a small function of your own devising which simply raises an exception. Then when the code-under-test is called, it calls what it thinks is file.open(), but is actually raisePlease().

This makes it a fairly idiot-proof two line test. We use it literally every day. I can't think of a single instance of a monkeypatch with *global* scope in our production (ie. not testing) code.

Hugs,

Jonathan Hartley on July 13, 2008 03:16 PM

spot on Jeff...

Steve on July 13, 2008 03:18 PM

InfoQ has a write up with an overview of solutions for scoping extensions (C# is one example):
http://www.infoq.com/articles/ruby-open-classes-monkeypatching

murphee on July 13, 2008 03:26 PM

RE: Alex Martelli on July 13, 2008 11:11 AM:

Prototype plays very nice with any other framework and has for some time. I've used it with YUI and jQuery. Several years ago (v. 1.4, I think) Prototype added some methods to Object.prototype at which point users rebelled and the author, Sam Stephenson, rightly removed it.


James on July 13, 2008 03:29 PM

I love people. They're wonderful.

Year after year us C++ programmers keep hearing that the ability to overload operators is the most awful crime against humanity ever committed.

Now it turns out that overloading operators is still evil, but somehow monkeypatching operators and methods and constructors and core language features is the most wonderful and exciting thing to ever be created in the entire universe.

Thanks a million, guys. I knew that C++ wasn't as bad as you all made it out to be. ;)


Also, I simply love the modern approach to language design: take C++, subtract the bits that are hard to explain in the "teach yourself in 21 days" book, add in some bytecode and a garbage collector, and then piece by piece add in all the bits you took out because it turns out they're actually useful.

Well, ok, the "everything and the kitchen sink" approach where every string class seems to end up with a hundred thousand manipulation methods and regular expression processors and XML parsers and every type of string oepration shoved into the interface is funny, too.

C# and Java have the "everything is an object" thing, and the better development would be to make free non-member functions into first-class citizens, so that a "string" can be effectively an atomic little unit that does one thing and does it well (with a small and testable interface and no BFUD "think of everything anyone will ever do with a sting") and all the manipulators can exist separately. Then when the internal implementation of "string" changes, it doesn't beak all the monkeycrappers.

But that won't happen - cos operator overloading is evil! It's evil, I say! And therefore C# is perfect!

Bob on July 13, 2008 03:38 PM

Total agree. In addition, using such extension will likely be nightmare for code review.

Andy Wong on July 13, 2008 03:41 PM

> And remember, kids, ALWAYS MOUNT A SCRATCH MONKEY!

I had never heard of this, actually. Interesting.
http://edp.org/monkey.htm

> It's worth mentioning that monkeypatching is an incredibly valuable technique for writing unit tests.

Ah, you're right -- dynamically altering methods would be very useful for testing.

> [criticisms of Left() code]

That code doesn't actually exist; it's only used as an example. I do agree with all the amendments / suggestions people have made, but critiquing dummy code is a little beside the point don't you agree? :)

Jeff Atwood on July 13, 2008 04:07 PM

This reminds me of the old arguments of the usefulness of multiple inheritance in C++. When I was managing a team of programmers we had the rule that if you wanted to use multiple inheritance in your design, you had to convince two other team members that it was the best design for your problem otherwise you had to do it another way.

It's a useful solution for a very small set of problems.

Evan on July 13, 2008 04:28 PM

> Full-blown dynamic language monkeypatching goes even further; it might even be the ultimate expression of programming power. Is there anything more pure and godlike than programming your own programming language?

It'll only truly be programming your own programming language if you're able to extend the syntax of the language itself. Which reminds me of Lisp :-) and this: http://xkcd.com/224/

> That code doesn't actually exist; it's only used as an example. I do agree with all the amendments / suggestions people have made, but critiquing dummy code is a little beside the point don't you agree? :)

Perhaps then you could mark it as such? I.e. being dummy code.

Giel on July 13, 2008 04:33 PM

"That code doesn't actually exist; it's only used as an example. I do agree with all the amendments / suggestions people have made, but critiquing dummy code is a little beside the point don't you agree? :)"

Jeff, Jeff, Jeff.

Have we learned nothing from Stairway to Heaven, a/k/a FizzBuzz? if you write a blog post about Foo and include a few lines of dummy code, you can expect a widely distributed code review and programming contest to ensue.

Expecting otherwise is very much like opening your eyes in the morning and switching on the radio to hear if Global Peace has been declared while you sleep.

Laudable Optimism.

Reg Braithwaite on July 13, 2008 04:58 PM

You only post this kind of stuff to scare me, don't you?

Every 5 years or so we have a new batch of programmers, and many re-invent previously discovered and, often rightly, discarded concepts. The idea of extending core class definitions like this is not new and, one may note, not prevalent. Why? Imagine the amount of undocumented extensions soon to be written. Really, you don't want the functionality to implicitly override existing core functions on top of that. Give us developers enough rope and we will hang both ourselves and everyone downstream of us ...

A certain level of pre-defined, ensured, common functionality means we can pick up on new code bases quicker. If youwant this, use inheritance.

I view this right up there with Shuggoths -- can you hear the voices on the wind? Tekka-li-li! (okay, that reference may be a bit obscure).
Flee the Mountains of Madness!

John Storey on July 13, 2008 05:11 PM

Because of monkey patching I'm VERY biased against Ruby right now. My introduction to Ruby was through a very complicated program, written by some consultants. That thing was atrocious, and used monkey patching like it was going out of style. For example, anything remotely related to Arrays was added to the Array class. You need to count the number of red cars in an array of mixed objects? Call Array.CountRedCars! Sure, you COULD create a class for car arrays, but that would just make things TOO EASY.

I'm sure Ruby is a very respectable language. The kids sure like it. But that program left such a bad taste in my mouth that I wont touch it again if I don't have to.

(I've probably got my syntax wrong, but that's okay)

Gary on July 13, 2008 05:53 PM

Gary:

"My introduction to _____ was through a very complicated program, written by some consultants. That thing was atrocious, and used _____ _____ like it was going out of style... I'm sure _____ is a very respectable language... But that program left such a bad taste in my mouth that I wont touch it again if I don't have to."

You know, I could tell almost EXACTLY the same story about several different languages and applications written by consultants. I suspect that when someone is paid $250 an hour to write an application, he feels compelled to make it buzzword-compliant.

And you know what? At one time "the kids" sure liked PCs, and BASIC, and Java, and C, and then C++... I suspect you were trying to be dismissive, but in my experience, dismissing something because the people who use it make you feel old can backfire, badly. be careful out there.

Reg Braithwaite on July 13, 2008 06:14 PM

I'm surprised that C# added this as a language feature. There are reasons that string is a sealed class; preventing monkeypatching is one of them, or so I am given to understand.

Well, OK, that's one of the reason's it's final in Java. I don't do much work in C#.

Powerlord on July 13, 2008 06:33 PM

Doesn't C++ already provide this via overloading? As Jeff always says, it's a people problem not a compiler problem.

Brian on July 13, 2008 06:44 PM

For those with short memories; what M$ tried to do to java (and caused Sun to sue) amounted to monkeypatching. In the java/M$ case, it was done to destroy Sun's control of java. Caveat Emptor

buggyfunbunny on July 13, 2008 07:10 PM

most monkeypatching done is made out of sheer demonstration that they know metaprogramming (especially for ruby). i find it less smart to instantly resolve to monkeypatching the language itself rather than extending it within the application or its library.

most importantly, monkeypatching without documenting it is pure evil. it makes it harder to debug the application imho.

sweetperceptions on July 13, 2008 07:47 PM

Good post. To me, Monkey Patching seems like, if you're having to do it, there's some other issue which you're having to work around which needs attention. Say there's a function in the language which is actually broken: maybe a Monkey Patch would work but it would also warrant investigating if there's a bug fix on the way and putting a reminder to remove the patch when it comes through. The problem I've always found is, though it seems easy to remove any sort of patching like this now, given time, anything becomes harder to roll back.

David from Oz on July 13, 2008 08:31 PM

Why not just add a reference to Microsoft.VisualBasic (which is included in the .NET Framework) and then use Left() function?

You can use Left, Right, Mid, etc. in C# by just adding a reference to that library that is already included in the Framework. It's a not VB.NET specific library and it is included in all Frameworks.

Jim on July 13, 2008 09:38 PM

Well... at first, I think that the debugging madness is mostly a hint towards debuggers that are not strong enough. In my opinion, if you run some program with maximum debug symbols, you should be able to say something like "inspect monkey_patched_string" in your debugger and see like:
Attributes: [list]
Methods: [list]
- Left added in monkeycorp/monkeyserver/monkey_jeff/monkey_file.monkeyextension
- Inherit modified in evil_joe/evil_plugin/evil_file.monkeyextension
[more list]

That way, at least *finding* the problem is not that hard. :)


On the other hand, I agree that monkey patching in a global scope is bad, because it is in the global scope. Whoever worked in large C-applications knows the pain of not having namespaces (usually, one creates his own namespaces by using fun names like "SNetUtlDebugIterCurrentDefined" (SNet Utilities List Iterator Function).
If one applied this to the monkey-patched methods over there, you would rather have sometihng like MonkeyServerLeft and EvilPluginInherit and the problems are gone.

Thus, I think, monkey patching needs to be tied to the scope you are in or needs extra syntax to import monkey patches in order to really work for larger things.
The first version would require some nested module system, and a monkeypatch applies to all module that are inside the current module - but the runtime system removes it as soon as a monkeypatched is passed to a sibling or a parent. That way, EvilPlugins inherit would not interfere with MonkeyServer, unless some other really kinky stuff is going on.

The other way is a bit more general, but requires more work. That way, MonkeyServer *could* say "keep monkeypatched inherits from EvilPlugin". If he did that, he would get patched inherits, but it would be documented. If he did not do that, patches would be stripped.
The drawback to this more general method is that you need to add code for each method added (even though I am still thinking if added methods need to be annotated in this way)

Greetings

Hk on July 13, 2008 10:45 PM

Welcome to the Monkeyhouse. It looks like modern languages have invented new ways of keeping old challenges unresolved.

BugFree on July 13, 2008 10:57 PM

public static string Left(string s, int len)
{
if(s.Length == 0 || len == 0)
return "";

return s.Length <= len ? s : s.Substring(0, len);
}

Dan on July 13, 2008 11:31 PM

> Even if C# allowed Monkey Patching, unless you were defining your
> Monkey Patching in a system namespace somehow, it would be easy to
> find by checking out what using directives you had to find the
> offending overwriter.
>
> Extension methods in C# roll like that. You don't get them at all
> unless you opt in for them. You can't universally add .Left to string
> that works in code you don't control unless said code subscribes to
> it.

This still poses a problem. What if C# 4.0 added a 'Left' method to the String class -- and one with a different interface or one that worked slightly differently? Then the extension method, and all code that relies on the extension method, breaks. OK, this is unlikely to happen for 'Left' and the String class, but it could very well happen elsewhere. I agree with what Jeff said in the article -- extension methods make me nervous.

> It's worth mentioning that monkeypatching is an incredibly valuable
> technique for writing unit tests.
>
> Ah, you're right -- dynamically altering methods would be very
> useful for testing.

Yes, dynamically altering methods would be very useful for logging, testing, and debugging. But how do you allow this without allowing Monkeypatching? The only thing I can think of is to allow one to call a function (or object method) prior to entering and after exiting the core function -- much like aspect-oriented programming. And those functions must not throw or otherwise alter the behavior of the original function. *THAT* would allow some useful runtime behavior (like dynamic logging) while preventing the bad behavior seen in Monkeypatching.

Here's to hoping some language writer and/or compiler writer can figure out how to accomplish this.

Kevin on July 13, 2008 11:43 PM

> That code doesn't actually exist; it's only used as an example. I do agree with all the amendments / suggestions people have made, but critiquing dummy code is a little beside the point don't you agree? :)

Just write real abstract dummy code then instead of C#.

Jens on July 14, 2008 12:30 AM

Will the people that are confusing monkey patching with overriding virtual methods in subclasses please NEVER touch a compiler again?

Joost on July 14, 2008 01:00 AM

> That code doesn't actually exist; it's only used as an example. I do agree with all the amendments / suggestions people have made, but critiquing dummy code is a little beside the point don't you agree? :)

I don't agree :)

One of my few gripes about Coding Horror is the lack of actual code, so it's nice to see some. Won't Stack Overflow be all about voting on code snippets?

Also, as someone who has never used C#, I've learned in just a few lines: 1) it does look like Java 2) but it's more powerful 3) you can use var 4) you can have free functions 5) the extension syntax is clean and 6) what code in a real C# codebase might look like.

Thomas Guest on July 14, 2008 01:01 AM

> public static string Left(string s, int len)

I've only started learning C# for a few months now, but one of the most annoying things I find about it are these ambiguous method names. I mostly program in Java and C++, and I have no background in BASIC, so maybe I'm just not used to it, but the method name "Left" has no meaning to me. Maybe, getLeftSubString. I dunno, maybe I'm crazy.


Ryan Thames on July 14, 2008 01:06 AM

Interesting. Is it the OO exposure that makes people prefer (sometimes strongly)

s = s.Left(5);

to the much more functional (and to me, sane)

s = Left(s, 5);

? If you are going to use function overloading on that, you really should have derived your own StringEx class for ease of understanding. I do agree that it makes applying test harnesses significantly easier, though.

Tepsifüles on July 14, 2008 01:12 AM

"That code doesn't actually exist; it's only used as an example. I do agree with all the amendments / suggestions people have made, but critiquing dummy code is a little beside the point don't you agree? :)"

Jeff, have you learnt nothing from the "FizzBuzz" incident?
http://www.codinghorror.com/blog/archives/000804.html

You post code, we'll post fixes. Always.
Nice post, though. "Fear the Power of the Patched Monkey. Coming Fall 2009..."

Tom on July 14, 2008 01:16 AM

As you said and well the problem isn't giving the programmers the ability to override a core class, but the programmers that do it. In Lisp you can do pretty much whatever you want to to the core language but you should still adhere to conventions for sanity sake. I spend my days in ActionScript 3, and it might come as a shock to some of you it's actually a very evolved language, and altough, like c# it only allows to add, not override, methods, i prefer to either wrap the core class or to have a utility class.
This is because it's a) easier to document and mantain and b) because it gives the user(programmer) the choice to use it or not.
I think that in it's essence it's a bad practice to add to core classes as it also may result in conflicts with future language updates, what was to happen if in the next version a Left method was to be implemented, that said I much prefer either a wrapper or utility class.

pedro on July 14, 2008 01:19 AM

I know this post is really about monkeypatching but I'm going to go off at a tangent anyway because the first part of the post reminded me of a conversation we were having at work last week.

What I'm going to talk about is example code. Now it's not a critique of the code shown itself, that's a function it does a job that I understand. It's to do with code snippets being shown without scope or context, or to put it another way what's not shown.

For example your extension function should be defined in a non-nested, non-generic static class (according to msdn - I've not had chance to use them much myself), and bought into scope with the using directive if not in the current namespace. But without this small but important bit of information it could leave some people scratching their heads over where this function should actually go - should it be a partial class of the string object, does it have to be named in a particular way, or located in a particular place.

I hope this is something people will be thoughtful of when posting code in future as it really can reduce the helpfulness of well intended examples. Perhaps it's something you may like to try and educate people to do when they eventually start posting their examples onto stackoverflow.com

MarkM on July 14, 2008 01:32 AM

I'd like to share some practical uses of monkey patching.

1) 'hot patch' of libraries being old, buggy or lacking a feature

For instance, I'm using platypus ( subpackage of reportlab ) to generate PDF documents in python. The problem with this library is the use of a global instance, making it buggy when generating multiple documents in multiple threads. I've overridden a few methods to add a locking mechanism. Essentially, this makes the deployment easier ; I don't have to provide a patch for the platypus library along with my application, making it a deployment nightmare. Patch is made at runtime, and the admins on the production server just have to make sure the standard reportlab packages are installed, not a patched release that might interfer with other applications.

Another example is a cherrypy monkeypatch I use in a turbogears application, allowing to check upload status : http://www.cherrypy.org/attachment/ticket/546/uploadfilter.py . I hacked it so it checks the content-type of an uploaded file before continuing the upload. I don't want to provide a patched cherrypy either, I just monkeypatch it at runtime.

These monkeypatches enable to patch libraries without having to provide a patch along with the application, avoiding deployment headaches.

2) I'm just in love with RoR duration monkeypatches, allowing such things as Table.count(:conditions => ["created_at > ?",24.hours.ago]). For sure, it's more evil as it implies changes on builtin types, reminding us those overloaded operators nightmares in C++. But it's giving us incredibly short and readable code !

Vincent on July 14, 2008 02:02 AM

"Ya know, you coulda just said "...nature of .NET extensions...""
It's still a specific language feature, not a .NET feature. Nothing in the rest of the architecture other than the language compiler need change for the feature. Sure he should perhaps have said "C# and VB.NET extensions", but definitely not ".NET extensions" ;)

"Just write real abstract dummy code then instead of C#."
It's quite difficult to write abstract dummy code when talking about a specific language feature.

[ICR] on July 14, 2008 02:41 AM

@vincent: Sure doing it at runtime sounds excelent, but it just moves the problem further down the line, what happens if a developer inherits your code and applies it at compile time? or if it becomes a part of the core class?

pedro on July 14, 2008 03:01 AM

Actually you can monkeypatch in Obj-C as well. The language itself doesn't allow it (it only allows to add functions to a class dynamically), but since everything is very dynamic in Obj-C (much less is static as in C++), you can pretty much modify whatever you want, even things you are not supposed to modify.

But there is something about monkeypatching you missed completely in your post. Sometimes it is necessary to make software work correctly. There are situations where the real implementation is buggy and while you could subclass and use your subclass only, if you offer a plugin/framework/library for other people to use, you can tell them "Please don't use class XYZ, but instead our subclass of it, as the method ABC of XYZ is buggy", you can't force them. By monkeypatching the broken method, you kind of fix someone else's bug, which is sometimes really useful and just makes things work.

The problem with it that one day the developer of this class sees the bug and fixes and, then you monkeypatch a method that isn't broken any longer. However, in a dynamic language this is no big deal, as there you can check the version of a plugin/lib/framework and depending on outcome you either monkeypatch the method or you don't.

Mecki on July 14, 2008 03:41 AM

Mary Poppins has spoken. :)

I don't like, endorse or do mutiple return statements.

public static string Left(this string s, int len)
{
if (s != null)
if ((len > 0) && (len <= s.Length))
s = s.Substring(0, len);
return s;
}

Christo on July 14, 2008 03:45 AM

Mecki expresses the same point I wanted to make.

@pedro, if I get your point, if I find a compiler for ruby or python, every method or package member would be virtual, and thus, overridable. For sure it's meant only for dynamic languages. For statically ( and thus faster ) compiled languages one would have to fallback to more classical approaches : either provide a patched library, or hide the whole buggy library behind an adapter interface which cares of the workaround( provided the monkeypatch is used for fixing an existing library ). I see it as a cheap shortcut which has to be used wisely. If I'd needed absolute security at any price, I'd avoid buggy library or would debug it thoroughly, indeed.

Vincent on July 14, 2008 04:36 AM

Monkey patching is bad, right? Consider a world where the effects of monkey patching is the default, and all classes are "open" by default.

More in http://mikael.jansson.be/journal/2008/07/extension-methods-greenspunning-generic-functions

Mikael Jansson on July 14, 2008 04:41 AM

Yes monkeypatching is bad... but at least C# doesn't let you do it. :)

I think the point is always overlaboured about not reinventing the wheel. Yes its a waste of time, but those "great" programmers who wrote the first code were probably nothing special either... just as prone to making mistakes and doing things badly. Further they probably comprimised to find a best fit function to as many scenarios as possible.

Monkeypatching is bad, but reinventing the wheel is not necessarily that bad.

Yes.. I have made my own programming languages in my spare time. I do love reinventing the wheel... but on the other hand at least I used flex, bison and loads of the stdlib functions to do so... at least after the first time. :)

Still... why even bother with monkeypatching. Derive a new class or something... then at least if you reinvent a wheel or two, your buggy code is not likely to break other things left, right and centre. :)

Jheriko on July 14, 2008 05:11 AM

Cool - I decided to write a String.Right() extension method just last week: http://harriyott.com/2008/07/extension-methods-in-c-again.aspx

Simon Harriyott on July 14, 2008 05:21 AM

Anders Heljberg is a respected language designer? WTF? He implemented Pascal, then Java, then Microsoft-Java (aka C#). When did he become a language designer in the same league as Wirth, Stroustrup, Codd, etc?

I wonder what's next? Duck typing probably - after all, no-one wants to write code that has to be designed in the new world of 'its so easy' .NET, not when they can just hack it about until it (fingers crossed) works like they want. I just pity the poor engineers who have to perform maintenance on all this code.

AndyB on July 14, 2008 05:28 AM

@AndyB:"Anders Heljberg is a respected language designer? WTF? He implemented Pascal, then Java, then Microsoft-Java (aka C#). When did he become a language designer in the same league as Wirth, Stroustrup, Codd, etc?"
Because God knows that "AndyB" is in a position to comment on the quality and the difficulty of writing 3 of the most well-known and used languages in the world.
And it's Heljsberg, not Heljberg.

Tom on July 14, 2008 05:51 AM

I find it deliciously ironic that in the "horrible language" you refer to, you wouldn't experience this problem at all. It implements substr() just as syd suggests.

bobby on July 14, 2008 05:58 AM

Don't be scared of a language's features. Learn how to use them.

A week ago a friend of mine related to me that a TCL primitive opened a program to a DOS attack. His answer was to rewrite the primitive in a manner immune to this. At the time he was in a reflective mood, and he did not call it monkey patching.

He also made a comment regarding teaching people, "I could teach you TCL in five minutes. It would take you years to master it."

Ben T on July 14, 2008 06:22 AM

> Oh it happened to me once in a huge JavaScript file, where I accidentally redefined "window"...it took me hours to track that!

Here's an advantage to not be a native english. You can use the variable class... "window" in your native language without problem :-)

Luc M on July 14, 2008 06:30 AM

I've come to the conclusion that the best programmers are not the smartest ones. The 'mediocre' ones stick to regular practices and don't adopt new features all the time for the hell of it. Their code is more consistent, better written/commented. The smart-arse mavericks always seem to want to prove themselves in their code by unneccesarily using techniques that they don't fully understand themselves and don't appreciate the full ramifications. Then they think that documentation is not for them and two years down the line, it's their shite that someone else has to figure out.

Andy Turner on July 14, 2008 06:48 AM

Urg, scary. The scope for those changes are way too wide. That would surely make debugging a nightmare. Never would I use those.

Kris on July 14, 2008 07:34 AM

The experience of maintaining C# code has just diminished to the equivalent of performing a colonoscopy. Way to go. VB is better.

Josh Stodola on July 14, 2008 07:41 AM

There's a reason some of the most deeply respected computer scientists in the world end up as language designers, and there is ALSO a reason why PHP is more popular, and has much more useful code, than Scheme.

The design goals of those scientists are strangely set apart from the design goals of programmers.

Anyway, those examples sound downright tame next to Forth, where you could change "0" to mean something else (for example). And, yet, Forth programmers learned to deal with it.

It's a matter, curiously enough, of tradition. Of learning what works and what doesn't, and passing that unconscious knowledge to your fellow programmers. We are seeing a spurt of innovation which will bring evolution to these languages, and like any evolution, some results are evolutionary dead ends. :-)

Daniel on July 14, 2008 08:08 AM

An example that comes to mind is a JSON implementation I've seen in javascript that modifies the prototypes of all the built in types, breaking anything and everything and that uses a for(... in ...) loop and much more.

David on July 14, 2008 08:23 AM

Could you use a color other than red to highlight the differences? It's very, very hard for us red-green colorblind folks to pick up red text mixed in with black text.

See point 3 here: http://jfly.iam.u-tokyo.ac.jp/color/#assign

Daniel Cormier on July 14, 2008 09:05 AM

Jeff I definetely agree with this one...monkeypatching is certainly more trouble than its worth.

o.s. on July 14, 2008 09:50 AM

Multi-inheritance, function overloading, even lambdas used too much: all are Clever Code.

Clever Code is bad.

Richard Haven on July 14, 2008 09:59 AM

@Daniel Cormier on July 14, 2008 09:05 AM
"Could you use a color other than red to highlight the differences? It's very, very hard for us red-green colorblind folks to pick up red text mixed in with black text."

Unfortunately, to people who are not color-blind, highlighting in red comes naturally. Red is an extremely vibrant color and contrasts well with both black and white - more so than any other color.

Which color do you think he should have used? Or maybe instead of changing the color, he should have just made the highlighted text bold as well?

KG on July 14, 2008 10:08 AM

@ James:
"Prototype plays very nice with any other framework and has for some time. I've used it with YUI and jQuery. Several years ago (v. 1.4, I think) Prototype added some methods to Object.prototype at which point users rebelled and the author, Sam Stephenson, rightly removed it."

First off, the mere fact that the author of Prototype.js thought it would be a good idea to modify the base object prototype is a pretty clear indication of problems at the core.

The use (and overloading of) the $() function also indicates problems, given that the $() function has a specified use, according the the ECMA specification.

I have to say that Prototype.js is a perfect example of monkeypatching.

Tony on July 14, 2008 10:53 AM

I have mixed feelings about monkeypatching.

Generally, it terrifies me. You can do that kind of thing in ActionScript/javascript, and I see it as potential security hole. If any random person can come by and redifine one of your private methods to do something different, then nothing is safe (though hopefully browsers will prevent things like that through XSS prevention).

However, it is also the only reason our product works. We depend on auto-generated SWF files built by a 3rd party tool, and the generated files are incredibly buggy. Monkeypatching allows me to fix the bugs at runtime by redefining the methods that don't work properly (and also allow me to fix a couple bugs in the built-in Flash UI components too).

Andy on July 14, 2008 11:28 AM

I think stuff like this is really useful. It lets you fill in functions that should be there, and that cut down on the amount of code in many other places. Imagine not having a left function, and having to write those 6 lines of code 100 times througout your project in various places. The .Net API now contains the String.Contains function. Which is very useful to find out if a string is contained within another string. In .Net 1.1 Of course, there's always .indexof, but that doesn't really make it completely explicit what's going on. Using indexof is only really used because there wasn't something more correct to use. Allowing developers to extend the core functions provided by the API can be extremely useful. Just make sure you don't go over board, and start adding stuff like String.To1337Speak().

kibbee on July 14, 2008 11:58 AM

Patching built-in or widely used classes definitely seems evil but modifying specific instances of a class can be extremely useful. Which is a good time to remind ourselves that 99% of the time we're actually doing class-oriented and not object-oriented programming. What you really want to do is modify the behavior of objects and not the behavior of classes - otherwise they'd be a different class, wouldn't they?

queisser on July 14, 2008 12:41 PM

Personally I'm fine with Prototype.js (or other framework such as MooTools)... Some might consider it monkeypatching (and I guess it is since they both redefine native objects etc..), but at least both framework are well-documented and adds a lot of functionality to javascript...If you decided to work with Prototype or Mootools, you pretty much accept that they both redefine native object as it's pretty obvious to anyone who spends more than 2 minutes looking at the code or the doc... The lack of documentation is what makes monkeypatching hard to bare with imho...

Jean-Nicolas Jolivet on July 14, 2008 01:08 PM

I've seen this in other languages although I've never actually 'monkeypatched' myself (although I have written a fair number of operator functions in C++ classes).

My question is: what happens in case of a collision? Let's say you define String.Left in one namespace, and then import another namespace that also has defined String.Left. Which takes precendence? Will the C# compiler catch you? Will it complain? Or will it just choose the last String.Left method it imported?

Michael Acobas on July 14, 2008 01:40 PM

@Powerlord - C# Extensions are just a compiler shortcut. The method has no access to the internals of the extended class, and the behavior can not be unintentionally called unless the source 'uses' the namespace+assembly that the extension is defined in.

Please note I'm not a fan of Spartan Programming, and I'm shortening to 'p' only for blog readability :P

e.g. We use a vendors point class, and need to compute area, but the vendor doesn't provide it. I'm assuming we don't have control of the point class, otherwise we'd just change the class.

public class Point { public double A { get; set; } public double B { get; set; } }

Note that the following two methods are semantically identical, as is the calling IL.

// a point class, lacking a useful area method.
public sealed class Point { public double A { get; set; } public double B { get; set; } }

public static class PointExtension
{
// old way
public static double PointArea(Point p) { return p.A * p.B; }

// new way
public static double Area(this Point p) { return p.A * p.B; }
}

public class Program
{
public static void Main()
{
Point point = new Point() { A = 2.2, B = 1.1 };
double areaOldWay = PointExtension.PointArea(point);
double areaNewWay = point.Area(); // same as old way, just looks nicer.
}
}

Note that if a name conflict occurs (i.e. Point later defines Area() method), the class implementation gets preference, and the compiler (currently) doesn't even warn you that your extension method will never get called.

All in all it's a nice language addition when you don't want full blown containment / subclassing.

greg on July 14, 2008 03:24 PM

Jeff,
If you're looking to beef up your library, I just releases the first version of the .NET Extension Library: http://www.codeplex.com/nxl/.

It largely consists of pretty run of the mill stuff, but the focus going forward will be to add classes and interfaces beyond extension methods that form the basis of a good framework (such as an abstraction to logging so you don't have log4net code spread out everywhere).

The first such enhancement is already there, in the form of an ICache interface which abstracts away an improved caching API (so it's easier to test). Here's my favorite sample code from it:

User user = CacheFactory.GetInstance.Fetch
(
"user.{0}".Sub(userId),
() => _dataStore.GetUser(userId),
3.DaysFromNow()
);

The 2nd parameter is the callback to invoke if the value wasn't in the cache - saving you from having to write repetitive code.

Karl on July 14, 2008 05:00 PM

Hi Karl,

While your heart is in the right place, what you've done feels unnatural in C#. Keep going, but try to preserve and augment the C# coding culture rather than change the way C# reads.


Sub: Confusing. Perhaps you could rename Sub() to AsFormat() or FormatFor() because it then reads 'treat this string as a format string' (except only will format 1 value). C# dev's already grok string.Format(), so there's no point introducing new terminology, especially when it's overloading the same concept (conceptually). When you wrote Sub I thought SubString, not substitute (pesky Spartan programming techniques). Unless you've copied all of the overrides as well, you really need to have IFormatProvider overloads as well, and you've just made it a lot harder for code introspection tools like FXCop or even resharper.


DaysFromNow: There's already a really good string format for TimeSpan's, it's of the form "[ws][-]{ d | [d.]hh:mm[:ss[.ff]] }[ws]" and in a library you'd need to provide heaps of overloads.

Considering cacheing should ALWAYS be configurable, you could then load the value from the (dreaded) Xml config.
TimeSpan expirationWindow = TimeSpan.Parse(ConfigurationManager.AppSettings["UserCacheExpirationWindow"]);


In practice I'd never write 3.DaysFromNow() or int counter; counter.DaysFromNow(); Use extensions as if you were adding a method to the class itself. Plug (perceived) holes in the shipped framework and make the syntax easier to read, not just different. DateTimeExtension.DaysFromNow(3) is a lot easier to understand in my opinion, but then is that easier than DateTime.Now.AddDays(3)? Not really.

another example of the kind of code I see all the time:
DateTime startOfMonth = new DateTime(DateTime.Now.Year, DateTime.Now.Month, 0, 0, 0);

versus
DateTime start = DateTime.Now.BeginningOfMonth(); // <-- this is the extension method

bobby on July 14, 2008 08:18 PM

If you love strings (okay.. some strings are kind of sexy, but thats a completly other topic) why do you use them in a safe manner?

Don't do "s.Length == 0" use "String.IsNullOrEmpty(s)" instead and put a little bit more safety in your code. And use curly brackets { } with if .. then .. else. And please. If functions return something follow the microsoft standard and only use one return per function. It's looking a bit longer but...

public static string Left(string s, int len)
{
string returnValue = "";
if (len != 0 && !String.IsNullOrEmpty(s))
{
if (s.Length <= len)
{
returnValue = s;
}
else
{
returnValue = s.Substring(0, len);
}
}

return returnValue;
}

offler on July 14, 2008 10:52 PM

Alright, in order to weigh up for all the {} flying around, I'll repost the final code from my post a few comments up (s/_/ /g):

(defmethod left ((sequence string) (length (eql 0)))
__"")

(defmethod left (sequence (length (eql 0)))
__nil)

(defmethod left (sequence length)
__(cond
____((>= length (length sequence)) sequence)
____(t (subseq sequence 0 length))))


CL-USER(1)> (left "" 3)
""

CL-USER(1)> (left '() 3)
NIL

CL-USER(1)> (left "foobar" 3)
"foobar"

CL-USER(1)> (left '(foo bar baz quux) 3)
(FOO BAR BAZ)

There, now we have a sane model where classes don't have special privileges, so methods can specialize on /any/ argument. Much better!

(Of course, the example code itself is rather silly, but it's just a direct translation of the original code.)

Mikael Jansson on July 15, 2008 01:41 AM

And oh, you should define the interface for the methods -- generic function in Lisp -- as well, in order to be a good citizen.

(defgeneric left (sequence length)
__(:documentation
"Return length items of sequence, starting from left,
or the original sequence if length is longer than the sequence.
Zero length yields the empty sequence."))

Mikael Jansson on July 15, 2008 01:43 AM

I'd like some clarification about monkeypatching being a security hole ( @Andy ). Ok, while overriding a private method is defeating the whole purpose of code visibility and leads to some mess, it's an issue in code design, but as far as I know, private method were never meant as a security measure towards external exploits. Because anyway, if anyone gets as far as being able to override a private method through monkeypatching, you're still as screwed up as if monkeypatching wasn't available in the first place. It's as much a security hole as reflection is.

And as for the XSS attack, it's on browser side, and I don't see the point either. Security doesn't belong client-side, and as far as an XSS attack is possible it's already too late, the same nasty stuff could be done with or without monkeypatching.

Vincent on July 15, 2008 02:33 AM

> Several years ago (v. 1.4, I think) Prototype added some methods to Object.prototype at which point users rebelled and the author, Sam Stephenson, rightly removed it.

Um if you are talking about the extensions to Array, then yes there was a (mini)rebellion by some. But they were not taken out as they only broke badly written code which did not follow standards. See Associative Arrays considered harmful by Andrew Dupont for more info.

Aaron Bassett on July 15, 2008 05:14 AM

@Bobby:
I agree with Sub, it's a shame you can't re-use a static name on an extension method - the extension actually compiles, but in some situations you can't use it (it's really odd, almost seems broken).

Aside from that point, I agree there's value in keeping things the C# way, but I also think there's much to be had by expanding that as well. If we only followed the C# culture, we wouldn't have unit tests until 2008, would be using datasets instead of NHibernate, and would use XML for everything. Some of the extensions that I added were heavily influenced by Ruby, which has a far more open concept and far more elegant syntax. Writing a view in rails is easily an order of magnitude cleaner than in ASP.NET MVC w/C# or VB.NET. For this reason, I think it's good to challenge that status quo, so to speak.

To keep the C# culture the way it is for the sake of making it easy is highly questionable. The alpha dogs are pushing hard towards two extremes - specification-based languages (Spec#), and dynamic languages (Boo, IronPython and IronRuby). I'm not saying C# will die out, but I do think you'll see more and more people take up specialized tools for the job (O/R mapping for your DAL, Spec# for your domain layer, and IronRuby for your presentation - heck most of us are already doing this in some form or another).

C# is by no means a holy grail, and the C# culture - by it's very strong ties to the .NET culture - is far from perfect. To quote a former Microsoft developer, "there are too many people these days calling themselves programmers who have no inclination to test the limits of their languages".

Karl on July 15, 2008 05:28 AM

@Andy and Vincent,
In actionscript 3 the compiler won't allow to override a private function, and the core features that are considered essential are defined as internal or as final which sets a common ground for future versions of the api.
@Kibee, well it's actually not as bad to have a toL33tSp3aK as you can be sure that the language will, most likely, never have a native method with that name.

I honestly think that all of this has been lived through by the older generation of programer with Forth and Lisp, but somehow there's a lot of new programmers that are pushed by the corporate world to just code as fast as possible and disregard whatever consequences the future might bring to their code.

pedro on July 15, 2008 05:51 AM

@pedro

What are you on about regarding Lisp and Forth? "Yeah, those old people don't know any better"

Mind clarify yourself?

Mikael Jansson on July 15, 2008 08:26 AM

I prefer to call it "duck punching", so that you get the appropriate violence invoked.

Note the problem with duck punching is that you immediately lose stability, unless you happen to know every other instance of duck punching being done anywhere else in your entire application.

Classic example:
http://weblog.raganwald.com/2008/07/ive-seen-things-you-people-wouldnt.html

And that's from a smart person who surrounds himself with smart people. Us lowly knuckle-draggers who don't regularly delve into the bowels of Rails are really in trouble.

There's also the problem of tracking changes, even if you manage to get a stable build. I discuss that scenario at length over at my blog.
http://enfranchisedmind.com/blog/2008/04/14/useful-things-about-static-typing/#comment-33124

Limiting duck punching to otherwise undefined functions is a *good idea*, because it gives you a limit on the ways you can shoot yourself in the foot, and a stp when you're about to.

Robert Fischer on July 15, 2008 02:02 PM

@Karl I appreciate your response and surely didn't mean to imply that you shouldn't push the boundaries of the language.

It has nothing to do with the technology you use, and everything to do with how you read code (code clarity).

_I_ don't want C# code that reads like ruby. I want to borrow the concepts, but not necessarily names, especially redefining existing names and concepts. That just makes code harder to read and maintain by someone else.

To me, the point of coding using the accepted idioms is because you accept that language, and you want to read the code, and not read a java person writing ruby, or a ruby person writing java, etc. I write c# to look like c# and javascript like other javascript, and python to look like python. If you want to write python, why are you writing lisp?

There is truly a staggering amount of thought that goes into language design, despite what most of the punters on this blog go on about.

By all means make it better, just be careful you don't invent an unneeded wheel. :)

bobby on July 15, 2008 06:10 PM

@Mikael
I meant that the older generation of programmers (I'm 27 and coding since 11) have been through this before and they managed to handle it (and that's a good thing :) ) and provide us with an insight on how to work with overloading, overriding and so on... Recent languages are usually stricter because _I_ believe the people that designed them have had these problems in the past.
In a perfect world we'd all be coding away in Lisp (a language that allows to redefine it's own syntax) and be awesome programmers, but we're not.
So to cut to the point, I feel that the biggest problem is that the new generation, and even mine, have somehow forgotten to look to the past for answers and somehow keep forgeting that 'monkeypatching' isn't a new problem and has indeed been tackled by plenty of programmers before us. Throw that together with the ever growing demand for shorter development cycles and programmers that don't work hard at understanding and you have the mess you have now.

pedro on July 16, 2008 12:50 AM

Old versions of Fortran let you redefine numeric constants in certain cases. The ultimate in Monkeypatching.

http://coding.derkeiler.com/Archive/Fortran/comp.lang.fortran/2005-01/0487.html

Oliver Jones on July 16, 2008 05:47 AM

@queisser

"""
What you really want to do is modify the behavior of objects and not the behavior of classes - otherwise they'd be a different class, wouldn't they?
"""

Well, sometimes you really want to modify the behaviour for a whole class - like, when the objects of this class are instanciated by a library or framework.


@all_the_bondage_&_discipline_programmers:

As usual, the problem is not with the feature, but with how it's effectively used.

Bruno on July 17, 2008 06:49 AM

@KG on July 14, 2008 10:08 AM

Pretty much anything. Even a lighter shade of red would be better.

Daniel Cormier on July 18, 2008 01:58 PM

The problem isn't monkey punching. The problem is programmers doing stuff that doesn't work.

Several people have mentioned Forth allowing this. Doesn't C allow similar things with macros and the preproccessor?

Forth lets you redefine absolutely anything you want to. For example:

5 constant 4

After this is compiled, every place your source code has the number 4 except inside strings, it will compile a 5 . I can't think of any situation where this is useful, but it's available. Every word can be redefined.

: DUP DROP ;

: : BYE ;

The only place this gets used is for compiling incompatible code together. Code written for different Forth versions can be run together, you set up different meanings for the same words in different scopes.

Forth79 definitions
: NOT 0= ;

Forth83 definitions
: NOT invert ;

The same word got defined to do two different things, one was a logical not that returns a true flag if the flag it receives is zero.

The other is a bitwise invert that flips every bit. If you have ancient Forth code from two different sources and you want to use them both withour rewriting, you probably can.

All this power that hardly ever gets used. Why not? Because you usually don't have a real use for it. But there's no need or reason to add code to the compiler to prevent it, either.

If you make a new function that does something new, usually you do better to give it a new name and not the same name as something else. Less confusing that way. You say what you want and the compiler does it. Simple and easy. You have to know what you want, but when you do know what you want you can get it a lot easier.

J Thomas on July 23, 2008 11:49 AM

Yeah, the scope thing scares me too. I think the fundamental problem is that without monkeypatching, I can always tell where the code I'm calling is defined -- find the definition of Class X, find the function, bam. But there's no obvious browser to show me where the cobbled-on function is defined, and if I don't extensively segregate my namespaces (I routinely have entire 20-class projects where all files are in the same namespace), I could really shoot myself in the foot. It's an interesting idea, but I think we need more experience with the idiom.

James on July 31, 2008 08:15 AM

You're evil. Seriously.

Stefan on August 2, 2008 03:55 AM







(hear it spoken)


(no HTML)




Content (c) 2008 Jeff Atwood. Logo image used with permission of the author. (c) 1993 Steven C. McConnell. All Rights Reserved.