February 7, 2009
The introduction to Head First Design Patterns exhorts us not to reinvent the wheel:
You're not alone. At any given moment, somewhere in the world someone struggles with the same software design problems you have. You know you don't want to reinvent the wheel (or worse, a flat tire), so you look to Design Patterns – the lessons learned by those who've faced the same problems. With Design Patterns, you get to take advantage of the best practices and experience of others, so that you can spend your time on … something else. Something more challenging. Something more complex. Something more fun.
Avoiding the reinvention of the proverbial wheel is a standard bit of received wisdom in software development circles. There's certainly truth there, but I think it's a bit dangerous if taken too literally – if you categorically deny all attempts to solve a problem with code once any existing library is in place.
I'm not so sure. I think reinventing the wheel, if done properly, can be useful. For example, James Hart reinvented the wheel. And he liked it:
I reinvented the wheel last week. I sat down and deliberately coded something that I knew already existed, and had probably also been done by many many other people. In conventional programming terms, I wasted my time. But it was worthwhile, and what's more I would recommend almost any serious programmer do precisely the same thing.
But who's James Hart? Just another programmer. If that doesn't carry enough weight for you, how does it sound coming from Charles Moore, the creator of FORTH?
A second corollary was even more heretical: "Do it yourself!"
The conventional approach, enforced to a greater or lesser extent, is that you shall use a standard subroutine. I say that you should write your own subroutines.
Before you can write your own subroutines, you have to know how. This means, to be practical, that you have written it before; which makes it difficult to get started. But give it a try. After writing the same subroutine a dozen times on as many computers and languages, you'll be pretty good at it.
Moore followed this to an astounding extent. Throughout the 70's, as he implemented Forth on 18 different CPUs, he invariably wrote for each his own assembler, his own disk and terminal drivers, even his own multiply and divide subroutines (on machines that required them, as many did). When there were manufacturer-supplied routines for these functions, he read them for ideas, but never used them verbatim. By knowing exactly how Forth would use these resources, by omitting hooks and generalities, and by sheer skill and experience (he speculated that most multiply/divide subroutines were written by someone who had never done one before and never would again), his versions were invariably smaller and faster, usually significantly so.
Moreover, he was never satisfied with his own solutions to problems. Revisiting a computer or an application after a few years, he often re-wrote key code routines. He never re-used his own code without re-examining it for possible improvements. This later became a source of frustration to Rather, who, as the marketing arm of FORTH, Inc., often bid jobs on the assumption that since Moore had just done a similar project this one would be easy – only to watch helplessly as he tore up all his past code and started over.
And then there's Bob Lee, who leads the core library development on Android.
Depending on the context, you can almost always replace "Why reinvent the wheel?" with "Please don't compete with me," or "Please don't make me learn something new." Either way, the opponent doesn't have a real argument against building something newer and better, but they also don't want to admit their unhealthy motivations for trying to stop you.
More seeds, more blooms, I say. Don't build houses on kitchen sinks. Reinvent away. Most of our current technology sucks, and even if it didn't, who am I to try and stop you?
Indeed. If anything, "Don't Reinvent The Wheel" should be used as a call to arms for deeply educating yourself about all the existing solutions – not as a bludgeoning tool to undermine those who legitimately want to build something better or improve on what's already out there. In my experience, sadly, it's much more the latter than the former.
So, no, you shouldn't reinvent the wheel. Unless you plan on learning more about wheels, that is.
Posted by Jeff Atwood
There are two sides to this coin, and in my recent experience.
On the one hand, there are folks who continue to patch 30 year olde codebases, just because they think it's too much work (it isn't, if you know what you're doing) to do it right this time. So, they keep adding bells and whistles to square wheeled trikes.
On the other hand, the xml/XQuery folk are desperately attempting to make an old fashioned hierarchical datastore as agile as the relational database. They've re-invented the square wheel that was replaced 35 years ago.
The moral: only re-invent if you're *really, really* smarter than the guys who did it first.
Moore's Law. Double your workload every 11 months.
Moore's case here is different. It's not reinventing the wheel. Their product is/was Forth, so it was a technical / strategic decision not to rely on 3rd party code. Their core competency is the language and the compiler and how well the code runs, so it's in their best interests to rewrite that particular wheel in a platform-specific, controlled way.
As others pointed out, Moore's decision to rewrite Forth for every platform might not have been the best use of his time. While he personally may have benefited, it would have driven PHB's insane.
If I'm just making a vanilla website that runs on any old server, I don't start by writing a web server, I use a platform. If I want graphical buttons I don't start with rasterizing routines, I start with the GDI routines or whatever is provided. I only step outside and do it myself when I've determined what's out there is junk.
Maybe I need a web server on small devices. I'm writing the web server because I can't or won't use IIS.
Reinventing wheels does come at a cost. Library code has been debugged, and usually if there's an issue, it's because you're doing something wrong and not the library. If you create the code yourself, you give yourself more points of failure, more code to maintain. You don't want to go to the extreme of doing everything yourself because you otherwise spend all your time redoing what's already done instead of making progress.
FWIW, Forth is embedded in a lot of places where it isn't labeled Forth on the box. Also, Forth influenced a whole lot of designs, in ways you might never think.
I'm often reminded of the early days of the Macintosh, when someone said it would eventually take over the world. In truth, it did. It just wasn't labeled Macintosh on the box, but Windows.
And yeah, I programmed in Forth, and have written various Forths and Forth-like systems. Some were toys. Some were reinventing the wheel. But some are embedded in places you'd never think of.
@ pumpitup Library code has been debugged, ...
Sometimes yes, sometimes no. I am often amazed and astonished at the craptaculosity of some libraries. Yes, some open source falls into that category. The number of open source contributors is no guarantee of quality.
Matt pretty much hit it on the head for me:
Reinventing the wheel is fun for the developer and good for him (he gets to learn how to make the wheel, etc.)
But for the customer, 99% of the time, the original wheel is good enough and all you are doing is wasting their money/time.
Sure - re-invent away if it is on your dime. If you're working on someone elses you've got to have a pretty lucid reason for why you're re-inventing the wheel and get them to buy into it.
On a side-note, once you've seen one wheel surely re-inventing it is not that hard.
Reminds me of an old Joel on Software post: http://www.joelonsoftware.com/articles/fog0000000026.html
And that's where I learned a key lesson in software architecture: for your most important, mission critical stuff, you have to use a tool that is one level lower in abstraction than ideal. For example, if you're writing a cool 3D shoot-em-up game (like Quake, around the same time period) and your key number 1 differentiator is to have the coolest 3D graphics, you do not use whatever 3D library you can find. You write your own, because it's fundamental to what you do. The people who use 3D libraries like DirectX are using them because they are trying to differentiate their games on something other than 3D performance. (Maybe the story line.)
That was a different line of argument, though -- he was suggesting that for stuff that really has to be completely perfect, you write it yourself, because then you can fine-tune it until it's good enough. You argue that reinventing the wheel is good from a pedagogical point of view. I guess both are useful.
Is the reinvention thread-safe?
Who cares? Threads aren't need in most programs.
Does it scale?
Who cares? Most code doesn't need to scale, and when it does, you can always change it then.
Is it reusable?
Who cares? I'm making it for me, not anyone else. If I need to reuse it later, I can always change it then.
On almost every project I've ever worked on, the requirements were a lot easier than they first looked.
Interesting that you bring up forth. As an educational exercise, I did the same thing. I took http://www.annexia.org/forth (Jones Forth) and coded it up on an AVR microcontroller. What did I cover - Subversion, AVR assembler, threaded code and shift division. I've done shift multiplication but shift division was all new. http://www.tofla.iconbar.com/tofla/arm/arm02/index.htm
Think of replicating experiments done by others.
Can't agree more - I recently sat down and created a REST library from scratch to manipulate data on Amazon S3 on Win32 without having to rely on the .Net library. I know that many coders out there would say Duh - the .Net library makes it a piece of cake to write SOAP enabled web clients, but I think I have learned more about HTTP protocols and socket threading from my (sort of pointless) exercise than I ever could by just calling .Net library code.
I cannot put a value on the knowledge I have gained from this one project, and I know that it has made me a better programmer in many ways.
Sometimes, reinventing can get you inspired to think about solutions in a whole new way.
I'm currently working on an existing custom app for a client. They've reinvented the wheel other the years. A lot. They've reinvented syslog. Very badly. They've reinvented SysV init. Poorly. They've reinvented message queues. Weirdly.
And it's full of bugs. Unmaintainable.
I guess the lesson should be, you may reinvent the wheel IF you've looked at existing wheels and they didn't work for you. Not because you weren't bothered to do 5 min of Googling to see if wheels had already been invented.
I think you are ignoring one core aspect of the programmers mind: wanting to do thing Right. Clean. Proper. I often enjoy reinventing wheels that are already existing because I enjoy knowing that my wheel is clean, and round, and doesn't have an adjustable size and 17 sets of tires and a complex AI that decides when to roll. I don't enjoy using third party libraries that may or may not do what I want and may or may not do it in the way I want it to. I'm certain I am not alone with that sentiment.
Reinventing wheels is not always bad, especially not if you are programing for fun instead of productivity. As long as reinventing the wheel is a conscience decision, and not lack of knowledge about existing wheels, it's perfectly fine to do so.
Having said that, I think I could do without programming another linked list in my lifetime ;)
I always want to reinvent the wheel, but you gotta know when it's appropriate and when it's not.
I guess that you should reinvent the wheel when the wheel wasn't invented properly yet. We're all coders, we're good and we should be able to know when the wheel is broken and when it's done to perfection. If it's perfect, you can try reinventing it, but consider using the perfect version.
Hmmm, interesting, interesting, but it all depends on the context. If you are trying to learn how something works under the covers, reinventing the wheel can be fun and rewarding. However, if you write code for a living (or is that professionally?) then this is likely to be a bad thing because:
a) Depending on the size of the wheel, your (teams) implementation is likely not to be as good as another team that has been doing the same thing for years, refining the wheel so it runs smoother and smoother, etc. While this is your first time to the rodeo, prepare to fall down a lot with your square wheel.
b) Writing anything of any size is hugely risky and massively labor intensive in our software development world. Writing “quality” code is especially hard, which is why you see so little of it. It is easy to make a prototype – really, really hard to make it industrial strength – cost lots of time and money.
c) Customers are not interested in you reinventing the wheel, they want it done as soon as possible, as cheaply as possible and as best quality as can be had, as long as it does not take too long or cost too much. See b).
Wow, this post came at a highly apropos time for me. A friend and myself are just beginning a video game project, searching out engines to make our lives a little easier. I'm utterly new to programming, but have a lot of experience in other fields of video game design.
Every single time I sit down to write a script, or start to program, I end up re-inventing the wheel. Was just talking with him about this last night while dredging through libraries and comments, looking for the elusive wheel to fit our car.
I want to reinvent it, while at the same time I have no idea how to start. Our game does the wheel differently, from what I have seen (lots and lots of video games under the belt :D) and I WANT to do it better.
So as I said, very apropos.
I have recently been reinventing a .NET service bus (see nServiceBus, Mass Transit, and Ayende's reinvention Rhino Service Bus). When I first grabbed nServiceBus, I had a hard time figuring out why all its pieces are built the way they are. Taking Udi Dahan's (awesome) SOA course certainly helped there, but I figured that building a service bus myself was the best way to really understand it. With that knowledge, I might just abandon what I'm doing and go back to nServiceBus or one of the other existing solutions.
A secondary motivation is, as J. Stoever said above, that I might want to remove a bunch of the flexibility, in order to enforce my own opinions of how it should work on the users of my bus.
I guess I'm reinventing not just the wheel, but the whole bus.
Well well, what a load of fatuous cockwaffle. Particularly this :
Don't reinvent the wheel unless you're one of the 0.05% of people smart enough to get it right
Hey, sweetcakes, for your information I personally _AM_ one of that percentile that you've pulled out your ass. I am the annoying bastard who can sit down and casually, over a couple of slow afternoons with lots of smoke breaks, sit down and code complicated, correct, software modules in some language that I only heard of yesterday.
I didn't get to be the coding ninja that I am by using some whiney freetard's homework project of a library every time I've bumped up against a hard problem that isn't solved in the language or framework I'm working in. You bone up on the problem domain until you live and breath it, and then you write your code. If you couldn't code it, you certainly don't understand it sufficiently to be throwing implementations of it around. If you aren't capable of doing this in a manner that provides value to your clients, that's a failing on your part, and your clients should come talk to me.
Let me give you some clues you can use :
1) if you think that only 0.05% of coders are competent to do this stuff, you've got a serious problem with your own argument. If I wet my pants every time I see some hard maths and go pick up a library, I've got little chance of finding a good one, since the vast majority of contributing developers will be outwith that percentile, and the majority of code will be crap.
2) Whatever the percentile, what makes you think that we're going to be packaging up and distributing our shit and sharing it with you ? What would be the motivation ? Fuck the open source love fest, because we don't play well with others, and there certainly isn't a financial incentive, being better than you is our USP.
3) If you honestly believe that all the problems a coder will come across have been addressed, solved, understood, and neatly packaged up for everyone to share, you probably spend to much time drawing UML models in crayon and drooling into an HTML reference. Get a real job.
The problem in software today isn't that there are too many wheels, it's that there are too many shitty wheels. Certainly it's possible to reduce the rate at which shitty wheels are created by reducing the absolute rate of wheel creation, but this just skirts the problem.
The solution should be, rather, to produce higher quality software and higher quality parts. Moreover, if there were higher quality components out there (and there are starting to be), people will use them naturally. An excessive abundance of quality libraries and wheels out there is a problem that every software engineer would be eager to have.
Don't reinvent the wheel unless you're one of the 0.05% of people smart enough to get it right.
EXACTLY. Chances are, you are not one of those people. You are not Charles Moore. Charles Moore did not have Visual Studio, .NET and a ton of hardware horsepower.
Who cares? Most code doesn't need to scale, and when it does, you can always change it then.
Arguments agains reinventing the wheel:
1. You are not the brightest bulb in the pack, otherwise your job would be inventing wheels, but instead, you write database front ends
in ASP for a small retail bank, so don't do it.
2. Even if you were the brightest, you probably do not have enough experience to invent wheels, so don't do it.
3. Even if you are the brightest and have experience, the truth is, the already-invented-wheels are good enough for 99% of programming tasks, and you would be just wasting everybody's time inventing obscure 1% wheels.
I think it's safe to say that if you are re-using someone else's wheel, you should expect to have to tune it up before it works for you. In that process you might decide to write it yourself or refactor it enough that it hardly looks like what you started with.
I think what Jeff and the SO devs went through with their editor control is a good example of this.
EXACTLY. Chances are, you are not one of those people. You are not Charles Moore.
And you never will be with that attitude.
There's another reason to write your own wheels: maintenance and comprehension.
If I use somebody else's library/module and they change it, I might have to rewrite MY code. If I use someone else's module and something goes wrong, I have to debug code that I didn't write and that's usually much harder than figuring out what *I* might have screwed up.
Yes, you can take that idea too far. But I think most people don't take it far enough. In general, I'd rather have code I wrote - even if the other guy obviously did it better (which is usually true).
One of the nicest posts coming out of Codinghorror in days. I agree with you completely.
More than half the time the wheel is reinvented because of developer ego and not because there is a deficiency, realistic risk in using the third party library, or willingness to learn. And in my professional experience I have found that the bigger the ego the crappier the code. just because your code is complicated does not mean that you are an artiste.
Good posting Jeff, and I think this speaks to a broader idea in software development: we should always question the rules and basic assumptions that permeate the industry. I don't mean to say we should always break those rules, or ignore the assumption, just question them. Just because Granddad did it and it worked for him doesn't mean I should. Indeed, I propose that the more ingrained or accepted an idea or technique is, the more suspect it becomes. As some famous activist once said: Progress comes one funeral at a time.
Reinventing the wheel is fun for the developer and good for him (he gets to learn how to make the wheel, etc.)
But for the customer, 99% of the time, the original wheel is good enough and all you are doing is wasting their money/time.
Please - from a client of many software projects, build me something that works in the quickest time possible! Don't reinvent unless it is a critical success factor FOR THE CUSTOMER!
There might also be a parallel with optimization here. Unless you exactly know that you'll need a new kind of wheel and how it has to look like, you'll quite certainly be better of starting with an existing wheel. And when your program runs, then you may consider the alternatives - and one might be to reinvent the wheel you started with.
On the other hand, I can't help thinking that the reinvention of the wheel is one of the major problems of our industry. It's hard for me to imagine how the electronic industry would have reached anything if every electronic engineer had wanted to reinvent the transistor before building a new TV or amplifier. But that's a complex issue that would require quite many blog entries to discuss event superficially.
And never forget that the tendency to reinvent the wheel often comes from the not invented here syndrome, or the fact that It’s harder to read code than to write it (Joel on Software http://www.joelonsoftware.com/articles/fog0000000069.html) or not so seldom from the plain ignorance that wheels already exist etc.
My final two cents: don't take the advice not to reinvent literally (as Jeff said), but never think you're so smart that you don't need to learn from the existing wheels and wheels makers - as this would be a sign of abysmal stupidity...
That was really strange seeing the title of the article this morning because I just wrote about the same topic a couple of days ago (a href=http://rgoddard.blogspot.com/2009/02/reinventing-wheel.htmlhttp://rgoddard.blogspot.com/2009/02/reinventing-wheel.html/a).">http://rgoddard.blogspot.com/2009/02/reinventing-wheel.html/a).">http://rgoddard.blogspot.com/2009/02/reinventing-wheel.htmlhttp://rgoddard.blogspot.com/2009/02/reinventing-wheel.html/a). The problem with the advice of not to reinvent the wheel, is that it is over used and does not recognize the fact that in many situations, it is better to redo something yourself.
Meh, some wheels are easier to reinvent than to research.
Often I go to the web to find something to use premade and end up taking more time googling and RTFMing than I take just writing the code myself.
Programming is easy - spending all day reading useless docs or searching the web is the real waste of time.
There are several problems with reinventing the wheel:
* It means your implementation is completely different from other implementations. When it comes time to support your code, the people who have that arduous task will now have to figure out how your wheel works.
* Your implementation will probably stink. If you are talking about Open Source libraries, they have been honed by hundreds of users over time and architected by people who are experts in their fields.
* You could be spending your time elsewhere. If I decide to rewrite some basic modules, I am not concentrating on the parts of my application where I should. I've seen too many projects where weeks have been spent on reinventing the wheel because someone thought they could do it better when they should instead be concentrating on their application and how it works.
This is not to discourage people from exploring and poking. One of my major complaints is how little most developers understand what they are doing. Too many language implemented features, frameworks, and other things designed to make our life easier means we no longer understand how basic stuff works. How many developers can explain the difference between a bin sort, a shell sort, and a bubble sort?
However, before reinventing the wheel, why not poke around the current wheel and see how it works? This is easy to do with Open Source projects. Sign up on the developer's list and download the code. Ask questions. Learn what is involved. Munge the code a bit. Submit improvements. Then, once you understand the basics of this implementation, and you think you can do better, start your own open source project, and provide a bit of competition.
I sympathize with your argument. Right now I'm sitting in my cave, writing permission-related code in ruby on rails. The very idea of using a plugin for this part of the application made me feel very uneasy. I just wanna know everything about it - in detail.
However, throughout my career I did encounter some grossly reinvented flat tires that I don't want to go unmentioned:
- A Webserver written in Java. Using jetty would just have been too boring, I suppose. Oh, and it couldn't do HTTP1.1
- An ex-colleague decided to spend days to write some nifty encryption algorithms that were supposed to prevent crackers from pirating the software. What do you think... did he succeed? ;)
- Months were invested in developing a form layouter, written in Cocoa. I was the guy who was supposed to maintain that code after the offender took off. After discovering that it rewarded resizing the window with a segfault I spent the better half of a Tuesday morning replacing the functionality with WebKit. Amazing how these browser components do form layouting these days! Scriptable.
So before reinventing the wheel, please ask yourself the following questions:
1) Are you trying to learn or just avoiding to read other people's stuff?
2) Is there really something to learn in the topic?
3) In most cases your reinvention will cause a delay. Is it worth the learning experience?
... and of course:
4) Can you do better, or at least just as good the alternatives?
I tend to reinvent the wheel all the time. It really does give you useful insight and experience. It lets you branch out and work on things outside of the wheel afterwards in a more useful way.
Interestingly, the evidence supporting your point is brought forth by two people who are some of the best coders in their field. There is no doubt that whatever wheel they come up with is very likely to be at least as good, if not better, than what existed before.
I sympathize with the idea of reinventing a wheel with the intention of learning more about wheels, but unless you are one of these exceptional programmers, once you're done, you and your team will be better off discarding your work entirely and use the more established version.
(who works with Bob, and yes, he's an amazing Java coder)
Sometimes it is pointless. The existing solution works and works well and it saves time to revinvent the wheels that may exist but do not work well at all.
I fully endorse doing something over, but only if you have the time to learn or the need for something better than what already exists.
All too often, we don't have the time and it's often considered quicker and faster to either live within the pre-existing limits or even live with its flaws.
If you ever wondered why so much code is crap, it is because it is filled with compromises - workarounds for things that fall short and patches for things that are just plain broken all to meet timelines and budgets.
I have read the maxim 'don't reinvent the wheel' so many times, but I'm a mathematician, not a programmer - so the programming I do is about enjoying a hobby! To me it's all about 'reinventing the wheel'; both to learn more about wheels, but also to learn about programming.
I'm glad to finally see a caveat about 'reinventing the wheel in order to learn more about wheels' (and I would add, 'to learn more about the tools one uses to make wheels').
I agree with David W.'s comment: Your implementation will probably stink. Look at http://thedailywtf.com - many of the posts are developers who reinvented wheels (often those provided by standard libraries) and did so poorly. Is the reinvention thread-safe? Is it secure? Does it scale? Did you write tests? Has the code been reviewed? Is it reusable?
I've certainly encountered buggy and ill-fitting third-party libraries. One size does not fit all. But that seldom justifies redeveloping the feature set on a project. Often another library will be a better fit.
Many code smells and design principles can be boiled down to Don't Repeat Yourself. Remove duplication when you find it. Reinventing the wheel is the ultimate violation of the DRY principle.
Everybody loves reinventing wheels.
Lots of people here are geniuses; just ask them.
And are so much smarter than everyone else. I might not be smart, but I sure am used to being on change control boards.
Here is an engineering decision making algorithm. It's not mine: just common in engineering professions.
1) Are you going to SELL the (X) ?
2) Is (X) part of your core business ?
Because we do software, we have the option of
3) Can you find an open source (X) that does the job ?
If its open source, you never need to upgrade just because the vendor tells you to.
Use an open source library that is good enough. ( remember, worse is better)
If there isn't one, write the code. Budget to have to maintain it forever. After all, it's unique. But that's okay, because you're a genius.
This algorithm does suggest never using anything that isn't open source. Not bias, it is merely an exercise in engineering configuration management. Can you spell engineering configuration management ? Our next phrase will be professional liability. A sentence using that word: Aren't we lucky that software developers have no professional indemnity.
If the customer is not interested in long term support ( less than a couple of years or so) then grab whatever vendor provided crud you can, take their money.
(Then in two years take it again.)
Learn by doing. Try doing engineering. Coding is scratching an itch. Don't scratch yourself in public.
I guess it's all a question of attitude. If you attempt to reinvent the wheel for learning and improvement, after knowing what's out there and without any tight schedule, great.
But don't do it out of lazyness to understand the existing solutions or on projects with serious time-constraints.
Also, don't try to reinvent everything at once.
Pure hubris to say it's worth the time to reinvent the wheel. I've been on too many projects where the architect said, Hey, let's create our own data structure kit/UI layout template manager/time date libraries because the ones out there just don't do what we need...
In EVERY case, they wasted their time, and ultimately, the company didn't get product out the door because of it. Every time.
Don't reinvent the wheel unless you're one of the 0.05% of people smart enough to get it right. The time and effort were already spent getting it right by people who needed it more badly than you.
With that said, the JoS post stands--if it's your core business, you'd better write it yourself rather than depend on someone else. But 99% of the Reinventing I've ever seen, was not this type of code.
Perspective is key here.
Doing something that you know can be done is not invention.
Sounds like a Mikrosopht .NET propaganda. Anyways, does writting a compiler for a new programming language means reinventing a weel? If not, how come there is so much computer languages out there? :P
Sometimes you have no choice but to reinvent wheels though.
Look at how many productized wheels cannot be relied upon because the vendor turns them upside-down from release to release, with releases coming and support dropping on a 2 year cycle.
Things like Biztalk or SharePoint from Microsoft. Things like FileNet or WebSphere MB from IBM. Unless you're creating one-off demos you can't even use these as building blocks, certainly not in enterprise applications. They're moving targets.
The same has even become true of underlying platforms, like .Net, Java, and their ecosystems.
good post. personally, i love reinventing the wheel as a learning experience, and sometimes its possible to outperform existing solutions... but 3rd party libraries are still an invaluable tool. :)
reminded me of this:
Reinventing the wheel certainly is annoying, but as long as all other
wheels are square...
To just throw a word into this discussion that wasn't mentioned before: Copyright.
The good thing about reinventing the wheel is that you can get a round one.
Douglas Crockford's response about JSON as reinventing XML's wheel.
This is fine advice for the business owner who is not consulting for someone else, but don't reinvent the wheel on the clock. Unless you and the person who pays the bills have explicitly agreed that a whole new wheel is one of the necessary deliverables of a project, your self-indulgent exercises in software experimentation are not appropriate. Also, keep in mind that your individual work may be part of a larger effort in a long-lived product. If you leave the beaten path, you had damn well better leave some breadcrumbs behind you for the next poor developer to follow. Good, clear documentation of some sort is essential. Otherwise, your super duper, way cool code is going to be unceremoniously dumped shortly after you turn your back. Finally, be absolutely certain that you are actually doing it better, and are not missing good reasons for using a more standard approach. For example, don't make changes that will require a total rewrite if an underlying framework is to be upgraded.
Wow, what troll bait. I'm ashamed to post a comment on a blog so desperately posting purely to start a flame war. However this needs to be said.
Your blog posts are read by many people with impressionable minds. Please stop spreading idiotic ideas like this. While I'm sure in the .NET world there is much less choice for free high quality libraries, for the rest of us, we have smart people writing good libraries that we can reuse in our projects to save us time and make us more money on the bottom line. Should I write my own openid implemention or use someone else's, which has been tested and debugged by hundreds or thousands of devs? The answer is pretty obvious.
Time is the concern in this argument - low hanging fruit and wheel reinvention are both ways to speed up development time while making an attempt to keep quality up as well. If you're under a deadline, and know you won't have much time for testing before release, sometimes it's a better bet to go with something established and 'safe'.
But you're totally right - if you don't have that constraint, then it's completely to your advantage to 'DIY'. You learn more by doing then anything else - and if you always do it yourself, you're going to keep learning. Which is important, if you're a programmer.
I think reinventing the wheel in a sane way is more complicated than just IF you.want_to_learn(wheel.internals) THEN you.build(wheel) ELSE you.sleep).
There are certainly strong factors that make you rebuild a wheel.
a) Think licensing trouble. Let's say, you have invented some sophisticated image recognition tool that is far better than all the others on the market. Do you want to use that GPL-library, that makes you give out all the source-code (and thus, all your secrets?) Certainly not. Thus, reinventing the wheel, ho.
b) of course, as Jeff said, learning about wheels. If it makes you a better user of the library or generally a programmer, rebuild it. However, if you rebuild something for yourself, you should judge very very carefully where you use it. You certainly don't want to use a 3D-engine you built in a few days for educational purposes in order to build the glorious Quake5 or Quake 6 or use your own HTTP-incompatible webserver in production applications.
c) think of maintainability and especially extensibility. For a certain size of software or software components, it is actually easier to just build your own wheel instead of searching and using an external component. Fixing a bug in your own solution will be something like Adding a unittest, searching the codepath thatis bad and fixing something. Of course, you do not want to reimplement matlab or pthread with that, but many logging frameworks contain a lot of cruft.
Of course, and I will repeat this, you need to think carefully if you want to release your new wheel into the wild, because if you release something into the wild, prepare to add to the pool of low quality / awkward solutions, prepare to maintain, prepare to extend. But if you keep this in mind, then you can reinvent the wheel and release it.
I went out on a limb a month or so ago, and wrote an article about the exact same thing. If I only knew I was in such good company, I would've written it sooner.
I've been successfully reinventing wheels for a few years now. My code almost always work better *for my purposes* than any of the preexisting options. But as you said, the only reason that's true is because I exhaust all of the preexisting alternatives before I write one line of my own code.
Bob Lee's quote hit home for me most of all, saying in just a few sentences what it took me a few paragraphs to get out.
Thanks for the great research Jeff. If anyone's interested:
So there is this point:
This later became a source of frustration to Rather, who, as the marketing arm of FORTH, Inc., often bid jobs on the assumption that since Moore had just done a similar project this one would be easy -- only to watch helplessly as he tore up all his past code and started over.
And I have to agree with Rather. In the end you need to ship something in a reasonable amount of time. Users don't are about the internal prettyness of the code. Compiler and library writers get a bit of a break because of the hopefully massive reuse of their systems but general programs, ha!
Just look at Forth, while there are were a billion implementations where is it today? Maybe if the focus was on getting something out the door things might be very different for Forth. I wonder how many contract where soured or outright lost due to Moore re-writing code that was working.
Don't get me wrong, as a programmer I love re-writing code. Polishing it to a nice shine. Getting the change to work it over again in the future. But I have to eat and it's not my decision anyway. It's whoever is paying the bills. So I code in my spare time. Sometimes for work, sometimes for fun projects.
I have to ask: is this post motivated by the fact that stackoverflow.com contains a lot of reinvented wheels and you feel a bit guilty about it and you are in need of some positive reinforcement?
The sacred 80/20 rule applies here!
If the wheel is in the 80% which does 20% of work and that don't make competitive difference, don't waste time reinventing the wheel, spend it to do something more useful.
If the wheel is in the 20% which does the 80% of work, and usually make the competitive difference, you would be fool to not, at least, try to reinvent a better wheel... since you are selling wheels, or, you are selling for how good your weels are!
build me something that works in the quickest time possible! Don't reinvent unless it is a critical success factor FOR THE CUSTOMER!
Ah Matt, do you mean works right now or works for a long period of time and is easy to maintain?? If the stuff is throw away then yeah, but otherwise it's better to take a little time thinking. Hardly anything is throw away, even the stuff that's meant to be.
Design Patterns - I use frameworks built on them. Very rare to actually have to dust the books off. I find the whole DP debate rather stale, I do have recourse to the law of Demeter and avoiding train wreck dependencies blah.hlsd.blu.gordfe.yyy == 3 kind of thing. But that's just common sense after it's burned you a few times.
i try to explain this basic concept to a friend of mine who always gets in over his head with big grandiose projects - how are you going to do something that has never been done before if you can't even competently do something that HAS already been done before?
I agree, reiventing the wheel is part of learning and i IMO essential (but do it on your own, at your own expense since it's most often to your advantage alone)
I'm never confortable with using a library to do something that i haven't done before, i have to know what it's doing (70-80% at least, i don't need to know the exact implementation).
If you were building a car, you'd want to know everything about the wheels so that you can find the most suitable set and learn about how they impact the product.
I *ALWAYS* reinvent wheels, pretty much on a weekly basis. Why would anyone do anything that stupid? The first wheel was made out of stone. The person creating the first wooden reinvented it. The person creating the first rubber one reinvented it. The person creating the first rubber one filled with air reinvented it. So if you need a wheel today, will you buy one made of rubber and filled with air or will you buy one made out of solid stone? I'll guess you go for the first one... but if nobody had ever reinvented it, your car would still use stone wheels.
When I go out to buy an actual wheel, I don't reinvent it. Why? I have absolutely no idea of wheels. I don't know how to make a good wheel. I don't even know how to make a poor wheel. And I'm absolutely the wrong person to come up with a brilliant idea how to make better wheels.
I'm a programmer. I have no idea of wheels, but I have ideas of code. Taking code from someone else means:
- I must assume he created the fastest code possible
- I must assume he created the most resource friendly code possible
- I must assume he created bug free code
- I must assume he created the best possible interface
and then I just use his code and it saves me hours of work. Have you ever considered that none of the above might be true? His code may be way slower than possible, using way more resources than necessary, it may have (still undiscovered) bugs, and the interface might really suck. What makes you believe, just because he has written some code, that this is actually perfect code? Because he's a genius? The programmer of an API may have half the programming knowledge you have, one quarter of your experience, and his grade point average in math might have been two grades worse than yours throughout his whole education career. Do you know that is not the case? However you are willing to just blindly trust his code? Why?
Don't get fooled by the fact that thousands of other projects use his code. Just because thousand other team leaders are too lazy to reinvent that wheel and just take the library already there says nothing about the quality of this library! The only you code of that you can control the quality is your own code (unless you do code audits on other people's code).
I have already reinvented the wheel just to find out that when benchmarking my code, it beats the original code in speed, it uses less memory, it is multithread-safe (while the original one was not), and the API exactly meets my needs. I already reinvented the wheel just to find out my code sucks in speed, it eats memory like popcorn, it has no advantages over the other code and my API doesn't look much different to the original library. Wow, big waste of time, right? Not necessarily. First of all I proved that the exiting library is of high quality (or let's say the quality is better than the best thing I can produce)... alone this insight was worth the work. And the second interesting thing is that even knowing my wheel is worse, it may be still good enough for my project and the advantage is that I have total control over my wheel, while the development of the other one takes part behind my back.
A huge community is busy re-inventing the wheel with GNU Linux, every UNIX utility was entirely re-written and in many cases re-specified differently.
Standing on the shoulders of giants.
As a scientist, I know the humbling experience of thinking something up, looking for similar work in the literature, and finding stuff that far outstrips my simple thoughts. From 1973. I would say that the more complex the wheel you're reinventing, the more likely it is that there is a better solution out there, that works in more cases, in parallel.
Hopefully if you reinvent a wheel, or at least reimplement one, you'll see the size of the giants on whose shoulders you were standing. If they turn out to be small, great. If they turn out to be tall, be prepared to actually use the 3rd party version.
Here's to the day you can just contact the guy who implemented this particular wheel before, and discuss the weaknesses and strengths of it.
I hope people will read it correctly, particularly the learning part... :-) Ie. they won't read it: Always to your own code!
There are several cases where one shouldn't re-invent the wheel: the most obvious is lack of time. If my manager asks me to add PDF export to our Java application, and do so while meeting the deadline, obviously I won't download the PDF specification and start coding the implementation: I just will hop to iText library (at best look at some alternatives) and learn how to use it.
While I would love to do the former, probably lot of fun, it isn't realistic, we don't have years ahead of us.
Now, in my free time, I will code my own algorithm of Delaunay triangulation, because it is fun and interesting and I will learn a lot in the process, and I have no deadline and bugs are mundane here.
And as others point out, to re-invent the wheel, you have to have some capacity to do it right, at least if you must use it later... I wonder how many posts on StackOverflow ask: Is my regular expression of e-mail validation right?. Often, it is just No, even ignoring extreme/rare cases.
I remember your post where you state It is my work, I live out of it, I must code it myself and master all I do. It was about sanitizing input, if I recall correctly. I must agree here, and I suppose you did lot of research, looking how others did, what crackers uses to attack, etc.
Now, you are not a fool and use good ready-to-use products, like WMD and such. :-D
Happily, Sturgeon's Law characterizes all human endeavour. Ninety percent of what we do is crap. The trick is to figure out which ten percent to keep. Of course, ninety percent of that then becomes crap, by definition. So, don't be surprised if ninety percent of the library you're linking to is crap. In many cases your executable will only invoke a small portion of the crap library, but rest assured ninety percent of that will also be crap.
Of course, if the standard deviation is small, the best ten percent isn't much better than the worst ten percent. In any case, if your customer is foolish enough to pay for it, re-invent away as long as they'll give you a charge number. (Why does Firefox think endeavour is mis-spelled? Isn't there a Space Shuttle known by that spelling?)
We're programmers because we love to solve problems; who cares if the problem has already been solved.
I'm really glad you posted this, Jeff.
I have gotten the vibe from many of your posts that you advocate never writing your own code, choosing instead to mash as many libraries and frameworks together as possible and writing the minimum amount of code yourself.
I have always felt this approach to programming was wrong, and on some level, even... offensive? Not quite the word, but there was something about the vibe I got that was seriously off. That programmers should never do anything new because they'd save time if they just reused everyone else's code.
Unfortunately, nothing new will ever get written under such an approach. Nothing existing will ever be improved upon or polished, either.
There's definitely a time and a place for reusing code, or for purchasing components that can slot into your project to speed development time. Charting components are a good example of effective reuse - in most situations, the pre-built ones offer all the features you will need.
The reuse mantra easily goes too far, though. Sometimes, it really is better to write it yourself, if for no other reason than it eliminates a potentially crippling dependency. Relying on external companies for critical functionality in your system introduces unnecessary risk in ones business. Hopefully the bean counters won't interfere too much in making these sorts of decisions.
So that's why I'm glad you posted this - I think it presents a much more considered view of how programmers should think. I think it is very useful to create new wheels from time to time, and the ability to create wheels is a necessary condition to being considered a serious programmer.
@Bobby, do you write your own jpeg decoding routines? Nah didn't think so. Soapbox--
I don't know how people started flaming on Java/.NET with this post.
As Simon Boulton wrote, the key point is the last sentence:
So, no, you shouldn't reinvent the wheel. Unless you plan on learning more about wheels, that is.
It's about our own attitude as programmers. And it must be a conscious choice, each time:
1. You understand that you are talking about a specific wheel
2. You see which models are already out
3. Do you want to improve yourself, by learning how to do this kind of wheel?
Of course, the answer to this question cannot be given, if you don't create a context for it:
- Are you doing it because other wheels only do 90% of what you need?
- Are you doing it for fun or are you getting paid for it?
- How long will it take to understand and reinvent?
If you are doing it as a real job then remember this: your customer doesn't care about the wheel; he only cares about the car color.
- Does it show on the UI? Then it's always the most important thing in the universe
- Is it a library? A protocol? A sort pattern? the customer simply DOES NOT CARE. It took you 1 week to do it? Good, next time he'll give the work to someone else who does it in 1 day.
There is no simple answer to should I reinvent this wheel?. So, what's the point of trolling on this?
As for me, my solution is: does the existing wheel cover 100% of what I need? Is it sufficiently used and tested? If yes, I use it, go home 1 hour earlier, and go play soccer :-)
I think it is very important to know what libraries exist and how they work. I have always looked to improve on lame methods and simplify wherever possible. That meant analyzing the wheel in front of me and saying, I think Nitrogen filled tires would last longer! - LOL!
If the tire is blown and you are part of a pit crew in the Daytona 500 (the client/company depends on results), that is not the time to plant the rubber plant seed and wait for it to grow. Grab a new tire and get on with the race!
btw, is anyone else sick of people who throw around Moore's Law any time they want to talk about any sort of quantity doubling over any sort of time period?
You are trying to sound smart and intellectual, but you in fact sound like a try-hard idiot. Stop it.
All generalisations are bad :)
In production: Pick which wheels work the way you want them too, or can be modified to do so more easily than re-inventing. Re-invent the rest.
When learning: Do whatever the hell you like, as long as it teaches you something. (Brainfuck, anyone?)
For leisure: Do whatever the hell you like, period.
It's that simple. Next?
Too many people don't want to reinvent the wheel, but don't have a wheel to start with.
uh, just to clarify, my previous comment was not directed at Jeff, who in fact did not mention Moore's Law at all. ^_^
I would agree the reinventing the wheel is a valuable educational exercise. But to do it continuously when there are better options wold indicate that you have no deadlines and or no lives.
We dont reinvent the wheel, we implement wheels of sizes and materials that we need.
Yes, it's another one of those strange 'memes' that programmers seem to get stuck on. If you take a mostly good idea, and go completely off the top with it, to some extremely remote point-of-view, chances are it's not such a good idea any more.
If you wrote it, it will be easier to deal with. The state of our industry is such, that just because the professionals wrote it, doesn't intrinsically mean that is will be any better than you're own solution (especially if you do your homework first).
Yeah, my problem is that reinventing wheels is FUN! I would much rather code a wiki engine from scratch than drop one in. But I have grown to realize that this isn't always best for my customers. So a happy medium is to _spike_ reinventing wheels, and only use it for production code if your spike can differentiate itself from other products and justify more time. More often than not, this helps understand why others' wheels are built a certain way...
Good post. I can now sell this at work! :)
I've been saying for the longest time, that reinventing isn't necessarily a waste. Atleast it helps you discover the right tool for the job. Even if you decide to throw away your wheel, you can ensure that the one you pick instead will suit your needs better and you can be 100% sure if you started with trying to re-invent it :)
Btw, for all those who say that the projects that have tried re-inventing the wheel never went on time, here are some possible reasons why?
1. If you get your scheduling/priorities wrong your project will be over-budget and not on time. It has nothing to do with re-inventing the wheel.
2. Don't try and re-invent something that isn't absolutely necessary to re-invent. Well actually, try to re-invent but do it quickly, and drop it when you realize you are better off using an existing implementation.
For instance, I had to re-write hashmap implementation in Java once, since I needed synchronization but one that uses semaphores. In that case I had to implement it. For all other times, I stuck with the standard implementation.
Richard Feynman always advocated reinventing the wheel.
What I cannot create, I do not understand. --Richard's last words on his blackboard at the time of his death.
I find that people who stick with design patterns tend to never be able to think outside of their patterns (outside the box), and great creativity comes from being able to make the leap to a new way of thinking.
My position has always been more of Re-invent the wheel if you need a better one. Wheels are generic in purpose, and sometimes you need a more specialized version. The wheels that go on nascar vehicles probably don't belong on your skateboard, for instance.
Of course, the danger of that approach is that you start to think of anything you want to create as a specialization/customization of something that already exists- IE, the situations where you don't really need a wheel in the first place.
We tried using the wheel lib, but our app kept rolling away!
That's easy. We'll just write our own Square Wheel.
Yeah, we'll call it a 'block'!
In my response to Joel Skeet Facts ( http://stackoverflow.com/questions/305223/jon-skeet-facts/316229#316229 ) someone edited it by removing one of the facts in which I used the world porn. He made a comment on my response saying:
I have removed the quote about p*rn, because I find it really offensive.
I, on the other hand, do not find it offensive. I will not edit it back to the original because I don't want to start a flame war, but still how you deal with situations like that?
@otherguy: Of course not... he's too busy working on his OS.
@Lepto: Ha! That was my username for a brief period way back when :)
@otherguy - that's exactly what I'm currently doing in my spare time. And yes, I do it because I like to learn how JPEG works, not because I need a JPEG decoder.
The flip side of third party code is that it can work well 95% of the time, but the 5% of the time it doesn't work can soak up 95% of development time. This can be due to actual bugs, gaps in functionality (something you think the library *ought* to do, but doesn't), or my favorite, unstated assumptions about input or program state that are not covered by the documentation and cause apparently valid code to fail.
So when deciding whether to reinvent the code or not, it's a question of how much leverage you get (time invested vs time saved) and whether the time spent / pain endured of debugging other people's code is less than that of debugging your own.
Of course, rewriting things for fun is its own reason. And also you should write your own code if it is central to your business or product. If you outsource your core code, you are not longer building your own product, you're just customizing somebody else's product and lose any competitive advantage that goes along with having your own code tuned to your needs. This goes for both proprietary and open source applications.
Your implementation will probably stink. If you are talking about Open Source libraries, they have been honed by hundreds of users over time and architected by people who are experts in their fields.
Um, that's not *always* the case. Yes, the Linuxs of the Open Source world have a bunch of contributing developers (less than you'd think actually, the number is in the 150 range IIRC), however there are those projects out there created by 1 or 2 people that for one reason or another get put into mainstream. Folks who come along and use it later can and usually do discover numerous bugs in said project.
What I'm saying here is that what happens when you're one of those experts in your field and you're working on a project that hasn't been honed yet?
I think the real key to all of this is knowing *when* it's time to reinvent the particular wheel you're using, or at least when it's time to tear down the wheel and rebuild it from scratch.
The attitude of meh, it's good enough will swim up and bite you in the ass every time.
I posted something similar to this on my blog. Rather than use libraries all the time, I frequently write redundant methods. I see it as a way to expand my skill-set and become a better programmer.
As much as I like efficiency, I like my pride more.
It's when we as developers stop knowing how the wheel even works, that we start going downhill. If you want to understand something, try building it. I'm not saying go with it afterwards, but without that knowledge can you even truly appreciate using that design pattern, that framework, that fill-in-the-blank? Speaking of course for myself, I can't.
Applies to everything for me. I don't yet (well there are some exceptions that I have gone through) have an excellent set of tools (not speaking about programming tools but physical tools like screwdriver, saw, drill, etc.) - actually have really crappy ones. When I reach the point of really knowing how crappy they are, i can *then* rally value and cherish having a really good set.
I finally reached the point of buying a variable-speed drill + screwdriver - wow it's nice to have! I don't want to buy sod, let me cut my teeth and try to grow my own grass. I don't want to buy something new, let me first buy something used and try to fix it up. I'm sure one of these years I will learn the value otherwise, but until then, I'm up for learning a thing or two.
It really comes down to: Reinventing the wheel is bad, unless really knowing that much about wheels is more valuable to your project than just having a wheel.
Software development is given a lot of different analogies, one of which is often that of a craftsman or carpenter. The techniques of carpentry have been refined and improved over thousands of years. But a carpenter, facing a new project such as a bookshelf, has to take that knowledge and apply it to a physical piece of wood. This is not as true in software engineering, where the knowledge of a thing and the thing itself can be wrapped up in a package and shared across projects.
I always thought it'd be neat to tstart with a project, such as a chess simulator, and write it from the ground up. The ideas for algorithms could come from other sources, but the actual work would be done by hand. This project wouldn't ever really have an ending point; as you learn better and faster algorithms you could apply them back to your code. Kind of like a car enthusiast tinkering with his Mustang on the weekends, trying to eek out the last bits of speed.
This type of a project would also help with learning new platforms. Want to build an iPhone app? You could take your core chess program, expose it as an api, and then build an iPhone app that queries the api.
I enjoyed this post.
I agree that the best way to learn is to try something that has already been done.
I created a Notepad clone in .NET that both works tabbed and as a Multiple Document Interface. In a couple of hours, I learned a about working with GUI front-ends, opening files, saving files, closing files, pre-print processing, passing objects through the constructor, overloaded constructors, etc... It was one of my first .NET projects and in the 4.5 hours it took me to write it I saved probably 20 hours of reading books and studying examples on the web. Not to mention that I reused 90% of the parts on a future (and paid) project either directly or as more advanced features (hardcopy to file, print screen to printer, post image processing to create inverted gray-scale, etc...).
I'm a firm believer in having my own collection of programs that are diverse and interesting to me. I use them to learn and discover the features and capabilities of each language and later as references when I forget exactly how something works. Modeling it after an already existing program or concept is like skipping the specification phase because it's already easy to visualize the implementation. If you extend it and make it a lot better than the original, it may become good material to spark an OSS project. Without that concept there would be no Winamp, no Notepad++, no Songbird, no Juice, no Miro, no Firefox, etc...
You don't want to re-invent the wheel, so you look at design patterns ? I thought the patterns were a tool, not the wheel.
@Evan Plaice: you should put it on the net. Who knows it might become popular and you might get stinking rich or famous and have all the girls.
It really comes down to: Reinventing the wheel is bad, unless really knowing that much about wheels is more valuable to your project than just having a wheel
I think you're ignoring a key aspect of the programmers mind: trying to make things "right." "Clean". "Correct." I often use the wheel that already exists, because I know that I enjoyed the wheel is clean and round, and does not have an adjustable size, and 17 sets of tires and complex AI that decision, when the roll. I do not enjoy the assistance of third-party libraries which may or may not do what I want, and may or may not do it how I want it. I am certain I am not alone in that sentiment. http://rapidbeetle.com
you can reinvent the wheel like the latest inventions have reinvented it. There are triangular wheels and square wheenl but the only difference is that they are on robots and they have strips of metal chainmail surounding them.
The key thing is that every coder should know *how* to create a wheel. Because if they don't understand the concept well enough to re-implement it, how can they understand it well enough to build more complicated things on top of it?
Jeff ended his post with:
So, no, you shouldn't reinvent the wheel. Unless you plan on learning more about wheels, that is.
For me, that's the point. Anyone who cares about coding as a craft (which I do) will go and re-invent wheels on the basis that the exercise makes you a better coder. Occasionally you even come up with a better wheel, but for me, that's not the primary reason.
I've always hated that particular cliché, not only because it discourages potentially useful endeavours, but also because the analogy is completely and unequivocally wrong.
The wheel is not an *implementation*, it's a *concept*. Creating a new implementation is not reinventing. And in fact, we re-implement the wheel all the time; bikes, cars, trucks, luggage, dollies, wheelbarrows, robots, and office chairs all use completely different types of wheels with completely different specifications. We would not have gotten very far as a society if all of these designers had said, oh, we don't need special wheels for this, we can just use those big stone ones.
This is the biggest difference between engineers and so-called developers. Engineers believe in specifications: what exactly do we need this component to do? If something actually meets the spec fully, then you buy. Otherwise, you build your own or you re-evaluate the design for appropriate trade-offs. You never, ever just look at the glossy brochure and say oh, that one looks good, let's use that.