July 6, 2009
Remember that Stack Overflow thing we've been working on? Some commenters on a recent Hacker News article questioned the pricing of Stack Exchange -- essentially, a hosted Stack Overflow:
Seems really pricey for a relatively simple software like this. Someone write an open source alternative? it looks like something that can be thrown together in a weekend.
Ah, yes, the stereotypical programmer response to most projects: it's trivial! I could write that in a week!*
It's even easier than that. Open source alternatives to Stack Overflow already exist, so you've got a head start. Gentlemen, start your compilers! Er, I mean, interpreters!
No, I don't take this claim seriously. Not enough to write a response. And fortunately for me, now I don't need to, because Benjamin Pollack -- one of the few people outside our core team who has access to the Stack Overflow source code -- already wrote a response. Even if I had written a response, I doubt it would have been half as well written as Benjamin's.
Developers think cloning a site like StackOverflow is easy for the same reason that open-source software remains such a horrible pain in the ass to use. When you put a developer in front of StackOverflow, they don't really see StackOverflow. What they actually see is this:
create table QUESTION (ID identity primary key,
UPVOTES integer not null default 0,
DOWNVOTES integer not null default 0,
USER integer references USER(ID));
create table RESPONSE (ID identity primary key,
UPVOTES integer not null default 0,
DOWNVOTES integer not null default 0,
QUESTION integer references QUESTION(ID))
If you then tell a developer to replicate StackOverflow, what goes into his head are the above two SQL tables and enough HTML to display them without formatting, and that really is completely doable in a weekend. The smarter ones will realize that they need to implement login and logout, and comments, and that the votes need to be tied to a user, but that's still totally doable in a weekend; it's just a couple more tables in a SQL back-end, and the HTML to show their contents. Use a framework like Django, and you even get basic users and comments for free.
But that's not what StackOverflow is about. Regardless of what your feelings may be on StackOverflow in general, most visitors seem to agree that the user experience is smooth, from start to finish. They feel that they're interacting with a polished product. Even if I didn't know better, I would guess that very little of what actually makes StackOverflow a continuing success has to do with the database schema--and having had a chance to read through StackOverflow's source code, I know how little really does. There is a tremendous amount of spit and polish that goes into making a major website highly usable. A developer, asked how hard something will be to clone, simply does not think about the polish, because the polish is incidental to the implementation.
I have zero doubt that given enough time, open source clones will begin to approximate what we've created with Stack Overflow. It's as inevitable as evolution itself. Well, depending on what time scale you're willing to look at. With a smart, motivated team of closed-source dinosaurs, it is indeed possible to outrun those teeny tiny open-source mammals. For now, anyway. Let's say we're those speedy, clever Velociraptor types of dinosaurs -- those are cool, right?
Despite Benjamin's well reasoned protests, the source code to Stack Overflow is, in fact, actually, kind of ... well, trivial. Although there is starting to be quite a lot of it, as we've been beating on this stuff for almost a year now. That doesn't mean our source code is good, by any means; as usual, we make crappy software, with bugs. But every day, our tiny little three person team of speedy-but-doomed Velociraptors starts out with the same goal. Not to write the best Stack Overflow code possible, but to create the best Stack Overflow experience possible. That's our mission: make Stack Overflow better, in some small way, than it was the day before. We don't always succeed, but we try very, very hard not to suck -- and more importantly, we keep plugging away at it, day after day.
Building a better Stack Overflow experience does involve writing code and building cool features. But more often, it's anything but:
- synthesizing cleaner, saner HTML markup
- optimizing our pages for speed and load time efficiency
- simplifying or improving our site layout, CSS, and graphics
- responding to support and feedback emails
- writing a blog post explaining some aspect of the site engine or philosophy
- being customers of our own sites, asking our own programming questions and sysadmin questions
- interacting with the community on our dedicated meta-discussion site to help gauge what we should be working on, and where the rough edges are that need polishing
- electing community moderators and building moderation tools so the community can police and regulate itself as it scales
- producing Creative Commons dumps of our user-contributed questions and answers
- coming up with schemes for responsible advertising so we can all make a living
- producing the Stack Overflow podcast with Joel
- helping set up logistics for the Stack Overflow DevDays conferences
- setting up the next site in the trilogy, and figuring out where we go next
As programmers, as much as we might want to believe that
lots_of_awesome_code = success;
There's nothing particularly magical about the production of source code. In fact, writing code is a tiny proportion of what makes most businesses successful.
Code is meaningless if nobody knows about your product. Code is meaningless if the IRS comes and throws you in jail because you didn't do your taxes. Code is meaningless if you get sued because you didn't bother having a software license created by a lawyer.
Writing code is trivial. And fun. And something I continue to love doing. But if you really want your code to be successful, you'll stop coding long enough to do all that other, even more trivial stuff around the code that's necessary to make it successful.
* Although, to be fair, I really could write Twitter in a week. It's so ridiculously simple! Come on!
Posted by Jeff Atwood
Will someone just do me a favour and clone this thing (properly, with polish, better than the Chinese clone) and just shut these guys up.
Writing code is trivial. Writing code that results in a great product is a lot more work, but still doable. Coming up with what actually makes a great product, *that* is the real work, the part that open-source projects often get wrong. Unfortunately, you've already done that work, and it's in the public; an open-source project would not have to do the hardest part of what makes Stack Overflow awesome.
"And, they're written in Ruby on Rails instead of ASP, so they're much cooler too!"
The logic escapes me. Trying again...
Programmers are nothing! Designers and executives are everything! Open source can never compete with closed source!
Actually, I'm far less worried about the "I could write that in a weekend" syndrome as I am about the "YOU could write that in a weekend" syndrome (usually spoken with complete confidence by a liberal-arts sales and marketing type) It's one thing to convince a fellow developer that something that appears simple is actually very complex. It's quite another to convince a business executive, and unfortunately in the business world, they're the ones most often making our lives hell.
Good points, it would take more like 2 weekends.
Good points, it would take more like 2 weekends.
What makes Stackoverflow successful? Err... 129k subscribers to Jeff's blog. That's 5+ years of hard graft on his part. Nothing to do with code. The code made people come back to it after trying it.
Seriously though, well done Jeff and Joel - a master-stroke.
Yeah, but i think that you're all forgetting what actually got SO off the ground in the first place. It was two people that did some podcasts and whatnot and let people in on the whole thing from the start. Those two people just happened to be rather "famous" amongst the "coding population".
Sure SO is a polished experience but creating something like that isn't all that hard, at least not for someone who can actually manage a project. Gathering critical mass to start things up is difficult.
Well, code might be simple to write (although not so simple if you want to make it robust, not a toy running on localhost), but indeed lot of work needed to make a clone have been done already: the hard work is to:
- Have the base idea: your application is truly original in the end, even if it is a mashup of various concepts (forums, notations, wiki, etc.);
- Come up with the detail and implement them;
So, coding is only part of the work; making good specs and design might be the harder part. And if somebody clone SO, he doesn't have to be innovative, just scrupulous...
It is a bit like those people making clones of games like Mario: they do a good job, but the genius comes from the original idea!
Just kidding. Nice work on Stack Overflow.
A few comments:
1: Dilbert's boss assumes that anything he doesn't understand must be easy. This kind of boss does not have a monopoly on the assumptions.
2: One of the recognized marks of virtuoso performance is that it looks easy. Go to a world-class ballet performance, and the dancers seem to move effortlessly. Take a ballet class, and find out that much simpler performance is hard.
Therefore, to say, "SO makes it look easy" (which I would hardly challenge) is another way of saying "They have done a virtuoso treatment of user experience."
Lastly, I'm thinking about my own website; I don't know if I've managed to make it look easy, but I can say that it (http://JonathansCorner.com/) would not have nearly the same success unless I had spent a lot of time on usability and user experience touches that are rarely noticed if you do them right. And I can say from that experience that SO is easy to take for granted in a way that reflects a great many things that should not be taken for granted if you are trying to understand how hard it would be to make a good clone.
I don't get a lot of the OSS sentiment. I someone creates a piece of software, its theirs, it doesn't belong to the public yet the OSS crowd seem to try and bully people into opening it up. I support lots of OSS ideas, and hope to start my own project soon, but that doesn't mean every popular application should be open sourced just because you think it must. It's not yours. Why is making money and protecting your business such an evil thing in the software industry? I think it's quite destructive.
Great post, good points.
Now... if only I could convince my boss that the length of my weekend should be the time it takes to implement a system like StackOverflow...
I still think that programmers should not be marketing people and so on. Might work in a small company, but in bigger ones a programmer should be able to concentrate on programming, not all the other stuff.
It is not sensible to say that any code is trivial or meaningless. Does a 'hello world' program have any significance or value? Does a first step?
Yes, some developers do have an immature lack of restraint and discipline that predisposes them to blurt out '... can be thrown together in a weekend'. The trouble is that the audience to this unprofessional behaviour comprises a significant proportion of incompetent managers, sales and marketing types that will take such platitudes as literal truth. The consequences of this are manifest to all.
Steve McConnell, don't give up.
It's just webprogramming guys not rocket science!!!
1. Why would I copy a working application (this is like copying a tattoo)
2. When someone looks at StackOwerflow and things he can do in one week he should really try
Why would you do a open source clone of this site? It works and it is open for all. So what's the point. Except to capitalize on the success of someones idea.
You could build the base in two weeks but the finishing polish would take months.
Just get real and have your own ideas.
I've seen a car. You could build one in one weekend, you just need four wheels, a chassis, an engine and a steering wheel, and plumb that together :-)
Good work and remember that lots of people in this community are pissy because they don't have any good ideas themselves.
They crave success so much and they hate it when someone else does well.
Even if there were 10 clones it would hardly matter. The clones (so to speak) would need active marketing and be able to be found easily. Even then the organization downloading the software would have to be able to host and manage it including taking care of security flaws as they come up, integrating it into their design and fixing any bugs as they come up.
To those who post similar sentiments to
> I don't use Stack Overflow because it has a well polished interface (although that does help), I use it because it has a large user base of intelligent people.
It's not stack overflow that is what people want to clone - it's the engine. There are many potential applications of this type of voted Q&A format outside just programming questions that people would like to use. Hence the point of stack exchange. This is what Jeff et al would be competing for.
Found one great mistake in the create-table SQL:
Never use varchar but nvarchar if you ever want to go international ...
I don't think you mean to say "But more often, it's anything but: [...]". That introduction is saying essentially "the following list is never what we do".
No... not sixty eigth weeks
six two eight weeks..
Agree that it's not much to do with code.
SO needed to have enough critical mass of, basically, marketing to get going. The whole business model is based on sucker^H^H^H^H^H^H user generated content. Content not code.
I think it's safe to say that unless Jeff had readership on his techblog (basic tech Readers Digest / Seinfeld stuff of 'Why do get peanuts on airplanes', 'Why does my database query run slow', 'Stating the bloody obvious - part 2' etc) AND he hadn't of gotten Joel 'formally of Microsoft, did you know that? Wrote VBA, but the company has gone downhill since' Spolsky then it would almost certainly would have failed.
You need a focused form of niche celebrity to launch this kind of product, and Jeff and Joel were a perfect match. Tech geeks love their faux-buddy heroes, and the writing the writing bar ain't that high.
In case that's too much vitriol (I'm half kidding); The SO execution has been very well done, and it's all for naught without that, so kudos where it is due.
Nothing is ever simple, especially writing good code.
I don't particularly care for much of Benjamin's slamming OSS, but I can't login to comment on his blog because it won't accept my OpenID, so I guess I will comment here.
I hear this all the time. Hell, I used to have a boss (who barely knew how to use a computer, and never wrote a line of code in his life) who used to ask (rhetorically) all the time "How hard can it be?". If you had done it once you would know it isn't easy. Most of the difficulty is in making it look easy.
Then developers shoot themselves in the foot by claiming they can 'write that code in a week' (kind like 'I can name that song in three notes') and then take months to write it.
Proprietary or OSS - either way, it really has nothing to do with the issue- I've seen as much or more crappy proprietary software than I have in the OSS world.
I've seen crap proprietary s/w because I am behind the scenes working on it, cleaning it up, refactoring it, writing code to replace it - whatever - because most proprietary code is never scene by the general public or even other developers - it is used/misused/abused/cussed at, by a million small businesses somewhere to run their business, but almost always behind the scenes, not opened up to public scrutiny like OSS is. But that is all beside the point.
The next time someone says "I can write that code in a weekend", just smile and say "have at it" or "take your best shot", and laugh all the way home. That's what I do.
Except that when a developer clones a site like stack overflow, especially for internal use, they can simply clone to ui and the rest of the 'spit and polish' along with it.
Doing something like this would be far cheaper than an ongoing license at those rates. Sure people will pay those rates but that's not to say that the exact same site couldn't be cloned for internal use far cheaper.
That's also not to say that tossing a pretty ui onto the above sql create a competitive product for a company to use. When they're licensing software I somehow doubt the users of the software are going to care (or even know) if it's a Stack Exchange codebase or not.
This bears repeating. Why do commentators here think SO _couldn't_ be an OSS project? I vaguely recall Jeff and Joel talking about that on a podcast at one point, and rejecting the idea. But it certainly could have gone the other way. Project leadership wouldn't have changed, the ideas and polish would all be the same. Hell, they could open source the code RIGHT NOW, and nothing would be different.
"given enough time, open source clones will begin to approximate what we've created with Stack Overflow."
Get off that horse man, who do you think you are? stating that stackoverflow might be coded in a weekend is indeed an exaggeration. But your appreciation of your own skills is hugely overestimated and as an open source supporter you should know how good open source can get "given enough time...".
Though I would hope that given enough time you will begin to approximate what a humble and honest developer is.
'It is trivial' remark can only comes from an unexperienced developer.
If we are just talking of the code itself (not all the things around) when it comes to develop a professional software I would say that 70% of the effort is spent on reliability, performance and usability. These are not trivial!
+ OSS is irrelevant. It doesn't matter who wrote the code in the first place. What matters is who's gonna maintain the code now and in the future.
I don't see why so many assume that if you want to clone StackOverflow then you likely also want to use it in the same genre as StackOverflow.
Personally, if StackOverflow were opensourced I would port it to be tracker. I can even see a few features that might be useful for sites that aren't like SO (like required tags and optional tags; required tags would be something like 'select one or more from list x' and optional would be selecting from list y or create your own).
Although, if I were a betting man, I'd bet that if StackOverflow were *really* open sourced, I'd bet that many people who claim they would port it (such as myself) would likely end up not doing that as most end up being all talk in OSS, since many genuinely lack the drive to actually do it.
Well .... SW is trivial to code to some beginers .... not having enough experience with projects .... :)
A lot of comments here don't seem to make the distinction that the website Stackoverflow with its community is one thing and the software together with the hosting being sold is another.
The comment to me seems to be aimed at the fact that I have to pay a lot of money for a website that doesn't come with a community. It is my job to build that community. Most of the pros people have been giving about stackexchange seem to hinge on the community, but I don't see the logic behind it.
Focus on problem, stackoverflow is really a nice and sucessful project, but sucess don´t have a linear relation with good code.
So, when someone say that is easy to clone stackoverflow i really believe it´s easy, and maybe it´s trivial to do a better code than stackoverflow, what´s hard is doing a better sucess on the internet.
And i don´t think it just have to do with smooth experience... sof is still a little bit far from smooth experience.
Just tell them to go pound sand and have them write their own implementation if it is so easy.
It was a very nice idea! Just wanna say thank you for the information you have shared. Just continue writing this kind of post. I will be your loyal reader. Thanks again.
It's certainly success++, i.e. post-incremented. There's a saying in Poland to the effect of: don't split the hide while it's still on the bear.
@Michael -- I've never been a great fan of the PerlMonks site simply because it doesn't have all of the polish. Navigation is often slow and searching is *very* slow and often doesn't find what I want (or buries it in extraneous results), and it's dated.
(I'm talking about just the site itself, the community that lives therein is a whole other set of pros and cons.)
The blogging, podcasts, and whatnot associated with StackOverflow are just a sideshow (I ignore all of that anyway).
I guess it's the usual story - many things are trivial to do. Doing them *well* is an entirely different matter.
I always put my code related query to google and it tells me - "Hey, I found the solution on stackoverflow.com"
Keep up the good work.