July 6, 2009
Remember that Stack Overflow thing we've been working on? Some commenters on a recent Hacker News article questioned the pricing of Stack Exchange -- essentially, a hosted Stack Overflow:
Seems really pricey for a relatively simple software like this. Someone write an open source alternative? it looks like something that can be thrown together in a weekend.
Ah, yes, the stereotypical programmer response to most projects: it's trivial! I could write that in a week!*
It's even easier than that. Open source alternatives to Stack Overflow already exist, so you've got a head start. Gentlemen, start your compilers! Er, I mean, interpreters!
No, I don't take this claim seriously. Not enough to write a response. And fortunately for me, now I don't need to, because Benjamin Pollack -- one of the few people outside our core team who has access to the Stack Overflow source code -- already wrote a response. Even if I had written a response, I doubt it would have been half as well written as Benjamin's.
Developers think cloning a site like StackOverflow is easy for the same reason that open-source software remains such a horrible pain in the ass to use. When you put a developer in front of StackOverflow, they don't really see StackOverflow. What they actually see is this:
create table QUESTION (ID identity primary key,
UPVOTES integer not null default 0,
DOWNVOTES integer not null default 0,
USER integer references USER(ID));
create table RESPONSE (ID identity primary key,
UPVOTES integer not null default 0,
DOWNVOTES integer not null default 0,
QUESTION integer references QUESTION(ID))
If you then tell a developer to replicate StackOverflow, what goes into his head are the above two SQL tables and enough HTML to display them without formatting, and that really is completely doable in a weekend. The smarter ones will realize that they need to implement login and logout, and comments, and that the votes need to be tied to a user, but that's still totally doable in a weekend; it's just a couple more tables in a SQL back-end, and the HTML to show their contents. Use a framework like Django, and you even get basic users and comments for free.
But that's not what StackOverflow is about. Regardless of what your feelings may be on StackOverflow in general, most visitors seem to agree that the user experience is smooth, from start to finish. They feel that they're interacting with a polished product. Even if I didn't know better, I would guess that very little of what actually makes StackOverflow a continuing success has to do with the database schema--and having had a chance to read through StackOverflow's source code, I know how little really does. There is a tremendous amount of spit and polish that goes into making a major website highly usable. A developer, asked how hard something will be to clone, simply does not think about the polish, because the polish is incidental to the implementation.
I have zero doubt that given enough time, open source clones will begin to approximate what we've created with Stack Overflow. It's as inevitable as evolution itself. Well, depending on what time scale you're willing to look at. With a smart, motivated team of closed-source dinosaurs, it is indeed possible to outrun those teeny tiny open-source mammals. For now, anyway. Let's say we're those speedy, clever Velociraptor types of dinosaurs -- those are cool, right?
Despite Benjamin's well reasoned protests, the source code to Stack Overflow is, in fact, actually, kind of ... well, trivial. Although there is starting to be quite a lot of it, as we've been beating on this stuff for almost a year now. That doesn't mean our source code is good, by any means; as usual, we make crappy software, with bugs. But every day, our tiny little three person team of speedy-but-doomed Velociraptors starts out with the same goal. Not to write the best Stack Overflow code possible, but to create the best Stack Overflow experience possible. That's our mission: make Stack Overflow better, in some small way, than it was the day before. We don't always succeed, but we try very, very hard not to suck -- and more importantly, we keep plugging away at it, day after day.
Building a better Stack Overflow experience does involve writing code and building cool features. But more often, it's anything but:
- synthesizing cleaner, saner HTML markup
- optimizing our pages for speed and load time efficiency
- simplifying or improving our site layout, CSS, and graphics
- responding to support and feedback emails
- writing a blog post explaining some aspect of the site engine or philosophy
- being customers of our own sites, asking our own programming questions and sysadmin questions
- interacting with the community on our dedicated meta-discussion site to help gauge what we should be working on, and where the rough edges are that need polishing
- electing community moderators and building moderation tools so the community can police and regulate itself as it scales
- producing Creative Commons dumps of our user-contributed questions and answers
- coming up with schemes for responsible advertising so we can all make a living
- producing the Stack Overflow podcast with Joel
- helping set up logistics for the Stack Overflow DevDays conferences
- setting up the next site in the trilogy, and figuring out where we go next
As programmers, as much as we might want to believe that
lots_of_awesome_code = success;
There's nothing particularly magical about the production of source code. In fact, writing code is a tiny proportion of what makes most businesses successful.
Code is meaningless if nobody knows about your product. Code is meaningless if the IRS comes and throws you in jail because you didn't do your taxes. Code is meaningless if you get sued because you didn't bother having a software license created by a lawyer.
Writing code is trivial. And fun. And something I continue to love doing. But if you really want your code to be successful, you'll stop coding long enough to do all that other, even more trivial stuff around the code that's necessary to make it successful.
* Although, to be fair, I really could write Twitter in a week. It's so ridiculously simple! Come on!
Posted by Jeff Atwood
You know, I would actually pay someone to come up with a decent clone of this thing... will someone please start coding?
Coming across as a douchebag in this post.
I agree with Jeff. Your posts are usually awesome - this one sucks "enough to write a response".
Funny on Hacker News this morning I read a short snippet titled the "Mythical Man Weekend". All about how easy it would be to build something over the weekend. And the dude used Stack Overflow as the example.
Now I read this.
Again with that ?
Just because some Chinese copied your idea ? and worst it's open source.
Come on, tell the truth, it has nothing to do with OSS. It's just because you're loose the Chinese market.
It was bound to happen. Stop being childish and get over it.
The devil is in the details.
A weekend? Much less, I have written my version in DoubleSpeak++ in less than a minute. See the _entire_ codebase below:
All I need now is somebody to write a compiler smart enough to interpret the code!
Stackoverflow reminds me of McDonalds in that it's really optimised to its core purpose but at the expense of being utterly soulless
@Jonathan Hayward. I beg you, please tell me your kidding on your personal website. Tell me you did not really "spent a lot of time on usability and user experience touches that are rarely noticed if you do them right", tell me you are being sarcastic.
Your site looks awful, is hardly usable and breaks about every guideline in the book in terms of navigation, color theory, contrast, accessibility and so on.
Either you were joking and the joke's on me, or you have a very very long way to go.
I could still bang out a Stack Overflow clone with even better spit and polish in 4 weeks on top of a FOSS stack (Ruby on Rails, Lift or Mono's ASP.NET - you pick). I've done it end to end in the past - from DB all the way to Ajaxified UI and CSS. It's not that hard.
These clowns are the same people undercutting us on bids, because they undervalue their own time. I love and support open source and free software, but scoffing over the cost like that doesn't help any of us support ourselves, much less a vibrant and functioning industry.
I've said it many times - the most important things that make for a good website don't happen on the web server!
Great post Jeff.
Could a decent coder put together the skeleton of SO in a weekend - yep. Would it be crap - yep.
It's all the small details that make a good product, and why so many projects that are just put together by ubergeeks, who are only interested in writing cool code, totally suck from a user perspective, and correspondingly, no one uses them. Ergo, they don't count for shit.
There are some awesome open source projects out there, and the concept of open source is great, but I really don't agree that if you write something cool you should just open source it. Maybe, but it is the creator's decision to make, not the angry mob's. It's not your birthright to use other peoples work, it only happens if they want it to.
This is about business, Jeff put his neck on the block, walked away from a stable job, and (eventually :) ) made an awesome product. Why TF should he just give that away, to be bastardised by some retards who never risked leaving their parents house, and set them up as competitors?
This is not Disneyland people. Here on planet earth, there is this little thing called a financial crisis going on, and people still gotta eat and pay mortgages.
Unfortunately, we don't live in a utopian society where money doesn't matter. That would be cool, but it isn't here yet, and apparently does not happen until wayyy after someone develops a warp drive...
Some people will just want it and buy it. Pay your money, turn the key, done. That's called commerce. Everyone else does it, why shouldn't programmers?
Others will want to write their own, that's fine - go for it.
@DMB - good for you - you finished a project - how much money are you making out of it?
@Jonathan Hayward - my freakin eyes !!!!
Jeff, I want to have your baby!
Ahh, Stack Overflow, a pile of shit followed by some shit developers, with the occassional decent reply.
Stack Overflow is not a smooth experience, it's a nasty bumpy one from Australia.
That's why coders are not project managers.
Developers estimate with an inverse factor of dog/cat years.
Er - Phillip from Oz. Having a bad day? I don't have any issue with SO and I am accessing from Australia. Change your ISP, mate. You've got a lame one, something even us developers in Oz should be able to work out... ;-)
I agree. Code is always trivial until you start actually *writing* it. Then all those little pieces that you magically glossed over in your imagination start rearing their ugly head. All of a sudden your weekend is spent trying to figure out why you can't connect to your database only to realize that you forgot to open a firewall port somewhere.
I am just as guilty as anyone of saying "Oh man, I could write that in a weekend". Enough experience has taught me that I am dead wrong most of the time.
It's all about the experience and the community. Open source is interesting, but I don't think a stackoverflow clone could beat the original.
The most important fact is that the community builds itself and the site has a vision.
Hmmm.... So, you want money to host question on a messageboard and get responses? I think not.
You obviously never heard of Drupal.
Those commenters are exhibiting a very common programmer flaw. Most programmers don't realize that getting code 90% functional is only 10% of the work.
It's very easy to bang out a mostly functional body of code. That's called a prototype (or maybe alpha/beta if it's really good). And it's nowhere near production level code.
The real work comes after things are mostly functional: robust error handling, dealing with edge conditions, documentation, and a solid and streamlined user experience. This is the area where so many programmers and projects just completely drop the ball.
So is there no plan on releasing Stack Overflow into the open source world! Shame!!! :P
Nice post! I wonder if StackOverflow clones could become the next standard demo app for frameworks, like the ubiquitous blog app has been for some years, and the address book app was before that.
"Given infinite time, you lowly maggots might just begin to approximate* the consummate codeglory that is stackoverflow. muahahah."
* for very high approximation tolerance
Jeff says lots of stuff I don't buy (or quotes it...no .NET developer should ever *ever* talk about another framework being "a horrible pain in the ass to use.") But he's got the thing that matters most right, despite our "religious" differences: user experience is KING. It is 10x more important than whether you used .NET or PHP or RoR, whether you wrote procedural code or OOP, whether you built unit tests or didn't, .
User experience will frequently determine the ultimate success of your work, and very few people will much care about anything else.
Agreed in principle, though your list of 'polish' is a bit contrived. If I wanted an open-source version of stackoverflow, I wouldn't care about ads. I wouldn't care about the podcast. I wouldn't care about conferences. These things are pointless frippery even for the *real* site, they certainly wouldn't need to be replicated in an open-source clone.
If being a programmer was just about writing code then anybody could be a programmer. Writing code is easy. Writing code cleanly, efficiently so it’s easy to maintain, now that takes some skill.
Actually, an open source alternative to Stack Overflow has existed for *years*. Go check out http://www.perlmonks.org. Only difference is the tight focus on a single programming language.
Get over yourself. SO is not a complicated website to recreate. I do think development of it would take more than a weekend however; likely, 4.
Why even respond to the HN claim? Are you challenging people to try and clone SO?
Only the newbest newbies tend to think building a good/acceptable product can take a weekend..
I LOVE the twitter comment! and I agree. entertaining post
lots_of_awesome_code == 'success'
Great article. Love the note about Twitter. Most of us probably could implement all of twitter in a weekend. However, it wouldn't scale to millions of users, but that's probably not important, because I doubt any twitter clone would reach critical mass.
Also, I think that specifically because you are three programmers that have been working on it for a year, proves that a small open source team could definitely create a clone in short time. But really it doesn't matter that there are clones out there. So long as you can convince people that your product is worth the money you are charging, it doesn't matter that there are alternatives. Sure the alternatives, whether free or just cheaper will cut into your market share. But that's true with any business.
Not to nitpick, but wouldn't we rather believe that success = lots_of_awesome_code? That sure looks like an assignment...
success = lots_of_awesome_code;
lots_of_awesome_code = success;
Joshua, yes, but is it..
best quote: "Not to write the best Stack Overflow code possible, but to create the best Stack Overflow experience possible"
Some of the daily deal sites I visit just toss together some opensource forum software in an attempt to build 'community', but woot wins hands down with even their modest 'forum' engine.
Maybe what the 'trivial' comment implied was that they could get something 'started' in a weekend. Is it ever reallly finished?
You left out Benjamin's final grandeurposturing conclusion of "That is why an open-source clone of StackOverflow will fail."
Twitter comment was hilarious - perfect touch to the end of a post like that.
More to the point, I was once asked how long it would take to create a line of business handheld app. None of us had any experience in making these types of apps and I could easily recognize that this app needed major architecture work upfront to meet the long term goals of the project. My response was that it would take 1200 hours. The higher ups scoffed at this and I was sent back to my cubicle. A young programmer with significantly less experience in handheld apps proclaimed that he could easily do it in 200 hours. And so he was given the project.
1200 hours later he was finally done. But only after the customer had been put through hell. I had since left the company but was having lunch one day with my former coworkers. One of them had mentioned to my boss "Gee, I guess Matt was right afterall!" to which he had reluctantly agreed.
The "I can do that in a week" mentality is all too prevalent in our world.
Actually, the "polish" isnt contrived, nor is it something provided in Drupal. If you think so, I think you are missing the point of Jeff's blog entirelly. What makes StackOverflow isnt just the software, its also the effort put into the experience, and the effort put into the marketing. In short, SO is a business, not a software product. Even if you had the software, a literal exact copy of what SO has, you would be able to reproduce SO without a business plan and a good marketing strategy.
Hacker News is full of young bucks who are straight out of school and full of piss and vinegar. I know, I used to be one of them.
When you're young you don't think that time and money are fungible because you have more time than you know what to do with and very little cash on hand. Therefore the idea of paying for stack exchange seems silly when you could just sacrifice forty hours(or more) of your life and get it for free.
As you grow older, and especially once you have kids, time and money become much more fungible. I'd rather pay stackexchange a grand than spend a weekend away from my kids writing code that's readily available and fully functioning.
>No, I don't take this claim seriously. Not enough to write a response.
Oh, and what this post was about then?
Just shut up and do it if you think you can. I'll come use it if it's better.
Just shut up and do it if you think you can. I'll certainly come use it if it's better than Stack Overflow.
Good luck with that.
Well, I find no convincing reason for anybody going for an Open Sourced clone of SO. Considering that the question and answers of SO are available under CC license, I do not see the necessity for an OS community to go and write a clone of SO, as what they community is going to care is whether the knowledge part of SO which is in the form of questions asked and answers given is Open or not.
Secondly, writing a SO clone is not going to anyway hit the business because for a SO clone to raise itself to the level of SO it has to travel miles. Also, a mere software clone really doesn't help much, it also needs the reputation which SO has earned through all these time.
I guess code is trivial but only as trivial as anything else is. I know when I have to mow the lawn I have to get the mower out, check the oil, pick up everything in the yard while checking for stones or soemthing that may fly out and kill a kid or ding a car. Start the stupid thing and then decide how I am going to go over the lawn this time. Then I have to wack the weeds and think about putting something on the lawn to make it look good.
But mowing is trivial as well. The difference to anyone is how much you enjoy it or how much you dislike it. But you are right on with the fact that anything done well is done in a very small place. Success is made on tiny edges. Everyone has the same resources to get things done for the most part. Making small improvements, better distinctions is what makes a product good.
This is why people don't pay for ideas. Ideas aren't worth anything until they are actually implemented and working. So saying anyone could code up Stackoverflow in a weekend isn't really saying anything until they have done it.
On the other hand, this comment posting system could use some work. Apparently file copying errors still post but don't show up right away, leading to my double post. :^P
@Michael: Wow, after a brief look at that site I can say... it makes a GREAT example of what Jeff is talking about with regards to polish. Sure, you can ask questions and get an answer there... the rest of the SO experience is missing. User ratings, badges, usability, pleasing design.
It's an example of what you can churn out in a week, and what took the SO team another year (or however long) to turn into what SO is now.
@Russ: You wouldn't care about the ads? They aren't an important part of the site? Wow, I hope you expect to lose LOTS of money when you run your SO clone and pay all the bandwidth charges without any actual income!
"Although, to be fair, I really could write Twitter in a week. It's so ridiculously simple! Come on!"
If you just made a fail-whale webpage, it would be right about 20 minutes a day!
It takes a lot to bring code down to the trivial level. Most developers tend to complicate things. It is just difficult to tell people they do not have to write tons of code to make things work.
I say Kudos to SO.
PS: Dev's lack good HTML skills.
The correct estimate is, of course, six to eight weeks.
The word "doom" is spelled "how hard can it be?"
Agree with Pablo or Joshua. It should be
lots_of_awesome_code == success;
or at least
success = lots_of_awesome_code;
lots_of_awesome_code = success;
Sorry for nitpicking...
I think there's a bigger point to be made.
It's easy to assess things that we are familiar with, but it is impossible to assess things that we cannot appreciate. Therefore it's best to learn a little about a lot and assume you're ignorant until you can prove yourself otherwise.
BTW: I've always wondered how the badge system is implemented in SO. Could anyone shed some light on it?
the "stackoverflow-weekend challenge" is definately a prime example of the 80/20 rule.. or so i would imagine
Jeff -- I notices the link to serverfault is broken. Looks like you made a typo (serverfault.co,)
I only noticed cus it said I hadnt been there...which is definitely not true!
I never cease to be amazed at the number of working software developers who devalue their own work by insisting that all software should be free. That comment about SO goes one step further and presupposes that programming is monkey work and therefore worthless.
When people start taking cheap shots like that, you know you are on the way up. Take it as a compliment.
I wish it were only developers that thought this way. The number of prospective clients that arrive at my web development agency with the same mindset is slightly depressing. There really is no understanding that a good user experience takes time to put together, and just how much time and effort companies (like StackOverflow, Google, Facebook, Yahoo etc) have put into the user experience.
Other developers having this mindset I can understand - it's a common problem as you point out. Non-technical users... tell me where they get the idea from??
Getting the site functional is one thing but tuning it is quite another. Adjusting css classes by a pixel until things are perfectly aligned. Taking into account differences in browsers. Trashing a page's layout because what looked good yesterday suddenly today looks like your little brother got a hold of it. Those are the things that really take a long time. I think a good developer is never really done with a project.
@Brian, "Get over yourself. SO is not a complicated website to recreate. I do think development of it would take more than a weekend however; likely, 4."
Its easy to "recreate" something after something has already been created. Coding the website is the easy part. Coming up with the ideas is the hard part.
Looking over the StackExchange page, there's one thing that sets my alarm bells off. For all but the most heavy duty of plans, you do the hosting. But there's nothing promising that my data is mine.
If you don't make it possible to get your data off of StackExchange then you're leaving a huge window of opportunity for an open source implementation. While few companies would give up a polished, debugged product just to save the monthly fees many would give up a polished, debugged product to know that they own their data.
There's nothing scarier than thinking you've spent several years building up a knowledge base that you now have to pay perpetually to access.
In some sense, this is inevitable. To be a good programmer, it helps to be more comfortable with computers than with people. Otherwise, you're unlikely to spend the necessary thousands of hours hacking alone.
The flip side is that many programmers know the inner workings of people about as well as J. Random User knows about microprocessors. That was certainly true for me well into my 20s.
It's an easy leap from not knowing much to thinking there's not much to know.
When smart programmers without social smartness look at a site getting profitable by day, only thing they can thing is "I could have done this"!
So, let's move on.
So programmers are easily replaceable by monkeys, but the only real work is done by the CSS/JS/HCI guys?
Good post. Agree with pretty much all of it.
Apart from the points raised in this post, I also think it's indicative of the open source community that they would sooner be cloning other peoples successful ideas then coming up with something original.
One point though. How did you manage to go from “Open source software only comes in one edition: awesome.”, to “open-source software remains such a horrible pain in the ass to use” in five days?
I think the illusion of triviality is essential for people to get started on the project and create their first hack up - in the spirit of "don't worry be crappy". If we do really understand the full extent of the work involved, we may _never_ start on a path that can turn out to be a fascinating journey. ... as the Twitter folks probably discovered :)
As it happens, I'm not interested in hosting my own SO clone, I was merely being hypothetical. But in response to your question - if I WAS hosting an SO clone, I would hope it was attached to a genuine business model that would provide the required income, thus I wouldn't need the 'frippery' of ad-support.
Running a business on ad revenue is a mug's game - Joel clearly understands this, hence the StackExchange model. Every dotcom I've worked for has had a real business model and *buys* ads rather than selling them.
Writing code is always easy, but StackOverflow (and twitter and facebook) are successful because of the community that uses them. You can't build a vibrant active community in a weekend. Jeff and Joel have been working for years creating communities that they can leverage to make StackOverflow successful. StackOverflow could be horrible, slow, and buggy code and you'd still be successful because you have a great following of people who love to talk about how smart they are.
We all could do SO for less than a weekend. But design = 1 month, making sure buttons click the right way with pleasant animation and all other UI tweaks you cant even imagine = 11 more months. And when I try to sell it = no community, untested scalability vs Jeff's already popular product = total fail. the only way I can beat it is by distributing it like fogbugz - here is the code, send back the tweaks and bugfixes, pay some one time fee (My company is in need of SO clone at the moment,but providing only hosting solution is a big showstopper).
the missing login / logout kepps me away from using so.
think you need more polish.
The missing login keeps me away from so.
Bette rpolish it up..
and for stupidity:
programming and human factors - Jeff Atwood
Comment Submission Error
Your comment submission failed for the following reasons:
Publish failed: Renaming tempfile 'C:\codinghorror\blog\archives\001284.html.new' failed: Renaming 'C:\codinghorror\blog\archives\001284.html.new' to 'C:\codinghorror\blog\archives\001284.html' failed: Permission denied
Return to the original entry.
"There's nothing particularly magical about the production of source code. In fact, writing code is a tiny proportion of what makes most businesses successful."
If this is, in fact, the case, then it can't hurt to open source your software's source code so that people can make good use of it, recognizing that (a) the real work in competing with you on a business level has only a tiny bit in common with your actual code and (b) the follow-on advertising and awareness you might get from others using your source might even feed back into making your own product more successful.
The thing I like best about your Twitter footnote is that it exactly illustrates what you're saying: that success is not about the code, whether in length or in awesomeness.
Your right, it wouldn't take one week. It would take 3.
Isn't that what you said a year ago(roughly) to Joel?
I am so guilty of this when writing programs my time estimates are terrible even using the twice as long + an hour rule.
I think most programmers are guilty of this because they can imagine the solution and their brains run ahead of them considering only how long the main part of the implementation will take not all the form/layout desing, database table creation and changes sql writing.
Hacker News, huh. Are they always this jealous? Or are they just jackasses?
I don't use Stack Overflow because it has a well polished interface (although that does help), I use it because it has a large user base of intelligent people. Someone could steal the entire source and put their own version online next week and it wouldn't make any difference. Stack Overflow is full of users, useful knowledge and since I already have an account history there its a no-brainer.
An open source Stackoverflow clone?
* Three monkeys
* Four Days
* Written in Perl
Look, what makes a community based website isn't the code. It's also the community that uses it. Let's assume I'm an awesome super programmer, and I could magically recreate the entire Facebook website in Ruby on Rails in 3 days. Would you use my Facebook alternative?
Of course not! Why would you want to join a community based website when there's no one else using it? People use Facebook not just because it's easy to use, but because everyone else is also using Facebook too.
What makes Stackoverflow isn't the awesome code and the superb look and feel, but the fact that if I ask a question on Stackoverflow, it'll get an answer with in an hour. Two hours if I am looking for a correct answer.
Could you write your own Stackoverflow clone and use that instead of Stackexchange. Heck, there are already open source stackoverflow clones out there! And, they're written in Ruby on Rails instead of ASP, so they're much cooler too!
However, for $130/month, Stackexchange will setup a server, maintain that server, maintain their software, support your site, and generally make it so you don't have to worry about a thing. Can you do all of that for $130/month?
And, remember that $130/month gets you a million page views. If you aren't expecting that many pageviews on your site, you're not the audience for Stackexchange.
"It's so easy!"
I've heard that many, many times from the comments on the Paint.NET forums :) "Why can't you just add __________, it'd be so easy!"
People extrapolate from the simplicity of the user experience to believing that the development experience is also easy. But implementing "a new feature" is not just sitting down and writing code. You first have to design it with respect to the rest of the program, and all of its users. If I were to add every feature that everyone wanted, Paint.NET would be an overbloated pile of messy junk that nobody would want to use, and the extra/advanced features would get in the way of the basic/simple ones. "Oh I'll just add an Export command over there, and a right-click command there, and 43 new menu items into the Edit menu ... then I'll organize them into 18 submenus, and ctrl+right-clicking on the thumbnail will make it export to Picasa ..." That's how you end up with something like ... well, I won't name it ;)
This is a great post.
As a technical lead, I can understand the trivialization aspect as good as anyone. My developers have a tendency to over-simplify / trivialize a project, and hence underestimate it. The director I report doesn't understand the technical issues (he probably shouldn't) so he pushes back (HARD) when I give him an estimate.
So sometimes I feel like the guy trying to save everyone from shooting himself (not themselves - grammar police) in the foot. Pretty exhausting, but that's why the pay me the big bucks (not).
Just want to-reemphasize the point Jeff (& Benjamin made) -- getting 80% of the functionality duplicated is 20% of the work. Making the system stable, user-friendly, easy to maintain/administer, etc. greatly compounds the workload. Add on top of that all the peripheral activities - marketing, sales, support, accounting, conferences, blogging, evangelizing, etc... and the load is significant.
A website like Stack Overflow depends on having enough critical mass to get good results.
That's where the challenge will be if a team tries to clone Stack Overflow, not the coding.
Old programmers adage:
It takes you 90% of the time to do the first 90% of a programming job. To do the last 10% will take the other 90% of the time.
When web applications are "trivial" to write, the first mover has an advantage. If the first mover does not continue to improve, then the advantage will slip away.
Also, the problem with open source clones of commercial software is that many people who start them have no need to continue to improve them once they come up to their need. Sure it is open source, so others that have greater or different needs can improve it, but there is no desire to complete a project. The commercial vendors (most of them) have a desire (money) to complete the project and continue to improve it.
My first impression was that hosted SO is way overpriced. $150 a month for a small website? Wow! However, a company that needs a website like SO, what would they benefit if they built their own, or started with a half-completed clone? If it cost $10000 to develop a clone (for the sake of argument, but I think this would be a ridiculously low estimate of cost), for the same $10000 you would have access to a fully built and supported and hosted SO website for more than 5 years! Over this 5 years I would assume that the SO website would become more polished and any bugs found would be fixed. In addition, what will the state-of-the-art be in 5 years. Will SO have it or will another competitor have it. Either way it is not likely to be your own cobbled together website.
Good job, I'm impressed by the Stack Overflow. It's good looking - has a nice cool design, smooth cool JQuery functions and lot's of great content. Really good well-thought things overall. I propably wouldn't do it better..
I'm little inspired by the site design and the creative use of JQuery, - really good stuff!!
I can make a pretty good pizza for a few people coming round to dinner, and I can maintain a perfectly usable wiki for knowledge exchange across a group of about 20 people.
But when it comes to lots of people, I'd call in the catering, or software that's made for the task. I wouldn't be using my own oven for more than two or three pizzas, that's for sure.
I guess I don't really understand the obsession with open-source. Seriously Jeff, you sound like Microsoft did a decade ago.
The point that programmers suck at time estimation is valid, although a single link to http://www.joelonsoftware.com/items/2007/10/26.html would probably be a more convincing argument.
Stackoverflow's polish pushes it well beyond a single weekend project... but I doubt we're talking 52 weeks x 40 hours x 3-4 coders here. (Based on the above estimates of "about a year") The majority of the much vaunted "edge cases" would be taken care of in exactly the same way as yours were (a beta that lasted some number of weeks), and then the initial rush of scaling bugs.
I guess I'm sortof confused why you think SO couldn't have been an open source project, considering that all that had to happen is for you and Joel to decide to open source the code, starting on the second weekend or so. I do agree that you would have to have managed the codebase over the last year, but that's no different from what Linus does with his kernel.
You may not realize this, but EVERY major open source project does everything on your 13-point list, except #11 (and some of them do produce podcasts). Essentially, you just made the argument "We've finished with the initial coding, and there's still work to do! Can't do that in a weekend, now can you!?" Even the somewhat-overconfident people saying they could do the coding in a weekend will grant you that fairly obvious point.
That's what you can't understand. As a programmer I don't give a s**t about user experience. I don't care about gui, css, ...etc. You are 75% businessman and 25% programmer...
"BTW: I've always wondered how the badge system is implemented in SO. Could anyone shed some light on it?"
I would guess that it there is a function (stored or otherwise) that is called when certain events happen.
For instance, if I post a question, once my question is actually posted, the DB will then see if I qualify for any new post-related badges. Same with upvotes, downvotes, accepting answers (and accepted answers for the person whose answer was accepted)... heck, even viewing the front page (see the Woot badge).
Hm, shall I be the one to disagree then?
I wholeheartedly agree with your comments on the "I could do that in [some random and ridiculously short period]"-sentiment. I think we all fall into that trap once in a while. I've personally taken the habit of making an estimate and then multiplying it by two before going to the customer with it - that usually turns out kinda right, and often they end up just a little cheaper than they thought, which is good.
BUT - and here's a big BUT - I really don't think this is something only closed-source apps can fix, and open source apps could never. Come on. I know of MANY, MANY closed source apps whose user-interfaces really totally completely suck in ways unimaginable to virgins. On the other hand, one of the darlings of open source web-development is Wordpress. Their source-code gives me an ulcer every time I look at it, but it DOES look slick and works perfectly for the end-user. Point being, there are horribly coded open source projects out there with a good UI. And vice versa for closed source.
No, I don't like the 10% polish either, and it really is 90% of the work. Other people do that better than me :) As a programmer, I've neither the time nor the patience to focus on a slick user-interface. Designers have, let them do it. Software just goes terribly wrong when coders design (or designers code, for that matter). It's all about division of labour, and nothing to do with your software model.
lots_of_awesome_code = success;
of course everyone knows that success has not been defined yet, so lots_of_awesome_code is a random byte.
Thesis stipulated, it's interesting that both programming expertise and its lack lead to the same error:
non-techie manager: "It's easy! The user experience I'm imagining is simple, so the code underneath must be trivial. Show me a prototype tomorrow. [Which I will then deploy as production, because as far as I can tell, it's done.]"
hyper-techie hacker: "It's easy! The entity model is so simple, wrapping CRUD around it in any flavor you like is trivial. I'll show you a prototype tomorrow. [which we can put straight into production, because as far as I'm concerned, it's done.]"
Why on earth don't these two get along in real life?
Great post and very interesting replies.
Good point @giulio, I under-estimate all the time.