September 22, 2004
I got into an interesting discussion today about that recently published Comparing Microsoft .NET and IBM WebSphere/J2EE report. If you haven't read it, there's a summary at eWeek, but I definitely recommend downloading the full report for the details. If you're too busy to do either of those things, well, I'll just tell you: in this particular study, VS.NET is about twice as productive as either of the two IBM J2EE environments, at approximately one-tenth the cost. It's also slightly faster and performs more reliably, but after the key productivity and cost results, those things are just icing on the cake.
This is significant enough that I felt like sharing it with a few fellow developers and some management. Predictably, I got some pushback from some of the Java oriented developers: "Well, what do you expect from a Microsoft commissioned report." I don't consider that a valid criticism. Microsoft didn't conduct the study, they just commissioned it. And the report has this disclaimer on the third page:
1.4 Does a "sponsored study" always produce results favorable to the sponsor?
Our arrangement with sponsors is that we will write only what we believe, and only what we can stand behind, but we allow them the option to prevent us from publishing the study if they feel it would be harmful publicity. We refuse to be influenced by the sponsor in the writing of this report. Sponsorship fees are not contingent upon the results. We make these constraints clear to sponsors up front and urge them to consider the constraints carefully before they commission us to perform a study.
Disregarding the report sight unseen because Microsoft sponsored it is like disregarding someone's opinion based on their ethnicity:
Of course we expect that from Bob, he's an Eskimo. And everyone knows Eskimos are liars!
If you want valid criticisms, you have to disagree with the actual substance of the report. So after actually reading it (one would hope) this is the criticism I got back from a fellow developer:
Visual Studio, like Visual Basic and other Microsoft development tools and languages, provide ease of use and a low learning curve at a price: They don't impose any kind of discipline or framework, making it too easy to crank out poorly designed apps and horrid code. This is not helped by all the amateur Visual Basic/Studio "developers" out there who have no understanding of basic comp sci concepts. (Luckily, we don't hire those kind of developers, but I'm sure we've all worked with many of them in the past.)
On the other hand, J2EE by its nature imposes some rigid rules and forces one to use some kind of a framework to deliver an enterprise level app. This takes more time, planning, and skill. As a result, though, J2EE apps, by comparison, tend to be more robust, maintainable, and scalable. This is not to say that .NET apps cannot have those qualities as well -- it just takes a lot more discipline and some self-imposed rules to achieve this -- Visual Studio doesn't give you that out of the box.
Let me get this straight. The difficulty of developing applications in J2EE is, paradoxically, a good thing? Oh yes! J2EE weeds out the weak developers!
It rampages through the land, leaving a wake of crushed and broken developers weeping in its path! You want the productivity? You can't handle the productivity!
Obviously, I think this is a complete load of crap. Consider all the projects you've worked on in your career as a software developer. At any point in any of those projects, can you ever remember saying to yourself:
Developing applications is far too easy! If only my development could be made more difficult and challenging!
Dan Appleman also refutes this ridiculous argument in one of his blog posts:
The reason that so much bad VB6 code was written was not because VB6 was RAD, but because it was easy. In fact, VB6 made writing software so easy that anyone could be a programmer, and so everyone was. Doctors, Lawyers, Bankers, Hobbyists, Kids Ã¢â‚¬â€œ everyone was writing VB6 code with little or no training.
Now, I don't know about you, but I still have copies of a few of the programs I wrote when I was just starting out, before I'd actually gone to school to learn a thing or two about software development. There was some BASIC, some Pascal, and looking at it now, it's all pretty ugly.
So let's get real. Bad programmers write bad code. Good programmers write good code. RAD lets bad programmers write bad code faster. RAD does NOT cause good programmers to suddenly start writing bad code.
RAD tools can make a good programmer more productive, because they speed up the coding process without compromising the level of quality that a good programmer is going to achieve.
What I find amusing is that someone would actually try to invert this argument, proposing that bad tools are good because they force you to produce better code. And if they don't, well-- that's because you aren't smart enough to use them correctly, stupid!
Posted by Jeff Atwood
i would like to extend this argument a bit further... the CIO that i work with isn't able to differentiate bad code from good code, but he can certainly tell the difference between done and not done. at the end of the day, it really doesn't matter how elegantly a particular app was written if its not getting the job done, properly.
ive seen highly successful telcos determine least-cost-routing tables for long distance calls, successfully, using excel, access, and some of the ugliest VBA code i've ever seen... it may have been ugly, but it worked, and the company stayed in business for years because of it. if you were the CEO of that phone company, what would it take to convince you to take on the risk of ripping out a working system and implement a new, expensive, untested (but elegant) replacment?
I agree, code reviews can weed out the bad coders. In fact, I believe a good, empowered team lead or project manager would be the first defense line, with code reviews being the second.
But, what do you do if your PM or team lead is a pointy-haired boss without an iota of good coding skills? Worse, what if he's good but not empowered to make changes? But rather forced to make due what he has.
I had the displeasure of being in both situations. If you haven't had that experience before, then I envy you my friend. Is your company hiring? ;)
In the end, I don't think we are disagreeing here, but it is reality that often becomes diagreeable.
Don't forget offshoring. Do you think the legions of $5/hour Indian developers are producing elegant, maintainable J2EE code?
Most companies are not SOFTWARE companies, so to them, IT is simply another cost center burning through their income. (and "soft" efficiency improvements are always overlooked in favor of "hard" cost savings, but that's a different issue.) You better believe that these companies are weighting absolute cost of development over almost any other factor.
I'm not saying I am in favor of throwing out all requirements other than "it should cost less and be done faster" but those are critical needs in this day and age of offshoring.
Of course the really GOOD developers get it done efficiently, cheaply, and with elegance :)
A couple of points...
1. The difference in productivity between individual developers is enormous. Studies consistently show differences as high as 20x. This can easily overwhelm a '2x productivity boost'.
2. Recently there's been a push in medical literature to ensure the whole truth gets published about new medicines. Turns out drug companies would do a number of studies, but only publish those that were positive, greatly skewing the results. The proposal being floated is that newly published papers include the results of *all* previously unpublished studies by the sponsoring company.
This is exactly parallel to what's going on here: "We only write what we believe, but won't publish it if it's embarassing." You can bet Microsoft can afford to fund 10 negative studies they can bury, just to get one they can publicize as 'impartial and positive'. And they're almost certain to get one, just because of point 1, and the many other things that can affect 'measured productivity'.
So, this study may be absolutely right, but how can you possibly trust it? I can't.
The argument goes both ways. One can also conclude that Sun or IBM had sponsered several other studies but published none because the results weren't favorable. Unless you know of published studies that show Java or Java tools being more productive than .NET and Visual Studio. If you do, then please share them with us, because I'd really like to have a more balanced view on the subject.
To claim that tools that make programmers more productive are somehow "evil" is ludicrous to say the least.
However, I do concede to one thing: RAD tools in general, and Visual Basic 6 in particular, allow bad developers to linger unnoticed in a lot of organization. Some of them linger enough to be promoted to managerial positions. So, one might end up with a really bad coder managing one's ass. It has happened to me once, and I have seen it happen seveal times before.
My own approach to weed out such types is to use the C language as a filter when ever possible in job interviews. If a candidate can crach pointers, then he or she shows promise.
How about looking at the "weeding out" another way…. Over the last few years I’ve worked as an XML expert on both .NET and J2EE projects. Forget about the code, let’s look at what managers see, because they pay for the IT projects.
The trend with the J2EE projects I’ve worked on have been cost overruns, overtime, missed milestones and deadlines, diminished expectations and confidence in the final product, and exponential increases in stress, finger pointing and frustration as the project progresses.
The trend with the .NET projects I’ve worked on have been on-time delivery, deadlines met, happy developers who get their weekends off, a working final product with plans for future enhancements, and managers taking everyone out for drinks on Friday nights out of gratitude for making them look good.
So forget about elegant code, how about well-rounded lives? The J2EE developers on most of the projects I worked with were mental and physical health time bombs – they had no time for personal relationships, were often in bad health, overweight, and had huge levels of stress. So these “strong” developers will eventually be “weeded out” another way…..
Bad developers, VB6 or otherwise, will only linger unnoticed if they are allowed to. Peer code reviews and skill sharing meetings where each developer takes a turn presenting some sort of topic will ensure that each person on the IT staff has a chance to grow, and that those who chose not to can be sent off to write their bad code elsewhere.
"RAD does NOT cause good programmers to suddenly start writing bad code"
I agree to a certain point but RAD type environments can in some cause the "good programmers" to become lazy. A prime example of this is the WYSIWYG editor in your favorite development tool. I know this is just HTML but as a web developer I am just as concerned about how a HTML Table is rendered as I am of how Reflection was used in the code behind.
I am a notepad type developer and have seen the horror's that WYSIWYG editors produce; bad spacing on HTML, attributes that are unnecessary, code geared to only a certain browser, etc, etc.
I have seen "good developers" lose knowledge in how good HTML should be written because they relied 100% on the editors in their development tool.
I certainly agree that RAD tools in general can assist in producing solid applications more quickly; my major concern is making these RAD tools so that Jane in Marketing can now become a developer. Cause when issues arise, you know who Jane is going to!?!
"Disregarding the report sight unseen because Microsoft sponsored it is like disregarding someone's opinion based on their ethnicity"
Please. What a ridiculous argument. Microsoft is not an ethnic group; they are an entity with a track record of paying for studies which turn out to be misleading at best and deceptive at worst. (Take, for example, the recent ad banners claiming that the cost of a Linux server is ten times that of a Microsoft server, even though the hardware being compared was not remotely similar.) In 2002, one such study was performed BY the very same Middleware Company, which later admitted it was skewed in Microsoft's favor.
If Bob the Eskimo has lied to me several times in the past, I'm going to have a hard time trusting him in the future. Is that because Eskimos lie? No, it's because *Bob lies*.
I hear basically the same argument from the Java service offering *all* the time. I've given up trying to make them happy and instead am leading our Microsoft service offering toward making our clients happy. :)
Here's my take. First, that's a very small project they looked at; 195 man hours vs. 94 man hours. For a project that size, I have no doubt that VB.net is a more productive environment. 4-5 developer weeks of time is nearly trivial. How does that productivity gain scale to a team of 7-10 developers working for 6 months to a year on a project, spending thousands of man-hours?
So it's not that the report can simply be discounted because it was sponsored by Microsoft, it's just that you must always take such reports at face value, and not read into them more than what they claim. Comparing any two such environments is difficult enough to cause major problems in conducting a fair study, and the parameters of the study can have a huge outcome on the results.
i have used both visual studio and java, programmer in both, but i will give it a thumbs up to java that is is more enjoyable programming in java then any microsoft product, yes if new programmers are learning programming from vb then i think there programming experience is ruined and if those guys are not excited as they should be then those ppl will never like java, i think every beginner programmer must start from c and then come into java and then into microosft world, then we will see the difference
Entering the software engineering field with a Java background and now working for a .Net consulting company, I understand both sides of this story.
On the Java side software engineers pride themselves on knowledge of software process, computer science related theory, software patterns, software architecture, and having tools that allow them to build applications with all of this in mind.
Java tools are traditionally geared towards software engineers whereas Microsoft tools are traditionally geared towards software developers. There are a lot of software developers in the Java community but most of them struggle because the tools are not really geared towards them. The primary focus of the modern Java environments has been on integrated Unit Testing, Refactoring, Build Processing, Pattern development, and Collaborative processes.
Since the focus is not on providing easy to use and quick form generators (composition editors), and data access mechanisms… many software developers struggle in the Java world.
So then you come to the .Net side of the community, where the focus has been on providing developers with easy to use tools that will quickly get the job done. In a traditional Microsoft tutorial you will find dragging and dropping of controls, datasources, and event handling routines done through a point and click interface. Microsoft is attracting the larger demographic which happens to be software developers, and even the occasional business professional who desires to wade their feet in development pool for a little while.
Coming from the Java end of the spectrum, I have a very hard time liking Visual Studio because IMHO it lacks out of the box integrated software engineering capabilities. Where I come from my IDE automatically indicates to me whether a class is Abstract, Concrete, or an Interface (no matter what name I give it). It also allows me to quickly view inheritance hierarchies, and find implementation classes. It also has integrated refactoring so when I rename a class, rename a namespace, or move classes around my IDE will fix all of my references for me. I am also used to built-in Unit Testing, and build automation. When people tell me that MS Visual Studio is the best IDE on the market I immediately think to myself that this person has never tried some of the Java tools out there and if they have… they probably do not use software engineering practices and techniques very often. Which really doesn’t play a big role in very small projects, but for larger ones… forget about it… without these tools I find it very difficult to be productive.
So that brings to me address the following quote:
“What I find amusing is that someone would actually try to invert this argument, proposing that bad tools are good because they force you to produce better code. And if they don't, well-- that's because you aren't smart enough to use them correctly, stupid!”
Okay, first of all, you are totally incorrect when you say that the Java kids are proposing bad tools are good. If you honestly believe that Eclipse, and other Java IDEs are bad we need start another blog. Almost every new idea that is currently going into Visual Studio 2005 (or is it 2006… Who knows) comes from the open source community popularized primarily by the Java community. The Java community has great tools, which are very powerful for large team based projects, and because a lot of software developers have yet to realize the power behind these tools, does not make them bad.
Finally you can write good code in a very productive manner in both environments. It’s more about being a professional software engineer/developer than about using any one particular platform or tool. In general however Microsoft lags far behind IBM, and the Java community in terms of providing high quality software engineering tools, not to mention that Microsoft has a long history of destroying many software community processes. Since I am on the soapbox I will through in one last tidbit relating this… Why can’t Microsoft simply adopt the open source community already and integrate rather than rewrite and make these great tools obsolete? Talk about frustrating, I am just wait for my company to adopt Team Services and try to convince me that all of my wonderful unit tests which work great with NUnit right now must be ported over to some MS hack which essentially does the exact same thing as NUnit… adopt NUnit and propose changes as needed already! Okay I’m off.
Anyways, I think this a great topic.
For all of you Microsoft guys out there, here is a article that articulates the field of software engineering from Steve McConnell.
"The Java community has great tools, which are very powerful for large team based projects, and because a lot of software developers have yet to realize the power behind these tools, does not make them bad."
There is no evidence to support this in the study. It also sounds a lot like another restating of "our tools are awesome, it's those damn developers who are too stupid to use them correctly." Uh, OK. For a tool to be good, it must make you more productive.
"On the Java side software engineers pride themselves on knowledge of software process, computer science related theory, software patterns, software architecture, and having tools that allow them to build applications with all of this in mind."
Just to play devil's advocate: how many of those things does the *user* care about? A perfectly "engineered" application is still a failure if it is unusable.
"Just to play devil's advocate: how many of those things does the *user* care about? A perfectly "engineered" application is still a failure if it is unusable."
And what evidence does the study show that the 'productivity' offered by VS'03 increases the usability of an application?
"It also sounds a lot like another restating of "our tools are awesome, it's those damn developers who are too stupid to use them correctly." Uh, OK. For a tool to be good, it must make you more productive."
Isn't this all kind of subjective anyway? I think justin hit the nail on the head: the right tool for you might not be the right tool for me. Do I think that's because you "don't know how to use my tool or I don't know how to use yours?" I don't think so.. I think it's called choice and it's nothing but good for we as the development consumers right?
You're Wing Chun... Justin is Hsing Yi Chuan.. you're tool choices reflect your styles.. nothing more. Me? I am a student of Master Lee:
"you see, actually i do not teach Karate, because i do not believe in styles anymore... i do not believe there is such thing as like Chinese way of fighting or Japanese of fighting, or whatever way of fighting because... unless a human being has 3 arms and 4 legs, we will have a different form of fighting... but basically we have only two hands and two feet... so styles tend to not only separate man because they have their own doctrines, and the doctrines became the gospel truth, that you cannot change, you know? but if you do not have style, if you just say here i am, as a human being, how can i express myself? totally and completely... that way, you won't create a style, because style is a crystallization, as opposed to a process of continuing growth..."
BTW: Five Star has been prominently tagged with the Wumpus.
Thank you very much for giving me an argument in favor of Python and Scheme *against* the nonsense that dribbles out of both Microsoft and Sun. Anyone who has ever used Either of those languages would be at a loss as to why anyone would ever mistake Visual Basic for something easy.
MS is great at creating easy to use tools, that can create simple applications. But as soon as there is a need for more complex solution, it falls apart. VS2005 is great for rapid prototyping, but I would not use it to build the final application. I dont want to sit and correct bugs because a button is to small for romanian words, or that the Forms are not 100% dynamic. But I guess, .NET is great for us contractor as the need for us wont go away.
"Disregarding the report sight unseen because Microsoft sponsored it" isn't really applicable here, it's a distraction from the point: A report that favors Company A's product over Company B's product is not as reliable if Company A paid for the study.
If Company A pays for a study that favors Company B and then publishes that study, I think everyone would believe it. So your sentence isn't accurate, it is the outcome combined with the payment that causes doubt.
When IBM publishes pays for studies that favor them, I assume that IBM made the rules, tried to make a comparison that they would win, and then paid a large sum of money for it. If the company that did the study ended up favoring IBM's competitor, IBM could tweak the rules a bit and do it again, but how many times would they do that before deciding to use a different "objective third party" to do the next study?
I have never fully believed a study that favors the company that sponsored it because I have never seen a study that ruled in favor of the competitor.
j2ee encompasses a great deal of technology, its not necessary to use it all to create a webapp.
MS for example has no equivilent yet of the object/relational mapping of EJB's.
If you somehow thing the IDE has more to do with productivity than the language itself, and its frameworks you are kidding yourself.
People write ruby on rails apps in VIM in a fraction of the time a similar j2dee/dotnet app would take.