I too pity fools who don't write unit tests. But writing the tests first? Not for me, too limiting!

During development, I like to keep the code as loose and maleable as possible. So often I find that I really need to play with code and experiment with strucuture and idioms, to make the code clear and concise. Sometimes its not clear how to do that until you are well into the development cycle. I often massively rip apart and rearrange lots of code because things I didn't think of when first designing it became apparent much later.

I hate too much upfront design in the code, I think its unrealistic to rely on getting the design right with nothing concrete to validate assumptions. TDD creates all this inertia around the initial interfaces in the code, which start to affect the design. Or at least I think it does, I've never actually tried it. But that's my fear.

And dammit, I just don't want to...because I'm special and different. And you're not the boss of me.

I do see tremendous value in unit tests, but I think TDD is going overboard. I could be wrong, who has time to try out every amazing new methodology?.

Hi, Jeff. I'm glad you wrote this. You are correct that I didn't do much to sell test-first to developers who don't do unit tests at all. Actually, I was writing primarily to developers who already do test-first. For the other group, you're right that any step in the right direction is better than nothing. I myself did a lot of test-after-and-only-when-I-feel-like-it before I did test-first. And before that, I avoided automated tests completely.

So, yes, I've been there. I used to feel the way Damien does. And I tried a little testing, and a little more, and a little more, and now I'm a religious zealot. :-)

My one problem with test religion is that it can totally blind someone to design mistakes, or entirely unconsidered effects. For instance, if you don't properly study the API of a library you're using, or worse if it's badly (or un-)documented, functions can have side effects that the caller has never even considered, and thus code that passes all the tests still fails on other input. (Fuzzers can help where unit tests aren't sufficient.) Or the tester will simply never consider the possibility that a closed file handle (as a simple example) could be passed to the function. The permutations of failure cases can easily overrun the ability of tests to find them; what they can do is to keep a coder from forgetting a case they found six months ago, and document it for others who wouldn't see it.

(As for design mistakes, I must not be the only one to catch myself thinking "it passed all the tests, so it _must_ be a correct solution..." at some point.)

I'm with Damien. My coding process is a lot more unstructured and involves a lot of changes, code refactoring, deleting, reorganizing ...

When I've tried test-first I find myself frustrated that I need to go back and "port" the tests to the new design. You could argue that I should be thinking about the design more before I start coding, but I find writing code, trying out little snippets and getting some real output helps me think through the problem in a way that I just can't on paper or a whiteboard.

The times when test-first works is when I'm implementing something that's already designed (porting a class from another language, implementing an algorithm, etc.) But that tends to be the exception.

The _value_ is in having the tests. How you get them written is less important (IMO). Test-first/during/after? Its all good.

My focus in software development is always on economic value. Do you get more out of something than you put into it? For me (and each individual is, well, individual), it's not paying off. I wrote the unit tests for a 400 line class using NUnit. I spent about equal time writing the class and writing/maintaining the unit tests. I could have done as good a job with less effort without NUnit.

I believe this is true for me because of my coding style. I'm very careful when I code, making sure each line of code is exactly what I want. Never guess, never assume, always look things up if not sure, write lots of little test programs. This makes me a slow coder compared to others, but I think it pays back in the end.

Hopefully, the unit testing technology will improve over time so that I can find economic value in unit testing.

Ironically, the first half of Damien's post sounds like a TDDers description of why they TDD. Part of the beauty of unit tests is that you can refactor your code and still have confidence that it works. Otherwise, how do you know? Manual retesting? Stepping through the debugger? Also, using the tests to drive the design reduces the need for upfront design, not increases it.

One of the advantages of writing the tests first is that it guarantees that your code is testable. It's far, far too easy to write a bunch of code, go back to write unit tests, and realized that everything is so tightly coupled that it's extremely difficult to unit test. I've seen it time and time again.

My current project, we did a good amount of upfront requirements development and design -- and have a good idea of what we're doing. Usually we're taking a day or so to code up what we wrote down, and then the unit tests, and that's working pretty well.

Kevin,
"Part of the beauty of unit tests is that you can refactor your code and still have confidence that it works."
I think he was referring to low-level functions that may be radicaly changed or eliminated entirely and new ones created. Obviously a unit test is less than useless on each individual function in that case, but at a higher level you're entirely correct, assuming you haven't completely tossed out your API for a new one. If you go through a few iterations of that, TDD is just extra overhead guaranteed to waste your company's time. (It's like copy editing a first draft - you might not think of it that way because most professionals do their first few drafts in their head or on paper/meetings/email.)

In this case you would test with as much confidence as you have that code will last. Once you feel you've started to settle on a stable design, work testing through it.

It all comes down to how testing can fit into your - and your team's - code-writing style (assuming your style is useful and produces timely, working code), not making your process more inefficient by forcing tests into every step. Happily marrying the two is always the best solution.

I'm a big fan of TDD, and I don't feel you need to be an experienced unit test writer before you can start writing test-first. Certainly there are advantages to having unit tests, regardless of when they are written, but I feel the extra advantages of test-first are significantly greater. Case in point: just yesterday I was writing, well, a bunch of things - but one of them was a hashset of a new, small class. I wrote a test to put an item into the set twice, then wrote the obvious code to make it pass - and it failed. I had forgotten (this is Java, fwiw) to write a hashCode() function for my new class. A moment or two and it was working right. Here's the thing. Writing test-after, I wouldn't have bothered. I've got a set, I can put things into it, what could possibly go wrong? And when the bug surfaced, I'd have been blindsided - had a very hard time finding this stupid mistake. With TDD, I make (roughly) the same number of mistakes, but I catch them a lot sooner and they're much easier to fix. It's not a panacea, and not every programmer likes it, but it's mighty good for some of us.

I've just recently started trying TDD on a new subproject in an existing large project. I like the idea and I'm seeing some benefit from it. I even found a bug in pre-existing code almost right away, though it's something that would have turned up very quickly even without the unit tests, it was just code that hadn't been used very much. But it was still a nice plus that I found it without ever having run the actual program, just the tests, and thus felt like a compile-time error.

The bigger problem I'm facing is that the program has a great deal of database and network access going on, and that stuff is hard to test standalone. To really do a lot of it effectively, there would have to be a whole test-only version of the database layer so that everything that's calling into or out of it could be tested without an actual database connection. Writing that would be a large undertaking, and I am very skeptical that it would be worthwhile.

Nevertheless, I'm writing tests where I can, and often before code, but not always. Just about any metholodogy can be harmful when taken to extremes, and TDD seems no different. But there's still a lot of good that can come from it, and a lot of good times to use it.

Actually I thought it was an atrocious article, full of sloppy thinking and riddled with condescension. These twelve supposed benefits of TDD range the gamut from ridiculously untrue ("unit tests prove that your code actually works") to personal preferences ("it's more fun to code with tests than without"), to unscientific generalizations from limited personal anecdotes or perhaps selected case studies ("it's faster than writing code without tests"), to solutions to problems few people encounter ("it virtually eliminates coder's block"). He lost me LONG before the last sentence.

No, I'm with James Bach. There are No Best Practices; a "Best Practice" is a rule devoid of context[1]. As Andy Hunt points out in his awesome presentation, "Herding Racehorses, Herding Sheep"[2], context-free rules are only the first stage towards holistic comprehension. They're great for novices, but experienced practicioners violate these rules all time. In chess, for example, a beginner might learn "never lose a piece without winning a piece of equal or greater value" - and that piece of advice, if taken to heart, will improve their game a great deal. But it's not a silver bullet. In fact, their chess ability will reach an early plateau if they don't start learning the situations in which they NEED to sacrifice material in exchange for position.

When I hear someone say, "we should use TDD all the time", it's very clear to me that they're a software engineering novice. They haven't quite understood in what circumstances TDD works best, which circumstances it has no effect either way, and yes, in which circumstances it's just a plumb waste of time.

Where does TDD shine? TDD works best in new "from scratch" development, in code which lends itself to well-defined, testable outputs (eg, text or API calls, not-eg UI or AI algorithms), in code which is algorithmically simple enough to run in sub-millisecond times, and in code which is largely self-contained (rather than dependent on a network, database, or large object model environment or outside framework).

Let's face it. These are very limiting criteria, and there's a lot of real-world code which can't be shoehorned to fit this model. Don't think that because TDD worked so well for you in your college project or cool little app that you wrote, that it's the One True Way that All Development Must Be Done.

I have been written to write a developer test from time to time. Who hasn't? Sometimes it's a unit test, sometimes it's an system integration test (testing the interfaction of several units). But I don't write tests for everything. Like I said, some code I simply can't test. Other code is so simple, I'd be wasting my time trying to test it. But beyond that, I'm in the practice of looking at the bugs that I myself, or my coworkers have written, and asking myself, "how could this have been avoided"? And the answer is almost never "a unit test would have caught that". I just don't see where the ROI is in banging out all these tests that don't catch anything.

And quite frankly, the few merits TDD has ("unit tests make better designs") is a level of hand-holding I outgrew long ago. I've learned (mostly from painful experience maintaining other people's code) what code smells I should avoid. The question "how should this function be used?" comes far more naturally to me than "how should this function be implemented?". I don't need TDD to force me to think that way. I know to keep my objects small, to write referentially transparent code, to not write Rube Goldberg machines, to minimize my usage of if statements, to tell rather than ask, to avoid casting and pointer arithmetic, to avoid code which transfers ownership of objects, to use std::string and std::vector rather than fixed-sized C-style buffers, to avoid premature optimization and premature abstraction, to prefer composition and aggregation over inheritence, to be careful to distinguish between a count of bytes or a count of characters, and the zillion other things which practicioners of TDD eventually stumble upon.

For me, TDD zealots are just one of a long line of religious loonies knocking on my door, trying to prove that my way of doing things is irrepairably broken and the only path to salvation is Jesus, Mohammed, Krishna, Dianetics, or Unit Testing. And maybe there's a good idea or two to be gleaned from all the mountains of bullshit. But you know what? Keep your crazy religion, 'cause I don't need it.

[1] - http://www.rtpspin.org/Information/030327prs.ppt
[2] - http://www.satisfice.com/blog/archives/27

One could claim TDD to be our latest attempt to get written specification of what the customer wants. The bugs I run into are mostly the result of late design changes/feature additions rather then bad design/coding.
With TDD you get a sort of a contract (think interfaces here) between you and you customer (this is even more important in in-house projects). This assures that the deliverables are specified and that you are actally building something someone wants/needs. This eliminates the "Oh, how about we add this..." ideas at project progress meetings.

Yes, there's a good reason if I ever heard one. When the customer comes into the meeting to ask for some silly feature he dreamt up, the proper answer is:

a. "That's the dumbest thing I ever heard."
b. "While we could look into it, it would mean a lot of lost time and extra costs, and my gut tells me the benefit is not worth the cost."
c. "We have tests! Tests! One must not desecrate the holy tests!"

And if it really is a great idea you wish you'd thought of, you'll still want to work it in or shuffle things around so it can be worked in cleanly in the future, anyway.

I'm with Alyosha. 100%. Unit tests are for beginners. Beyond that they're a monumental waste of time and effort.

PS This is just an opinion. It's mine and there's no reason at all for it to be yours.

Sadly, it's the "senior" developers that are often most resistent to trying new techniques like TDD - years of doing things a certain has them set in their ways.

The fact is, there's very little code that can't be test-driven in some way. I've test driven complex web UIs, database code, OLAP code, Web Services. They may be trickier to get a handle on at first, but once you learn some tricks of the trade, it's totally doable. Maybe your coverage won't be quite as good, but as Jeff says, some tests are better than no tests. If not for you, then for the next guy who has to take over your work.

It's amazing how often your here "TDD doesn't work", followed shortly thereafter by "oh, but I've never tried it". Unfortunately, TDD is one of those things that you often can't convince someone the value of until they do try it - people are very attached to the way they are used to doing things, and the advantages of TDD can be subtle and difficult to appreciate without trying it out.

Foxy - while I generally don't think of unit tests as a "customer" specification (that's what customer acceptance tests are for), unit tests are great specifications about what the expected behavior of the code is.

Have you ever inherited a bit of code, gone to change it, and realized you don't actually know what the repercussions of your change would be? Having a battery of unit tests to validate your change is immensely valuable.

"Sadly, it's the "senior" developers that are often most resistent to trying new techniques like TDD - years of doing things a certain has them set in their ways."

I have an alternate explanation, one I think is much more credible than "old fogies don't know what's good for them". No, it's that TDD is a teaching tool which benefits beginners much more than it does seasoned veterans. Junior programmers are more likely to get fanatical about TDD because it actually did help them advance the learning curve from complete novice to journeyman programmer. They also likely did it on projects that were easily amenable to TDD, such as college projects or new from-scratch development.

And senior programmers are more likely to say, "huh, this is a practice which will teach me me things I've learned long ago and which will force me to program the way I always have. Alright, so why should I bother?"

Sure, you CAN shoehorn any sort of development in the TDD model. So what? It's still shoehorned in. You don't get the same benefit that you would, say, with your legendary suite of a zillion tests, each which run in under a millisecond, which cover every aspect of the code in complete isolation from each other. Instead you have something approaching the results of traditional QA, where you have limited tests that try to test things in isolation but usually don't, with big gaps in coverage and still don't catch the complex interaction bugs.

You know, the choice is not between "no tests and some tests". Whenever someone says that I want to slap them. Traditional QA is not about writing no tests. Traditional Dev is not about never writing an opportunistic test when they feel like it. The choice is between some tests and more tests. Are more tests always better? Hell no! I'd rather have no tests than a suite of fragile tests which never catch anything and throw off hundreds of false positives. Granted, more tests are USUALLY better, but you know, those "more tests" don't come for free. I may WANT to write more tests but choose not to for the same reason I may WANT to buy a Ferrari but I end up buying a Honda Civic -- because it's the best value, and my time and money is better spent elsewhere.

Okay, to sum up. I'm not saying "TDD doesn't work". Clearly it does work very well for some people and some projects. What I am saying is that TDD doesn't work equally well for ALL people and ALL projects. For some combinations of people & projects, it's not worth it for them to go through the exercise. And those people can do less with the wet-behind-the-ears TDD zealots with their condescension, their pity, their starry-eyed idealism and their "if you've never tasted the kool-aid you'll never know what it's like".

Alyosha said:
"And those people can do less with the wet-behind-the-ears TDD zealots with their condescension, their pity, their starry-eyed idealism and their "if you've never tasted the kool-aid you'll never know what it's like"."

But it is still true, if you never try kool-aid, you will never know what it is like. Every now and then you do have to taste the kool-aid to experience new things.

I am a Coke drinker. Back in the day when they had those taste tests between Coke and Pepsi, I always knew which one was which. So, I would always go to the Pepsi ones, and choose Coke. Why? Because that was what I liked. I didn't want to give those Pepsi drinkers the satisfaction of saying I liked Pepsi more.

To this day I only drink Coke. Does it limit my options? Yes, half the restraunts in the US serve Pepsi (might even be more than half). So, I am disappointed half the time. Maybe if I wasn't so set in my ways, I might be able to appreciate Pepsi every now and then.

I think you are picking a very hard line stance like I do with my drink choice. I feel that if you opened up a little, you just might be able to appreciate the 'other side' more.

You are definately allowed your own opinion of TDD, but from some of the stuff you said, you may not have found the correct techniques that make it successful for a great many developers. The reason I feel this way is because of these comments:

"When I hear someone say, "we should use TDD all the time", it's very clear to me that they're a software engineering novice."
and
"Let's face it. These are very limiting criteria, and there's a lot of real-world code which can't be shoehorned to fit this model. Don't think that because TDD worked so well for you in your college project or cool little app that you wrote, that it's the One True Way that All Development Must Be Done."

So, it doesn't matter if you do it or not. You may write some great code that doesn't need testing. I can appreciate that. I don't test everything, but I am moving to more testing in my development. This testing is in real-world line of business applications (inventory, accounting, etc). So to say it only works in college projects is completely false!

Hi Jeff

I read a lot of your articles in CodeProject like:
http://www.codeproject.com/aspnet/CaptchaControl.asp
http://www.codeproject.com/aspnet/WebFileManager.asp
http://www.codeproject.com/vb/net/MhtBuilder.asp

Reading your topic about Unit Testing, I ask: Do you unit test asp.net applications? Do you use NUnitAsp? Or what?

Regards
Fabio Alves

Fabio,

Not to answer for Jeff, but if you use the Model View Presenter pattern for testing, your UI should have almost no code in it that needs testing. Just a little info for you if you don't know about it.

Jeremy Miller is a pretty good resource for TDD. Here is a post on the MVP pattern for ASP.Net:
http://codebetter.com/blogs/jeremy.miller/archive/2006/02/01/137457.aspx

Sorry, but Jeremy Millder is totally, totally out to lunch. He either works on extremely simple web apps, or he doesn't know what the hell he's talking about. Either way, it is in fact possible to have a view that is a little bit more than trivial and needs to be tested. The totally contrived example he provides is near useless.

Thank you Brian, I will check about Model View Presenter.

Foobar, if you think J. Miller article useless, what you sugest then?

Jeff,

What does this mean ?

"... that unit tests are so important that they should be a first-class language construct."

Regards,

Justin

"any tests are better than zero tests"

I disagree; like comments, incorrect ones are actually worse than none at all.

> Foobar, if you think J. Miller article useless, what you sugest then?

There's nothing really out there right now that truly helps with comprehensively testing web apps. I'd say we're still about three-four years away before we get something that'll be ready for prime time.

But for now, NUnitASP would be better for testing, especially for existing applications. Even then, you'll still have to do a lot of the usual load-and-try-out-the-page testing.

I am 100% with Alyosha. "There is no silver bullet" can not be said often enough. I have seen unit testing work pretty well on front-end simple stuff you just have to write. But in every case that is more research than just development they are just bullshit. So, yes, you want your junior programmers to write unit tests, but that's it.

> Reading your topic about Unit Testing, I ask: Do you unit test asp.net applications? Do you use NUnitAsp? Or what?

I was late to the unit test party, so only my "latest" codeproject article has unit tests-- the one on encryption. And the unit tests helped immensely! I not only changed the API based on my external use of it, but I found a few bugs as well.

> unit tests are so important that they should be a first-class language construct.

Meaning, the unit test attributes should be built into the .NET framework itself, and you should be able to run tests as part of a command-line compile cycle. And the Visual Studio IDE should support TestRunner type functionality out of the box.

> Even then, you'll still have to do a lot of the usual load-and-try-out-the-page testing

You don't like the watir thing? It drives IE to run the tests, so Javascript, etc, is evaluated.

<a href="http://wtr.rubyforge.org/">http://wtr.rubyforge.org/</a>

Seems like a good idea to me.

Hmm, I find the main of advantage of writing the unit tests *before* writing the code is that it helps me concentrate on how the unit will be used, rather than how it works. That pretty much matches point 6, though it’s less to do with the units overall design and more to do with its interface. In the end design is something that you do in your head; and only experience and better mental models are going to improve that.

The other benefit I find of writing tests first is that it acts much the same as a detailed to-do list. Giving me a sense of accomplishment when I tick of a task/test, and guiding my activities.

Other than that the most of the points in the article simply spell out the benefits of unit testing in general, not necessarily test first development, which makes the headline "Twelve Benefits of Writing Unit Tests First" a bit of a misnomer. In fact the article seems to point out the opposite: That the benefit between writing the tests before vs. after writing the code isn’t that great.

Does anybody have some good links that show benefits/drawbacks of unit testing in web (browser) focused development.

I am coming from a typical LAMP environment and develop intrant applications in a small team of developers (usually 3-5).

Usually the majority of the bugs we face or related to the user interface (typically compatibility issues with browsers or bugs in complex client side scripts) and therefore i don't see a good use for TDD? Am i wrong? Are there automation tools for browsers and/or JavaScript engines? Or is TDD not suitable for UI focused problems at all?

Running static tests in a "perfect world" of a testcase simply does not match the stupid reality of so many bad and full of bugs implementations in browsers.

So, where does one get started in the whole Unit Test thing?

I use VS 2003 right now. Should be adding 2005 soon. But I also use a lot of VBA.

And, I am a solo developer with no budget for cool tools (even for the really necessary ones :( ).

Not a fan of TDD myself. My arguments against unit testing are:

1. Unit tests only prove that the code works for the specific inputs/outputs that the programmer has already considered. For experienced developers, virtually all bugs occur in the edge cases that aren't covered by unit tests.

2. Unit tests really do take a lot of time to write and maintain if done properly, and in informal projects or any project where the requirements change, it's easy to spend over half one's time just keeping the tests up to date. Sometimes you *know* that you're going to break old code, and then in addition to changing all the old code, you now have to change all your unit tests (or just give up on the unit testing for that code, which is often what happens in practice).

3. Unit tests do a very poor job when it comes to threading and timers, for reasons that should be self-evident.

4. Unit tests are mediocre for anything database-driven, where often the only way to verify correctness is to use production data and verify the results (it's possible to do this in a full-fledged test environment, but not in a unit test).

5. Unit tests do nothing to expose architectural flaws, which are the most expensive ones to actually fix (assuming the requirements are correctly defined). Those flaws are generally discovered during the coding process, which is obviously going to move slower if one is also dealing with unit tests, and that means the time it takes to discover such defects is greater overall.

6. In my experience, at least 1/3 of the bugs are introduced in the UI, which unit tests can never test properly. Maybe some day computers will be able to model the chaos that humans bring into the fray, but we're not there yet.

Given time I could probably think of 4 more points, but I think that's enough for now. :-)

My biggest beef with McConnell (Code Complete) is his religious fervour for unit tests. As other people have commented, and as James Bach has famously stated, there are no best practices, and unit tests are too often treated as a best practice. For extremely large projects that have massive APIs which are all supposed to work in deterministic ways, unit tests can be a great tool both for initial testing and regression testing. For anything smaller (I'd say anything 100,000 lines or less), less formal, data-driven, etc., they may merely create the illusion of working code and are not *always* such a useful practice.

I have yet to see any truly persuasive argument that unit testing is a best practice, or even a good practice, in the general case. Having said that, if they help someone write code more efficiently/accurately, then I would not by any means try to steer that person *against* unit testing. People should use whichever tools they find useful; I just wish they wouldn't prescribe their elixirs to everyone. If I were managing a team, I would not try to stop any team members from using unit tests, but I would not force the team to use them either. Code reviews and -actual- testing (by a good tester) will reveal deficiencies much faster and with better accuracy.

Sorry for the rant Jeff, you're usually spot-on (after all, that's why I'm here) but I think this one might have missed the mark. Can't win 'em all though!

Every anti-TDD zealot I've run across invariably has yet to get over the learning curve, the logistics of implementing and automating unit tests, and are either too lazy or too arrogant to do so. Once JUnit or NUnit is set up, the overhead of writing a new test is minimal. The "waste of time" argument is ridiculous -- the vast majority of unit tests I write take seconds to set up and run, and save me massive amounts of time I would spend otherwise mucking around in the GUI and debugger, setting up watch variables, etc. I can harness up the method in question so much quicker with a unit test. So its benefit as a development tool alone cancels out any percieved "cost" of writing -- what? -- another 10-15 lines of code per method? Just how slow do you people type? Give me a break.

Of course, excecution always trumps concept, and if you're not thinking through all the edge cases (yes, sometimes even changing APIs) then neither TDD nor anything else can help you.

To Aaron's points:

1. Excecution trumps concept. Unit tests only automate and expidite what a good coder should be doing anyway, i.e., thinking through inputs/outputs and considering all the edge cases. I have yet to come across a cases that I couldn't test for.

2. No. If you integrate unit tests into your daily workflow and coding habits, it's actually faster and at worse a wash on the really difficult cases. There's probably a library out there to automate for your environment (JUnit, NUnit).

3. Yes, there are some things Unit Tests can't do.

4. ???? Exactly what DB operations can you not test? Run a query. Make an assertion.

The app I write tests for is a front end for a Database. All I do all day is write code for a database app. The thought has never crossed my mind that "unit tests are mediocre for anything database-driven." I guess I just don't understand your point.

5. (see 3) Unit Tests don't design your code for you. They can't tell management to give you better requirements, and they won't wash your socks.

@#$%&*$ Unit Tests!!

6. I'm not following you here either. Write good MVC (or some variation) code and think through the edge cases for user input. Test the controller. Isolate the UI from the business logic. Test the model. I have yet to think of possible user input that I couldn't test against.

One last point to Aaron:

Size truly doesn't matter. A class is a class. A method is a method. They test the same whether they're in a package with 2 other classes or 100.

The database app I've worked on for eight years -- with tons of ugly legacy code that isn't unit tested -- is approaching a million lines, eight to ten developers committing to source control daily.

Aaron "Code reviews and -actual- testing (by a good tester) will reveal deficiencies much faster and with better accuracy."

This has been the opposite from my experience. It is SO much more expensive, process-wise, to have a tester or code reviewer catch a bug. If I can turn in solid, tested code the first time, that's a win, hands-down. The coders I know who unit test do this much more cosistently than those who don't, and they develop a reputation for doing so.

We've tried a "hybrid" approach for several years now, where unit tests were considered a good idea, but not enforced. It's been frustrating. The anti-TDD zealots who didn't adopt (mainly because they had such a high opinion of their own coding practices, and because they didn't want to take the time to get over the setup/learning curve) spent a lot of time complaining when they'd break somebody else's test; complaining that they'd rather be "actually coding," etc. Oh yeah, and they spent a lot of time quoting James Bach to anyone who would listen.

Over time, those coders developed a poor reputation with QA, and the "unit testing" crowd began to be held in high regard. We're still a "hybrid" dept. in theory, but -- funny thing, this -- a number of the non-testing coders aren't here any more, and most everybody else pretty much writes test for their code, not because they're forced to, but because it works.

> You don't like the watir thing? It drives IE to run the tests, so Javascript, etc, is evaluated.

Looks good, but it's still quite limited. The main problem with a web page is that even though HTML/CSS/etc on the surface looks easy, creating a layout and *adhering to that layout* is damned hard. That's why you still need to look at the thing to see what's been bungled.

As for Aaron's comments, creating unit tests is actually pretty easy. Oh well.

What's really funny is that there's two arguments going on here: The merits of TDD and the merits of unit testing.

TDD is merely another variant of the SDLC. Big whoop. Use it or not. I'm not convinced it's any better or worse than all the other gazillion SDLC variants out there.

Unit testing, on the other hand, should be mandatory. It's really the only way we have right now of formalizing, capturing and persisting testing procedures. What many people seem to have a hard time with is that you're actually creating another client for your API. Here's the catch. Most programmers, who are actually damned hacks, couldn't write a business interface layer to save their lives. For them, unit testing will be very hard, what with business logic strewn all over windows forms, web forms, http modules, event delegates, App_Code, servlets, JSPs, etc. So for them, writing a "client" for their non-reusable code is impossible, even from a conceptual viewpoint. Even for those of you who do work on project where you were fortunate enough to write a business API, I'd bet most of your co-workers don't even realize they're using an API.

What a red herring!

"Unit tests only prove that the code works for the specific inputs/outputs that the programmer has already considered. For experienced developers, virtually all bugs occur in the edge cases that aren't covered by unit tests."

Of course, that is true! It almost sound like one might be saying, "If unit tests are not going to catch all the bugs and edge cases, then we should not even do it." That is silly.

If one's tests miss a defect or edge case, then add a new test case which PROVES the defect has been fixed. If one is concerned about edge case, write more tests. Nothing in the process says a testing team cannot come back alter and add more tests.

Unit Tests before you write code only works if the design of the code has been done up front. Unfortunately, I think this is rarely the case. I think most developers don't know how things are going to work out until they start writing code. Now, whether that is a good or bad thing can be debated. But its hard to write tests if no design exists.

Ideally, the design of the application would be known so that technical testers could write the test harnesses to fully test the application. Ideally, you would want a structural test of the code (unit test) followed by a functional test of the code. The structural testing could be done by something like NUnit. Functional testing is having users actually step through the system and verify functionality.

Unit tests are valuable, provided both positive and negative test cases are supplied. Many bugs can be found during unit testing. Keep in mind during unit testing you are not only testing your code, but the numerous libraries you are employing across the application as well.

I guess my only conceern is that it compiles. Believe it or not, I have had cases where checked in code did not compile, so unit testing may be asking a bit much in any shape or form.

First off I love TDD. I get more satisfaction out of my code when I write the tests first. I feel like I fly through my task list and my mistakes are few and far between.

Second, I don't do much TDD. If it's something I've done a million times before, then I just write the code. But there's always that one little bug that takes time to root out and find, even though I'm an expert on this or that. TDD, I have very few bugs during and QA finds even fewer bugs after. No TDD, there's always something. And even if it only takes me 2 minutes to find the bug, that's 2 minutes too long, not to mention the tester's time I've wasted. Shame on me for being so darn brilliant.

I think a key difference between the people who embrace TDD and those that don't is your comfort level with code. I mean raw code, semicolons etc. If you use whiteboards and/or UML first, or need to get a rough design doc on paper before you can commit to code, then TDD probably doesn't resonate the same way. If code is your first language and you have to translate that to english to talk to other human beings, then for you, tests are the only form of design you can really embrace. They're always accurate, and let's face it, you don't trust design docs or comments anyway.

Now I'm also a huge refactorer and I like nothing better than ripping up huge chunks of code to accomodate a new wrinkle in the requirements. At that point TDD hurts, at least until you embrace the delete key. Those tests are going to need hours of rewrites? DELETE. First find the lowest higher level above the construction zone and make sure you have a safety net. Then just blow the rest away. TDD the reconstruction, or not, who cares? The tests are there to help you along, not hold you back. I've got the behavior I want to enforce at some level and if that doesn't break I don't really care what happens below now. Sprinkle in unit tests in whatever fashion you please until you feel good about the code.

I think any good developer writes "test programs" for the code they write. I mean how can the code be "finished" if you haven't see it perform its task correctly?

The best advise I was ever given (in this area):

Developers should spend a little more time on the quality of the code in their test programs and then Don't Throw Them Away when your done. Check them in!

I'd say Brian comes close to my developer test philosophy.

If something is complex, make it simpler. If it can't be made simpler, try to move as many complicated bits into a testable location and test that before you declare it "done".

But don't waste your time testing the simple parts. 20% of the code has 80% of the bugs; spend your time there instead. You don't have to TDD everything.

I don't have a problem with unit tests, per se. What I have a problem with is this one-size-fits-all, unit-tests-solve-everything attitude I hear so often from TDD/UT advocates.

My problem with unit testing is that it isn't testing the right thing. In fact, it is testing the part that is least likely to fail.

Pure code, the kind unit tests target, is easy to write correctly. It is very rare for me to have a bug code I wrote entirely myself.

Most of my bugs come from boundaries. Code that interacts with...
the database.
the file system.
the GUI.

Unit tests simply don't address those concerns.

Can you write repeatable tests that account for the database? Sure, but it isn't easy. It takes a tremendous amount of planning.

I my own line of work, I find that writing formal test plans that account for every boundary (GUI, filesystem, network) are far more effective.

And by the way, McConnell's Code Complete 2 cited a study that showed Unit and Regression tests to be among the least effective testing strategies. Other techniques, like formal code reviews, were far more effective.

This point has been made twice, and I don't understand it, so I MUST be missing something:

"Most of my bugs come from boundaries. Code that interacts with...
the database.
the file system.
the GUI.

Unit tests simply don't address those concerns."

These are all just sources of input, right? Sure they're variable, but that's what most unit tests are designed to test -- variable inputs.

Beyond that, the file system and DB are transparent to the testing environment through IO objects and SQL. Can somebody give me a concrete example of a DB, GUI or filesystem input/scenario that isn't testable?

Dave, I think you're using the term "unit testing" to refer to all kinds of testing. It's not -- there are tons of different testing techniques besides unit testing. Yes, unit testing tests variation of inputs, and traditional QA testing tests variation of inputs, and the difference between the two is a matter of scale.

Remember, received wisdom about unit tests is that they must execute QUICKLY -- or else your devs will get too bogged down running all the tests every time they go through a red-green-refactor cycle. Executing quickly means avoiding lengthy setup procedures and it means testing things in isolation from each other.

With network and/or database dependencies, you are essentially testing large, complex data sets. You will naturally get large, complex results back. Not only are your setup/teardown costs going to be much larger, but you're also not testing things in isolation -- it's harder to enumerate all the edge cases and provide good coverage.

It is bad testing practice to verify that an API call results in such-and-such database modifications and such-and-such XML being sent over the network. Because that's testing implementation details, details which are the first things to break once any refactoring happens. You should test what goes IN to an API and what comes OUT of it only.

No, they are not "just sources of input".

They are also destinations for output.
Databases can perform a lot of processing unto themselves. This cannot be stubbed-out, as the error may be in the database trigger or stored procedure.

Any time you touch data in a file or database, you run the risk that another application is going to also touch that data. That means you need to not only test your application, but also any application that uses the same data.

One has to also consider the difficulty in testing bad data in a database. Unlike a traditional unit test, where you can literally pass in anything you can code, a database needs to be pre-populated with both the good and bad data you need to test. This cannot be simply tossed in the build like a unit test. It needs to be carefully planned for.

As someone who has written a LOT of intregration tests against a database with a unit testing tool like NUnit, you certainly can write tests which resolve what happens when bad data is put into a database. I have done it many, many times and it fits easily into a build. It does not take as much planning as you think.

Boy I love reading the self righteous "I'm better than any test" posts. Clearly an experienced programmer won't get as much bang for the buck as a less experienced programmer will from units test... up front.

I'm going to assume that even experienced programmers are still not writing perfect towers of code the first time with perfect execution of every use case... since that is impossible in the real world with the realities of moving business requirements. Yet, that is exactly what we are being told by the "I'm better than any test" crowd: that they don't need unit test because they already are doing what the unit tests would force them to. They never write bugs. They never refactor. They write gleaming ivory towers of code that are unassailable by error, user requirements or time.

Hogwash. Anyone who told me they could do that would be shown the door quite quickly during the interview process, because anyone who touches real code knows that such a perfect utopia doesn't exist.

So what do we gain by tests? We gain the ability to refactor, with confidence. We gain the ability to ensure that error we fixed doesn't crop back up when some other coder touches the code and doesn't realize that strange case actually had a use. We gain the ability to give code to a mere mortal programmer and have some confidence that when they are done with it we know what damage was done (assuming that things didn't go well) and that we can go back to a known good CVS state... or we can be assured that at least those cases that are tested work.

In other words, tests provide those who live in the dingy, complex real world can write code with similar confidence to those who live in imaginary ivory towers were requirements never change and nobody ever writes buggy code.

"Hogwash. Anyone who told me they could do that would be shown the door quite quickly during the interview process, because anyone who touches real code knows that such a perfect utopia doesn't exist."

Beat up straw men much?

No, the utopia you describe doesn't exist. But it's also not the logical conclusion of "I try to get it right the first time". Design mistakes can and will be made by even the best programmers. The process should accomodate that fact without explicitly optimizing for it. I would be suspicious working in a place where refactoring the world is the norm because of all these wonderful unit tests make the cost of change almost free. That strikes me as a very amateurish way of working: churn out anything that works, we can fix it later.

It seems some people have this deep-seated disbelief that people can get better at programming with experience. Sure, you can always point to some 15-year old fogie who churns out crap day in and day out. I've known plenty of programmers like that. But I can also point out a few 15-year old senior devs that have good design sense, remarkably high productivity and remarkably low bug rates -- people who generally DO get it right the first time.

I admire how well you can identify the benefits of a good test suite. I agree with most of them. What I think you're a little blind to (and folks have been consistently missing this point I've been making) is that there are costs in writing such a test suite too. And sometimes tests emit false positives which can be another huge time sink. My point is that sometimes the benefits outweight the costs, and sometimes they don't. One size does NOT fit all. Unit tests do NOT solve all problems. We should have the freedom of saying that without being slapped with this "elitist anti-TDD crap-churning old fogie" label.

I would be amused by the posters that believe we should eschew unit tests if I was not so appalled. Of course unit tests are not the answer to everything. Of course TDD is not the be all and end all. Of course unit tests are expensive and time-consuming to develop. Does that mean we should not make use of them? NO!! Good grief, that attitude is unconscionable. Here's the news, folks: most developers out there are pretty clueless. Just because you are writing and maintaining your well-designed, careefully constructed, expert code does not mean you will always be doing that. One day, some clueless wage slave will be sitting at 3 a.m. in front of your perfact C++ code with a stream of drool escaping the corner of a slack mouth, thinking "WTF is bind2nd?", with a production outage he has to solve because some other clueless wage slave broke the code with a one-liner that could not POSSIBLY be wrong and did not bother to even compile the code before it was checked in. What, someone like that should be fired? Yes, of course. Are they? Not nearly often enough, if ever. People are hopelessly short-sighted if they think their great code (and I say that sincerely) is not going to be murdered for expediency. Would unit tests help? Truthfully, maybe, and maybe not. If they are integrated into the build and run automatically, they could save the day. If not, well, good luck. The clueless wage slave probably doesn't even know about them and wouldn't run them.
Really, I am not some ivory tower academic who is promoting unit tests because I read a couple of books and web sites on the topic. I have been in the trenches for decades, like many of you, battling some of the most heinous code ever to escape the worst developers' twisted minds, heaven help us all. I have also seen carefully crafted code exhibit some subtly nasty bugs because the boundary conditions were not tested, even though there were unit tests in place. Some that code was even mine, I am sorry to say. So unit tests are no panacea. You need other levels of testing - system level, end to end, and so on, absolutely. But doing without unit tests altogether in a real-life situation seems to me to be total insanity. You can reduce the expense of writing unit tests by focusing them on critical layers of the code. I have in general found that the lower, infrastructural layers of a system are the most critical, because everything else depends on them. The foundations need to be rock solid. Unit tests are the safety net that stops those foundations from crumbling, if I can mix some metaphors. Nothing has applicability to every single situation. Like Aloysha said, one size does NOT fit all. Agreed! But I cannot think of any real-world project that would not benefit substantially from some degree of unit testing.
And I have to tell you: having a good architecture without a good implementation can sink a system just as deep as the converse situation. However, a bad implementation is potentially correctable, while a bad architecture is practically impossible to fix.
Final thoughts: you can make use of unit tests without subscribing to TDD. If your management gives you the time ;-) just write them after the code is written (although you will have great fun refactoring the parts of your code that are untestable without having the benefit of, um, unit tests). For those people who talk about testing DBMS, UI, network, I have two words: mock objects. Again, not a panacea, and it can be a lot of hard work, but still... it's nasty how those pesky unit tests can show up irritating code dependencies, isn't it? It kind of forces better (or at least more flexible) design, I think.
Bottom line: you would indeed be foolish if you did not seriously consider incorporating a unit testing strategy into your real-world project.

I like how it is said I'm attacking a straw man and then the assumption is made that doing test driven work somehow requires that you not think ahead and do the design up front... or even to try to get it right the first time.

I see your "straw man" and raise you a "failure to address the points" such as the reality that not only golden children will touch code and thus designing a system that will detect failures of imagination is a good idea.

My question got lost in the debate. So I'll ask again.

Where does one get started in using Unit Tests? Is there a book I should be reading? or a website? or what?

I am a solo developer, with no budget. I don't even have a dedicated testing computer. And I have to test and deploy apps (all in-house) in ASP.Net, VB.Net 1.1, VBA (Excell and Access) and a bit of Word.

Test Driven Development by Kent Beck
Test Driven Development: A Practical Guide by Dave Astels

Also check out the pragmatic programmers, they have a book for junit and one for nunit. You can also take a look at JB Rainsbergers Junit Recipes, it's a great reference, even if you don't use junit (presuming you can translate from junit to whatever).

None of these books require that you be a TDD fire-breathing zealot, though it's usually implied that it would keep your ears clean.

If you want to test code that's already written and doesn't seem to be testable: Working Effectively with Legacy Code, by Michael Feathers is awesome. This book tells you what's the bare minimum you can do to make your life easier without having to start from an ivory tower of perfect tests, 100% coverage etc.

There's also tons of websites and tutorial surrounding the tools. Junit for java, Nunit for .Net and hundreds of others for all the other languages.

Good luck!

Reading the subsequent comments, I think the most common response to my arguments was "yes, but so what?" and the second most common was "nuh uh!" followed by a set of further utopian best-practices (such as "isolate the UI from the business logic").

There's a fundamental misunderstanding of the argument here. I said from the get-go that unit tests can be useful and I don't have anything against them. My problem is with those who:
a) Believe that they're a legitimate substitute for actual design and documentation
b) Honestly think that they can test for every conceivable bug
c) Recommend - nae, INSIST on it as a "best practice".

Each of these lines of reasoning is not only sloppy, but dangerous.

(a)
If your program is anything but a throwaway or one-off, then there's no excuse for hacking it out without a detailed requirements document and design/functional spec. I've already spotted a few ninja-style programmers in the comments who "just wanna code" and use "but I have unit tests!" as an excuse to prove that they've done it correctly even though there's no supporting documentation. This attitude will burn you in the end. Some people adore unit tests precisely _because_ they are simply code, and believe they've discovered a way to avoid the mind-numbing work of documenting things. This is patently wrong. Unit tests aren't understood by your customers, and they won't help a maintenance programmer much either (I know from experience).

(b)
Unit tests may prove that a particular class or method works, but contrary to popular belief, it does not -always- prove that it works *correctly*. As Jonathan Allen pointed out, you are testing the part of your program that is not only least likely to fail, but also the easiest to debug later on and the least expensive to fix. The real problems, the most difficult ones to fix, are the design flaws. Interdependencies between routines and classes. Semantic mistakes. Sequential coupling. Heisenbugs/schroedinbugs. Erratic behaviour under low-memory conditions. Strange errors getting propagated from library (i.e. DB or networking) code due to things like poor networking conditions. Unit tests test help with precisely 0% of these. If design problems account for 80% of the debugging (I'd say that's a conservative estimate) then even if unit tests halved the time spent on code problems, it's still only saved you 10%, and that may very well be offset by the time spent creating and maintaining the unit tests.

(c)
Unit testing is just one tool that can be used in a full QA program. If the people who preach Unit Tests got half as excited about other types of QA (i.e. exploratory testing and code reviews) then I might be more inclined to listen to them, but instead I get the very strong feeling that they consider it an adequate substitute for a full QA program, which it isn't. There are no best practices. Learn how to do your job, which means knowing precisely how much or how little testing your project requires, and what the best test methods really are. Don't just assume that unit tests will magically solve all your problems.

I have a real problem with Unit Testing gospel because I've seen the above three points in practice. In fact, these assumptions are implicit in many development and consulting shops who claim to use "Agile" methodology. I note that true Agile development can be extremely effective but I am referring here to the vast hordes who use Agile/XP as a buzzword. I've actually seen people state that they don't need documentation and that Unit Tests are just fine for a complete QA program. I've seen the result of this thinking in practice. It's always a disaster. The resulting product never works correctly. The evidence usually seems to indicate that the people who made it don't even completely understand what it's supposed to do - they just dove into the code without thinking and did what THEY wanted to do without heed for actual requirements. Ask for documentation and months may go by before you receive anything. Serious errors are fixed slowly. Enhancements, if implemented at all, can take ages.

Bottom line: Unit Tests are often good. Unit Testing Evangelism is always bad.

"Dave, I think you're using the term "unit testing" to refer to all kinds of testing. It's not -- there are tons of different testing techniques besides unit testing. Yes, unit testing tests variation of inputs, and traditional QA testing tests variation of inputs, and the difference between the two is a matter of scale."

Actually, I'm thinking of the Unit Testing environments I use on a daily basis to get my work done: JUnit, NUnit and RUnit, from the command line and inside Eclipse. I'm thinking of a variety of "asserts" I've written for load testing iterations through large recordset returns, using Schema objects and SQL to test for constraints, missing foreign keys, missing relational records, any number of weird edge cases I can think of.

Can I think of all of them, all the time? No. Have I EVER thought of a database or filesystem situation that I couldn't test for with an assertion from any one of the libraries I've mentioned?

No.

"It is bad testing practice to verify that an API call results in such-and-such database modifications and such-and-such XML being sent over the network. Because that's testing implementation details, details which are the first things to break once any refactoring happens. You should test what goes IN to an API and what comes OUT of it only."

This is another Red Herring. Yes, Unit Testing only guarantees that -- Shock! -- the I/O for the method in question is correct. It doesn't guarantee integration will be perfect and it doesn't put QA departments all over the world out of work. We know this.

But if there's an edge case you're worried about -- missing PKID's, missing constraints, missing records, questions of volume (maybe not TDD best practice, but I can do it, so sue me), in general you can write an assertion for it, and the overhead for doing so is minimal. It literally takes me seconds to write a decent unit test for 99% of the methods I code. And then after that it saves me many minutes, maybe hours, as I don't have to run the whole exe to test any changes -- I just have to compile the class and its test class and execute the test. No clicking through GUIs, no setting up watch varibles or breakpoints, no relying on my own "eyeballing" of the output through several iterations, hoping I remembered to check everything the same every time. Unit Testing simply automates and speeds up the things I'd be doing anyway.

Yes, if the recordsets are large and complex, the setups will take a little longer, but, once again, this is NOT the fault of the unit tests. The tests are STILL making my life easier by AUTOMATING what I'd be DOING ANYWAY.

"Bottom line: Unit Tests are often good. Unit Testing Evangelism is always bad."

I agree with this. I'm not trying to oversell TDD or make it out to be the panacea some people may think it is.

I just find it amusing when the anti-TDD zealots try to tell me what I'm NOT doing on a daily basis with unit tests.

Aaron wrote:

"If design problems account for 80% of the debugging (I'd say that's a conservative estimate) then even if unit tests halved the time spent on code problems, it's still only saved you 10%, and that may very well be offset by the time spent creating and maintaining the unit tests."

I've never seen design cover 80% of the bugs. Maybe you work in a department of 20-year veterans with immaculate coding practices, but in my world there are a lot of recent BS and young coders still working it out. But even if I were in a deptartment with a bunch of ubercodemensches, there are still things design couldn't do.

Brilliant design cannot write methods. You cannot design null checks. Design cannot guarantee that when other code monkeys break my code, I'll know about it within the hour. Our dept. does a lot of design, QA and code review. Testing is just one tool in the box, but it's an essential tool that covers things design and code reviews cannot.

And I'll beat this horse again -- it's faster to code with unit tests than without. It's even faster to code good design with unit tests. There shouldn't be a dichotomy between the two, but in my day-to-day reality, I don't always have control over what requirements I get, or who's in charge of the design process. But I do decide whether my code is going to be tested or not.

One thing I think many people who do not actually do test-driven design fail to understand is the design step that precedes (and follows) the writing of the code. In all the places I have done TDD, there is ALWAYS a discussion of how the new piece fits into the larger design paradigms.

The design discussions and diagramming still happens, but nobody thinks to put the information into a Word document or a Visio diagram. In many cases, that is a shame. Providing a short three to four page archiectural brief with a couple of diagrams provides IMMENSE value to the anyone who later look at the system. They are just a simple roadmap/mindmap to help people get oriented.

Also, do not forget that the last step of TDD is refactoring - improving the existing design without changing its behavior. This is essential because you re-visit your design decisions and make sure they still make sense. Many, many teams (even the good ones) often forget this step and move on to the next test. Again, this is a shame because when you are focused on the implementation details, one must consider the larger design considerations. Even if all the tests pass, you still cannot put a square peg into a round hole.

Also,

"The utopia of separating UI and business code."

Hmm, I'm about 95-99% in utopia then.

What a naive statement. Another damned hack.

Going back a bit here, but the latest version of Visual Studio Team System is geared explictly for unit testing. Microsoft sees the need for testing at the code/unit level.

I have worked on more than a few functional and system test projects in QA. The worst part about testing at this level is finding a bug in production that a unit test could have caught. They exist, they are not unicorns. But what saddens me is how quickly a developer will place the blame on the QA group. We cannot test everything, unit testing is essential.

Resistance to unit testing from developers is the first part of the problem. But lack of understanding from business sponsors and business analysts alike only complicates the issue. If you deliver a poorly designed system to a developer you will get a system, but it won't be what you really had in mind.

The politics of this makes me sick. Developers need to follow requirments, and code what is asked. The BA's need to deliver solid requirements and design specs. The QA group needs to over see the ENTIRE process, start to end. Otherwise your just doing Quality Control, at which point you are better off not bothering to test. The QA group is always relagated to the role of "tester", this is why bugs end up in production.

I used to be a developer, I know how painful it is to have your ability as an artist dumbed down to the science of the printing press. But it is a necessary evil.

Development groups need to be more open minded, this is why you see so much offshoring of development projects. Offshore resources will do as you say to a T. Keep in mind you will find holes all over your design when someone in New Delhi codes exactly what you sent them. Again QA has to sign off on that design before submission, but this goes back to the politics of software.

If you have ever approached a development team with the suggestion of someone else, maybe in QA, writing unit tests you know what reaction you will get. First they will ask "Why do unit testing?" Then they will ask "Who is going to write the tests?" And when you tell them QA/junior developer they tell you "They have no clue how to write a unit test, they can't understand my code, and they will ask me questions all the time, how do you expect me to get anything done?"

At the end of the day any company that is finding problems in their production apps needs to look at their SDLC. The groups that participate in the lifecycle need to have clear definion of their roles and clearly documented expectations. The groups need to have a HEALTHY amount of dependency on one another. Politics need to be kept out of the process. Without this you will ALWAYS find defects in production regardless of what methodology or "religon" you follow.

"One day, some clueless wage slave will be sitting at 3 a.m. in front of your perfact C++ code with a stream of drool escaping the corner of a slack mouth, thinking "WTF is bind2nd?""

Hahah! Nice mental image there. Don't think I've ever used bind2nd, but point understood ...

I think if your coworkers really are total monkeys, then you're just f*cked no matter what you do. No process or technique -- not even one as wonderful as TDD -- can save your ass in that situation. So for the sake of this conversation I'm assuming competent coworkers -- knowing full well that it's not always the case, but if you don't have competent coworkers then any process improvement is giving medicine to a corpse.

"But doing without unit tests altogether in a real-life situation seems to me to be total insanity."

I totally agree. Scroll up, you'll see I've never advocated such a thing.

"the assumption is made that doing test driven work somehow requires that you not think ahead and do the design up front".

TDD has many arguments for it, so not all TDD advocates use TDD as an excuse to write crap and then refactor. But some do. Of those, the argument that "TDD makes refactoring easier" is the most attractive one of the lot, because refactoring is what they do ALL THE TIME. Refactoring should be much rarer than that. One need not adopt a process that optimizes for constant refactoring.

Anyways I think we're having a Colbertesque "George Bush: great president or greatest president?" argument here. I think we all can cite specific unit tests or developer tests that have helped us greatly in the past. But there is still a lot of art to them, and there are still too many ways to write ineffective or unproductive unit tests. And, you know, people who point that out the limitations of unit testing keep getting branded as God-hating anti-TDD commie Luddites (even though everyone accepts that unit testing does have its limitations).

No religious debate here ...

Aloysha, I agree 100% with your last post!
Especially the part about being screwed if a team consists of total monkeys. Not that most of the team I work with are monkeys, but certainly some of them would qualify. My late father used to say, "If you pay peanuts, you get monkeys". That's the sad truth, although regrettably the converse is not always true: if you pay a lot, you don't always get a great developer (but you are more likely to than if you underpay).
The real simian tragedy is that by their actions, many corporations today demonstrate that they believe 3 coders at $33/hr to be better than one real thinker at $100/hr., and that you can just switch people in and out like relays in a fusebox without a serious loss of productivity.

Would Mr T write tests before he went and smashed stuff up? I dont think so.

Hi all,

"Would Mr T write tests before he went and smashed stuff up? I dont think so." - yourMateNick on February 20

hahahahahha lol. If you really think about it, he acctually did the tests and they were called practice. If he didnt, he might just get smashed up himself.

On a smilar note, however, I do think that unit tests are kind of waste of time. But I will definatly try them to really feel what all this hub-bub is about. From what I have read and understood, it seems more like unit tests as a tests which one can do while they code any way. Debugging and hacking works the best for me up till now, and writing more code specificaly for tests for situations I KNOW for a fact that will change will eventually require change in test modules as well...then why bother?

But as I said I will have a go at them.

I do a lot of my low-level (as opposed to end-user-oriented) testing in the debugger. In essence, I don't trust a piece of code until I have forced it through all possible code paths. This might sound daunting - even impossible - but it is not. With a good debugger - and I se Visual Studio which is truly excellent - you can put your code through hoops in short order. Simulate an error condition by changing the result returned by MySuperDuperDooWopADiddyFunction () to false. Use 'set next statement' to walk through a rarely used and difficult to reach code path. Ensure that a particular message box is displayed, at least once, to see that it looks acceptable on the screen and reads OK. Use edit-and-continue to fix a minor bug there and then and walk it through to check that it works.

The list is endless. If you don't work this way, you are missing out.

Paul Sanders
www.alpinesoft.co.uk

I do a lot of my low-level (as opposed to end-user-oriented) testing in the debugger. In essence, I don't trust a piece of code until I have forced it through all possible code paths. This might sound daunting - even impossible - but it is not. With a good debugger - and I se Visual Studio which is truly excellent - you can put your code through hoops in short order. Simulate an error condition by changing the result returned by MySuperDuperDooWopADiddyFunction () to false. Use 'set next statement' to walk through a rarely used and difficult to reach code path. Ensure that a particular message box is displayed, at least once, to see that it looks acceptable on the screen and reads OK. Use edit-and-continue to fix a minor bug there and then and walk it through to check that it works.

The list is endless. If you don't work this way, you are missing out.

Paul Sanders
http://www.alpinesoft.co.uk

I fail at unit testing.
I can never figure out what inputs to test and how to know what the corresponding outputs should be.

@Zeo

> On a smilar note, however, I do think that unit tests are kind of
> waste of time.

Pretty much everyone does until they actually try them.

> From what I have read and understood, it seems more like unit
> tests as a tests which one can do while they code any way.

Yes. But unit tests are repeatable, very quickly. You stepping through a debugger is both error-prone and slow.

> writing more code specificaly for tests for situations I KNOW
> for a fact that will change will eventually require change
> in test modules as well...then why bother?

That's the most important reason to write unit tests. You will find that the code being tested changes much more frequently than the unit tests. Having automated tests makes it far quicker to test, and allows far more confidence in the changes you've made.

> But as I said I will have a go at them.

Good :-) It's the single most effective thing you can do to improve your productivity. It is much quicker to write unit tested software than not (partly because of the vastly reduced amount of bug fixing you do, partly because you tend to end up with a better, more maintainable design)

@Damien

> TDD creates all this inertia around the initial interfaces in the
> code, which start to affect the design. Or at least I think it
> does, I've never actually tried it.

Ah, arguing from ignorance :-) The amount of debate over practices like this from people who have never actually done it is astounding :-)

> who has time to try out every amazing new methodology?.

TDD is not a technology, but a way of doing things.

"I'm with Alyosha. 100%. Unit tests are for beginners. Beyond that they're a monumental waste of time and effort."

Actually, this comment has it 100% wrong. It's the beginners who don't get unit tests. They may have been coding for years, but they're still beginners (in the Dreyfus model sense of the word).

You see this in two ways:

1. "Unit test coverage should be 100%". You hear this from recent converts mostly, but it's not true. Unit tests are very weak in some areas. I never unit test my UI or report layouts, for example. It's possible, but there are better tools to test those things.

Eventually, (some) people get to recognise when unit testing is appropriate and when it's not.

2. "Unit tests are a waste of time". You hear this mostly from people who've never tried it, or haven't tried it much. In fact, if you listen to people who are experienced at unit testing, they are pretty much all agreed that writing code is quicker if you unit test it, and particularly so if you write the tests first. Furthermore, the benefits accrue over the years you have to maintain the code you just wrote.

Unit testing tests your code's behaviour explicitly with little coloured lights, and its design implicitly, because writing tests for bad architectures is hard. Write the tests first and you test your design upfront and continually.

Writing code with fewer bugs and a better design saves you time and money both initially, and over the years when maintenance is easier and more reliable.

If you are not already doing it, then adopting unit testing is the single best thing you can do to improve your productivity. It's not a silver bullet, but it most certainly is a best practice. Just don't try and do it for every single thing.