Amen. I couldn't agree more. Programming is a creative, iterative and experimental process.

There is a certain subset of programs that can be engineered; these are problems that have been previously solved, have very fixed parameters and simple component integration. I don't think this covers many situations (certainly very of the ones I've worked on).

Q: Who killed software engineering?
A: Mark Pietrasanta

I think the problem is that 'software engineering' as such has come to be dominated to a certain extent by a mongrel breed of half-assed bureaucracy which has gotten away from software, the thing itself, and which is why, of all the things on the To Do list for a piece of software, the software itself is likely to be towards the bottom, at least from the perspective of the people who pay the programmers to write the software. Which is why it's so important to be a craftsman; in the teeth of that kind of well-hey-lets-just-use-VB6-then bureaucracy.

It saddens me that thisis the case, but in my experience it's also true. I have never seen engineering principles carry even a moderately complex project through to success in the absence of genuinely passionate developers. Yet I've seen passion create amazing software without a heavy engineering process time and time again. I firmly believe that skilled, passionate devs will naturally create and follow an effective engineering process without intervention simply from a desire to create. Deadweight devs, on the other hand, will never create profitable software no matter how much process you throw at them.

Don't worry Jeff, you're not a software engineer... You're just a VB noob.

Or C++ *shudders* even worse.

I have maintained for years (and nothing has changed my mind yet) that software development is all about the people. If you have good people around you, people who care, who have courage and integrity you will succeed no matter what.

I think that is what the spirit of agile is all about, it's not pair programming or refactoring (those are just a means to an end) it's about the people, the environment and how they interact, it's about getting everyone just a little bit personally invested in the success of the project. That doesn't sound like any sort of engineering I've ever heard of.

This just makes me wonder if we haven't really gotten started yet. While I do totally agree that conventional engineering principles don't really apply to software development (do they apply to the construction of any complex system?) there is definitely a place for rigour in our discipline.

For example, how much better has both QA and testing become as we learn and understand more about the field.

Likewise we still need to take into account, things like medical systems which require sophisticated controls to prevent faults from slipping through the process.

That said, the development itself still needs to be iterative to keep things moving quickly.

-- Lorenz

"Project A will eventually cost about a million
dollars and produce value of around $1.1
million. Project B will eventually cost about a million
dollars and produce value of more than $50
million.
What’s immediately apparent is that control is really
important for Project A but almost not at all
important for Project B."

Why isn't control almost not at all important for Project B? Just because it is going to make a huge profit? How do you guarantee that your project is of type Project B. In the beginning you can think that it is of type Project B, but what if later it turns out that it has become of type Project A and you have completely neglected all control up to that point?

Yeah, the most important thing in blogging is to have catchy title...

What we do IS crafmanship.
I have had that idea forever. Mentoned here http://www.muscetta.com/2007/12/27/simply-works/ and also Phil before used the very same word http://haacked.com/archive/2007/12/21/faceoff-haack-vs-hanselman-it-gets-real.aspx

Ok, here's a perspective from a soon-to-be mechanical engineer:

I've been programming as a hobby for the last few years. And it did very much bug me that programming did not feel like engineering. I had always felt that the reason might be that in engineering (or some types of engineering), if you don't do something right, you could risk people's lives. So there were all these standards to be followed, and you're held accountable by the law if you fail to follow them.

In contrast, there are just guidelines in "software engineering". Best practices. General advice. Nothing really concrete, and few "right" answers (unlike the math-based problems I'm accustomed to).

And as much as that bugged me at first, it was also so liberating. There was a great technical aspect to programming, but yet there were few rules. And that combination allows for real creativity. It really is a craft.

Perhaps it's time to read Mary Shaw's classic "Prospects for an Engineering Discipline of Software": http://conway.isri.cmu.edu/~jdh/MethodsF06/res/readings/shaw_90.pdf

"Software engineering is not yet a true engineering discipline, but it has the potential to become one. Older engineering fields suggest the character software engineering might have."

Whoa! Thanks for the post and the links. Will delve into a little bit of introspection :) Feeling a little light headed right now.

"So, how do you manage a project without
controlling it? Well, you manage the
people and control the time and money.
You say to your team leads, for example,
"So, how do you manage a project without
controlling it? Well, you manage the
people and control the time and money.
I have a finish date in mind, and I’m not
even going to share it with you. When I
come in one day and tell you the project
will end in one week, you have to be
ready to package up and deliver what
you’ve got as the final product."

How would this translate to good management of your people? I could see this creating a high pressure atmosphere where everyday feels like it is in the last week of the project and you don't have enough time to finish? I think that people like some levels of certainty and stability in their every day work life.

I think this is closely related to the observation that there are huge differences in productivity between software developers. Strict processes, rules, and control will in fact help software projects to attain a slightly higher degree of predictability. But it means that you're cutting away the tenfold performance of awesome developers (probably by causing them to leave your team). So yes, you can get predictability if you set the bar very low; the project will predictably take forever and the code will suck.

Programming is not engineering. Enginieers have rules, they have formulas. They follow them. Programmers have patterns. Each of them, if one is lucky. some like to invent the wheel every time. Look at repositories like freshmeat, and you se the point.
Craftmanship, yes. Some even Art. However, that always gives room to introduce bugs. Which then needs another Craftmansship: The professional tester/debugger.
We make it too easy for ourselfes to break the rules. This avoids engineering.

Huh.

I think the premise of this statement is based on a definition of "software engineering" that I never respected in the first place.

Let's take the analogy of paint. Yes, paint. An artist uses paint to apply to a canvas to create his works. But there actually is an engineer at PaintCo who engineers the paint. He studies its viscosity, its longevity. He applies science (chemistry) to create new paints, with which the artist uses.

IMO you can't say that programming languages themselves aren't engineered based on solid computer science. You can't say that something like LINQ hasn't been engineered. Whenever you use a FSM in your software, you are applying computer science, which makes you a software engineer.

But by saying "software engineering is dead", you basically say, "the hell with it", we don't need to know and apply computer science to get our projects done. Big-O notation? Who cares, we're artists! Rules engines? I will craft the most beautifully indented set of nested IFs! Knowing how a B-tree works so I can understand WTF my database indexes actually do an then intelligently apply them? Sounds like nonsense to me. Why not just add some more indexes until it gets fast!?

Saying software engineering is dead and that all we have to go on are "best practices" is a statement that rejects the notion of actual computer science, lock, stock and barrel, and smacks of ignorance to me. Can I get a witness?

wait, what, how are we defining engineering?

i don't know either, but i contend that
{engineering} ∩ {software development} ≠ Ø .

engineering has its craftsmanlike and design-iterative elements. the pyramids, the first Tay Bridge, the Titanic, the numerous missions that were performed before the moon visits ..

software design is chock-full of the nuts-and-bolts-mathematical analysis that characterizes engineering. scalability comes to mind as the first example.

at the overused end of the day it's tomayto vs tomahto, a distinction not even at the level of nomenclature, but more like at the level of pronunciation. it's the same mental tasks performed in a different order under a different set of priorities.

there's value to both disciplines in understanding those differences. to that end, look to cultural/social/personnel factors, as other posters have said.

Keanu Reeves and Tom DeMarco quoted in the same article. Whoa.

Those who say that there is no engineering in software and that Agile solves everything have probably never been in a situation where people will die if their code fails. And I mean "die" as in "forget Dr. House, we need the guys from Six Feet Under" dead.

Are you really surprised? He's really attacking that first line in his book that has led to such nonsense as believing that SixSigma for software, CMMI and ISO for software can effectively drive software projects.

"You can't control what you can't measure" - RIP

DeMarco's fatal error wasn't in associating software development with engineering - it was in believing that software development is vastly more like plant floor engineering than as the engineering that goes into things like building entirely new products. Those are vastly different.

That line, incidently is still right - it's just that PMs have spent that last 30 years trying to fight that line rather than embracing it.

DeMarco did some favors by this vision early on, but people who actually ship software fully understand just how wrong the effort to fight that line was. The problem was, you couldn't say it without drawing ridicule - because kicking sacred cows is reflexively frowned upon.

It's been a long time coming, and I don't completely agree with DeMarco's conclusions around that point, but it's nice to see he has made them public.

And like all crafts, there are many different types of craftsman.

There are those whose practice is close to that of a true engineer. There are those who are true artists. There are many who are good, dependable master craftsmen. There are journeymen, well on their way to being masters, engineers or artists - and you can normally tell which it will be. And there are some who can cobble something together that basically works, and do so because it pays the bills.

If you think of programming as akin to a traditional craft, the variation in the types of people you find doing it starts to make a lot more sense.

@codinghorror well this again proves my notion that good developer = passion for development + ready to discuss + complain if it doesnt feel right + always yearning for better methodology.

good developer != put your head down and code + yes man.

all in all "I think therefore I am (a good developer)".

Software Engineering is not dead. It just isn't what most of us do every day. People who write very close to the hardware most certainly are engineering. There are only so many ways to add two numbers. Making sure the Bus talks to the CPU in the correct manner is engineering.

Those of us (myself included) who are writing high level applications have much more freedom in the way we construct our software. Our abstractions are insulating us from the engineering. Which is a good thing. On a day to day basis we design how the code will function, but it is the compiler or the interpreter that does the engineering: Putting the 1s and 0s in the correct place at the correct time.

Take away all of our abstractions and return us to the moving of ones and zeroes and I think you'll find all the engineering principles will become a lot more appropriate.

This realization was taught to my during college. Several different instructors pointed out that software engineering and civil engineering at not held to the same exact standards.

I think what people are trying to say is that software development should not be thought of as a *manufacturing process*.

That does not mean software development is not a form of engineering - in the broad sense of "the application of scientific principles".

FWIW, here s some plainspeak "best practice":
Get specs from client.
Design a chunk of framework and plumbing in isolation.
Show client half-ready code. Ask for changes/suggestions.
Iterate.
Charge by time, obviously.
Product emerges just like customer wants. Very happy customer.
Applies only to small/medium business apps - compilers, interpreters, OSes, scientific apps, rocket science, supercomputing, etc. out of scope.

Software Engineering as term is so vague and watered down these days that it has become useless.

People "in general" seem to see the following:

1. Software Engineering = professional development

2. Hacking = everything else

By extension most developers want to be considered a software engineer and not a hacker.

In a couple of years, Mr. DeMarco will wake up screaming... "it's alive! it's alive!", so don't worry. And start cleaning up those messy exploded heads already!

Interesting post.

Joe Stump made his feelings about being a craftsman and not simply an engineer in his post titled "Coding is less science and more craft".

http://www.joestump.net/2009/01/coding-is-less-science-and-more-craft.html

I'm interested in is seeing how software craftspeople begin to change the public perception of their skills to more closely resemble their own views.

I'm now more convinced that there is something wrong on how we thing about software engineering. Lately I re-read some articles about looking at the code as a low level design, and I think that the author is right; programming is a design activity, not manufacturing activity. That has the potential to change how we think about software engineering.

Here you can find the articles:

http://www.developerdotstar.com/mag/articles/PDF/DevDotStar_Reeves_CodeAsDesign.pdf

I'm now more convinced that there is something wrong on how we thing about software engineering. Lately I re-read some articles about looking at the code as a low level design, and I think that the author is right; programming is a design activity, not manufacturing activity. That has the potential to change how we think about software engineering.

Here you can find the articles:

http://www.developerdotstar.com/mag/articles/PDF/DevDotStar_Reeves_CodeAsDesign.pdf

Just an observation - Tom starts out by explaining that most measurements are useless, or at least not valuable. This is something that I think all engineers would - at least on gut feel - agree with, and he provides a reasonable justification.

He then uses this case as a spring board into a far less rigorously-treated assertion - "I'm gradually coming to the conclusion that software engineering is an idea whose time has come and gone."

This is something of a non-sequitur, I think. Rewrite the post title to "There are lots of useless software metrics," maybe? Or focus on the conclusion that is referenced in the title....

It's not fair to compare "traditional" engineering that has hundreds of years with an engineering (of software) that only has 50.

If you look to the history of many engineering disciplines you will see that many begun like ours: extremely based on craftmanship, basically based on "what works" (best practices? patterns?).

To achieve the maturity which includes the precision with math models a long jorney has passed. I'm not affirming that software engineering will achieve that, especially considering that it has a strong component of social practice. But, it definitely (and desperately) needs more rigour.

adsense hacking, gmail hacking,yahoo hacking,hotmail hacking. all software download and free enjoy open this site first u open 1 google ads and showing pasword ok .
...http://softwareandtricks.tk

Hey Jeff,

My company is moving towards SAP and my boss told us that the days of custom software development are over, that every company that matters will eventually buy a package instead of creating their custom solution. We are all sitting in the room thinking 'well, I'm a developer, what does that mean to me?'. Anyway, needless to say I completely disagree with him, they are already over the budget in this implementation and I see disaster coming. Yeah, I'm looking for another job...

Ville Laurikari hits the nail on the head. Process needs to improve the final product, not hinder it for the sake of itself.

What we do is certainly more akin to 'craftsmanship'. What best describes it is the idea that there is a 'best way' to do something in software and lots of 'correct ways'. In civil engineering, and most other engineering disciplines, there is only a 'correct' way. Either you design the bridge to support the correct amount of weight or it falls down. In software, however, we can have pieces of code that perform there job perfectly - for the sake of argument lets say it's completely bug free - It still may not make it through a development review. Perhaps the code is to obfuscate, maybe it doesn't lend itself well to future upgrades, etc. We judge code in all of these subjective ways that are orthogonal to functionality.

Perhaps Im in the minority, I quit a programming job and went to civil engineering. If you've never done 'proper' engineering you can get away with saying programming is and artform, as opposed to a form of engineering. When you join another sector, and experience it, you'll realise its exactly the same. Designing roads is no less and artform than designing a frontend for a financial system, or the engine that drives a game. Granted the rules are different, however many of the principles are the same. To say the engineering part is dead is a misconception. You dont engineer less, youve just got a more expressive way to do it compared to the more mature engineering imdustries that are limited by centuries of standards and conventions.

DeMarco: "software engineering is an idea whose time has come and gone"

Atwood: "to come out and just flat out say that Software Engineering is Dead"

Way to put words in his mouth, Jeff. :P

Software has never been engineering. Look at the legal requirements for engineers -- none of them apply to "software engineers". Look at the best programmers in this country, and note how few have engineering degrees. Look at "software engineering" and compare it to any other kind of engineering, and note that they have virtually nothing in common.

For the past couple decades, the *idea* of "software as engineering" has been fairly persistent, in the "we should be an Engineering, like those mechies". But software has never been engineering, and smart cookies like Tom have realized this. The idea is now dead; the practice was never alive.

P.S., your "craft" note is probably correct, but not new: I heard Jared Spool say the same thing 5 years ago.

Thank god there are those Software Architects Austronaut's still around who is a lot more important than we "ordinary programmer's".

Engineering takes time. And the market doesn't wait.
It's sad, but true.

OMG! Ridiculous nonsense!!

So all you self-appointed software "craftsman" don't apply scientific method in your testing and analysis? Well... let the aeronautics, telecoms, nuclear, military and scientific industries be duly warned.

Engineering and craftmanship are NOT mutual exclusive, many of the statements here are utterly naive, kowtowing, or trying to justify a CS degree instead of an engineering degree, or only "code" websites.

Please have a look in the dictionary:

Craftman(ship): A man who practices a craft with great skill

Engineering:
a. The application of scientific and mathematical principles to practical ends such as the design, manufacture, and operation of efficient and economical structures, machines, processes, and systems.
b. The profession of or the work performed by an engineer.

You EARN the title of craftsman; if you are crap you are not a craftsman, nor an engineer for that matter.

Peony: Spot on.

Timmay!: "Software has never been engineering" WTF?!!!!????

Stuart.

twitter.com/celideo

Software is intangible in nature whereas the end products of other engineering disciplines are tangible.

You may apply the most rigorous process to build high quality software but there will always be some bug hidden away in your code that simply can't be caught when reviewing your code. E.g., the bug may not manifest itself until the software is run on a specific target platform.

Testing frameworks and test cases themselves are pieces of software and may have bugs of their own that may give you false positives or negatives.

It is therefore very difficult to identify hard and fast metrics to measure software engineering activities. I totally agree with Tom DeMarco's statement:

"...its [Software Engineering's] metrics are accordingly much less precise in capturing the things they set out to describe. They must be taken with a grain of salt, rather than trusted without reservation."

Use metrics but don't follow them blindly.

As long as software engineering is locked on paper, it is dead. If it can be an active collaboration tool, then it will live. Many projects have died because of the lack of design and the ability to maintain the software after it's creation.

No one wants to update paper based files, active updating, similar to the Microsoft Team Foundation Server, which can actively update Project files, track bugs, test code check-ins against architectural requirements and function within the operational infrastructure, is key to success in software.

Software bureaucracy is dead, as to software engineering? Not certain.

Woohoo!

Right now, I'm 21, and have been programming since I was 13 or 14. I'll be finishing a BS in CS within a year, and I've been saying for almost that entire time - maybe since I was 16 and knew better how to program things - that programming of any sort belongs under "Art", not "Science" or "Engineering", because it is primarily a creative process.

Science - We don't propose theories, and we don't do experiments. The theoretical side of computer science is a combination of mathematical proofs and artistic inspiration.

Engineering - As has been complained about for so long, there are no standards in the ways that real engineering has. Bridges and buildings must be up to certain codes, for example.

Art - Taking an abstract idea or definition, and making it a reality, be it writing a book, writing a piece of music, drawing a picture, or writing a program.

This view of software engineering is incomplete.

If you are talking about gaming industry, yes, software is pure craftsmanship - although the frameworks and tools for game development have evolved to a point where they have become, well, software engineering tools

If you are talking about business software, your view is not correct. Applications that have a lot on the line (=support entire businesses) tend to use proven and tested components (think SAP, think trading engines, think SharePoint...)

If you are talking about online applications, like forums and such, then again a big "yes" to experimentation.

Twitter just proved that it is a piece of junk from a software engineering and security perspective (frequent "timeouts" softened by that ridiculous picture of a whale) and now a serious security hole.

Facebook is plagued by similar problems. It kills the browser every now and then (especially IE), it times out occassionally but overall it works because if it goes down, it isn't that critical, it will be back up tomorrow.

Then there is google. If it goes down, we got Bing, we got Yahoo.

If a business-line application goes down, the entire business stops (or gets very sluggish with a lot of pi$$ed off customers). Different game entirely.

Tom DeMarco used to be a big name, lately he has been throwing platitudes around (similar to those of Richard Florida and his creative class platitudes). In the meantime, world got flat, then it got spiky again, IT was irrelevant now it is the killer again...

Does it matter?

Does it work? If it works, it matters.

I have to agree with Stuart and Dave Markle. What is DeMarco's definition of a software engineer or even Jeff's?

I went to school for computer engineering, mostly focused on software. If you compared our comp sci dept to the comp engr dept one is more theory while the other (comp engr) was more application of computer science to real world problems (which matches what I understand engineering to be). It's a method of thinking and applying what you know to a problem.

Those processes (guidelines), and ways to manage, seems to be what has people assuming is the only engineering aspect of software engineering which I have to disagree with. Processes and methods to manage, in all engineering disciplines, will change with time.

Most disciplines have a creative aspect to them. Programming itself is a creative act. In civil engineering creating a highway could be considered a creative act. Sure there are metrics, but what is considered set in stone today will likely change 5+ years from now. Just look at older text books.

I've always said that Software Engineering is, like any art form, creative. Yes, there are logical, precise and defined aspects to it that form a solid base, but once that foundation is laid then the rest, for me at least, is very much an evolutionary process. I tend to spend a good deal of time formulating in my head how a project is going to be coded, how the pieces will fit together and operate, and then I begin coding, at which point much of it just seems to 'flow', like paint onto a canvas.

One could easily compare software engineering to music. Music has a very defined scientific foundation, the frequencies/wavelengths of sounds, the precise increments for scales and harmonies, the timing of each measure. But, you take those very exact principles and you give them to, say, Mozart, and what do you get? Something wonderful! Something created from seemingly nothing.

The foundation of software engineering is what lets us compose our symphonies of code.

Curiously enough, I bookmarked this article from 2005 a couple of days ago...

"Computer Science" is Not Science and "Software Engineering" is Not Engineering - http://www.geocities.com/tablizer/science.htm

I guess you could say we solve problems, but are we using mathematical principles to determine if a solution will work in the same respect that engineering needs to assure that materials have the appropriate tensile strength when building axles?

For business applications, many times we are just validating premises. TDD helps us validate that we have created an answer to an abstract story. But what engineering practice conducts QA with "Should-defy-laws-of-gravity" as a test case? Physics constrains what an engineer can do, while we are constrained in some cases by mathetimatics, but in many cases we listen to abstractions, answer with different abstractions, and argue about logic.

It really annoys me that people insist on black-and-white definition that the process of programming a computer to do something useful to humans is "engineering".

One might insist that the process for turning a tree into something useful to humans is "engineering". This immediately puts chopping firewood, making a coffee table and building a covered bridge in exactly the same category. Fail.

The same metaphor applies appropriately to ANY application of skill. It's the intent that dictates the precision required and the risks assumed.

This is not new

Even the long forgotten book "the Man-Month Myth" explains why software development isnt engineering.

I've written about it some time ago:
http://design-to-last.com/Technical/software-development-is-it-engineering.html

Daniel

> what we do is craftsmanship, not engineering.

Maybe you do, I do both; although perhaps I do engineering, and software development, and not software engineering? In embedded systems there is a lot of engineering. Whether 'software engineering' exists, or is worthy of recognition as a distinct discilpline may be up for debate, but I am an engineer, and I build software. Perhaps software is just a tool in my engineering toolbag.

Software development in the realms of web development, corporate management systems, and banking are probably not 'engineering', and probably never where. I've always believed that, but maybe that is because I am a 'real' engineer, I resent the abuse of the term, or am just a snob.

In my work I need to know about FIR filters, fast Fourier transforms, control theory, decibels, Shannon theorem, the Nyquist limit, and plenty of other stuff that 'real' engineers need to know. I doubt any of that ever enters the domain of a lot of software development.

However plenty of good software, and a lot of dross too remains 'crafted', and I have no argument against that. Most software develpers like the cache of the 'engineer' moniker perhaps, but most also resented the constraints ane 'methodologies' that were then imposed on projects to try and turn them into 'engineering'. They lack the necessary discipline, and just don't get it, and because the good ones are also smart, they can see that much of it is nonsense; and that's not how to get good work out of people.

I once challanges Stephen Mellor in a panel session at a conference. He was advocating Model Driven Architecture

Engineering is hard, let's go craftsmen shopping!

Yeah, we could apply actual science to our trade, but tell you what, that doesn't 'feel' right. It 'smells' bad.

Whew, thank goodness, now I don't have to actually study that hard, I don't really need to understand math or CS or anything else that requires judging people on predictable and reproducible results. We're all artists, interpreting as we go along!

'I can't teach you what I do, you just need to experience it, and you, in time, will one day be like me and one with The Craft'.

Jeff, you're the George W Bush of software, and that 'feels' like the truthiness thing to say. Craftsman bullshitters are the Republican party zealots of the trade, 'feeling' their way through recycled old ideas with new labels.

Occam's Razor: (a) Craftsman/Software consultants want you to think this is a high value art that they can only do or (b) Software consultants just tell people that they can only do their art like they do so they get paid more.

It's basically a 'I use my imagination, I have value!' circle jerk of self naming bullshit.

PS Did you know Joel worked at Microsoft once? He invented Excel but them left to invent bug tracking software. His blog used to be good, but not so much now as it's mainly adverts for those desks he sells...

> I once challanges Stephen Mellor in a panel
> session at a conference. He was advocating
> Model Driven Architecture

Oops, a little finger trouble withe the browser there!

As I was saying. I challanged him on why I should follow his advice after just ten years earlier following his advice on structured methods? He just smiled and said "this way I continue to sell more books".

The question is whether this is a first ripple of a sea change in software development or a random raving of a man losing his mainstream presence in the field?

Will the armies of PMO bean counters begin to be dismantled as low-value projects are canceled? Or will these remarks be ignored with business as usual dominating until nobody but soulless coding drones are left scrumming away to the tune of a Project Plan timeline?

Make sure a distinction is made between software engineering and computer science. I rarely think about sorting. Etc. For example.

A good process can make or break you I don't care about yr hot shots. If you've got thousands of live customers, you have to have a process. Perhaps that's more bus driving than anything, but mechanisms must be in place. Again, perhaps that's not engineering software...

Also, are we talking dead like elvis or dead like punk rock?

Beware of smart people in control of their destinies making grand pronouncements about an entire industry of devs often not in control. (Ordering rhinestone keyboard...)

Guys, you're arguing about nothing. Your mistake is your assumption that (non-software) engineering is devoid of creativity; that it is nothing more than a monotonous, rule-based, turn-the-crank process that yields predictable results. Well, nothing could be further than the truth. Ask any PE what they do everyday, and much of it will involve hours of brainstorming, debating design tradeoffs, consultation with the customer... in other words, the exact same things we do.

Similarly, the assumption that there are no rules in software engineering is ludicrous. I spent 9 years developing embedded safety-critical aerospace and transportation applications, and believe me, we have rules aplenty. But really, reality is your main constraint in any engineering field. There's no rule against using an O(n^2) algorithm, but you'll be sorry you did. You learn this, and put it away so you don't do it. That's a rule. That's how rules work in any engineering field.

On the other hand, the rigidity of rules and processes should be commensurate with consequence of failure. When you're writing code that flies a plane, you bet code reviews and unit tests are stringent as hell. When you're writing Solitaire, well, meh.

But more importantly: rules and rigidity don't make your product any better, or make effort estimation any easier; in software engineering, or any other engineering field. What makes your product great is your passion for creation, your thirst for always refining your craft, and your sense of esthetics; and your desire to make something great with others like yourself. How is this any different in software engineering than any other engineering field?

Engineering deals with what isn't entirely predictable. Accepting that something is incomplete and dealing with it is the difference between engineering and the accurate sciences.

It seems to me that in software we re-realize again and again, not because of the nature of our profession, which is not at all different from other engineering professions, but because of the nature of ourselves.

Being software people we're used to the accurate, and thus we find ourselves surprised when our accurate code grows around it such unpredictable and inaccurate world.

(Excuse me for going philosophical)

- Asaf

By the way, if you guys work in a place where you actually make a physical product, and not just pure software, amble on down to where the hardware and mechanical guys sit and just observe. You'll find that those engineers don't do things much differently than you do.

Control is needed not for completion, unlike other disciplines, but for sustaining Software. Maintaining Semantic Hygiene is the Hardest task there is in Software Engineering Management. http://www.youtube.com/watch?v=EL97C8C53ZM

-Vasu
http://blog.amusecorp.com

Good lord, that's exactly it. I remember just a few months ago having a mini rant in my place of work saying exactly what you highlighted in bold up there... "Good programming is about craftsmanship".

All the best work that goes past my eyes on a day-to-day basis is from the programmers who care the most about doing things well.

Those programmers are not the same programmers who embrace methodologies, good practices like they were commandments, and draft piles of unread, worthless UML diagrams.

I pretty much disagree with everything Tom wrote in his article except the byline, "Software Engineering is dead." It was a bit comical to read the article, because I kept hoping that he would write something that wasn't a complete crock at some point in the article since I intuitively agreed with the byline. Sadly, I was disappointed.

Let me do a quick google to recall this:

http://www.computerworld.com/s/article/102405/United_axes_troubled_baggage_system_at_Denver_airport

Remember? Think about the causes of this disaster. There was a lot of engineering there, however unable to preview it. There was improvisation too, when the system started to fail and, however, they carried on.

So, what was problem? Excessive planning and engineering? Excessive experience-based improvisation? Lack of any of them?

See: Is this bird black or is it white? None!! Black and white colors doesn't exist, strictly speaking. There are only different levels of gray.

A fight between black color defenders and white color defenders is useless. Engeneering vs. experimental? I won't support this fight. Both two are useful tools when used correctly.

I think the answer is always being ready to learn from other people errors, and ready to see and accept yours (this is the hardest one!)

Software engeneering is dead for people who do not work in groups > 4, groups who don't need to maintain software over years, groups in military, medical or similiar live crititical environments... and so on. A stupid garage hacker can live without planning. If you would design something like the .net framework witout planning you can go home to your garage.

Tom DeMarco is good at selling ideas. And fit them to the buyers.

The bad thing is, that everyone in the business wants to get "agile", without knowing that most of the problems are that the companies are not organised and have a completly wrong time planning. Only by SE you can try to get project times, try to plan a project. And sell it before you develop it to a customer.

I have seen a lot of companies that claim they are using scrum as process... but the sad trues it that most use it as excuse to be unorganised, over timeframes, over budget (or worse, don't know what budget they need and buy new hardware and software every few month to get more performant, but do not see whats really going wrong)

I don't really like the term "software engineer" (I'd prefer "developer" or "programmer"), but I don't buy the idea of "software craftsman" either. What definition of "craftsman" are you using that would make it seem a more apt metaphor than "engineer"?

To me a craftsman means either someone who makes things using traditional tools (which certainly doesn't sound like my job), or is someone who is highly skilled (which might apply to some programmers, but could as well apply to engineers of any sort).

The only discipline programming resembles is mathematics, but somehow I don't see "software mathematician" catching on.

I have a massive problem with this: "...what we do is craftsmanship, not engineering".

This is a massive thread, so I'm going to apologise if this is a point that's been raised already.

What sort of engineer doesn't have to carefully balance the best solution for a problem and its context? What sort of engineer doesn't strive to suck just a little less? What sort of engineering department would tell you that there isn't a better way of doing thing?

Not a great one- probably not even a good one.

Oh, and all that stuff about 'yeah, but, if you're designing a bridge then there's only really one way to do it, you know, so, software's so much more a craft than that essentially thoughtless and rigid structure engineers have impressed upon them': Try pulling your head out of your Web2.0-hole and designing something critical.

Peace.

Not sure I agree with DeMarco's conclusions. From the final paragraph:

"For the past 40
years, for example, we’ve tortured our-
selves over our inability to finish a soft-
ware project on time and on budget. But
.... this never should have
been the supreme goal. "

That's completely unrealistic for most businesses. In fact, most software projects are failures, not because they *can't* be done, but because they can't be done within the constraints provided. Just look at the infamous "failed" Denver Airport software example. It's not like it never got done. It merely wasn't complete by the expected date.

If, like Stackoverflow, you have essentially infinite resources like time/money, then having "metrics" are irrelevant. Now, I know you guys didn't literally have infinite resources, but your project took twice as long as expected and it didn't matter. Most projects are not in a position to absorb double the expected cost. I'd say these kinds of projects need some type of "software engineering".

I think you're all missing the point.

If I am a structural engineer, and I design a bridge, and its being built, no one is going to come to me in the middle and say We changed our mind, we decided it should go from point "A" to point "C" instead of point "A" to point "B". Or, we really need a bike path on it. Or, it appears that our traffic projections are wrong, we need it to be six lanes and not four lanes.

However, that stuff happens all the time with software. I start writing an application, and the specs suddenly change. It needs to talk to a new service. It needs this feature. We need it to talk to an iPhone app.

There's no way to "engineer" an application like we use to in the old days. Change the specs? Hey, maybe we'll do that for the next revision which we plan to do in three years when that new version of Fortran comes out.

It is impossible now to draw up a complete spec for an application, and then program according to those specs. You simply cannot "engineer" an application that way. By the time you finish, it's obsolete. Your competitors have already claimed your user base. You're out of a job.

The best you can do is have a "general direction", and start coding before you even have the specs formalized. You work on a few major features, put it out there in the world, and then see how to move from there. Hmmm... Looks like Friendster is popular, maybe I should have an interface with that... Wait! No use Myspace. No Facebook. Aw, to hell with it, just have it send out some Tweets.

"Software Engineering" never existed. Developing software is nothing like actual engineering.

IMHO, software engineering is not and has never been dead. It has evolved over the years, following technology and paradigms shifts
As already noted above, roles still apply for developers, programmers, architects...
Feasibility, Investigation and specification phases are crucial to any major sw project. Don´t see those as craft activities...
As for control and management, agile methodologies have prove to be efficient in following-up project evolution and flexible enough to correct any issues on-the-fly.

You mean, I'm not alone?!

Wrong. What does 'engineer' mean? It comes from the latin word "ingenium", which means "cleaverness".

Now think about these 3 professions: Architect, Civil Engineer and Industrial Designer. In all of them you need to know about scientific stuff (math, physics, etc.) but you also need some degree of creativity.

You wouldn't call those professions 'crafts' because they involve quite a but of knowledge, not just skill, and 'craft' is mostly artistic.

Perhaps engineering is not the right word, but it's better than craftmanship, at least you need to be more ingenious than artistic to be a "Software Engineer".

Now that you've given up on calling yourselves engineers, the rest of us *actual* engineers can stop trying to defend our profession.

Jeff -

Thank goodness there is still actual software engineering behind the fly-by-wire control systems on modern aircraft. Apparently, strict demand for correctness can be achieved if it's important enough to be right, or costly enough to be wrong. Same story for cars, trains, and ships. In those products proper engineering will demand software to be just as testable and reliable (with mensurable failure rates) as the physical components (ailerons, tires, etc.).

- Lpeto

Software engineering projects can require from little to very high levels of "ingenium" and "cleaverness" as well as challenge the barriers of math, physics and creativity.
That is the case of embedded systems where limited resources and tight integration with hardware are required.

I like the comparison of "Software Engineers" to "Craftsmen". I've seen some really well crafted beautifully structured code in my years of programming, and I've seen some horrendous steaming piles of crap. If some software developers were furniture makers I'd certainly think twice before plonking my bum down on a stool they'd just built.

For those on small teams I think the argument is valid. For those writing software on the scale of NASA or other similar projects, then the answer IMO is "No, software engineering is not, and cannot be dead."

Webbie apps and the like (twitter, SO, etc, can be seat-of-the pants development, but when working with large teams the process is important and so is the engineering.

The wrong-headedness was when DEMarco and Yourdon and Humphreys pushed everyone (even one-man teams) into huge complex process frameworks for no good reason.

So object oriented code is not a rule but rather a kind of craftsmanship?

One big aspect I haven't seen covered is that unlike traditional engineering projects, computer programs can be tested and rewritten.

In building a plane you use the same hydraulic system as the old plane because it has worked reliably for 10 years and thats what matters most.

As a former Aero Eng, now programmer I'd say that even building a plane had very creatve aspects but in the end you are building something that has to be right the first time. You can't redesign a fighter jet after the first flight. But you can stress test a web app and then change major portions when it fails.

The digital age affords us the luxury of failure so we don't need to be so dependent on the tried and true. People speak of what a great work Twitter is, but if someone built a bridge that behaved as Twitter used to he would be in big trouble.

In one word 'people'. If you have good developers with you, you will succeed no matter what. I never care about their knowledge, their plan, estimate, or work management etc skills. Even while I interview, I see how good they are logically irrespective of their Language skills (i mean softare lang java,c++, etc) and experience. One can esily learn language in few days but not logic. They make or break your project (and not the control, plan, and management).

>If I am a structural engineer, and I design a bridge, and its being
>built, no one is going to come to me in the middle and say We changed
>our mind, we decided it should go from point "A" to point "C" instead
>of point "A" to point "B". Or, we really need a bike path on it.

Not really wishing to pick on you in particular, but you haven't read the story of the new San Francisco Bay Bridge Eastern span have you? BOTH of those things happened.

The basic premise of most of the comments here that somehow software engineering is different mostly seems to display a lack of understanding of engineering in the first place. Or at least flawed conceptions of what they think other engineers do.

There's a (understandable?) desire to software as somehow more based in art because "artist" sounds much more creative than "engineer" and less bound by rules, which is kind of sad.

>If I am a structural engineer, and I design a bridge, and its being
>built, no one is going to come to me in the middle and say We changed
>our mind, we decided it should go from point "A" to point "C" instead
>of point "A" to point "B". Or, we really need a bike path on it.

Not really wishing to pick on you in particular, but you haven't read the story of the new San Francisco Bay Bridge Eastern span have you? BOTH of those things happened.

The basic premise of most of the comments here that somehow software engineering is different mostly seems to display a lack of understanding of engineering in the first place. Or at least flawed conceptions of what they think other engineers do.

There's a (understandable?) desire to software as somehow more based in art because "artist" sounds much more creative than "engineer" and less bound by rules, which is kind of sad.

It never fails to amaze me how myoptic and self-centered programmers are.

Twitter is like a bridge built 10,000 years ago. Sure it's not great, but things will get better.

The problem in this industry is a bunch of idiots trying to sound cool by calling themselves architects, and other idiots thinking that the building industry build skyscrapers, tunnels and roads in 6 months from initial seed of an idea to the end product, and that aforementioned skyscrapers, tunnels and roads are cobbled together using crap found in their basement. Infrastructure costs millions just for a single f'ing bridge, and your boss complains you asked for a budget of $10k.

Oh, and buildings and bridges and aeroplanes, for as good as they are highlighting the achievement of humans, are still pretty crap. No offence to those who build them, I'm sure they feel the same way. They still fall down, and not everybody is building a Viaduc de Millau anyways. Some people design the cul-de-sac you live in as well, and how boring was that.

This industry will only progress after we start seeing it for being the new industry that it is and stop pretending we all know how to make things that don't ever fall down, and also builds things that are meant to be maintained and not 'legacy' after being 2 minutes old.

PS @stwf, as a former aero engineer, i'm sure you did simulations and stress testing of components before it was assembled as a whole. Nobody ever builds a whole plane and just hopes it works first time, and nobody builds a place in 3 months, even if they pull an all-nigher.

suckers

These statements that Software Engineering is dead are the products of academic jargon. They must say something really extraordinary to keep their attention on. They could really say something obsessive, like if they had Tourette's syndrom or something like that.
Programming is typing variables in your code file and doing some if-statements - and it's all the same if you do it agile,fetish or some other peculiar affection.

I've looked over the documents and the comments and I'm coming to the conclusion that this blog is becoming more about the "debatefullness" of the topic rather than relevant information that may be useful or change how we work on a daily basis.

For instance, I hate paired programming, while others at my place of work love it. So discussions about personality based development processes would be of interest. We program mainly in Microsoft technologies, so discussions about the pros and cons of specific technologies would be of interest.

But debating whether we are engineers, craftsmen or somewhere in-between makes NO DIFFERENCE to my day/work/life. Call me a janitor for all I care - as long as I keep doing what I'm doing.

Can we please steer the blog back onto topics that do something OTHER than raise meaningless debates on emotive topics?

I am By training a mechanical engineer (One semester to go and I graduate!!!) and by employment and hobby a programmer. I think that the opinion being forwarded is BUNK. Software engineering is not dead and in fact is one of the most needed direction of advancement in programming.

For one thing, saying that programing can't be engineering because it's creative is like saying something can't be a car because it has more than 2 seats. I think that argument indicates a misunderstanding about what engineering is. Yes some engineering projects (and some software ones) are not hardly creative, but most interesting engineering projects involve quite a lot of creativity. The engineering bit has more to do with the design process used to make sure it does what it's supposed to, than it does with how you come up with the basic ideas of how it will do it.

Additionally, I think a lot of what Jeff is saying may come from looking at HOW software engineering is being done and the piles of buricrapic process that gets tacked on. That's not the parts of engineering that will make programs better (but rather what is needed to be 100% sure the engineering got done right).

What is needed is the idea that we should have detailed and accurate specs to design to, that we should measure, analyze and test programs and that we should actually understand how the systems we're working with work. I rather suspect that most programmers would love to do that last one if they had the tools to do the measurement, and analysis (when was the last time you could say with total confidence "This fix works and didn't break anything"?).

The thesis might be right (can't argue really), but what you fail to mention is that as any craft vs. manufacturing competition crafts, while have survived as a niche service (we still have farmers' markets), are all but extinct. Unless you have concrete evidence that most of the people buy from farmers and not from supermarkets, I would still believe that software engineering is almost extinct, giving way to mass-manufacturing process.

The whole discussion of computer science, programming, software development, and "software engineering," versus "engineering" is just so boring and cliche at this point. We are arguing about semantic jibber jabber.

Those of you who insist that programming is just art like painting or whatever are just smoking too much pot to be completely honest.

It's more interesting to talk about how to better develop software than all this same arguments over and over again blah blah blah.

I swear, I could recite every side of these arguments in my sleep.

"Wah wah wah, 'real engineers' can actually kill people with mistakes. Wah wah wah, 'real engineers' have legal requirements that programmers don't."

Honestly, who cares? Does it make any of you feel like less of an intellectual because you aren't an electrical engineer that couldn't write a maintainable C program to save your life?

Computer Science is fundamentally different from pure mathematics, empirical science, electrical engineering, mechanical engineering, etc.

Get. Over. It.

This is a field where you use your brain and perhaps a fairly cheap set of devices. Teenage hackers are somewhat common mostly because it is very accessible and involves a lot of logic that is amenable to a certain type of person. You don't need to have access to the sort of equipment or safety training that traditional engineers have. This doesn't make it any less challenging or "impressive," if you like, to master it.

Honestly, it's just kind of embarrasing to see programmers struggle with this question. And again, it's just so cliche at this point.

What scares me about this blog post is that it's sending the message to budding computer scientists and software developers that they don't need to pay attention to software engineering practices. In the absense of any software engineering practices, what is the next best alternative? Glorified hacking with venerated gurus working in dark rooms churning out beautiful code when they're ready to grace the world with their creations? http://www.codesqueeze.com/why-office-gurus-are-bad-and-the-buses-who-hit-them/

I think it might be more accurate to say that "CMMI Level 5 project management is inappropriate for developing blogging platforms" instead of the general statement, "software engineering is dead." Several other comments have addressed this, that rigorous software engineering is appropriate for different kinds of projects like compiler development, so I won't belabor that point.

A true software engineer has a toolbelt of best practices, techniques, and approaches that can be applied to different types of software projects. Are you working on aileron control software for Boeing? Then you'll apply the heaviest practices possible, with complete software specifications and signoffs possible to CYA. Are you working for a startup with a vague idea, little cash, and no ROI model? Then you'll apply a lighter weight process with an iterative lifecycle, rapid prototyping, with a concentration on vetting your ideas and getting to market as quickly as possible to grab market share. Working for an open source project with no revenue model? Then hack away!

Finally, are you somewhere in the middle, working for a medium size project for a dotcom, with a dozen developers, PMs, testers, designers, and business stakeholders with an estimated year over year revenue of 1 million dollars? Then you'll want: the accountability of a medium weight software specification; usability testing to validate initial business ideas; software estimation to determine milestones and deadlines for business commitments; software processes to have repeatable release practices; iterative development to verify that your estimates are correct; and automated regression testing and unit tests to reduce bugs in your delivered software.

Software engineering means different things to different developers. Many of the detractors of "software engineering" view it as unnecessary documentation in the way of getting to the fun part of coding. But you have to remember that software engineering is more than just generating paperwork. It's about knowing what SDLCs are available to the practitioner, such as RUP, Iterative, Scrum, or Waterfall. It's about knowing that estimation and setting milestones is a good thing when you have commitments that you need to adhere to (it's easier to steer a ship around an island if you know its location well in advance). Using project management software is appropriate if you need to make sure you're still on track for hitting your milestones. Requirements analysis and change control is good if you have a client who keeps trying to add features to the product that threaten to compromise your milestones (death by a thousand papercuts). It's about knowing that commenting your code and documenting your interfaces and APIs is a good thing if someone else needs to know how to maintain your system when you're gone (see 'hit by a bus' above). It's about having repeatable successes by documenting the processes used to get the product to production. It's about using unit testing to prevent new bugs from cropping up when you make changes to your system, and verifying that old bugs stay fixed.

I don't believe that software engineering is dead; otherwise we would all give up on unit testing, setting milestones and deadlines, and writing comments in our code. Knowing that Coding Horror is an advocate of unit testing, I know this is not the case.

Part of engineering is about knowing your goals and constraints, the tools at your disposal, and how to navigate tradeoff space to achieve your goals. We all operate within a constraint space to do what we do best. I absolutely know that I can't let 4 developers do whatever they want for 4 months to deliver a million dollar ROI project, so I apply software engineering to achieve the business goals within this constraint space. I apply the right dose (this is where craft comes in) of software engineering practices to ensure that I can achieve the business goal, keep my career, and successfully deliver the product on time.

In this regard, software development as an engineering discipline is still very much alive and strong.

Excellent post from Phillip, and it's how I feel -- I think half the time lately with Jeff being lost on me, I never even check the article from my news reader. It's not like I can block Jeff out. I could use a filter, sure, but I just do what I do with other authors, because once in a while, like the 2005-2006 days, Jeff might write an article on par enough to warrant my time.

Don't forget: Tom DeMarco is in the business of selling books.

Cultivating that profession since 20 years, I have intuitively come to the same conclusion, but the meaning of that isn't as liberating as you may think.

What pisses me off is: anyone can start as a programmer, and be hired as such, without ever having served as a journeyman under one or several master craftsman, without being member of a guild, without any third party authenticated proof of his experience.

It is the typical wild west situation on a new frontier, and it is my conviction that only a proper regulation akin to the one associated with traditional craftsmanship professions can bring an unprecedented improvement in quality on all levels of software development.

What a load of guff! No wonder us sales people despair. No wonder we, and the upper management, want to outsource you useless lot.

Don't you get it? We don't care about whether the product is 'engineered' or not. Just deliver the product when we say it has to be delivered.

And what idiot is this Marco chap? Control? Sales and Marketing people control you nerds! Without us, you bunch of moronic navel gazers wouldn't have a job in the first place.

I only saw this web page due to going in the monkey-compound and seeing this web page up on the projector screen. While talking to the team-lead (just what are they good for?) I got reading it while not listening to him trying to explain why his 'estimate' for the delivery was a lie. Coding horror? I get that!

Ah! Marco is in the business of selling books, not such an idiot after all.

Get back to work losers!

Someone who works for a newspaper once asked we what I did. "Software developer" was my answer. To which she replied: "Oh, you must be creative then. It's almost like art, right?"

I still not sure if it purely "art". Maybe art where you're only allowed to use certain colors.

Don't worry! Everything has a end.

@codinghorror

I wonder if you've read this post "Meta is Murder". I think it explains why these discussions are dangerous.

"And consider: how is progress made in the world? By sitting around and debating the process of how things are done ad nauseam? Or by, y'know ... doing stuff?"

You dais "And yet, it's also a release. It's as if a crushing weight has been lifted from my chest...."
Don't you mean a RELIEF?

The article is talking about "software engineering" as a means to control projects.

From further along in the article: "I still believe it makes excellent sense to engineer software. But that isn’t exactly what software engineering has come to mean."

Call yourself a crafts-person if it suits your vanity, but don't confuse the engineering of software (what you are doing) with the project management process regrettably come to be known as "software engineering."

We're not standing on the shoulders of giants we're kneeling on the feet of midgets.

Software is almost an organic process, how many times have we seen huge monolithic software which over the years has become completely unmanageable, yet companies still persist in the "just add another feature" policy! :D and who'se responsible for that.... I point you in the direction of the Marketing department!....

( and why do some software "craftsmen" treat software like the computing equivilent of scrap-heap challenge! )...

Quoting David W:

'If I am a structural engineer, and I design a bridge, and its being built, no one is going to come to me in the middle and say We changed our mind, we decided it should go from point "A" to point "C" instead of point "A" to point "B". Or, we really need a bike path on it. Or, it appears that our traffic projections are wrong, we need it to be six lanes and not four lanes.'

You think this doesn't happen? Of course it does! And then the customer complains that YOU didn't think of adding a bike path, or going from 'A' to 'C'. A good civil engineering craftsman has learned how to deal with it. And so has a good software engineering craftsman.

So we are not Engineers.
Architects? Come on..
Being a Developer is quite lame too.

Programmer [x]

Federico is completely correct that programming is a design activity, not a building one. We have totally outsourced the manufacturing of our programs to our toolchain: compiler, linker, loader. We put in a detailed blueprint of what we want them to build and our tools assemble a flat-pack of components and a set of instructions that, when they get to our customers' systems, are assembled by the loader and the OS into one or more running processes. We have performed this automation so successfully that we do not even recognize it as the manufacturing activity. Instead we look at the last level of human involvement and decide it must be manufacturing, then wonder why it is not predictable.

Once we have successfully solved a problem once, it tends to go into a library of well-tested code, usually with parameters to control its behaviour. Would that a civil engineer had a self-assembing suspension bridge that could place its own footings! If we have solved a problem once, we often don't need to re-solve it again unless our original solution doesn't quite meet the criteria that we had before. This is where software engineering tells us to make the minimal changes to fit the new problem while remaining compatible with the old one; craftsmanship might instead tell us to design a new component that's incompatible.

In tools like the .NET Framework and our operating systems, we have vast libraries of solutions to our problems, but as people who want to believe that we are making a real contribution, we will tend to decide that such components somehow aren't quite right for our problem and waste time producing something custom. Software engineering is the discipline to resist this temptation.

If a software engineer is sufficiently experienced to know the wide range of tools at his disposal and to understand the problem sufficiently well to know exactly what to build, it is possible to predict and control the activity of designing it. If it's that well known, though, it has probably already been designed and built.

Engineers we can be, craftsmen (or craftswomen) we might but we are definitely not artists.

Anyone who says otherwise is a retard.

Let's not over emphasis our creative abilities to the extent that we believe the programs we create are works of art. Clever and sophisticated they may be but more than that they ain't Mr Knuckle head.

No one is going to hang a print out of your code from the Sistine Chapel, a polymorphic composition of 2 overloaded methods that converge in the middle touching at the angled brackets. Art transcends nature revealing hidden mysteries, software programs do not.

Those who work on projects with other programmers are more like engineers following standards and applying acceptable practices that others can emulate and understand. Those tinkering with code on the side or with full autonomy are free to ply their trade as a craft and enjoy the thrill of the chase.

But artists we are not. Did you get that yet Mr Knuckle head.

Easily one of the more dreadful articles about navel gazing.. sorry I mean software engineering I've ever read

while coding
iterative : yeah. (or recursive it depends)
artistic :someday (algorithms and maths are arts, rights?.. I believe so).
+ Art transcends nature revealing hidden mysteries ; s/Art/Computer Science/g

so maybe I am retarded...

but I agree on some points to. maybe there is art out there, but first we think, project, milestones, specifications and productivity..

I'd also say that programming is not art because software can be demonstrably wrong. Art can be good or bad, but cannot be wrong - a novel doesn't stop working just because of a misplaced semicolon.

I think many commenters are missing the fact that software development is an incredibly broad discipline. I would never dream of calling the process I follow as 'engineering,' but I am not writing sorting libraries or mathematical simulations - indeed if I were to require that functionality, I'd look for a library of pre-written pre-tested code instead.

What others are confused about is that they don't realise that other types of engineering are not perfect. Developing a new component for an aeroplane requires a significant amount of iteration, both in the design and development stages. Classical engineering is fortunate to have decades more 'patterns and practices' to build upon, but many of the fundamental issues are shared with software development. I also recognise that of the hundreds or thousands of engineers involved in building a large-scale project, such as a new bridge, they will have many different roles (at many levels of abstraction) - just like software devs specialise in a particular area.

What is also apparent is that there is a massive difference between the level of abstraction in a project and how it is perceived. Sure, working on a driver for a realtime embedded system may be regarded as engineering, but me contributing to my team's web-based application is not the same (and I wouldn't regard either piece of work as less worthy than the other). There seems to be a tendency for some people to think in black and white terms here, which is a shame. I feel a little aggrieved when some people above regard us as arrogant or lazy when we say we're not strictly engineers - the fact is that there is definitely a creative aspect to many parts of programming, and just because we're not writing fourier transforms doesn't mean that we're not worthy of being in the industry.

As for the original article, Jeff's guilty once again of over-egging the pudding. And what happened to a bit of research? This argument has been raging for years and there's much prior art.

I don't like his "one-week" horizon - it means you can never do anything that might take more than a week to complete!

A major issue is that web development has been around for "only" 10 years or so. And yet there are millions of developers.

That is a formula for a lot of issues.

In the document, Tom DeMarco defines software engineering:

I still believe it makes excellent sense to engineer software. But that isn’t exactly what software engineering has come to mean. The term encompasses a specific set of disciplines ... All these strive for consistency of practice and predictability. Consistency and predictability are still desirable, but they haven’t ever been the most important things.

I'm not sure when "engineer" got co-opted by MBA types to make their organizations sound smarter.

To be an "engineer", of any kind- in college you took; chemistry with chemists, physics with physicists, differential equations, linear algebra and buckets of calculus with mathematicians. You took your actual engineering classes towards the end, and you did most of it by hand so you can effectively eyeball an answer, and can tell when your model is lying to you.

Being able to think logically and solve problems is something some second graders can do, it does not qualify you to be an "engineer."

Great. On one side of the camp, we have the circle-jerk of self-centered geeks aggressively misquoting and editorializing authority figures in order to "prove" that they can't be actively managed, and on the other side we have mindless drones throwing out the same tired buzz phrases like "agile" and "best practices".

Can we please have some sanity?

Tom states, in no uncertain terms, "I still believe it makes excellent sense to engineer software." When he disparages software engineering (and he never says "dead", very irresponsible of Jeff to make that attribution), he is CLEARLY talking about the actual institution of software engineering that is actually project management and obsesses over metrics and controls. He is NOT saying that you shouldn't apply engineering concepts to software, and he is DEFINITELY not saying that development is a "craft".

As usual, the waters are being muddied by people with no engineering education or experience whatsoever, people who are not even remotely qualified to comment on matters engineering-related. When an actual expert writes something that looks, superficially, like the message these egotists want to send, they take it completely out of context and say that "a crushing weight has been lifted."

I'm in software by trade, but was educated as an engineer, and I can tell you that they are EXACTLY the same. All engineers have to deal with vague requirements, scope changes, scope creep, faulty components (materials), clueless management, project failures, process failures, experimentation in general, and vast differences in competency. That's engineering. That's what engineers do.

The real message from Tom is that these suits who co-opted the term "engineering" to create a project management discipline have their heads up their asses. The ISTQB did the same with "testing". This abuse is all over the place. The only worse travesty in my opinion is when somebody reports it as the norm and uses it as a straw-man argument against the actual, legitimate ideas.

Engineering in other fields is tied to the scientific limits and equations that are developed for the materials being used. There are certain Physics involved that are STANDARD across the board. There are only a few ways you can do things successful in those engineering fields.

With Intellectual property rights and abundant creativity, there is no STANDARD set of libraries that everyone uses, there is no STANDARD OS that everyone uses, software engineering is a misnomer. It is still the same craft of ideas you see on Hack-a-day or any other site that will show you new innovative ways to re-use or remake something from things you never thought about.

Software development is still the same CRAFT as when I was 6 years old with a TI-994a banging out BASIC code to cassette tapes. You have an idea and passion, you make it happen. Just like society in general, once you start adding rules and government it takes the FUN out of it and strips the people of their passion unless they are WISE enough to know how those rules will benefit them in the long run.

Essentially the commercialization of knowledge has yes expanded it to an unreal extent, but has debased it as well to the lowest common denominator.

Recursive division has that effect.

Most architectural engineers never build the same house twice. That doesn't mean what they do isn't engineering; the underlying physical rules remain the same, regardless of what house they build. The fact that we haven't yet had thousands of years of knowledge to figure out what exactly these rules are doesn't mean what we do has to be a craft.

Saying "well, it's not engineering, it's a craft" is the cheap way out. Sure it's a "release", like saying "I don't have to worry about my life anymore, I'll just believe that god will guide me" is a "release."

Agreeing with others that say this debate is pointless. As an electrical engineer by training, we just have, oh, about a hundred years of experience over this brand new software development thing. So we have very specific design specs that need to be met, very specific testing requirements based on the laws of physics (is this lumped capacitance going to affect things? is this electric field too high in practice and going to interfere with this antenna? is this control system sensitive enough?) I think that we are much more able to automated quantitative tests than software guys, to be able to give very accurate assessments of the quality of the device in question.

Does that demean or make software development any less intellectually challenging? Hell no. Does that make it "engineering"? Who cares, semantics.

Also you'd be absolutely fooling yourself if you think that other engineers don't have fickle clients that change specs at the last minute. Imagine how much worse it'd be when designing an engine or an integrated circuit.

That's the business, engineers have been dealing with that for decades, it's nothing new. You don't see too many of them inanely blogging about every little design mod they have to make...

I'm justing adding to the long line of "hear, hear"s on this post.

As somebody who took Software Engineering in university, I realized this a long time ago. Firstly, that nobody wanted to pay the money, and take the time to do a properly "engineered" piece of software, save for NASA and a couple other organizations. And that even if you did all the engineering process stuff, it doesn't help when you have a bunch of meatheads doing the coding. People make the mistake all to often of equating coding with assembly level or construction work. They think, if you put enough controls in place, that somebody with minimal amount of training can create good software. This doesn't happen. What you have to realize instead, is that each person doing the coding, is doing their own design, their own "engineering". You have to have really smart people doing the coding. I think this is pretty much the road that Joel has taken with Fog Creek. Hire really smart people, and you don't really even need so much process, because really smart people who get things done will figure it out. Same reason that Jeff, Jared and Geoff were able to create Stack Overflow in a few months, and up with a good product despite not having that much unit testing, and not really even having a project plan. Saying that programming is a craft is very true. Throw 100 woodworkers together and get them to design a nice hutch and you will never get a good product. But give a good craftsman 1 month, and you will probably have a really nice end product.

I gotta say I disagree that Software Engineering is dead. The classic processes are dead. But, I've found a difference between a programmer, doing development, and engineering a product.

Clean consistent APIs that make sense and use good patterns are engineering.

Do people actually agree on what the definition of "software engineering" is?

Just because one programs, one is not an artist.

Programming many times is a "McGyver" experience: you have some tools, a goal, a time frame -- see what you can toss together.

Before making grand pronouncements you guys should read a bit about other engineering disciplines. I recommend you start with Henry Petroski's "To Engineer is Human".

What we call "hacking" in software, happens in all other engineering disciplines. Sometimes because the science is wrong, sometimes because the requirements are missing, sometimes because there is no science (eg. flood control and civil engineering).

Engineering is the craft and the art of designing useful artifacts that do not fall down....

I agree with Aaron G. DeMarco seems not to be against engineering but the larger issue of how software projects are managed.

In an interesting article from Computerworld a year or two back, there was a write-up on think tanks like DARPA. One of the principals they interviewed who'd been there when they did so many great things in the 60's said that today DARPA has changed and would be less likely to achieve what it did then. Some of his opinion might be egotistical self-promotion for the past achievements of his team, but I found it interesting that he called out the management, not the newer generation of workers. He specifically referred to the leadership there being not engineers, but project management people who have a focus on the short-term delivery; he felt this would not allow the environment to encourage the same developments the teams in the past delivered. A lot of great things happened when engineers were allowed to spend a lot of time just "thinking" and he's got a point; most project management types would not be happy to see a room full of people not delivering the latest status report.

I think this is what DeMarco is referring to when comparing the two projects in his article: one is heavily project managed and only delivers the small, containable, measurable, justifiable thing, while the other was less project managed and happens when people think big and projects aren't just canceled because they didn't deliver anything in the first 30 days.

That said, I also got a laugh out of Rod's responses and thought he had some good questions in the thread above.

Great writeup :)
I've been advocating for years that SW development is *art* and not "engineering", but I think I'll from now on actually start adopting "craftsmanship" instead in fact. I think that word more accurately sums it up in fact ... :)

Saying that software engineering is dead for the reasons stated here, is akin to saying that literature is dead because everyone and their dog has a blog that is, largely, a piece of chit.

Believe it or not... but there's still people out there who know what they're doing. (No, I'm not necessarily one of them)

I think you hit on the key, Jeff...

"The guys and gals who show up every day eager to hone their craft, who are passionate about building stuff that matters to them, and perhaps in some small way, to the rest of the world -- those are the people and projects that will ultimately succeed."

If you have a solid team in place with a culture of support for team goals and teammates, the debate over art versus engineering becomes moot.

Mostly I agree with @Federico that software development is a design, not *manufacturing* activity.

When I hear people say "why can't programming be like building a bridge" they aren't talking about whether building software is like civil engineering. They mean that programmers should be able to use business specification as a blueprint and turn that into working software a manufacturing process. I guess in civil engineering this would be like they mayor saying that he wants a new bridge and then going out and hiring a construction company to build it, and then expect it to come in under budget, without first having an engineer do all the technical drawings. Then again I wouldn't put that past the current mayor of my city.

I have always believed that the closest craft to programming is poetry(not the free verse crap). there are definite rules to follow but the writer must figure out a way to piece the language together so that it both meets the rules for the type of poem, makes logical sense and is pleasing to to the reader.
In the last few years we have been seeing right brained designers and left brained developers and a manager trying to get the two types to work together. Good programmers have the ability to use both the left and right hemispheres of their brain.

Maybe it is, and certainly now that companies try to conserve money, it's basically only used as guidelines because the actual programmer has to link the ends. And sometimes it's used for software documentation but that's it basically

There are more moving parts in the average SQL Query than in the Big Dig. Computer programming relies heavily on automation to assemble these moving parts into a working whole, usually with 1% of the time and .5% of the budget of a comparable (in complexity) mechanical engineering project.

This doesn't leave a lot of room for getting things perfectly right, if that's even possible. The body of running code out there could be humanity's greatest achievement, an 8th wonder of the world. Don't let it go to your heads, though :)

The first version that does what you want is done through craftmanship. The second version built to fix all the maintenance problems is done thorugh software engineering.

I'm pretty sure that for the first few hundred (thousand?) years, building bridges, large buildings, cities, etc... was all just as much an art as programming is now.

Engineering has come to mean formulas and reliability because it's been done so much that it's predictable--just about every variation has either been tried or can be understood based on mountains of existing data.

Programming won't always be like it is now. I'll leave it as an individual exercise to imagine what it will be like, that's outside the range of my crystal ball.

Giving up on being able to reliably build things is a huge step backwards. Sure, it's popular and all the rage, but we're never going to exploit the potential of our hardware unless we admit that consistency requires discipline and monitoring, which is pretty much the opposite of making it a free-for-all. We know, and have known for a long time, that a massive chunk of unstructured code is nearly worthless. Even if it runs, it's impossible to extend; it's at the end of it's life. The only way to add structure is to actually build it in initially or add it later. Either way, we have to know what good structure is first, which requires a design. Accidential structure is unlikely.

Paul.

"Software engineering is the application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software, and the study of these approaches; that is, the application of engineering to software."

http://en.wikipedia.org/wiki/Software_engineering

Licensing Software Engineers?

http://www.computer.org/portal/cms_docs_software/software/homepage/2008/s6car.pdf

One of the differences between software engineering and other forms of engineering is that software engineering rules get codified into tools. For example, Apple's development tools force you into using a Model-View-Control design. Ruby on Rails does the same, as well as enforcing DRY (Don't Repeat Yourself), and encouraging you to use a database rather than arbitrary files for back-end storage. Java developers using Eclipse get bad code style flagged as you type.

In contrast, a mechanical engineer needs to know the rules up front, and then test each design against those rules.

And people wonder why Microsoft said it will launch Windows 7 when "it is done", and their primary goal this time is to do it right...

Old news.
For quite a while now, programming has been more about the creative side of creating software than the engineering methods of applying form A to problem B to get solution C. (not that this is necessarily bad, and many of those methods exist, but in a much more flexible state than most engineers would be comfortable with)

It seems that most people here don't even understand what Software Engineering is. Forgive me if I read the first few comments and skipped to the end, but SE is not about "putting your head down to code". It's actually as far from that as possible.

Software Engineering is not a descriptor of how passionate you are about your work; to claim that "craftsmanship" and "engineer" are a contrasting duality is naive. Engineering is about how much responsibility you take, planning you perform, risk you assess, your methodology of requirements gathering, and more. They are part of engineering because they are truly quantifiable/measurable properties. In fact, the only truly "crafty", "creative" part of software development is when you sit down to write code-- but any good developer will tell you the code is only the "20%" of the process.

If you want to call yourself a craftsman, fine; but what you're essentially doing is calling yourself a glorified codemonkey. Someone who might be great... at only 20% of the software development process.

@Jon - Well put.

Unless you have concrete evidence that most of the people buy from farmers and not from supermarkets, I would still believe that software engineering is almost extinct, giving way to mass-manufacturing process.

The economies of scale that drive real-world mass production simply don't exist for software. Mass production historically required orders of hundreds of thousands of units (muskets during the civil war) to get started. Even with hardware engineering techniques refined from hundreds of years of experience, and it being the default way engineers are taught, businesses producing below a few hundred units annually are generally based on skilled craftsmen:

http://www.morgan-motor.co.uk/

There is no software company in the world that produces for sale ten thousand units of software a year. A few organisations (USAF, NASA, NHS) approach that scale for internal use, and might just hit the break-even point for mass-manufacturing techniques.

To summarize a bit of the better-thought-out above comment mess...

1)Engineering is about using a messy situation and producing a working design, to tolerance. This involves both creativity and correct practice.

2)Software is more abstract and has almost no correct practice(only better/worse). This is probably caused by the newness of the discipline.

3)Engineering is partly about the process as well as the work itself.

4)DeMarco was flaming the project bean-counter mindset, not the idea of rigorously constructing a project.

That's much less flame-baity than Jeff....

Computer science is mathematics.
Programming is the act of writing programs.

Computer science != programming. It's maths.
Software development != programming != software engineering.

Software engineering is the engineering field of creating software. For example, a software engineer creates control systems for nuclear reactors. A software developer is just someone who develops arbitrary software, like some payroll app. Both program.

I have to agree with Dave Markle and others. Maybe not everything all software developers, including some who call themselves software engineers, is engineering. But what I do certainly is, and I've seen others do it in many situations.

And I am actually a "real" engineer originally. Not a software engineer, not a computer engineer or an electrical engineer. I dealt with physics and materials and stresses and so forth. And then I developed software, and of course it's different, but it absolutely is still engineering that I'm doing.

Sadly, much of the audience here assumes that large projects contain only highly skilled and motivated software developers. Eager to produce, despite the efforts of idiot managers to curtail them. Sadly, that just isn't the case. The majority are ordinary, unmotivated and uninterested in really improving their skills. The audience reading this is simply unrepresentative of the real world

If developing software is craftsmanship, then the boat is sinking. It seems like this is an excuse for an industry that has been slipping for years. I have been in technology for almost 20 years. I remember working next to Digital developers and being in awe at the level of ability, intelligence, and care. Today I see software as being slapped together. The tools that do so much for developers seem to limit their ability to solve problems.

Maybe this is because I work in Health care, the software that is created for Health care never mentions up to some of the stuff I saw come out in my early days. Lack of error checking, dependence on pre-compiled wasteful code, and a total waste of resources.

How about we cut through the crud and give ourselves new names that have no historic categorisation and therefore we can define what these new titles by our actions and profession.

Software Developer: Ni
Software Engineer: Peng
Software Architect: Nerwong
Database Administrator: Ekiekiekiekifertangzerbongmfrm

or something like that...

I believe their should be a distinction between what software development should be and what software development is in reality.

I still believe that software development should be a combination of craftsmanship, art and engineering.

In reality it is unfortunately often none of the above.

Don't panic. Software engineering and software project management are two different things. DeMarco complains about the second one.

My father is a retired structural engineer. He worked for a few large companies and lectured, but spent his last few working years at a small consultancy.

When I was a kid we built meccano together, and I can remember him working out how to get something I was trying to do to work, and saying that engineers are the people who work out how to make people's ideas real.

Most of the work he did as a consultant was talking to the clients ( "requirements capture" ) and making sure that the builders built the right thing ( "validation" ). I can't think of a single use of metrics in his engineering practice, which is what the 'no, no, and no' of the article relates to.

My career went through electronics to software ( particularly modelling of hybrid systems ). I've worked a bit in safety critical circles; the commenters who say that software doesn't have design codes simply like physical engineering doesn't work on anything that will kill people - off hand I can think of def.stan., JAA, AEC and MISRA standards for critical systems.

Fundementally, the difference between engineering and craft is that engineering artifacts are products of societies and crafts are products of individuals. You have teams of hundreds of engineers on typical systems engineering projects, some of who are software engineers. Even for small projects, you need to apply the engineering principles of breaking down a system into subsystems by purpose, function, structure and behaviour in order to reduce it to a tractable problem.

Sorry, but translating an article saying metrics only matters on unimportant projects to rejecting all engineering is bollocks. Much of what you've done on SO is engineering - making an idea real and habitable.

I think it all depends on the maturity of the craft itself. Sure, if you do someting for the first time, it's craftsmanship as you haven't done it before therefore you need to be inventive, etc.

Just as in the case of the first cars - see, there was this amazing variety of solutions for the same problem. Cars had any number of wheels from three to like ten, more types of engines you can imagine, etc. After a while, patterns started to emerge (the McPherson suspension, the Diesel engine, whatnot). Patterns are easier to use and more beneficial to refine than starting from scratch.
Fast forward to today: all the cars look the same. (still it takes 4-6 years for a new model to get out of the gates)

I see similar things in software: you have databases so you don't need to write one. You have compilers, toolchains, standard libs, whatever. You don't need to do everything from the ground up. Sure, there is still plenty of work to do, but a good percentage of code in probably any given software is not written by your software team (or yourself).

Sure, software is inherently easier to develop and distribute, so there is a greater diversity. However, there ARE some standards (*ML, SQL, just to name a few) and best practices you can follow. Testing is not as thorough (this is why all sw comes with a long EULA - you can't get away with a serious problem in case of a car this easily...)

However, engineers do break out of the rules and formulas sometimes - one calls it innovation. The rest is engineering.

What I seem to see in a lot of code of clients is if it is not embeded machine code, safety critical or on a huge project like government etc then the code is bloated, full of quick hacks and generally a nightmare to read never mind work with.

So I think in smaller projects there is less care for the engineering involved.

Programming is intrinsically meta, because programs are a form of data. This means whenever you have some non-creative programming process, you can write a bit of software to automate it. Thus you tend to be left with just the creative parts. That's why it feels especially "artistic".

Normal engineering has the creative parts, but they also tend to have more non-creative parts. You can automate a factory with robots, but if you are building a bridge out in the field, you pretty much need a bunch of men with welding kits. That's hard to automate. From a distance that kind of labour can mask the creative side.

I think this turned out to a pointless discussion of what it means being an engineer or a craftsman.

If this discussion doesn't bring any improvement to the field, so we're all losing time reading and answering all of this.

To me, building a software is constructing something that does what it meant to do. But I can build a wooden brigde, a stone one, or even a steel, or I dont know. The point is, it's going to work ? Yep ? So let my creativity run free. (as long as the project doensn't suffer.)

If you are reading this article, you are part of the minority that care genuinely about their craft. But that isn't the majority of programmers.

On a real project, you won't get all burning, committed craftsmen. You won't even get that as a majority. And it is easy to poison a group.

You won't get an open ticket to innovate/create... at least not forever. You won't get turned on to, "go build the Taj Mahal"... You'll get turned on to build a new community center in Podunk.

So how are you going to energize your programming Narcissists when it's just a community center?

You

Dave Markle: "Big-O notation? Who cares, we're artists! Rules engines? I will craft the most beautifully indented set of nested IFs! Knowing how a B-tree works so I can understand WTF my database indexes actually do an then intelligently apply them? Sounds like nonsense to me. Why not just add some more indexes until it gets fast!?"

Craftsmanship doesn't just mean artistry, and even if it did, artistry doesn't mean "arranging things in a pleasant way". Being able to work out what to implement, which parts should be involved and how these parts should function internally and externally is exactly what programming as software craftsmanship is. It's about using your knowledge of programming and of the situation at hand to create a good solution. There are tons of solutions for most problems, and those that work better than others are both finer crafted and better engineered; as far as I'm concerned, it's the same thing.

The reason it sounds like nonsense to you is because no one actually said that. It was your straw man. Don't make the mistake of tying craftsmanship or artistry to "beauty" or "usability" or "fanciness", or engineering to "performance characteristics". They are tools to be applied across all facets of the design, not handles by which you wage ridicule at poor solutions.

If you think of the "craftsman" thing in terms of a specific craft - clockmaking - it becomes even more interesting.

Imagine the difference between designing a lot of "clock parts" that are engineered to be beautiful, precise and interchangeable, and buying these parts to build a unique clock with a beautiful hand-carved body, hand-calligraphed face, etc.

Most of us are the clock-making artisans. Many of us still need the true engineers to make the precise, interchangeable parts. We understand what an escapement is and how it works, but we couldn't build a respectable one to save our own lives. We can use it well, though.

Sad to see an obviously highly regarded person come up with this revised view. I think he has left the real world.

The real world (where I am a project manager that needs IT now and then) has only so many chances to build Google Earths or Wikipedias - both of which have started as free/not-for-profit apps.

It also consists of many businesses that try to make money by improving/evolving through "relatively useless [marginal] projects", as in mature businesses there are not many "tranformation" break-throughs.

The real world also changes not that often.
The whole world is talking about continuous improvement and kaizen, but DeMarco thinks it's useless?

Agile is a great approach, but the inertia and ignorance in the IT world is also great - try and get a system built where customer focus and involvement are paramount and the system works incrementally from early on. Many IT companies still love the Big Bang.

DeMarco is coming up with a similarity of the lamest excuse in the book: you can't standardise MY work, I am an artist.

See what crappy, ever-changing interfaces these artists build! Try and train users to work with several different behaving apps. You artists are consistently reducing efficiency.

I think DeMarco's lost his marbles. Maybe we need to write a wonderful product called MarbleSearch!

Smells like beardy sandal spirit.

Thanks Jeff for your great site and work over the years!

OK, SE is supposedly dead. But the notion that we should just throw up our hands and despair of ever managing programmers, while perhaps attractive to certain workers, is ridiculous. Craftsmen are easily managed.

If coders are craftsmen, then we have ample precedent for how to "make" them. It's the same as any other building trade. Send them to school for a few months to learn a bit of theory, then apprentice them to a master who teaches them how to do things in the real world. Eventually they become a journeyman capable of working on routine stuff without immediate supervision; after many years, they become masters themselves capable of supervising the work of others. This system has been around for centuries and it works quite well at producing skilled craftsmen.

Lately, even in the building trades the process has become short-circuited. In the interest of cost-cutting, contractors have started using labor that, to put it charitably, is less than properly trained. Sadly, this works because most Americans care more about getting the lowest possible bid than getting quality work. I see the effects all the time in the residential and landscaping fields. I shudder to think what civil structures might be built with such labor. *Most* software is built with similarly poorly-trained programmers. No wonder most software is so flaky.

Programmers are fundamentally no different than carpenters or other skilled tradesman. The fundamental problem with "software engineering" is that we try to have the carpenters also be the architects and the civil engineers and the plumbers and everything else. This may work for simple projects, just as a skilled carpenter can probably adequately design and build a typical house that involves no unusual features. But putting an ironworker in charge of architecting a skyscraper would be madness. Unfortunately, this is exactly what happens in most software projects.

The central problem with software is not that it involves craftsmanship. Nor does this fact disqualify it from being an engineering discipline Nearly every engineered system involves craftsmanship at some point in its construction. Tin must be bent. The problem is our routine conflation of craft roles with engineering roles.

Finally someone said out loud. I've been always wondered why the cs department belongs to the engineering faculties if this shares few things in common with another engineering, and so it does with sciences. Well, thanks jeff

It's fine to just wing it sometimes when building apps, especially when it's a new idea and you aren't even sure exactly what it will become.

But I also think software engineering principles, designs, documentation, requirements, use case libraries, etc. are very important when they are used to build the foundations that all of us rely on (the .NET framework, or an OS for example).

I always considered a large part of SE way out of my picture for reasons as:
1- I feel that most, if not all, concepts after The mythical man month is academic. It is sometimes useful, but not even closely as they want me to believe.
2- The project schedules and budgets I have learnt to deal with does not support even 1/10 of the effort required to write the docs, test the app, review it and such as they tell me I should.
3- I know for sure my competitors don't use them at all, yet they win over us.

I know (3) for sure because a customer choosen a different studio to go for a SW he needed. After a few weeks he came back here to ask for a technical assessment.
Wonderful.
In retrospect, we should have charged him way more.

Blogging: dead

As seen before nobody had a benefit from reading blog posts. In most cases it costs the reader time which we all know is money. The only people getting money for this are bloggers and merketing weasels.

I don't want to participate in the argument over semantics here because it seems trivial and could go on ad infinitum.

However, as a side note from a global perspective, in many countries, you are legally restricted from using the word "engineer" or "architect" in your title unless you have a very specific degree or license.

I had to cringe when someone of the baby-boomer generation recently referred to me as an engineer. It just sounded so ... wrong. As a Web applications developer, I consider myself more of an artist or something more poetic. "Engineer" just sounds so stuffy.

Great and when your craftsmanship crashes because of poor design, testing, implementation and usability I will still feel safe and secure knowing it's not an engineering problem. Why is software any different from hardware engineering? That's not a craft. When you build something physical it has to do the job and not crash. When you build lines of code you can just patch it so it doesn't matter?

What the article is implying is that you can't impose engineering principals on guys wbo couldn't do engineering in the first place...

If you had a magical replicator that could make copies of any design for free how would that change your process to make a car?

That's software engineering in a nutshell.

It's not that software engineering doesn't exist it's just that we haven't even come close to figuring it out. The lack of physicality of software is both liberating and baffling.

Izkata: Right now, I'm 21, and have been programming since I was 13 or 14. I'll be finishing a BS in CS within a year, and I've been saying for almost that entire time [...]

That's because, until now, 100% of your programming has been done by yourself, in your bedroom.

"Why is software any different from hardware engineering?"

As an actual engineer (Aerospace) I'll answer your question:

Physical solutions are built upon immutable knowledge - the tensile strength of steel, the conductivity of copper. There is nothing immutable about software.

It is interesting that you can get a Software Engineering degree (e.g. http://www.bseng.uvic.ca/ ) and you can become a licensed Professional Software Engineer ( http://www.apeg.bc.ca/reg/pengappdocsrequired.html ), yet something like half the comments on this blog indicate that there is no such thing as Software Engineering. I wonder why that is?

Hi Jeff,

Read Object Thinking by David West (Microsoft Press) if you haven't already.

Seriously opened my mind on this matter.

Regards,
Matt.

> Normal engineering has the creative parts, but they also tend to have more non-creative parts. You can automate a factory with robots, but if you are building a bridge out in the field, you pretty much need a bunch of men with welding kits. That's hard to automate. From a distance that kind of labour can mask the creative side.

In the UK, welders, bricklayers and other skilled technicians are called craftsmen. It's a different role to an engineer, and does not denote lack of science. There's a different argument between 'you need some craftsmen to build it' - a hell of a lot of software projects are made by welding existing components together - and 'software engineering is dead'.

The word engineer comes from the Latin 'ingenium' meaning 'Cleverness'. As is the word 'ingenious' (marked by originality, resourcefulness, and cleverness in conception or execution - Merriam Webster)

I would argue that software engineer is the most appropriate title for what we do, and it is other types of engineer that are mis-labelled

I don't agree - programming in the higher level very often is engineering, in the sense that one applies science to solve problems.

If the issue is that programming projects often are messy, I don't at all see how that makes it differ from other engineering areas.

coole Site many thanks

I agree with the idea that software design is experimental. but this doesn't imply that you can control it. What it is needed is a different control approach, more similar to the one used in scientific research: propose an hypothesis, define an experiment, test and measure, evaluate results and go back to hypothesis.

Software engineering is not dead, it's just waiting for the next step.
When the software engineers itself.

Don't think I'm way out there in the whacked out AI world.
We are approaching sufficient horsepower where self modifying and adaptive code is goiing to become a reality. And not just the little 'toy' examples we've come up with so far.

I just hope it does not knock on my door and ask me if I am Sahra O'Conner.

The biggest difference between typical "engineering" and "software engineering" is that all the "engineering" results in automation.

Standard Template Library - Done - now I don't have to worry about how to garbage collect lists of arbitrary objects.

Visual Basic - Done - now I don't have worry about graphically coding buttons and developing sophisticated call-back mechanisms.

OpenGL - Done - Now I don't have to worry about projecting 3d space to 2d screens.

SOAP - Done - Now I don't have to worry about how to develope a platform - independent API.

Developing the tools is engineering. Using the tools is programming.

-CF

Damn straight. I'm an artist. My medium is code. Have been for 32 years.

Sadly I have to agree with "Sales and Marketing". The big problem with programmers is that we think businesspeople actually CARE about "engineering" or "design principles" or all of that good stuff that we repeatedly say is the "right" way to develop software, when in fact ALL that matters is to get working software that meets the business' needs out and into the hands of the people who need it.

This is why a hack who writes spaghetti code is infinitely better than the "software engineer" who takes his good sweet time making sure everything is properly decoupled, and implements the right design pattern; the hack's code is ugly and inefficient, but it works and it's completed fast.

The sad truth is that software exists as a supplement to existing business functions, and all this talk about "craftsmanship" ignores the real goal.

so what then is craftmanship vs. engineering. Can you explain? In some ways engineers are craftsman and vice versa.

First off - I've called myself a "master software craftsman" for nearly 20 years now - for much the same reasons alluded to by various posters (and TFA).

Secondly - it never has been and likely never will be engineering in the legal sense of the word - using known, repeatable processes to transform known raw materials (e.g., data) into a result provably knowable in advance. That's pretty close to the Texas definition, if I recall correctly - which is why it's /illegal/ for "software engineers" to be called such in TX and several other enlightened jurisdictions.

We're not engineers. We're barely craftfolk; we've started work on various iterations of the Cathedral at Reims any number of times - and now people think we can bring the Sydney Opera House on spec, on budget, in half the time reasonable.

We're not engineers, and we likely will be, barring a transformational tragedy roughly analogous to the New London school explosion of 1937 for which software, preferably COTS software, is left visibly holding the bag. We won't be, barring such an event, because the culture and folklore that have grown up around software development reinforce the traits that actively militate against engineering - namely the lone heroic coder, 'code complete', "don't worry, be crappy" and other such garbage. And it IS going to take an event where a sizable number of people wind up dead; we've shown great impassiveness as an industry in the face of large and expensive (but non-fatal to humans) software failures. If I had a penny for every million dollars spent on projects that ultimately were delivered but failed catastrophically, I could retire and put six kids through Harvard Law.

Back about 20-25 years ago, I used to think we were 20-30 years away from a 'true' software-engineering discipline. Now, I'd say it's going to take at least twice that long from now - even if a New London-class failure of software were to happen tomorrow. Some people call it the Microsoft Effect. I think it's just a byproduct of us as a non-profession, not having any way to overrule technical decisions made by egregiously unqualified people and making them stick.

Prove me wrong. Not with statistically-improbable events 15 or 20 sigma out from the mean - but AT the mean.

This article is great. After 15 years involved in the Enterprise Software Development field I am utterly convinced of the fact that Software Engineering has in its deepest internals noise wrapped by fancy concepts. Also it's true that many corporations are doing money selling courses, methodologies, certifications, software development based upon "state of the art software engineering techniques", but I bet for motivated people with talent AND commitment instead. I'd like to meet some day a project manager with these abilities together.

Keanu said whoa (on tape even) way more than once...

I don't think true arty and crafty people do anything for the sole purpose of function. I think we are far closer to engineers than artists. We have a science and a toolset and a lot of knowhow, and rules/standards to follow, even if they aren't bound by law. We design and build things to order that have - first and formost -function. Making the function look good is secondary, as it is in engineering. Making it not fall down is our main priority.

Maybe 'architects' is a nice middleground as it tends to combine function with form.

I agree, people who only program do not typically understand the engineering that goes on behind a project and are not software engineers. However, anyone who actually have to run a project, estimate a project, convince a prospective customer about the true cost and value of a project, bring the project in on time and allow it to exist in a cost effective way for a further 10 years. These people are engineers and not craftsman. It just requires hard slog and dedication and applying various sensible principles of leveraging the right people at the right time in the project. Yes it might sap some of the enjoyment out of the project for people being forced to do things they feel is ‘beneath them’ but the results are usually satisfying, if a little boring.

I say “craft” is still too good. I might concede “software craftsmanship” for something like building a new system with carte blanche to use the best techniques via the best tools, but that’s exceptionally rare in the corporate world (i.e. this is software craftsmanship). What we are paid to do is generally to fix other people’s programs, to glue one program to another, to massage a program into working, to decipher the mysterious behavior of badly-written software. I think better names would be software plumbing, or software carpentry.

No HTML? Where I said «this is software craftsmanship», I meant this: http://schematics.sourceforge.net/ .

@Glenn: craftsmen certainly know about function. Craftsman make suits, clothing, tableware, ceramics, knives, furniture. Software is really close to craftsmanship, except most craftsmen care more about the quality of tools and results than most software-oriented companies.

Architects are even further away from programmers. When was the last time you saw an architect being rated by number of blueprints produced in a month, or by the number of housing problems solved? Yet programmers are routinely rated by lines of code or bug count. Dijkstra or Knuth would perhaps rate as software architects. The industry has no jobs for Dijkstras or Knuths.

@Robin: designers and marketing people also works under the same restrictions for cost and value estimations, project maintenance, and results. Nonetheless, no one calls them «usability engineers» or «sales engineers». Go ask a real engineer (say, a civil or chemical engineer) what’s his job like, then you’ll be able to spot the huge gap between it and so-called «software engineering».

Hi Leonardo,

You misunderstand me. The software engineers are not working within the constraints of cost and value. They are the people who will say what the cost and value will be. The better they are at this then the less constrained they will be when they actually build the system that they just quoted for.

Anyway, I like your analogy of a computer programmer being a plumber. It is very fitting and it is also a job that anyone can do with the right training. A craftsman does things in a beautiful and perfect way and there are many finely crafted software components (which probably have a very low ROI).

A software engineer is neither a craftsman nor a tradesman. He is not the project manager, the sales person, the business analyst, the QA person, the programmer or the support person.

He is the person responsible for the technical delivery of a software project from start to finish.

He tells the project manager how much it will cost and he makes sure it meets the customer requirements at the end.

That's it, simple.

Anyone who is not prepared to take full responsibility for these two things is not engineering the software.

I hope its more dead than alive. I'm tired of doing everything but creating software at my company. :)

How about lean/agile software practices?

Regression testing
Continuous integration

is that software engineering?

Because I think everyone starts being a "craftsman" in programming ... individuals should be craftspeople, but teams should have a structure.

I lean towards craftsmanship myself, along with being an artist and architect. I explored this topic on my Software Results blog (http://softwareresults.blogspot.com/).

The whole discussion so far (apart from a few comments) goes in wrong directions thanks to the ambiguous meaning of the term "Software Engineering".

I once took a university course called "Software Engineering", expecting to learn how to design and implement software "the engineering way". My initial fascination came from the wrong impression that I will approach building software in the same way as an engineer designs a car or a plane - picking the optimal solution given a set of constraints. I quickly got vastly disappointed and what I realised was the course should well be named "Software Manufacturing". Why I came to that conclusion? Well, the last thing that mattered in the assessment of the coursework was the structure and the design of the delivered software. Instead, a trivial kind of Java application was to be developed through a mindlessly formalized development process, producing hundreds of mindless formal artifacts irrelevant to the problem domain, totally eliminating any room for ingenuity and innovation. In short, this course just tended to reduce software to something that comes out of an assembly line being assembled by robots. Unfortunately software developers are not robots.

Indeed, trivial software can be manufactured through a formalized process, but the word "engineering" comes in play only when you have challenging constraints to consider. Fantastic pieces of software, much like some great cars and hi-tech planes, are a product of human intelligence reaching its highest levels of reasoning in the battle of struggling against constraints -- be they gravity, logic, or finite computer memory and computational power. This is in its essence a creative process that cannot be blindly formalized.

Software engineering, in the real meaning of "engineering software", is the way for the future of this field.

Read this statement "Software development is and always will be somewhat experimental. The actual software construction isn't necessarily experimental, but its conception is. And this is where our focus ought to be. It's where our focus always ought to have been " and totally agree

I love watching all these non-engineers telling me what engineering is and isn't, and how "software is special" because, like, we don't know the solution before we start.

It isn't, anymore than any other real engineering starts with solution as a given. If it is, then you don't need an engineer (except, perhaps, to check that the standards have been followed)

Engineering is about the unknown and/or hard maths. Everything else is management-guff; technical documentation or skilled-technician grade work.

It's not that SE is dead - it just was never alive. It was a name for some university course - a way for academics to scoop out some pocket money from wannabes to make them feel like professionals.

Real professionals knew for a long time there is just a hard work with a lot of thinking about every step you take.

Tom DeMarco's article is interesting, however I believe it has been made intentionally questionable. It just shows the trend in the history of SLCMs: from nothing to waterfall model, then to iterative, then to early XP and Scrum, then to modern Agile, and ... What's next? Tom suggests it's commom-sense-based anarchy. I doubt it really is. More probably it will be kind of mix: Like AUP, but better defined than it is now.

The way that a software developer wants the user to get the information is “craft” but the way that the software developer has to write the code to perform that function is “engineering”.

In 2001 Richard Gabriel wrote about a Master of Fine Arts in Software (http://www.dreamsongs.com/MFASoftware.html)

Software Engineering has, in general, been the futile attempt of bureaucrats to get rid (to not need) creative, irreverent programmers. It's sad that so much money got wasted in CASE, UML, and now BPM tools, instead of in workshops, study groups, mentoring, and training.

This all sounds very trendy but it's ultimately pointless, pedantic, and unhelpful.

I much rather tile a bathtub surround when it comes to "craftsmanship."

Love this Atwood - thank you for writing exactly what I've been thinking but not knowing how to put into words. Particularly like that you called out the arguments about reliability and scale. Software is craftsmanship with some additional parameters - but the additional parameters do not imply that it is engineering. Thanks again, man.

Excellent post, this is something that's been on my mind ever since I took my first programming class. It might be the professors, but a course designed to introduce people to programming feels out of place in a university. Especially when the instructor tells you to launch VS 2008, check w/e boxes, type, and press the compile button.

If this is a craft, where is the guild?

If you go towards crafting instead of engineering, it just means that you have not achieved even the crafting level. But sure as the sun shines, after crafting comes engineering.

I think this man might have been a brilliant developer at some stage, but forgot that the world revolves around people with individual needs. The second fact that he is missing is that with all the new knowledge coming into being a new breed of developer with a much higher understanding of the problem, ie: how to test for drugs with sophisticated algorithms involving various technologies cannot be handled with a standard piece of code.

He is trying to reduce the cost of development to reasonable proportions again. If one looks at the sort of income stream that Microsoft will be collecting for W7 I do understand his concern.

nice article thanks for sharing, wish you continued success

post thank you for the beautiful share with us, respect

It was always a mistake--except possibly as striking rhetoric--to speak of "software engineering" alongside "bridge engineering".

Bridges can truly be engineered in that there is a small and fixed number of solutions and the engineer's task is to apply whichever solution is appropriate to a situation. The software equivalent is not writing a web application or browser extension but porting gcc to very slightly different platform.

If we translate the other way from software to bridges, then the equivalent request is to build a bridge with two or more endpoints, where one or more endpoint can be crafted, and halfway through construction requests will come to hang motel-like rooms on the sides of the bridge.

Such a "bridge" cannot be *engineered*, even if one could undertake the risky proposition of trying to *invent* it. The engineering work in software is handled by COTS and Freshmeat acquisitions; if a team of developers is asked to do it, then it's probably not something that could be engineered in the first place.

I'm glad we don't have software *engineering*: if we did, most of the work that draws me would be *dead*.

@Jonathan Hayward: Of course software is different in nature than concrete constructs. But that doesn't mean that we shouldn't try to make software construction sane and efficient. Both buildings and software have different kinds of projects, in some people create tools and in some end products.

At the same time in architecture the architect is a kind of an artist. There exists many fancy looking buildings. But even the fancier buildings have been engineered more or less.

There are also software projects that try to create something big but fail. Maybe they didn't realise all the dependencies beforehand or something. But they could have used some strict principles to improve the design and construction process to begin with.

Artists and crafters have more room for creative mess than engineers. But big software projects cannot be playgrounds of creative mess. Some creativity can exist eg. in the specification level and inside modules, but in general the construction should be orderly and standard. Even if every function is different, there can be a standard of naming the functions.

For example I don't want to hear like "hey Jack, are these documents up to date?" nor "there are no documents of this?", because in engineering of course the documents should be up to date without asking. In software development it is often more like "who reads the manual anyways"-attitude, but that doesn't work in bigger projects and in larger scale of things.

Abercrombie & Fitch on Sale, Hoodies, Jeans, T-Shirts, Pants, Polos abercrombie and fitch abercrombie fitch abercrombie cheap abercrombie fitch Abercrombie Men Tee abercrombie womens polos Abercrombie & Fitch Men, women, and children's clothing

Please, go purchase and read Roy Miller's, "Managing Software for Growth (Without Fear, Control, and the Manufacturing MIndset)". It is a small book that on par with Peopleware and The Mythical Man-Month. It is in the Software Engineering section, ironically. Guess you have to attack from within.

/s.

I am not a fan of software engineering and i do think that "passion always creates good projects" without heavy engineering them.
But that's just not true for all kind of software.
History can offer so many examples: the Linux kernel, is the result of passion. But a lot of critical "pieces" have been designed, analyzed and discussed before typing C on a keyboard; software for online banking and trading have a lot of engineering inside; avionics software (thank god) has a lot of engineering too.
I do not understand why a civil engineer is a must to build a bridge and a software engineer is not even considered to build a critical system.
I don't agree with the conclusion "Software Engineering is dead". That's just a polemic conclusion.
I'd rather say "it depends", as a good engineer would say :)

I think the reality today is that software is not an engineer science, at least in my own experience. I have never been a project where, after having the 80% work done, a lot of the initial requirements were changed. How you could manage this in an engineer discipline? Again, with the example of the bridge, I can't imagine the engineer of a new bridge to accept changing the design of that bridge when the work is 80% completed. This is the huge difference.

Perhaps one day "computer engineering" will be a reality but until then, what we do every day is craftmanship.

I think the reality today is that software is not an engineer science, at least in my own experience. I have never been a project where, after having the 80% work done, a lot of the initial requirements were changed. How you could manage this in an engineer discipline? Again, with the example of the bridge, I can't imagine the engineer of a new bridge to accept changing the design of that bridge when the work is 80% completed. This is the huge difference.

Perhaps one day "computer engineering" will be a reality but until then, what we do every day is craftmanship.

"How you could manage this in an engineer discipline? Again, with the example of the bridge, I can't imagine the engineer of a new bridge to accept changing the design of that bridge when the work is 80% completed. This is the huge difference."

When a bridge is in CONSTRUCTION, the Engineering Design is completed.
When a piece of software is being CODED, the Engineering Design ideally should have been completed.

Yes you can craft software without Engineering Design, just like you can craft a bridge without Engineering Design, as long as the customer is willing to accept the potential faults and defects.

Really. Should we say ALL engineering disciplines are dead because if you are crafty enough you CAN build anything without engineering design.

I think the reason why we have this discussion here is because everyone and their dog calls themself Software Engineers as soon as they can code Hello World.

"Physical solutions are built upon immutable knowledge - the tensile strength of steel, the conductivity of copper. There is nothing immutable about software."
Have you ever consider the fact that software runs on hardware therefore software inherits / is constrained by most if not all of the host hardware's so-called "immutable characteristics"?

If you ask me, the "death of software engineering" really means that the average software application has become little more than a trivial time-wasting web site, an unnecessary toy for the rich, rather than anything socially important.

Imagine I'm writing the software that controls the Airbus A380 that's about to take you out over the ocean, or the medical device that's about to shoot gamma rays into your head to treat your brain tumor. Would you prefer I adopted a casual non-engineering approach to my code? Let's dispense with unit tests and code reviews, after all, it's only software? Let's try out some new untested ideas and "see if they fly", literally?

I dropped out of college in the 80s to write software. Back then, we thought software was a revolution that would change the world for the better. Instead, I've spent the past 25 years helping companies get rich selling junk to rich kids. It's very sad.

I just hope that ehsanul and others who made similar comments never work on any safety-critical systems - fly-by-wire control software for passenger aircraft, for example!

I then to think that everything Tom DeMarco said was prett correct except for two bits: the idea that software engineering is useless or dead (as opposed to the control-freakery the "engineering" word has been used to justify) and his link to "Agile" (which is just another boring rehash of stuff that all competent software peopel have known since the 1960s or maybe earlier, with the usual added soundbites intended only to make it look like something new but actually encouraging the empty suits to impose yet more stupidity on the real developers.

an interesting article, but in my little corner of the software world, I don't think that is true.

I have been trained as a hardware electronics design engineer and am recently writing code for an embedded platform, and I see very little difference in engineering one or the other.

Using software libraries to build code is very similar to what I would do in the hardware world, building schematics, laying out PCBs and manufacturing.

The only difference is that the software cost nothing to put right when a mistake is made.

I can't really comment on the 'soft' area of software as I have no experience in it.