*
programming and human factors
by Jeff Atwood
*
In Microsoft Project and the Gantt Waterfall, many commenters wondered why software projects can't be treated like any other construction or engineering project:
I am not sure why it is so difficult to estimate software development? Is it a mystery, magic, is there a man behind the curtain that every project depends on?I mean it's simple, come on! It's like a designer and general contractor coming over and estimating how long it will take to remodel your kitchen.
But software projects truly aren't like other engineering projects. I don't say this out of a sense of entitlement, or out of some misguided attempt to obtain special treatment for software developers. I say it because the only kind of software we ever build is unproven, experimental software. Sam Guckenheimer explains:
To overcome the gap, you must recognize that software engineering is not like other engineering. When you build a bridge, road, or house, for example, you can safely study hundreds of very similar examples. Indeed, most of the time, economics dictate that you build the current one almost exactly like the last to take the risk out of the project.With software, if someone has built a system just like you need, or close to what you need, then chances are you can license it commercially (or even find it as freeware). No sane business is going to spend money on building software that it can buy more economically. With thousands of software products available for commercial license, it is almost always cheaper to buy. Because the decision to build software must be based on sound return on investment and risk analysis, the software projects that get built will almost invariably be those that are not available commercially.
This business context has a profound effect on the nature of software projects. It means that software projects that are easy and low risk, because they've been done before, don't get funded. The only new software development projects undertaken are those that haven't been done before or those whose predecessors are not publicly available. This business reality, more than any other factor, is what makes software development so hard and risky, which makes attention to process so important.
One kitchen remodelling project is much like another, and the next airplane you build will be nearly identical to the last five airplanes you built. Sure, there are some variables, some tweaks to the process over time, but it's a glorified factory assembly line. In software development, if you're repeating the same project over and over, you won't have a job for very long. At least not on this continent. If all you need is a stock airplane, you buy one. We're paid to build high-risk, experimental airplanes.
The appropriately named happy-go-lucky left this comment on the same post which explains the distinction quite succinctly:
The problem originates in the fact that "Software Project" shares the word "project" with "engineering project" or "construction project.""Ah-ha!" our hapless software managers cry, as they roll-out the money for a copy of MS Project. And the software developers cry all the way 'til the end of the "project."
Software invention is just that. It has more in common with Mr. Edison's Menlo Park laboratory than the local hi-rise construction site.
Of course, there are far fewer variables on a work site than in software invention. Gravity remains the same, elements don't change and concrete has known properties. Things aren't that stable when operating systems, hardware, software and business processes come together in "the perfect speculation."
In software engineering projects, you aren't subject to God's rules. You get to play God: define, build, and control an entire universe in a box. It's a process of invention and evolution, not an assembly line. Can software development be managed? Sure. But it's a mistake to try managing it the same way you'd manage a kitchen remodel.
Thanks for posting this.
I liken software development to driving to a strange town, in a vehicle your only marginally familiar with, without a map, and not knowing the driving conditions.
Yes, you can get there. You can define steps. You can state your goals. But you're not likely to _know_ every step you'll take. When you get there, you have success or failure (depending on your definition of "success").
Not a perfect analogy, but it fits every software project I've ever worked on.
Jim Lang on November 17, 2006 1:07 AMIt's interesting that you talk about software being invention rather than construction. I've been told that the german word for software developer (which is software entwickler) actually translates to software inventor.
Helen on November 17, 2006 1:12 AMAs someone who works in software in the construction industry, I think you're making a lot of assumptions and generalizations. Construction has a lot of the same lifecycle issues that software has - including bad or incomplete design, environmental issues, and insane/stupid clients.
Construction also has a lot of issues that software development does not have. One being weather. Construction sites get rained out, or a storm does some major damage, or it's too hot to safely work. And with construction starting to get more tech-savvy, they've got a lot of the same issues we have in software development. Bad internet connection, the server is down, the server is dead, poor source/document control
Finally, construction jobs are often completed late and over budget. This is due to some of the things I've already mentioned - but it also includes: materials shortages, equipment shortages, overbearing owner/client/government requirements for minority/female/apprentice/local workers.
But the thing that construction has over software is the abundance of skilled laborers. In construction you can have a night shift come in to help get the job done on time. In construction you can get specialized workers fairly easily. These things are either impossible or very hard to do in software. In construction you can add another electrician to the job at 8am and have him be fully integrated into the project in a number of hours. In software it can take days for a developer to get completely up to speed with the project.
But overall, construction carries much higher risks than software. That's why most firms have Risk Management and Legal departments.
From my friends who have had contractors do work for them, almost every project has been overrun either in time or costs.
So using kitchen remodeling as an example only bolsters your original point.
Haacked on November 17, 2006 1:23 AMWow, to use remodelers as an example of an industry who makes accurate estimates is mind-boggling. Ask anyone who's been through the process...it takes longer--A LOT longer--and costs more than the contractor ever estimated. Sounds a lot like software engineering to me!
Then throw in all the other software engineering-unique issues and there you go. We might be able to figure out accurately and repeatably how long it takes a bag of concrete to dry once mixed, but how long to develop a user interface where how much concrete is needed to even start is a question neither the business nor the technology sides can anticipate 100% up front.
michela on November 17, 2006 1:23 AMOh, and hey, can anyone spell Big Dig? Why does software engineering alone get such a bad wrap in this area?!
michela on November 17, 2006 1:27 AMHave to disagree with much of this.
I have remodeled several kitchens and they are never the same.
Part of what developers do is always new, but perhaps not as much as we think. Of course many times the shifting sands of dev tools affects us too, and so you are not god. You have an OS, a framework, a fancy UI, etc., that you must obey.
A kitchen and a software project are not the same of course. But just as you will be penalized for not thinking thru a remodel, you will pay the price for half-baked specs and users who are allowed to change their minds at will with no penalty -- and you especially will pay the price if you assume that software projects cannot be managed well, waterfall or otherwise.
Steve on November 17, 2006 2:10 AMI think part of the problem is that generally people aren't willing to pay for software engineering like they pay for other kinds of engineering. It's a lot faster and cheaper to do a shoddy job, and since nobody's life is on the line, people are willing to pay substantially less for software that only sort-of works.
As a result, if you're building software on top of other, existing software, and that existing software only sort-of works, it's difficult to predict how the resulting beast will react when it's subjected to your particular set of circumstances.
Software development will become predictable when the "standard parts" we use become reliable.
Dwayne on November 17, 2006 2:19 AMBrooks: you're still trying to compare construction - where generally unskilled or low-skilled labourers take an actual design and build it. They're not changing the properties of something as they build it. They know how many (identical) bricks they can lay in a certain amount of time, but the weather determines how long it takes for your concrete to harden, so the project isn't going to be entirely predictable. They also have to allow for mistakes in the design; if this occurs, and part of the structure turns out not to be strong enough, they have to go back to design.
We have unskilled labourers to construct our software. They're called compilers, linkers, make tools, continous integration tools. We've so automated our construction activity that it doesn't look like construction anymore, so our detailed design activity is confused for construction. If we make a serious enough mistake in our design, the labourers go on strike (produce error messages). But when we make a mistake that the labourers don't catch, it's not as readily apparent as when part of the house falls down when you're trying to build it.
Architects are also not generally trying to construct houses from individual grains of sand. They don't normally design their own bricks. Designing basic building blocks is all too common among programmers. Sometimes it's justified. Most of the time it's not.
Mike Dimmick on November 17, 2006 2:20 AMMike's onto something - that's the scale, from the large to the small details. On my current project, I've designed the architecture. I've also implemented assembly code.
Meanwhile, as it happens, my new house is being constructed. They're right on schedule, actually ahead I think. The architects of the house certainly did not have to develop particle board or paint in order to pull this off, which is essentially what I have to do (I'm working in an embedded environment with no OS or third-party libraries).
I think some of the readers are missing the point, it's not that you it is not possible to manage a software project, you can. The point is that the degree of accuracy of determining the length of a COMPLEX project is lower then that of a SIMPLY project.
The analogy of building an airplane is a good one, an airplane is a complex thing, but each line worker has a relatively simple task, bolts, wiring, etc. Once you complete the first plane you are able to say how long it takes to complete the next one.
But if you are building a prop plane today, a jet tomorrow, a cargo plane after that. Can you really say you will know exactly how long each will take if you are building each one for the first time.
You read someones blog, you want to build that cool widget and someone says "IT WILL TAKE YOU 5 HOURS". And when it takes you 8 hours, is it poor estimation or poor execution?
Tim on November 17, 2006 6:03 AMIf God's name is John Von Neumann, well maybe. Even so, you *will* obey his laws, because the computer you execute on does. It is more strict than Turing, which is Vapor.
BuggyFunBunny on November 17, 2006 6:08 AMEven engineering projects have the same problems - different root issues, but very similar. Check out "Critical Chain" by Eli Goldratt - David Anderson's book "Agile Management" takes these concepts and ties them to software development. Both worthwhile reads.
S Hebert on November 17, 2006 11:16 AMWhen discussing building construction vs. software development, the key in both cases still comes down to the degree of planning that you are able to do prior to sitting down and writing the first line of code or digging the first hole. Both rely upon standardization, something that is increasingly true in the software arena as componentization continues to push its way up and down the abstraction stack.
Moreover, while there is still a certain level of art necessary to be a good software architect, its worth understanding that most software projects are going to be ranch houses or fairly standard office buildings, not Fallingwater or the Guggenheim museum.
Kurt Cagle on November 17, 2006 11:28 AMI love these posts and find them very interesting - but when are we going to get back to some coding horrors? Seriously - can we have some code issues too please?
Ronnie on November 17, 2006 12:32 PMSoftware Engineering is like all other engineering, but all engineering is an art when it is young.
Imagine the first person to build a stone bridge with a support across a river. That person was probably an artist and visionary of immense talent. Many artists with extreme talent end up in engineering, but even so, almost all the people you encounter in "Engineering" today are simply copiers.
They take components and pieces designed by artists and place them in patterns that are copied from something an artist once made.
In software engineering, we are still creating the components and patterns. This means that without an artist on your team, you are very likely to fail. This will not always be true, but until artists create enough tools for programmers to use, the jobs will be difficult to schedule and results will be bug-ridden and sloppy.
Waterfall works pretty well if you have an artist on your team (which was more common when Waterfall was created, but less so as we moved through the 90's and the tiny pool of artists diluted.)
Agile development is an acknowledgment that very few teams have artists on them and a set of methodologies to compensate.
Consider the difference between hiring Michaelangelo and having him pick talented young laborers to paint a Chapel roof vs going out yourself and hiring a bunch of house painters and saying "None of us really understand this process, so let's allow an extra month in the schedule".
The house painters don't have a chance, and it's this fact that causes the 80-95% of software projects that fail.
I guarantee there are at least a few at SUN, that beautiful, clean Java API didn't write it self. No matter how many times you iterate, without the vision you will not get to beauty, you will just (barely) get to functional.
Also, My opinion is that like any other art, much of your aptitude is set before you are a teen. If you aren't eating code for breakfast, lunch and dinner after your first 3 years of exposure to a PC, you'll never be an artist.
Just like if you are not playing decent piano within the first few years of full-time exposure to one, you'll never be a master. You can be good, you can play with a group, but you are not going to fill a concert hall alone.
Bill on November 17, 2006 12:43 PMAnd... folloowing last comment... once you have done a few easy buildings (easy software projects) you know more or less how much time is needed. Most of the project failures are due to lack of initial analysis and task breakdown.
A lot of people out there go straight to software arquitect after coding in a few big projects. Hence, they don't have the required experience.
that was one of the reasons for the dot com boom, too many twats promoted without experience or knowledge. If you are doing rocket science, yes, it's extremely difficult to be accurate on your estimations, if you are doing regular office building, you should know to certain extend how complicate is going to get...
argatxa on November 18, 2006 2:43 AMIt all comes down to the 90/10 rule. The last 10% takes 90% of the time.
Software fills the last 10% of the tricky logic problems. Solving the tough problems tends to unwind average designs. Changes force feed to a design that is failing, and the last 1% almost impossible.
Many systems (e.g. many construction problems) tolerate non-perfect results and can ignore the last n%. Tolerances in logic are slim.
Kenny on November 18, 2006 2:49 AMAccurate estimation of a task involves
1. Understanding the problem
2. Understanding how to implement the solution.
If you give the same problem to two different teams in the same company to solve, they're most likely to come up with different solutions, depending on how they have understood the problem, and so will have different estimates of how long it will take.
With software, there are many ways to solve problems, and each solution has its own set of constraints.
Paul Broadfoot on November 18, 2006 3:14 AMWell I think the comments show more similarities than difference between mid to large scale software development and other more "physical" kinds of development/construction.
I suspect that software is more burdened with unrealistic expectations. The customer compares custom development with the effort required to select, acquire, and deploy shrink-wrapped products marketed to 5,000 to 5,000,000 users and has no visceral feel for the effort that's amortized over all those sales.
Bob on November 18, 2006 9:01 AMOf course software project is like any project and can be managed! For me this type of thinking is typically a developer's way to glorify and mystify softare development.
True, there are some never-been-done-experimentary-really-hard-to-estimate ventures. However, an overwhelming majority of projects do not fall into this category. Most SW projects are about organizing GUI elements, storing and fetching data from database, integrating some 3rd party components, implementing business logic and flow. In most SW projects there is nothing new or groundbreaking in the code. It is the business function that might be revolutional.
These projects are not impossible to estimate or design. Difficult, sure. But if there are good requirements and good design, implementation is mostly "just run-of-the-mill work". Work that is often inspired and fun, creative and challenging. And sometimes really tedious :-) But still, saying that most software is unproven and experimental, is to me a far stretch.
It seems that many developers have some kind of natural aversion to proper design and implementation guidelines. This typically leads to chaos, odd experiments and delays, but the reason why most IT projects fail is not that software development is too difficult.
Ok, off my high horse :-) Thank you for your thought-provoking blog.
karen on November 19, 2006 5:44 AMComing from both an aero-space and software background, I can tell you that both the risks and challenges in the development cycles are extremely similar.
No airplane is identical, and even within a given line (eg: F-16) there are many blocks which differ substantially in how the components integrate with each other. Their software analogy would be version upgrades.
Next up from that are variants (A 747-400 is substantially different than a 747-400ER), which is similar to the differences between say Visual Studio Standard and Enterprise editions. Each variant of a 747, while based on the same historical airframe and air-tunnel research, starts branching off and adding different features based on their unique target markets.
Hardly anything in aero-space engineering is repeating the same thing over and over. From the airframe designers to the line engineers, there’s always a new challenge they have to incorporate with their existing knowledge.
And funny enough, the high-risk, experimental airplanes are the staunchest adherents to the waterfall model.
1. Research Historical Data and Conceptual Design
2. Prototype
3. Scale Component Testing
3a. (loop research again if planned/needed)
4. Full scale prototype
5. Testing
6. System Integration
7. Flight Testing.
Conversely, a move towards away from the pure waterfall model can be found in Multi-disciplinary Design Optimization (MDO), which appears in more mainstream engineering efforts. Here, they are attempting to iterate more frequently in order to achieve better use of materials and structures. The waterfall models do attempt some iterative loops, but they’re low in number and generally pre-determined.
So, why has the aero-space industry generally use the waterfall model and gantt is planning overall projects? Because the correct use of these require only two pieces of information:
1. Historically, how long has similar tasks usually taken.
2. What are the risks? How well do I know this problem space? Are we innovating substantially here?
That is all that an accurate (if not entirely precise) estimate requires. Any more information and it’s likely a biasing factor because of wishful thinking or political reasoning. Techniques such as MDO makes it harder to estimate a project with any reasonable accuracy because the dependencies added from the feedback loops cannot be enumerated. How can anyone before hand, say how many iterations will it take, before we get it right?
For the aero-space industry, the cost savings of MDO promises to make it cheaper in the long run for production to offset this uncertainty in development cost.
So, can software development learn anything from the aero-space industry? Quite a lot I must say. The same risk-analysis, and structured development methods can easily be applied to software development.
1. Gather requirements up-front and prioritize. What are the primary goals of the design.
2. To get an accurate global estimate, one must strive to reduce the number of dependencies between tasks.
3. Recognize which tasks have historical correspondence, and use that data.
4. Explicitly write down your risks. Give an estimate to them, and that PLUS your original estimate is the time required. (*1)
5. Don’t change the requirements and then assume the costs will be the same. All the other engineering industies have already learnt this lesson. Ref Change Management.
6. Make sure a design for testing is in your development plan. The earlier risk questions can be answered earlier, the better it is. It gets progressively more expensive to test later.
7 Where it makes sense from a long-term cost, use MDO. (*2)
Footnotes:
*1 - Ironically, despite the fact that most programmers should have studied network theory, the software industry is the only one where I frequently see people try to do a 60%-40% rule or some obscene optimistic fudging for schedule+risk estimates. Scheduling is a longest path problem – any fudging is called wishful thinking. If a value is too long for your liking, plan to answer the risks early enough so you can re-estimate, not arbitrarily change the value.
*2 - My personal opinion on #7 is that frameworks and language designs are probably the only application of this rule. And as every good developer should know, unless you’re in the frameworks business, you should not be building frameworks.
ted on November 19, 2006 12:50 PMTo echo Dave Mackersie comment that "Software never leaves the design phase", have a look at this article from 1992 called, "What is Software Design" http://www.bleading-edge.com/Publications/C++Journal/Cpjour2.htm
"I concluded that we are not software engineers because we do not realize what a software design really is. I am even more convinced of that today."
Even though the article was written 14 years ago, I believe it still holds true today. I wonder how many of us that write code realizes that, "A program listing is a document that represents a software design. Compilers and linkers actually build software designs.” Seems obvious, but…
As Brad Cox (inventor of Objective-C) wrote 16 years ago in his article, Planning the Software Industrial Revolution, “I only wish that I were as confident that the changes will come quickly or that we, the current software development community, will be the ones who make it happen. Or will we stay busy at our terminals, filing away at software like gunsmiths at iron bars, and leave it to our consumers to find a solution that leaves us sitting there?” http://www.virtualschool.edu/cox/pub/PSIR/
Software Industrialization is one of my favorite topics when I am not filing away at iron bars :-)
Software Engineering? “A study akin to numerology and astrology, but lacking the precision of the former and the success of the latter.” I wish it weren’t true, but coming from an electronics engineering background, it seems like we have a long way to go for software development to be called “engineering.”
a href="http://developer.valvesoftware.com/wiki/Valve_Time"http://developer.valvesoftware.com/wiki/Valve_Time/a
Right...
P.S. Be sure to check out my recent post on my blog about this subject - blog at ggglog.B_L_O_G_S_P_O_T.com
gjsmo on November 20, 2006 2:30 AMBy the way - take out the underscores (_) in the URL of my blog.
gjsmo on November 20, 2006 2:32 AMI've noticed something interesting. People seem split into two camps.
1. Software construction is an art and cannot be estimated well.
2. Software construction is a science and can be estimated as well as can be expected.
Here's my two American cents: I work for a large company and I have done lots of work on well established, large software packages. When adding new features or module enhancements to established packages, estimation is much easier and waterfall/BDUF works surprisingly well. As a software publisher, we're responsible to a large customer base, so having a good design helps. There aren't many surprises, so the waterfall isn't a hindrance. Projects, when well managed can come in close to time and budget.
Since the company is so large, I've also done a lot of work in more cutting-edge areas on newly established products. Waterfall and BDUF aren't so good in that arena. A more agile approach is necessary since the plan tends to change much more often. Nobody (the company, managers, or developers) has a large stake in the new product, so rapid and large change decisions can (and do) get made often. That's okay... a professional developer should be able to be agile enough to handle it (in my opinion). Estimation of completion at the start of that project is next to impossible. What to do? Work with management and provide ongoing status updates often. Make sure that they are aware of how all change decisions are affecting the time line and where the project stands and give re-estimates often.
My opinion is free and probably worth what you paid for it.
g
Garret on November 20, 2006 5:58 AMAs a software engineer who has worked closely electrical and mechanical engineers for many years, I think the key difference is this:
Software never leaves the design phase.
Now, I know that as software engineers, we often make a distinction between design and coding. But unlike mechanical engineers, our work never needs to go to a manufacturing floor to get built. For the mechanical engineer, the transition from design to manufacturing is an important milestone, and represents a significant commitment of monetary resources. In fact, each step in the waterfall reperesents a major decision point to commit more financial resources.
In software, we can implement the work in relatively tiny incremental iterations, with each iteration representing a relatively tiny financial commitment of a programmer's time. Of course, a lot of those tiny iterations can quickly add up to a lot of money. But my point is, there is the potential for a lot finer grain control over the project, for someone who is able to stay on top of it. I don't think any other type of engineering discipline can do the same, but if they could I'm sure they would.
Dave Mackersie on November 20, 2006 6:19 AMI think, finally, Dave Mackersie hits on the key point: Software is an iterative process. That is all.
And whoever Bill is, your little speech about what defines art and what is an artist is frickin' hilarious. Your romantic vision of yourself is so funny I threw up a little in mouth.
kferron on November 20, 2006 9:12 AMEvery time I hear somebody comparing software building with say physical house building and criticise "Well, people have been doing houses for ages, why can't coders do it well and on time?", I remind myself the house my parents bought took three times more to deliver, and probably costed much more than original planned, though we didn't have access to those figures.
darksith on November 21, 2006 5:08 AMThe problem lies in that software is a new invention every time and in building it you invariable "improve" and change what you had done in the past. Nick Bradbury (a href="http://nick.typepad.com/blog/2006/06/start_coding_li.html)"http://nick.typepad.com/blog/2006/06/start_coding_li.html)/a describes it like this, "... after I'm comfortable that the memetracker is working as intended, I'll then throw the code away and start a more formal design approach. Cowboy coding is a great way to find problems, ... the goal here isn't to complete a feature quickly (although that's often a side effect), but instead to discover the gotchas up-front so you can design the feature correctly."
You often don't see the gotcha's even in a well defined project. In the case of a kitchen remodel or any remodeling of an older home, you don't know what is inside the walls/floor/ceiling. You don't know if Jimmy did the wiring himself and daisy chained it or cross wired something. It's the gotcha's that happen all too often in software that change things. The 3rd party tools and widgets don't work as advertised and you have to build your own. The customer doesn't want to spend for the widget and you have to change the design to meet what you have, etc. Sure the construction process might be the same, but getting the individual parts to work together isn't.
Mindfulgeek on November 21, 2006 7:30 AMEngineering (like build planes) and software follow same risk patters.
It is risky to build a copy of a working plane ?, really not. It is risky to make a copy of a working running software ?, really not.
It is risky (very) to build a new designed plane. As it is risky build new designed software (a lot less than a plane).
What is different : Thousands of new software are make every year, a handful new planes are designed every year, ergo, new software problems arise often, not new plane problems.
All this talk of construction, projects, and engineering is confusing everybody. Software development is much more like making a movie than any of these.
All the effort is up front and then once released (we even use the same word) you start seeing some income.
Once done, you can sell the work again and again for almost no cost.
Just as many adventure films have been made, so have many accounts payable systems, each with its uniqueness.
If a movie's a hit the imitators arrive from all directions, just like in software development.
You can patent software about as well as you can patent a movie. Same applies (in the other way) about copyright.
Both have the same attitude about deadlines.
Both industries are legendary for cost overruns and massive flops.
Both industries are legendary for cost overruns and massive flops.
David Douglass on November 23, 2006 8:44 AMHey gal's/guy's, can you tell me what is the difference between CPU processor ENGINEERING and Software ENGINEERING, or maybe Chinese WATER HYDROELECTRIC POWER ENGINEERING, this are BIG projects.... they all take long time, there are some really silly things - like taking one year to write several hundred lines of code like in previous Microsoft rant ;)
BTW if you are building a web site DON'T take a dozen of developers - man that will eat your soul and your pocket out.... like someone before said don't try to emulate a Michelangelo if you ain't one -- go to school!!! -- then work for several years as an apprentice and if you don't cut it in 3 to 5 years with hugely popular and PROFITABLE project then man/woman hide under the rock and die as a poor programming schmuck and don't droll and be jealous of people who make a killing in software business or any other.... ;)
Mayo on November 30, 2006 5:06 AMI read a good description of one key difference between software and engineering projects once: when you're designing a new engine you don't usually have to worry that the pistons will leave the cylinder block, destroy another component and pop back in again. And when physical components break, they tend to break in predictable ways.
JohnD on February 7, 2008 10:10 AMDon't be a pussy. Software is no more challenging than machine design. I'd sit in the A/C and write code all day if I could. Programming is the easiest part of the job.
Matt on July 14, 2008 6:58 AM@Matt
Ya know, I completely agree with you. Software development isn't easy, but then again neither is say, designing/laying out/manufacturing a processor. There just aren't many outspoken hardware blogs whining about how their profession has it rough :). No one I know that is really good at it got to where they are by professing REAL hardware engineers started when they were 9 and live/breathe/eat hardware engineering. That's just dumb and I'm sick of it from the software community.
Software development (and I say development, we are still very much in the dark ages) still has a long, long way to go.
chris on January 8, 2009 11:33 AMYou buy a plane, you buy software. Engineering methods are still needed even if software engineering is project-like. You cannot blame a project failing for being a unique project, if you do not use proper engineering methods. A piece of software is still a piece of software - so software engineering methods are needed to keep that piece intact and proper. You also can recycle smaller parts of program code from other projects. Then there are design patterns in software engineering, but design patterns already existed in for example building architectures.
There really are lots of successful unique scientific projects in the world. Science always builds on earlier science, and new software builds on earlier software. We rarely need to start from scratch, at least we have high level programming languages. The language providers are the basic level inventors. Then there are application level inventors who make basic components. And then we have some trivial stuff like packaging, which feels like assembly lines.
Because we build unique programs, we get money on the unique level, so we can afford good methods. Though there are lots of basic programming, too. Not every program require tons of inventions a'la interstellar travel, but tons of normal features. And even if a software project was more like inventing some mind exploding algorithms, the more important it would be to use scientific methods together with the engineering methods.
If there are lots of things in the project like software, hardware, operating systems, people, paper, and business processes, then these all should be considered - with engineering and with science.
Don on February 6, 2010 9:51 PMHey! Jeff, you read my comment and made today's post from it. Cool.
Grant Johnson on February 6, 2010 9:51 PMPrior to changing careers and becoming a full-time programmer, I was a research scientist. We carried out research "projects" which bore even less resemblance to engineering "projects" than do software "projects". In research, you very oftendon't even know how you're going to accomplish task B until you understand the results of task A, which is an investigation of some physical behavior of a new material or the mechanical behavior of a new configuration. Sometimes, if Task A1 gave inconclusive or negative results, you would have to carry out task A2, and maybe A3 and A4, before you knew how task B would or even could proceed.
Yet we had managers who would try to plan the project out to the day, to the dollar. But it is hard to plan in advance for synthesis, for innovation, for invention. You keep trying new things, and studying what went wrong with the last new thing so the next one had a better chance to succeed. Many managers didn't understand this, so they'd lose patience and interest, and complain about how the researchers never finished anything on time or on budget.
Sound familiar?
Jon Peltier on February 6, 2010 9:51 PMEverything depends. If you select a small part to develop, then you can be more certain that if you give some time, the time is enough. And the amount of time allocated depends on what you are doing.
Like you select that one part of an enterprise will be automated, you use this and this technology, those technologies are familiar to you, and you expect no or little business process re-engineering. Automation should be straightforward, maybe some batch processing and an computer-machine interface. You allocate fair amount of time, money, commitment, and skilled experienced motivated familiar people with a good leader. The project goes ok.
Bigger projects has to be divided into smaller pieces. Then it is easier to estimate the time consumption of those smaller pieces.
If you are faced with unfamiliar technology, like you have some unknown hardware, then that is just a real life issue. Its difficult for a project to succeed, if there are unknown hardware or hardware that doesn't work. So, the solution is to use well documented known hardware that has the reputation of working. When you program some software and blame hardware problems, then its like "I am a good progammer, but we do not have a good hardware person, we just trust luck with hardware".
Don on February 6, 2010 9:51 PMAnd there are many ways to achieve the understanding of a problem. When a customer wants some problem solved, he might not know all related issues about it. And the customer might think that the software or the engineers are the ones who solve the problem, so why the customer has to think something about it. But the customer is needed in analyzing the situation and in providing information. But this can't either go like only the customer states what will be built, because the customer rarely knows the possibilities and limitations of software engineering.
Don on February 6, 2010 9:51 PMWell, thanks for using my quote, I hope it made for some interesting discussion. I have to disagree that every project is a leap into the great unknown. All projects I have worked on have 1 thing in common.
Get data from user
Store it
Read data from store
Show data to user
Now, most if not all programs read and store data, so I don't see how every program is an unproven entity. The methods used to implement the program may be unproven, but the overall concept is not. The fact that the technology changes every 6 months is certainly a problem for this industry.
There are many cases when it is entirely unknown, like writing a new compiler, a new OS, a new piece of hardware. I don't think the kitchen remodel fits for those situations. These are more R and D projects.
The concept of equating a kitchen remodel to software development I think holds true if the design is not revised.
However, most projects the design is not complete or proven by the time the coding has begun. This leads to a lot of iterative development (as many people have suggested).
Most engineering projects the design and solution are redfined before any implementation, once the design is complete, it is not changed. Try building a 5 lane bridge after you have designed a 2 lane bridge!
With software development the design and solution are refined throughout the implementation.
So, It is much easier to hit a stationary target (other engineering) versus a moving target (software engineering).
We should spend more time on making the target as stationary as possible before coding. This probably isn't entirely possible, but it is something we should strive for.
Yes, there is like UML-documents, which can be considered as the design. Then the actual program code would be the construct. Each iteration costs when they design and test airplanes with models. Each iteration costs when software design is iterated and program code updated accordingly. But such is life. Though you don't have to iterate the design, if you catch the change already at the requirements level.
Well maybe pioneering engineering isn't quite art, but lots of creativity is needed when new things are tried to create. Then others can copy existing design patterns without inventing new wheels.
Don on February 6, 2010 9:51 PMi have read a lot of literature on this subject but none impressed me as much as this awesome article thanks planning jobs
The comments to this entry are closed.
| Content (c) 2012 Jeff Atwood. Logo image used with permission of the author. (c) 1993 Steven C. McConnell. All Rights Reserved. |
Subscribe in a reader
Subscribe via email