Fifty Years of Software Development

September 20, 2006

O'Reilly's History of Programming Languages poster is fascinating reading.

Detail from O'Reilly History of Programming Languages poster

If you trace programming languages back to their origins, you'll find that we've been at this programming stuff a long, long time.

C is roughly as old as I am; FORTRAN is as old as my parents. But what about the new kids on the block? the TIOBE software TCPI metrics page provides some data on language popularity going back to the year 2001. Consider the tender age of many of the newest, hippest programming languages:

Ruby is barely a teenager. JavaScript hasn't even hit its teens yet.

Now correlate the ages of those modern languages with the publication dates of a few books that represent current thinking in modern software development:

1994Object-Oriented Design (Booch)
1995Design Patterns (GoF)
1997UML Distilled (Fowler)
1999Extreme Programming Explained (Beck)
1999Refactoring (Fowler)
2001Agile Alliance is formed

Modern software development is a recent development. Even though we collectively have over fifty years of experience under our belt, the profession of software development is still very much in its infancy.

Consider source control as an example. Source control is the absolute bedrock of software engineering. I believe that source control was not widely prevalent until 1999. Here's why:

  1. Although CVS has been around since the late eighties, it was widely popularized through SourceForge, which didn't exist until the year 2000.
  2. Microsoft's SourceSafe was available starting in the mid-90's but didn't hit mainstream acceptance until it was bundled as a part of Visual Studio 6.0 Enterprise in 1998.

Clearly, source control existed before the year 1999. Why did it take so long for this essential tool of software engineering to filter down to the mainstream? The answer lies in the Redwine-Riddle maturation model (pdf):

Redwine and Riddle reviewed a number of software technologies to see how they develop and propagate. They found that it typically takes 15-20 years for a technology to evolve from concept formulation to the point where it's ready for popularization.

They identify six typical phases:

  • Basic research. Investigate basic ideas and concepts, put initial structure on the problem, frame critical research questions.
  • Concept formulation. Circulate ideas informally, develop a research community, converge on a compatible set of ideas, publish solutions to specific subproblems.
  • Development and extension. Make preliminary use of the technology, clarify underlying ideas, generalize the approach.
  • Internal enhancement and exploration. Extend approach to another domain, use technology for real problems, stabilize technology, develop training materials, show value in results.
  • External enhancement and exploration. Similar to internal, but involving a broader community of people who weren't developers, show substantial evidence of value and applicability.
  • Popularization. Develop production-quality, supported versions of the technology, commercialize and market technology, expand user community.
Redwine and Riddle presented timelines for several software technologies as they progressed through these phases up until the mid-1980s. I presented a similar analysis for the maturation of software architecture in the 1990s.

CVS was released in 1986. It took another fifteen years for CVS usage to become mainstream, exactly as predicted by Redwine-Riddle.

The model Redwine-Riddle proposed in 1980 is very much alive today. Mark Dominus, in Design Patterns of 1972, reaches back nearly thirty-five years to illustrate how we're still struggling to evolve our programming languages today:

Had the "Design Patterns" movement been popular in 1960, its goal would have been to train programmers to recognize situations in which the "subroutine" pattern was applicable, and to implement it habitually when necessary. While this would have been a great improvement over not using subroutines at all, it would have been vastly inferior to what really happened, which was that the "subroutine" pattern was codified and embedded into subsequent languages. Identification of patterns is an important driver of progress in programming languages. As in all programming, the idea is to notice when the same solution is appearing repeatedly in different contexts and to understand the commonalities. This is admirable and valuable. The problem with the "Design Patterns" movement is the use to which the patterns are put afterward: programmers are trained to identify and apply the patterns when possible. Instead, the patterns should be used as signposts to the failures of the programming language. As in all programming, the identification of commonalities should be followed by an abstraction step in which the common parts are merged into a single solution.

Multiple implementations of the same idea are almost always a mistake in programming. The correct place to implement a common solution to a recurring design problem is in the programming language, if that is possible.

The stance of the "Design Patterns" movement seems to be that it is somehow inevitable that programmers will need to implement Visitors, Abstract Factories, Decorators, and Faades. But these are no more inevitable than the need to implement Subroutine Calls or Object-Oriented Classes in the source language. These patterns should be seen as defects or missing features in Java and C++. The best response to identification of these patterns is to ask what defects in those languages cause the patterns to be necessary, and how the languages might provide better support for solving these kinds of problems.

I do think the pace of change in software development is quickening, thanks to exponential increases in communication over the last fifty years-- television, satellites, cellular phones, and of course the internet. As software developers, we've grown accustomed to computer hardware doubling in speed every 18 months. What we haven't been able to cope with so well is how long it takes for the human beings to catch up with the hardware.

Posted by Jeff Atwood
42 Comments

I can't believe that source control wasn't popular until 1999. I've never worked for a company that didn't use it. Before cvs there was rcs, and before rcs I'm sure there was something else.

Maybe source control is new for the hobbist.

FigBug on September 22, 2006 7:12 AM

-- What we haven't been able to cope with so well is how long it takes for the human beings to catch up with the hardware.

arguably, not. the Wintel monopoly is so named because of the death spiral they dance. Intel has no (at least, little) engine of growth without a need to suck cycles. M$ has no (at least, little) engine of growth without a source of cycles to suck up with new "features".

They desperately need each other. the rise of linux, and the $100 machine, is their collective worst nightmare.

how to *productively* use the cycles courtesy of Moore's Law, now that's a different issue. your recent articles about Vista's requirements and "features", should give those in bed with Wintel the shivers.

speaking as one who is currently embedded in commercial applications, a BCD multliplier is a much better way to use that real estate and cycles. hasn't happened. less pixel dust and more real computation. gamers likely don't see it that way.

a more disturbing thread has been bothering me of late. to wit: "The Soul of a New Machine" was published in 1981. the notion that a bunch of folks could, or even know how to, design a processor today from a parts list (which probably doesn't exist either) is farfetched.

hardware is becoming a mono-culture. unless, and i just don't know, hardware engineers are having a party in the embedded space. i hope so. if not, then where are we really headed? how soon will we stop even training hardware engineers, because it is decided that X86 is good enough for all purposes?

buggyfunbunny on September 22, 2006 7:14 AM

I agree that 1999 sounds crazy late for source control.

I started using PVCS, a commercial package, at a game development company in 1991. Game programmers are quick to tackle new technology, but pretty slow to get up to speed on development practices, so I'm assuming even in '91 we were late adopters.

Granted, we used it primitively, i.e. no multiple checkouts and no branching, but in retrospect it was much more powerful and reliable than SourceSafe, which I ended up using for most of the late 90's, until things like Perforce came along.

bb on September 22, 2006 8:29 AM

I'm with bb. I've worked as a game developer since 1995, and even back then working without source control would have been almost unthinkable. And still with bb, we're usually pretty far behind on good devleopment practices. I cannot believe source control wasn't widespread outside game development well before 1995. I don't have much evidence either way, though I did breifly work for a non-game developer in 1996 where CVS was already well entrenched.

I've also been present at two different game developers for a switch from SourceSafe repositories to CVS. One of those SS repositories predated 1995, but the other maybe not.

Joe Rumsey on September 22, 2006 8:47 AM

One of my colleagues used to work at Boeing at the turn of the century and they had to do some work on VAX machines. VAX (at least this is what he told me) has source control built into the operating system, and I would have said VAX machines have been around since the late 70's early 80's.

Andrew Davey on September 22, 2006 9:05 AM

It's not that the VMS filesystem has source control built in, exactly. When a file is changed, the file's version number is automatically incremented, so instead of foo.c you'd have foo.c;5. If you needed to go back to a previous revision of that file you could open foo.c;4 and save it over foo.c, creating foo.c;6 with the same contents.

Deleting a file deletes all revisions, but in the normal usage pattern you just plain don't delete most files.

Angstroem on September 22, 2006 11:32 AM

Hi

I read the blog in an online news reader and all images are replaced with a "WTF" image. I tried it in several online readers and it's the same.

Any chance you could authorise images when displayed on newsisfree.com domain?

Mike Krus on September 23, 2006 2:24 AM

Perhaps instead of 'mainstream' you meant small teams?

Source Safe was used 1993 at Corel.
RCS was used at my present company, a unix company, at that time.

However I worked at a small company with a team of developers below 10 and they were just using a source copy on the network, and doing diffs with norton commander..

Small teams definitely think of source control as being an hinderance rather than helping.
Today I use ClearCase and I can't live without source control -- I check in every day a snapshot of my work

ulric on September 23, 2006 2:41 AM

I've never worked for a company that didn't use it.

I worked for plenty of small businesses that had no idea what source control was. I stand by my assertion that *mainstream* programmers didn't have ready access to source control until around ~1999.

There's a very good reason why source control is item #1 in Joel's twelve-item checklist.. which was coincidentally written in the year 2000.

http://www.codinghorror.com/blog/archives/000643.html

Jeff Atwood on September 23, 2006 2:50 AM

A lot of those new languages I'd consider as scripting languages and they are used to get over platform dependence on the internet.

So yes people who use these don't necessarily need to know about hardware, but do need to know about the abilities of the framework that they work under - which could be considered a virtual machine.

Paul Broadfoot on September 23, 2006 3:17 AM

Our repository goes back to mid-1996.

By contrast, I've worked with software companies well into this millenium who were stubbornly ignorant of source control.

I'm really not sure what this means, except that the issue is probably a lot more complicated than it looks, and I doubt any one person's individual experience is likely to shed any statistically meaningful light on the subject.

Apoch on September 23, 2006 6:51 AM

Instead, the patterns should be used as
signposts to the failures of the programming
language. As in all programming, the
identification of commonalities should be
followed by an abstraction step in which the
common parts are merged into a single solution.

That's a very good point that never occurred to me before. But it makes so much sense.

Take for example the "once" keyword in Eiffel: it complete does away with the need for Singleton patterns. Or rather, it provides the Singleton pattern with a single language keyword.

Obviously there's a lot of fluff going on with "Design Patterns" today, but I wonder what other patterns would be considered so essential that would justify providing them as a language construct.

RiX0R on September 23, 2006 7:11 AM

I tell you, kids these days...think they know everything. Maybe you Microsoft folks only took to source code control in 1999, but it's not like that with everyone.

I was working with *old* SCCS controlled source trees in 1986 (a system which dates from the 70s...look for Rochkinds' paper). Almost everyone I delt with then working on non-trivial programs used SCCS or eventually RCS and considered tracking changes essential.

Maybe that's just a Unix thing...:-)

kjs3 on September 23, 2006 7:49 AM

You whipersnappers make me feel old sigh. Back in the day, we, too, had "source control." It was called "paperwork" and organizational "procedure." Mind you, this was back in the 70s in the military. But, for us, no one touched the code until we assigned them to it. The only way their modifications could get into the official source code was at the end of the release cycle when we accepted their Hollerith cards for inclusion into the official source. That's after all the myriad forms of testing of that particular hunk of code and it's forward and backward dependencies (which we kept track of on a paper "chart").

This is another example of how all the stuff people figured out in the mainframe world half a century ago was just ignored and then had to be re-invented in the PC world when it started to hit the fan. "1999:" if I could find my dentures, I'd say "pshaw."

David A. Lessnau on September 23, 2006 8:01 AM

Your point about patterns being examples of what's missing from a language is well made, but your comments about source control are way off-mark.

I've been in the business since 1980, and my department was using source code control well before I started. Self-built systems on a Vax, with proper check-in and check-out. We even had a part-time librarian (yes, they were actually called that). Shrinkwrap PRODUCTS might not have been commonplace, but the need for control was well understood.

There may well be organizations that don't use source control: but they're not professional, and I wouldn't work for one.

Bob Moore on September 23, 2006 9:57 AM

I stand by my assertion that *mainstream* programmers didn't have ready access to source control until around ~1999.

Define *mainstream*. I too have used SCC since the early nineties...PVCS '91-'95; VSS '95-'97; ...

You Don't KnowMe on September 23, 2006 10:43 AM

Add me to the list of people who were using source control long before 1999. I began using it in the early 1990s, and it was already old-hat then.

Part of the problem is you don't seem to realize that CVS is a 3rd generation software package. CVS was a layer on top of RCS. RCS, in turn, was basically scripts to extend the original system, SCCS. RCS basically made some SCCS operations simpler, CVS added real branches to it.

Michael Heinz on September 23, 2006 12:05 PM

Bob Moore - reading your comment, you reminded me of something; back in the mid-80's I co-oped for the US Navy. My project was to implement the DIFF algorithm in Fortran IV, so that they could easily see the changes made *between revisions* in their *source control system* - they actually tagged every line of code with the name of the programmer who last altered it.

I'm also hard pressed to understand what any of the "new" languages listed offer that the older languages don't...

Michael Heinz on September 23, 2006 12:14 PM

I agree with Mark Dominus comments, but what are we mere developers to do. I took a compiler construction course at Uni (which was by far and away my favourite subject), but I'm smart enough to realise that creating my own 'perfect' language for the typical projects I encounter just isn't feasible. Maybe I'm too smart.

I work in C# every day, and the above is one of the many reasons that I wish the language supported macros. Not C/C++ style text replacements, but honest-to-goodness syntax tree modifying macros (with intellisense support thrown in too thank you very much). I HATE typing 'string.Format("output the value of {0}", blah)' (much like a sprintf in C). Such a waste of time. If I could, I'd make a macro that allowed me to do this 'F"output the value of $blah$"' in a heartbeat.

I remember reading a post by one of the C# language developers who was quite adament that C# would not support macros anytime in the near future (reasons being misuse). I remember when I first started in C# I thought that was smart. Now I must admit I think its a little arrogant. Leave the decision in the hands of the developer as to whether macros are a good or a bad idea. I think sealing your classes or C#'s way of not making all methods virtual by default is arrogant. Let me decide what I want to change and override. How can you possibly know what my needs in the future are going to be.

Now that I've finished my mini-rant, I wish C# supported macros. I wish it would be easy for me to build my language from the ground up. I wish that I could just do away with all boilerplate code. (seems like the ranting could start again so I'll leave it there)

Andrew Davey on September 24, 2006 4:05 AM

I believe Borland Delphi had PVCS integration in the IDE when it came out in 1995.
And maybe Turbo Pascal before that; I'm not sure as I wasn't using it, but Borland Paradox for Windows came with a 3d party source control system in the Developers Toolkit in early 90s...
If those are not mainstream?!

Franois GAILLARD on September 24, 2006 4:48 AM

Do languages ever die out?

No, they just fade away. Take Icon, for example. It's cute and had some nice features (like text processing and generators). It's on the list, but it's all but unused. You can go find it if you want, but most of the really good or novel ideas are now in newer, more popular languages.

Jeff Mather on September 24, 2006 6:05 AM

Do languages ever die out? Are there languages from the early days that are no longer in existance and should be on the chart?

I'm amused that scripting languages are included, but 4GL, database reporting languages (RP*) and are not. Also, there are no calculator style programming languages. I recall programming a HP calculator that was the size of a desk in the mid-seventies. I guess this is just for general purpose languages.

Miles Archer on September 24, 2006 11:28 AM

I agree with Mark Dominus comments, but what are we mere developers to do.

Nothing. We just do what we do now - we work with what we have, and hope something better comes along in the meantime.

foobar on September 24, 2006 11:44 AM

Mark Dominus is a saint. I am so happy somebody else in the universe finally figured out the problem with "design patterns" - they're stop gap solutions to a broader problem - the languages we're using today are woefully inadequate for the tasks we're using them for.

I love the Fowler, et al, crowd, but they're directing their energies towards the wrong audience. They should be pressuring programming language providers to integrate design patterns right into the language, just as Mark Dominus mentioned.

foobar on September 24, 2006 1:26 PM

Jeff -

Your reflections on historical use of source code control are just wrong.

My personal experience was that we were using it at a small, mixed development shop in 1989 - full SCCS/PVCS integration into our scripted build process.

Others have commented on the maturity of text control systems in Unix.

I see this again and again w/PC developers. Just plain ignorance of the past.

mjh on September 25, 2006 5:36 AM

I think Jeff's right about the widespread use of source control. Remember, even if every mainframe programmer alive used it until 1980, there were still perhaps 10 times that many Visual Basic programmers just about a decade later. My current shop implemented source control a few months after I arrived in 1997, in part because after I had fixed a particular bug, had the fix overwritten accidentally, and then fixed it for the second time, I drew a line in the sand and said, "We have to stop writing code for a little while and get past the Sneaker-net era." Visual SourceSafe has not been perfect, but it has been far, far better than nothing at all.

Andy on September 25, 2006 6:55 AM

We used to use PCVS(?) which was a DOS based source control system way before 1999. But the real reason that no one really used source control before then was that modern networking really didn't come along until about that time. All of the source control tools available only ran under DOS. And they were pathetic. You didn't have good networking capabilities in products prior to Win95 so it was simply easier to just make backup copies of you source rather than try and share some common code base across a network that took forever to try and setup. People really didn't trust networks at that time.

matt on September 25, 2006 7:49 AM

The technology in question should probably be "revision control" in general, not CVS.

I used SCCS in college back in the mid 80's (on cooperative projects). Some kind of revision control was mandated on every job I've worked since I graduated in '89. It sure seems to me that it was "in widespread use" since at least then.

CVS started life as a bunch of scripts built on RCS, which has been around since the early 80's. However, SCCS was created in '72. So the start date should be at least '72 for revision control. That would give a date of '87 or later for general acceptance by your 15 year rule. That jibes pretty well with my experience.

T.E.D. on September 25, 2006 7:56 AM

I had missed the part about Mark Dominus, thanks posters for pointing it out.

Is anyone doing something about this recommendation of enhancing a languages with partterns being built-in? Are there good working examples of this? (beside macros to hide the tedious code...)

ulric on September 25, 2006 12:06 PM

I hope you're aware of that OO is a bit older than that book from 1994. About 30 years older actually...

geir on October 1, 2006 5:02 AM

Have to agree with a lot of the posters here re. the use of SCC pre-1999 -- our company (Cimlinc) was using RCS on the Unix platform starting 1988-ish. To think about how we had to handle our source code prior to that time makes me shudder.

dh on October 2, 2006 9:26 AM

I used "SourceSafe" before it was owned by Microsoft in the early 90s. We used it for DOS-based application source control. I believe the company that Microsoft bought Source Safe from was called "One Tree Software" at the time.

Today's version essentially looks the same with a Windows UI.

Daryl on October 28, 2006 4:38 AM

Just another data point: Microsoft was using source code control internally when I started there in 1989. (A proprietary, internal use only, tool, but that's another story.) Before that I worked at Control Data where source control was a whiteboard - you wrote your name by the files you were working on.

Randy on January 24, 2007 2:41 AM

Somebody asked "What new languages have to offer that older languages don't have?"... well, the answer is quite complex, but basically I want to say "a lot". New languages like Ruby have a nice mix between old and brand new ideas that probably will change the way we code forever. I just challenge those who think that "these languages are just scripting toys", to go ahead and dive in one of them, especially Ruby (and Ruby on Rails)... you'll really surprise on what you find!

Nando on June 25, 2007 6:43 AM

"I believe that source control was not widely prevalent until 1999."

Uh, no. RCS (and it's ancestor SCCS) has been around since the 1970s:
http://en.wikipedia.org/wiki/Source_Code_Control_System

Anybody doing development anywhere near a Unix box was using SCCS or, (after 1980 or so) RCS. There were many home-grown variants of it (DSEE, SourceSafe, MPW Projector). In nearly 30 years of professional software development I've only witnessed one shop that didn't use it (and I fixed that in a hurry when I joined...)

J. Peterson on November 26, 2007 1:37 AM

Interesting article -- though you might want to rephrase "current thinking in software development" as "dominant doctrine in recommendations for software managers working on consumer / PC projects." The latter involves an entirely different generation of coders, from what I understand: people who started their careers on (relatively) inexpensive personal computers, perhaps even as hobbyists. They generally lacked awareness of and/or experience with the software tools (versioning filesystems, source control systems, GUI debuggers, "real" programming languages and libraries) common to professionals and academics working on large mainframes and expensive workstations, and also knew little about formal software development methodology (which was arguably a mixed blessing; a few infamous projects failed perhaps because of the old formal style of management).

I can't imagine that they had a strong desire to talk to old-school Lisp hackers or defense contractor programmers, for example, or even that they had much awareness that such people existed. Why else would Bill Gates have picked BASIC as a language to implement on the Altair? It's a lot harder to parse than Lisp or Forth, and memory for the parser was presumably at a premium, so it was either out of ignorance or because he wanted to target the hobbyist market.

mfh on November 26, 2007 4:22 AM

I have RCS files dated 1984. Professional use of some method
for controlling source versions have been in place for a long
time. The new part is using those tools for collaboration
between institutions.

Lawrence Crowl on November 30, 2007 3:34 AM

I remember having to explain source control to a team of consultants at a telecommunications company in 2000. They'd never heard of it before.

Will Sargent on January 7, 2008 8:42 AM

Source/Version Control has been around for a long time, but it was mostly in academia and large business. The dividing line is really at hobbyists, small, and medium sized business. While those tools would have been readily available and known to anyone who obtained a formal degree in Computer Science or worked for a large software vendor/defense contractor like IBM or Boeing; they would have been completely unknown to the average person in the 1980-1990s. The average computer users in the 1980s and 1990s were not mainframe programmers and by that point academia had begun to shift to Pascal, C, C++ and ADA. Whether or not they were available before then is really rather moot due to the gradual shift of who the "average" computer user was. So his original argument about the "popularization" of RCS software is still fairly accurate since it was really tied to the proliferation of the Home PC market. As low-cost home computers which were more powerful and readily available for personal businesses so did the market for small and medium size computer businesses.

J. R. on April 2, 2008 3:56 AM

I've been in software development since the mid-eighties. We used PVCS integrated with BRIEF for C development. I did work for large corporations in those days, so I can't speak for what smaller shops were doing.

Since then every company I've worked for has used a version control tool of some sort - VSS, Clearcase, CMVC, TFS, and so forth.

sulla on April 6, 2008 9:09 AM

I've got to agree with the sccs going back further than the mid-90s - I was working on a legacy system for a big company that had deltas going back to 1978. Of course, I didn't get around to putting my home directory under source control (subversion) until a couple of years ago.

bolobot on June 9, 2008 10:21 AM

"Perhaps instead of 'mainstream' you meant small teams?"

Small teams have been the vast majority of professional programmers for a long time. For every company with a hundred developers there are a thousand companies with 10 or less developers.

So in this context, I think that "mainstream" and "small teams" means the same thing. If only large teams were using source control, then it was niche.

Sparr on June 18, 2009 9:15 AM

The comments to this entry are closed.