*
programming and human factors
by Jeff Atwood
*
Source control is the very bedrock of software development. Without some sort of version control system in place, you can't reasonably call yourself a software engineer. If you're using a source control system of any kind, you're versioning files almost by definition. The concept of versioning is deeply embedded in every source control system. You can't avoid it.
But there's another concept, equally fundamental to source control, which is much less frequently used in practice. That concept is branching. The Subversion documentation has a decent layman's description of branching:
Suppose it's your job to maintain a handbook for a particular division of your company. One day a different division asks you for the same handbook-- but with a few parts modified specifically for them, as they do things slightly differently.What do you do in this situation? You do the obvious thing: you make a second copy of your document, and begin maintaining the two copies separately. As each department asks you to make small changes, you incorporate them into one copy or the other. But you often find yourself wanting to make the same change to both copies. For example, if you discover a typo in the first copy, it's very likely that the same typo exists in the second copy. The two documents are almost the same, after all; they only differ in small, specific ways.
This is the basic concept of a branch-- a line of development that exists independently of another line, yet still shares a common history. A branch always begins life as a copy of something, and moves on from there, generating its own history.
If you don't ever use the branching feature of your source control system, are you really taking full advantage of your source control system?
I find that almost every client I visit is barely using branching at all. Branching is widely misunderstood, and rarely implemented-- even though branching, like versioning, lies at the very heart of source control, and thus software engineering.
Perhaps the most accessible way to think of branches is as parallel universes. They're places where, for whatever reason, history didn't go quite the same way as it did in your universe. From that point forward, that universe can be slightly different-- or it can be radically and utterly transformed. Like the Marvel comic book series What If?, branching lets you answer some interesting and possibly even dangerous "what if" questions with your software development.
Parallel universes offer infinite possibility. They also allow you to stay safely ensconced in the particular universe of your choice, completely isolated from any events in other alternate universes. An alternate universe where the Nazis won World War II is an interesting idea, so long as we don't have to live in that universe. There could potentially be thousands of these parallel universes. Although branching offers the seductive appeal of infinite possibility with very little risk, it also brings along something far less desirable: infinite complexity.
The DC comic book series Crisis on Infinite Earths is a cautionary tale of the problems you can encounter if you start spinning off too many parallel universes.
Prior to Crisis on Infinite Earths, DC was notorious for its continuity problems. No character's backstory, within the comic books, was entirely self-consistent and reliable. For example, Superman originally couldn't fly (he could instead leap over an eighth of a mile), and his powers came from having evolved on a planet with stronger gravity than Earth's. Over time, he became able to fly, his powers were explained as coming from the sun, and a more complex backstory (the now-familiar "last survivor of Krypton" origin story) was invented. Later it was altered to include his exploits as Superboy. It was altered further to include Supergirl, the bottled city of Kandor, and other survivors of Krypton, further watering down the original idea of Superman having been the sole Kryptonian to survive the destruction of his world. There was also an issue of character aging; for instance, Batman, an Earth-born human being without super powers, retained his youth and vitality well into the 1960s despite having been an active hero during World War II, and his sidekick Robin never seemed to age beyond adolescence in over 30 years.
These issues were addressed during the Silver Age by DC creating parallel worlds in a multiverse: Earth-One was the contemporary DC Universe, which had been depicted since the advent of the Silver Age; Earth-Two was the parallel world where the Golden Age events took place, and where the heroes who were active during that period had aged more or less realistically since that time; Earth-Three was an "opposite" world where heroes were villains, and historical events happened the reverse of how they did in real life (such as, for instance, President John Wilkes Booth being assassinated by a rebel named Abraham Lincoln); Earth Prime was ostensibly the "real world," used to explain how real-life DC staffers (such as Julius Schwartz) could occasionally appear in comics stories; and so forth. If something happened outside current continuity (such as the so-called "Imaginary Stories" that were a staple of DC's Silver Age publications), it was explained away as happening on a parallel world, a premise not dissimilar to the company's current "Elseworlds" imprint.
Start juggling too many parallel universes at once, and you're bound to drop a few. In most source control systems, you can create hundreds of branches with no performance issues whatsoever; it's the mental overhead of keeping track of all those branches that you really need to worry about. Your developer's brains can't exactly be upgraded the same way your source control server can, so this is a serious problem.
I find that the analogy of parallel universes helps developers grasp the concept of branching, along with its inevitable pros and cons. But it doesn't get much easier from there. Branching is a complex beast. There are dozens of ways to branch, and nobody can really tell you if you're doing it right or wrong. Here are a few common branching patterns you might recognize.
Branch per Release
Every release is a new branch; common changes are merged between the releases. Branches are killed off only when the releases are no longer supported.
Branch per Promotion
Every tier is a permanent branch. As changes are completed and tested, they pass the quality gate and are "promoted" as merges into successive tiers.
Branch per Task
Every development task is a new, independent branch. Tasks are merged into the permanent main branch as they are completed.
Branch per Component
Each architectural component of the system is a new, independent branch. Components are merged into the main branch as they are completed.
Branch per Technology
Each technology platform is a permanent branch. Common parts of the codebase are merged between each platform.
You may notice a few emerging themes in these branch patterns:
But that complication is often justified. The more developers you have on a project, the higher the chances are that one of those developers will check something really bad into source control and disrupt everyone else's work. It's simple statistics. People make mistakes. The more developers you have, the more mistakes you'll have. And the more developers you have, the greater the consequences when everyone's work is simultaneously disrupted by a bad checkin. So what are our options?
The answer usually lies somewhere between these two extremes. Like everything else, branching can be abused. Chris Birmele notes that branching has its own set of anti-patterns you should watch out for:
| Merge Paranoia | Merging is avoided at all cost, due to a fear of the consequences. |
| Merge Mania | The team spends an inordinate amount of time merging software assets rather than developing them. |
| Big Bang Merge | Merging has been deferred to the very end of the development effort and an attempt is made to merge all branches simultaneously. |
| Never Ending Merge | Merge activity never seems to end; there's always more to merge. |
| Wrong Way Merge | A software asset is merged with a previous version. |
| Branch Mania | Branches are created often and for no apparent reason. |
| Cascading Branches | Branches are never merged back to the main development line. |
| Mysterious Branches | Nobody can tell you what the branches are for. |
| Temporary Branches | The purpose of a branch keeps changing; it effectively serves as a permanent "temporary" workspace. |
| Volatile Branches | An unstable branch is shared by other branches or merged into another branch. |
| Development Freeze | All development activities are stopped during branching, merging and building new baselines |
| Berlin Wall | Branches are used to divide the development team members, rather than divide the work they are performing. |
If you've managed to read this far, perhaps you can understand why so many software development teams are completely sold on version control, but hesitant to take on branching and merging. It's a powerful, fundamental source control feature, sure, but it's also complicated. If you're not careful, the wrong branching strategy could do more harm to your project than good.
Still, I urge developers to make an effort to understand branching-- really understand it-- and explore using branching strategies where appropriate on their projects. Done right, the mental cost of the branching tax pales in comparison to the benefits of concurrent development it enables. Embrace the idea of parallel universes in your code, and you may find that you can get more done, with less risk. Just try to avoid a crisis on infinite codebases while you're at it.
| [advertisement] PhotoDrop.com makes it simple to create and share online photo albums. Upload your full resolution pictures via the web site or with the free Photo DropZone utility, and you're done. No fees. No storage limits. It's the fastest and easiest way to share photos and create albums. |
Posted by Jeff Atwood View blog reactions
« Pushing Operating System Limits Revisiting Programming Fonts »
Beware of letting Project Managers know about branches. They end up jealously guarding their branch and refusing to allow merges.
Asd on October 3, 2007 02:51 AMWe use branches primarily to deliver slightly different functionality to various clients, and it's quite a havoc I must say. :-)
unton on October 3, 2007 03:23 AMVery interesting simile, and one that I had though about before. Actually, last week I discussed with some other programmers (*almost* entirely in jest) creating a time traveling game with a subversion back-end, where branches would be alternative time-lines, which could be merged, and wher conflicts would be bad for the overall space time continuum. ;)
eric casteleijn on October 3, 2007 03:40 AMWe're using ClearCase and have around 6 active branches. One branch is as old as 1 year, and has been all the time merged into the main line. But we have huge problems when merging between branches. We do a big bang merge for every version that is shipped of to QA.
There are many times that the merge tool has managed to an automatic merge as it was simple, but the result was that we lost code that did not show until the customer sees it again. Many times the automatic merge has produced code that didnt compile, cut of half of a cs file.
Is the automatic merge in CC just plain stupid or is this a problem with all merge tools?
What is the best way to counter problems from merging?
Unit tests, but what happens if the unit tests must be merged? (This makes me believe that less code reuse in unit tests is better and is there is less to merge)
redsolo on October 3, 2007 03:51 AMBranching is sometimes a necessary evil, but it's still that one feature where you're likely better off never using it. Call me branch paranoid.
It's not even the concept that's difficult, it's the implementation. I can understand what I want to do; it's getting the marriage of source control ui's and development tools to also come together and see my point of view (without taking an eternity).
1. Do I completely trust my source control. If using VSS, then please answer no.
2. Do I trust other developers not to screw up. I'd like to say yes, but nobody's perfect.
3. Will source control allow me to do things the way I want?
Maybe it's just the terminology and the crappy tree-surgeon metaphor that confuses me. Branching is Hard (TM) - let's call it O(n^3) - easy enough on hello-world applications quickly ramping up to mentally exhausting as the project starts to get hard. (Yes we use agile, run tests and have build machines. )
minty on October 3, 2007 03:55 AMBranching is not evil. Merging is evil.
The solution to the branchingmerging problems is distributed source control systems.
I have to laugh. I'm in the middle of merge hell today caused by a mix of premature branching and source code reformatting.
I strongly believe that for branching and merging to work well a coding style must be rigidly enforced (preferably by the source control system).
Steve Bosman on October 3, 2007 04:42 AMIn the day job, we have a team of 40-50 engineers working on a common codebase. We maintain a branch for active development of future product revisions and a branch for each historical version that is in maintenance. This makes three branches. Additionally, for one customer we maintain a branch that mirrors their live environment for critical hotfixes that can not wait for a maintenance release to pass full UAT.
In a recent phase working on a new major release, a significant development was isolated to it's own branch. As work was active on the trunk, and a number of defects were raised by customers and fixed over multiple branches, the resulting Merge was a hideous nightmare and has now invoked "Merge Paranoia" at certain important decision making levels.
Branching is in theory great, but, if we're branching for more /production/ versions of our product, then that increases the overhead in any defect fixing massively. We have to have a build and test environment for each branch. The maintenance team have to merge and retest the fix in yet more locations, which decreases the number of issues they can resolve in parallel and affects our ability to hit SLAs.
If we branch for development to isolate a number of trunk based developments, we run into hideous merges that need to be performed. With a large codebase and development team, branching needs extreme planning and consideration.
THEMike on October 3, 2007 04:59 AMI might point out: for many corporate developers, branching is something of a solution in search of a problem. Everything I'm working on has *one* currently running version, in one environment. Usually with just one person working on *that* particular component at a time.
Heck, I've never actually done a branch. I keep getting tempted to just do one on my personal Subversion server just for the sake of trying it - except every time I think of it I'm kinda busy :p
Jivlain on October 3, 2007 04:59 AMEven that Conan story you reference was a second revision: http://the-isb.blogspot.com/2006/06/hypothetical-adventures-of-conan-part_19.html
Tom Clancy on October 3, 2007 05:23 AMNice article.
A previous employer now requires manager approval to make a branch, but thats not the only thing that off at that place.
I agree except on the branch per target OS. Yes, maybe while developing for that target OS, but your build system should be able to handle all target OSs from the same branch. Maybe its a bit zen like, but I enforce it on all projects I manage.
It keeps an OS particular branch from getting stale.
Jason on October 3, 2007 05:25 AMThe section on branching in Brad Appleton's ACME project http://www.cmcrossroads.com/bradapp/acme/branching/ is one of the best references available on the web, and his SCM Patterns book (http://www.cmcrossroads.com/bradapp/acme/#ScmPatterns) written with Stephen Berczuk should be required reading for anyone who's looking to "do version control" properly.
Brian Sadler on October 3, 2007 05:25 AMBranching is a solution to the problem Jivlain doesn't have - that of having several people working on the same thing at once. If your application is split into many discrete modules, and you have 1 person who works on each module, then you never need branching. You have a single-user control problem that by definition isn't a problem.
The next step downwards is to have some of those modules shared by other modules (eg a database schema that everyone uses). if one person changes the schema, everyone else has to upgrade to it, or their code will not work. This is still not much of a problem, you just need a bit of communication among your colleagues.
The next step though is where you have several people working of multiple modules at once, and this is where branching really makes sense. This is doubly true if your project is going to take a long time to complete.
In this case, you branch off, do whatever you want with a stable (to you) system, and merge your changes in when your system has been completed and tested. At no time do you break anyone else's code.
Using branching to handle released versions is a mistake IMHO, and is where Subverison falls down. I don't think making a branch that is designed never to be merged is a good thing, you should 'snapshot' the code for this using a Tag or Label mechanism instead (and use patches to update historical versions).
The time I used branches in anger was with a large application, we used the 'branch per task' model which worked well. It wasn't perfect, and we sometimes still had pain at merge time, but I think you wouldn't get any better.
AndyB on October 3, 2007 05:30 AMVery timely post!
Here where I work we are currently researching source control solutions, and have stumbled on the complexity of branching.
The overall negative responses here so far worry me a bit. Isn't there anyone out there with a positive experience of branch (and merge)?
Would I be right in saying that having automated builds and tests reduces the risk of big bangs, or at least lets you know sooner if something is broken?
We had a look at TFS, but given the price, and that (as I understand it) you can't visualise branches graphically, it's a no-no, at least for the moment.
The remaining options we a looking at are SVN and AccuRev. AccuRev does 'streams' instead of branches. Still trying to get my head around what the difference is - quite difficult, because as Jeff points out, even with branches there are n different ways to branch...
Ben on October 3, 2007 05:35 AMI like this article, which was in IEEE Computer a while back: http://www.accurev.com/product/docs/SCMBranchingModels1.pdf
And that's why one should use a branching based source control system like Mercurial! It's no publicity, but using that kind of software implies good habits, whereas using softwares like CVS implies missing some of the better habits.
loki on October 3, 2007 05:49 AMAt the outset, I should say that we're a contract development company working basically as an outsourced IT department producing custom software, with a few components that are reused between solutions which are run a little more like commercial software. Because the projects are relatively small, it's rare to have multiple developers working in the same codebase concurrently, and even then generally only for the first release or major additions of new features.
We operate like this:
All new development work is done in trunk. Each release is labelled.
On the rare occasions that a customer needs a patch to a previous version that has to happen out-of-stream, i.e. while developing the next major release, a bug is discovered in the previous release that can't wait, we branch at the label and make the fix in the branch. The check-in MUST only contain that one fix - keeping the check-ins small keeps the merge small. The fix is then merged into the trunk.
On even rarer occasions we've already found and fixed the bug in developing the new release, but the customer then asks for the fix earlier. Then we do the merge the other way, but it's less likely that the fix was isolated, so it may need the fix to be implemented manually.
If you're unlucky the bug is in an area that has been heavily revised so you have to work out whether the bug even occurs in the new code and if so, how to fix it.
Anything truly experimental is generally not checked in until validated. If it looks like it's going to need to be backed up - it covers more than a few days of work - or if the experimental work requires more than one developer, then we would use another branch for that, even if no development were happening in trunk. At some point the experiment will either get accepted and merged back into trunk, or get thrown away.
In a few cases we've sold very similar systems to customers, where we've developed it a little more proactively, and these are also organised as branches in case a fix is required in common code. Generally I'd prefer to make the common code a project in itself, with per-customer customisations, but that wasn't possible in the timescales agreed.
Tools: SourceGear Vault 3.1 with the companion SourceGear DiffMerge tool.
Mike Dimmick on October 3, 2007 05:49 AMThank you for this pedagogic presentation of branching.
What makes git so different (better?) than subversion on this aspect ?
Excellent post!
It seems many development houses approach source control in a haphazard way, and do not bother to learn about the power and dangers of various features, branching especially.
Another interesting guide to SCM and branching is from Perforce: http://www.perforce.com/perforce/bestpractices.html
I'd like to share a success story about branches, since so far it seems most commenters had negative experience with branching.
I used to work in a place that used ClearCase (and ClearQuest) for SCM. We had a branch per task system, with an "integration" branch which was the ancestor of all development branches. We had about 20 developers, and about 10-15 branches. We used this system for a few years (afterwards I left, they are still using it), and it was a great success.
Yes, there were troubles, and ClearCase is in my opinion a suboptimal tool. But!
We had many branches, and many merges between them (up and down the branch tree). Some developers worked on several branches. Some worked on just one. Some performed frequent merges (every week or two), some infrequent (every two months or so). Some developers shared tasks and
branches, some had a branch all to themselves.
Yet with all the chaos, the product (a very large, complex, embedded system) was progressing well. All the branches were almost always stable (it was a rare occasion when someone broke the build, but it happens). We had nightly builds of the integration branch. Through all the many times that source files had to be merged, we almost never had to perform manual merging, and I can remember maybe one time where an actual merge error was introduced.
Compare this to the alternative, and I say hurray for branches. They saved us from the sort of inevitable chaos that would have happened otherwise.
Of course, your tools need to be industrial strength for this to work. Don't even THINK of trying it with a tool that doesn't support atomic changesets, changeset tracking, hierarchical branches, etc.
Not all tools that claim to support branching do it well enough for branches to be useful. I know of one tool which supports branching, but its merges do not always work as advertised.
Branching and merging is no problem for our team since we have started using SnapShotCM ( http://www.truebluesoftware.com )
Based on my limited experience, distributed version control and Branch per Task are the way to go. Merges are not a problem when your tool is designed to facilitate them, especially if your branches are focused (in purpose) and short-lived. It's a matter of using one's tools properly. (And yes, sometimes that means not using them at all.)
Felix Pleşoianu on October 3, 2007 06:02 AMEric Sink, the founder of SourceGear, also wrote a good article on branching at http://software.ericsink.com/scm/scm_branches.html as part of his 'Source Control HOWTO' series.
Mike Dimmick on October 3, 2007 06:04 AMJust to go off topic a bit.
After reading your blog for over 2 years I never know you were a comic book fan. That is so awesome!!
...back on topic
My company is also researching source control solutions. We're a Microsoft shop so we automatically were going to use Visual Source Safe because of its integration with VS.
I wanted ask anyone here what they thought about VSS and there experiences with it? Also if anyone has any other solutions they recommend that plays well with VS?
Donny V. on October 3, 2007 06:07 AM> I strongly believe that for branching and merging to work
> well a coding style must be rigidly enforced (preferably
> by the source control system).
> Steve Bosman on October 3, 2007 04:42 AM
A friend of mine had what I consider a brilliant idea: let the version control system operate on the abstract syntax tree, and let each user check it out via her own formatter. There are issues, of course - like how to deal with code that doesn't compile, and how to keep comments in the mix - but I really hope someone puts such a system together.
Carl Manaster on October 3, 2007 06:25 AMDonny V.: For the love of your source code, don't touch Visual Source Safe with a three-hundred-foot pole!
Visual SourceSafe: Microsoft's Source Destruction System is an excellent article on everything that's wrong with Source Safe. Even Jeff's written an article on the subject.
Most source control tools out there provide some sort of integration with Visual Studio. If you're looking for something similar but less flaky than VSS, SourceGear's Vault is likely a good choice.
I'm personally a fan of Subversion. I'm not a fan of VS integration, but I know that there are plug-ins available; I just haven't tried them out to speak to their quality.
John Price on October 3, 2007 06:25 AMI worked for a medium sized software house who catered for different customers using branching. It was hell on wheels. Don't do it unless you have a compelling reason - even then I'd rather have if statements or something like that in the code base ... so when you're trying to test a problem for a specific customer you know you're testing the right thing.
I'd also use subversion, even with Visual Studio, Tortiose SVN gives you a nice windows front-end and it doesn't lock files or get lost and start daydreaming when you're in a hurry. VSS also didn't used to like lots of activity (as in 30+ people checking in), it may have been fixed.
Subversion has a lovely facility where you make a note of the check in number and can just say "give me build 24", or whatever. Great for testing or trying to work out when a bug was introduced. I think you have to annotate VSS by hand? Unless it's changed since I last used it, which is a long time ago.
Francis Fish on October 3, 2007 06:29 AMSorry, didn't realize my links would get stripped out (the bright red "no HTML" maybe should have tipped me off).
Visual SourceSafe: Microsoft's Source Destruction System http://www.highprogrammer.com/alan/windev/sourcesafe.html
Jeff's Article: http://www.codinghorror.com/blog/archives/000660.html
John Price on October 3, 2007 06:30 AM@Donny:
remember that VSS has a limit of 5GB of file changes database, you can go way over that but corruption will pester you quite frecuently.
Here we are using VSS with a combination of branching and pin&share. For each new version we branch, and if we fix a bug on a later version we can move the pin up for earlier versions.
If it is a MS only shop, go for TFS, if you can get away with it and you feel adventurous, give a go to subversion.
argatxa on October 3, 2007 06:39 AMDonny V. I recommend not using Visual Source Safe. We just moved from Source Safe to SVN. Using VSS we had multiple database corruptions and the whole process of pinning was very inefficient. We now use SVN with Visual SVN plug-in for visual studio 2005.
Read his previous blog: http://www.codinghorror.com/blog/archives/000660.html
Bobby on October 3, 2007 06:56 AMEven single developers like Jivlain can benefit from branching. Any time you want to experiment, start adding something new, anything, just make a branch. You can even switch back and forth between working on two (or more!) different features without having to worry about an incomplete edit screwing up your ability to compile and test in your other branch. I find this quite nice because sometimes I get a bit bored working on just one atomic task at a time.
Jake McArthur on October 3, 2007 07:13 AMI'm in a similar boat as Mike Dimmick. Four developers, all sharing a very large C code base for a financial application. 85% legacy code (> 15 years old), 15% "newer" code.
Anything experimental isn't checked in until it's at least proven harmless to the build and the basic functionality of the code. We are frequently in each other's code working on overlapping modules and features.
Long-term development projects are either done alongside of existing features and not "activated" until needed or are replacements for existing functionality and don't interfere until configured. Testing is very thorough and automated and done once a month whether needed or not -- though all done at the application level. Hardly any unit tests exist.
Once a month, on Staging Day, we branch into Release X from Main. Then the automated tests are run against Release X. If there are bugs found in the Release, we patch it, confirm the fix, and then merge just that change back into the Main branch. When the Release is delivered, we basically abandon that branch forever unless an emergency fix is needed during the month (rare).
We've done 60 releases over 6 years so far without problems. It's an effective system for a shop our size.
Im looking for a good, simple and free version control system for a small team (around 5 persons) that is compatible with Visual Studio (no Visual SourceSafe). Im going crazy trying to find something easy to install and configure to no avail.
Do you guys can give me a suggestion??
PD: Jeff, sorry if im derailing the discussion. If I am, please delete this post
Carlos G on October 3, 2007 07:19 AM@John P:
Thanks for the links. Didn't know VSS had so many issues.
@argatxa:
I just looked at Microsoft's site for Team Foundation Server and it looks like its using VSS as its source control portion. Am I reading that right or do they have a new version that's better?
http://download.microsoft.com/download/1/0/4/1046448f-3775-4c29-940c-4a2ac9ee5a02/Team%20System%20Datasheet.pdf
Looks like everyone favors Subversion using TortoiseSVN. Is Subversion hard to setup, maintain or use?
Branching is not evil, merging is not evil, SCM tools that can't do merging well are (creating branches is usually ok, making them interact with each others usually isn't, CVS can't do it, SVN can't do it, ... let's not talk about SourceSafe please).
That's why trying a distributed VCS such as Mercurial or Git is, in my opinion, a breath of fresh air: since they're *based* on the concept of merging (each checkout is basically an anonymous branch, each local clone of a repo is an other anonymous branch, and you can easily create branches of branches of branches), they *have* to have good merging models, or they just don't work.
And they do work.
> What makes git so different (better?) than subversion on this aspect ?
First of all, *it tracks merges*, which means that Git knows which revisions you have and which you don't (when it talks to another Git repository). Subversion doesn't, so you have to manually track revisions in order not to repeatedly merge them in, and it's error prone, and it usually doesn't work.
Second, when it merges two branches/trees, it doesn't take all the changes of one branch and put them in the other one in a single, ugly, horrible, unusable revision, it leaves the two branches in parallel and creates a "merge revision" in which you basically handle the disparities and incompatibilities, if any (most of the time, there isn't any and the merge revision is only there to say "this is the union of branches A and B at point C"). Plus that tracking allows you to create wonderful committrees such as http://repo.or.cz/git-browser/by-commit.html?r=git/repo.git
Third, it has very aggressive merge matching, which means that it resolves merges fast, it finds conflicts only when there *are* conflicts (usually) and it mostly doesn't find conflicts, especially when there aren't any, so merging between your branch and someone else's branch boils down to pulling his branch into yours and calling something along the lines of `git merge`.
Masklinn on October 3, 2007 07:29 AMFound a VS plug-in for Subversion...
http://ankhsvn.tigris.org
I thought the following blog posts about branching and merging would be of interest to you Donny.
Good luck.
Mark
Mark on October 3, 2007 07:44 AMWhere I work, after each release, the current code is renamed to a new production project. Upon the next release, the current production is renamed again with a release date.
Ongoing changes to the production code (the renamed project) that must also take place in the current development code are simply duplicated.
When I first started, I thought this avoidance of merging was silly. After doing it awhile, I think it's probably much easier than merging.
Great article. I've also found that companies rarely use branching. I suspect it is because management can easily understand the need to have a source code repository, but managing different lines of development is a bit more abstract and requires some rigor to do properly.
I have always wondered how companies get along without branching/merging - I would think that at a minimum, most shops will have to deal with a 1.1 production patch while a 2.0 release is under development. Even a one-man shop will have to deal with this situation at some point.
I think every development team needs to have at least one person who has the responsibility of managing the source repository and pays close attention to branching activity. It is even better if the team has some policy in writing about when branches and merges are performed, and the purpose of each branch. The branching/merging scheme affects more than just the developers - QA needs to know when merges are performed and if retesting is needed, and project management needs to know that time is being spent doing all of this.
Ken Liu on October 3, 2007 07:51 AMI belive team system has a new source control system, I don't belive it builds on VSS. [That said I haven't used it]
Subversion is a bit painful to install [on your server] if you want fancy things like SSL and to use your domain logins instead of logins in a text file...
Configuring clients is easy.
here's a vs extension for subversion: http://ankhsvn.tigris.org/
Sometimes people recomend you configure tortiseSVN to start the svn folder names with underscores to work well with asp.net, if you choose to do that you'll need to configure ankhsvn the same way if I recall.
I wish I could help you more, but I've only spent like a weekend playing with subversion, my work isn't about to change to something else even if it is better and free.
[Oh, the book pragmatic version control using subversion is good, but somewhat assumes you haven't used source control at all before. And for brevity/compatibility it describes the command line tools, which isn't a problem because graphical tools use the same language typically. The installation chapter is really helpful though, and so is the reference. ]
Sven on October 3, 2007 07:56 AMyes yes yes use git, bzr, mercurial (hg), etc. - subversion and other centralized systems are broken by design when it comes to merging.
Baczek on October 3, 2007 08:20 AMIt was a nice article. Actualy Visual Source safe was the only tool i have used as a source control. It's a very small tool and has many limitations. Even till Visual Studio 2003 they hadn't improved anything in the area of source control. When I got a chance to work at my client office... The Clear Case source control system was a fantastic new experience for me. Even it has a small learning curve, it's really nice system which help to smoothen our development especially dealing with a gigantic source structure. It's simply awesome. Thanks for such a nice article.
Sarath on October 3, 2007 08:44 AM>if your application is split into many discrete modules, and
> you have 1 person who works on each module, then you never
> need branching.
not true
branches are the perfect tool when you're developing an application, and continue to make support fixes to the previous version. This is probably the most common scenario.
Companies that don't use SCM properly will do things like make a full copy of the source code in a network drive.
In some very trivial case you can get away with just labels, and move labels when a fix is made, but this really doesn't work in the long run.
>Many times the automatic merge has produced code that
>didnt compile, cut of half of a cs file.
>Is the automatic merge in CC just plain stupid or is
>this a problem with all merge tools?
ClearCase's automatic merge is excellent, and well proven.
If you're having trouble with this, it's probably that
>Many times the automatic merge has produced code that
>didnt compile, cut of half of a cs file.
>Is the automatic merge in CC just plain stupid or is
>this a problem with all merge tools?
ClearCase's automatic merge is excellent, and well proven.
It remembers what's been merged and at what point, that's very important for good merges in the future. ClearCase Details is the tool to master here, for tricky merges.
If you're having trouble with this, it's probably that you're doing too drastic or conflicting changes between branches. In that case, merging can't be expected to work too well.
For example, if a dev in your team has the habit of moving code around in a file, or changing the coding standard for fun, this will cause major merge problems if you have concurrent branches of the same files. Obviously. If that's your environment, you have to code with merging in mind.
In my experience I never had problems with branching with Subversion. For branching to actually practically work you have to do whatever it takes to avoid problems with merging. That means you have to merge frequently (continuous integration anyone?) and enforce a coding standard. If not then chances are that you will end up with an unmergeable hell.
A professor in configuration management once told me that tools are not the problem and people using them in the wrong way are. The things he saw what developers did was using branches for other reasons than parallel development which left the branches unmerged.
Then again, if git and whatever actually enforces merging, then probably tools are the problem (speaking from a usability standpoint)... Again, unless people use the tools in the way that was not intended.
Spike on October 3, 2007 08:56 AMAnother reason to really dive in and understand branching and merging is that anytime you check out code to your local machine and start working on it there, you've created an implicit branch.
http://haacked.com/archive/2006/06/02/ImplicitBranchingAndMerging.aspx
You're already branching and merging, so might as well understand good patterns for doing so.
Haacked on October 3, 2007 09:13 AMThose alternate timeline branches are childsplay compared to Primer (http://www.imdb.com/title/tt0390384/). Check out this timeline chart: http://www.freeweb.hu/neuwanstein/primer_timeline.jpg
I'm going to go out on a limb and say that's not too huge of a spoiler, because it's really hard to follow even if you've seen the movie. The thing that's cool about it is that the writer's thought through things like avoiding pesky time travel questions most people gloss over. Example: if you went back in time 24 hours, you'd be floating somewhere in outer space since the earth would have moved on.
As to software branching, it got to be a big deal at a previous job at a financial company. We used StarTeam, which was pretty hard to use but did handle branching well. I agree with everyone else who's suggested Subversion - it's by far the best source control I've ever used, and just happens to be free.
Complex branching scenarios start to get complex enough that they require a substantial amount of time and planning. If you count all the time required to handle different branches (including meeting hours), I'd say that the development team of ~30 devoted the equivalent of 2 or 3 full time resources to managing it.
There are many times when it's preferable to work off the same branch and use configuration, resource dependencies, and other software tools to solve the problem rather than trying to branch and merge via source control.
Oh, more lightweight source control solutions like tagging (http://svnbook.red-bean.com/en/1.1/ch04s06.html) and shelving (http://blogs.vertigosoftware.com/teamsystem/archive/2006/01/18/Shelving_and_Branching.aspx - on your Vertigo blog) are often preferable to making an entire branch to separate a feature.
Jon Galloway on October 3, 2007 09:19 AM> For branching to actually practically work you have to do whatever it takes to avoid problems with merging.
And that's the core problem.
You shouldn't have anything to do to avoid problems with merging, the tool should take care of that.
Subversion, of course, doesn't.
Masklinn on October 3, 2007 09:26 AMWe have SVN and what we do is have a "current". When there's a complex new functionality, we branch for that, but the guy in charge of that must make sure that he portes and merges when needed. Basically, what he does, is porting from current to his/her branch. That way, the later merge (from the branch to the current) is not difficult (in theory).
When we release a version, we just create a TAG (in SVN tags and branches are the same). So we have a rigid structure of "release/version/"
We also move that "release" to a stable "branch".
We continue to add new stuff to current. But when a Bug is found, it's either applied to stable and later ported to current or viceversa. Stable always compile. When enough bugs have been fixed in stable, we might release a "minor" revision. Again, this gets "tagged".
When Current is ready for a new "major" release, it goes to "stable". It gets also tagged as release-xxxx and the process is all over again.
The day we ran out of bugs and new features. We'd need to find a new job. ;)
So far this has worked fine por 95% of the merges. The problem is that VStudio and the .NET tools in general do not tend to play nice with Source Control systems. Too many user files you have to exclude and stuff like that.
But so far, it's nice to have a subversion under the hood. We don't use ANKH (dunno if it has been updated) but we use TortoiseSVN and do all the stuff from the command line.
My two c.
Martin Marconcini on October 3, 2007 09:35 AM> anytime you check out code to your local machine and start working on it there, you've created an implicit branch
This isn't true at all-- your local work is not revision controlled, it will have no history, so it's not a branch. A branch is an alternate set of history. No history, no branch.
> You shouldn't have anything to do to avoid problems with merging, the tool should take care of that.
> That's why trying a distributed VCS such as Mercurial or Git is, in my opinion, a breath of fresh air .. they *have* to have good merging models, or they just don't work
How exactly do these distributed version control systems make merging easier, though? I think merging two people's independent work is hard no matter how you slice it. This sounds awfully silver bullet-ish to me. Can someone explain this better?
Jeff Atwood on October 3, 2007 09:43 AM> This isn't true at all-- your local work is not revision
> controlled, it will have no history, so it's not a branch. A branch
> is an alternate set of history. No history, no branch.
That's true if your local environment does not have local history. That's limitation is becoming pretty much just an MS thing now.
Eclipse has had local history for years, and Netbeans 6 does as well. :)
Jess Sightler on October 3, 2007 10:24 AMHave you seen this video? http://www.youtube.com/watch?v=4XpnKHJAok8
Linus Torvalds is sharing his opinions about SCM, branching and merging. It's brilliant.
Tapio Kulmala on October 3, 2007 10:43 AMThere was this video with Linus Torvalds talking about git, and explaining the philosophy behind it.
http://www.youtube.com/watch?v=4XpnKHJAok8
I can't remember the details, but the idea is that if the tool's philosophy is that everything done is a merge, and that you can work "offline" without connecting to a central server, then merging becomes much easier.
I am not convinced that merging is particularly hard for strong tools, whether centralized or not. My experience with CC was that merging is pretty effortless. Google have thousands of developers all over the world, and they seem to be quite happy with Perforce. Both of these tools have centralized servers.
Maybe it's because we employed some discipline (not that much) when merging, though.
M on October 3, 2007 10:44 AMI just searched for other YouTube videos about SCM. This one is histerical. Historical? Well, you be the judge.
http://www.youtube.com/watch?v=vpILRv_tZy4
Jen
Jennifer on October 3, 2007 10:57 AMI think the real trick here is knowing when to branch and merge. Branching works great, its always the merge that gets you. Try and have developers not to work on the same code files between branches, if possible. If not, sometimes the automerge fails and manual merge is required. This can sometimes be painful, especially if you are doing the manual merge and did not implement the changes. Even with manual merge, tools are decent to help you out identifier and merge changes (GUIFFY, FileDiff, etc.)
Jon Raynor on October 3, 2007 11:46 AMOne branching mistake you left out of your Anti-Patterns that I have had to deal with is developers copying code to make a "branch", but they don't set it up as a branch in the source control system.
Depending on which Source Control system you have this can be a huge problem. Visual SourceSafe, for example, has significant problems with this when it comes to merge time. You have to trick it with unsupported command line calls to make it think that the copied files should be "Merged" into to the main line.
Stephen on October 3, 2007 11:55 AMJust an FYI, Microsoft's Visual Studio Team Services is a COMPLETELY DIFFERENT source control system than VSS. It uses SQL Server to store revisions, has all the goodies for branching and merging, and even includes shelve sets for getting local changes put into a shared storage area without affecting the main source control. Once one gets used to it, it's very easy to deal with.
YMMV...
David Scheidt on October 3, 2007 12:23 PM"A friend of mine had what I consider a brilliant idea: let the version control system operate on the abstract syntax tree, and let each user check it out via her own formatter. There are issues, of course - like how to deal with code that doesn't compile, and how to keep comments in the mix - but I really hope someone puts such a system together."
I can up with a similar idea when there were discussions about the possibility of making Visual Basic keywords lower case. To avoid conflicts between machines files would be saved with upper case keywords, but if the preferences were set so they would be displayed with lower case. The same could be done for a variety of style options.
[ICR] on October 3, 2007 12:32 PMAt work, we use ClearCase. Creating a branch is difficult, time-consuming, and error-prone. Merging is also difficult, time-consuming, and error-prone. All of these are faults of the tool.
I moved my development to a Mercurial repository -- I use a separate branch for each major task, and employ a couple of scripts to keep things synced with my colleagues' ClearCase checkins.
I've never looked back.
Merges now take mere seconds and are conflict-free. The SCM tool cleans up my mistakes, rather than the other way around. Best of all, I don't need to beg permission from the tool before editing a file.
Echoing David's comment on TFS/Team Foundation Server...I've used VSS, CVS, StarTeam, and TFS. TFS is the real deal...it's the source control product the MS SCM world always wanted.
While other SCM products integrate with Visual Studio.Net, the real power of TFS is its OOtB integration with work item tracking (tasks, bugs, etc.), unit testing, analytics, builds, check-in policies (e.g. can't check in without passing all unit tests, without a comment, etc.), and project management. When implemented correctly, your PM will never again bother you with Bill Lumbergh inquiries about your percent complete on tasks.
Jeff Hunsaker on October 3, 2007 01:02 PMNice article.. well written.
Kashan AHmad on October 3, 2007 01:10 PM> This isn't true at all-- your local work is not revision controlled, it will have no history, so it's not a branch. A branch is an alternate set of history. No history, no branch.
Sorry for the harsh tone Jeff, but frankly, please talk about DVCS once you've used them.
Local work *is* revision controlled and *does* have all the history you want, and you can branch it further by creating clones of your local working copy. In a DVCS, each working copy is a repository.
> How exactly do these distributed version control systems make merging easier, though? I think merging two people's independent work is hard no matter how you slice it. This sounds awfully silver bullet-ish to me. Can someone explain this better?
I'll just redirect you to the "official" documentations.
Here's the relevant part of the Mercurial Book, http://hgbook.red-bean.com/hgbookch3.html#x7-530003
Didn't find anything as clear cut for Git, it works pretty much the same way as Mercurial except it's faster, and Git tracks the whole tree as one single blob instead of tracking individual files.
Anyway yes merging can be hard when there's a conflict, but it should be trivial when there isn't.
DVCS make the "no conflict" merge pretty much automatic and painless, and make it much easier to deal with the "with conflict" merge. And keep full history of all the merged branches.
Crap, thought the first quote was a response to the first part of my previous post, but after posting it (...) I checked and it wasn't.
Sorry Jeff, I'm the stupid guy.
Masklinn on October 3, 2007 01:15 PMA few people have mentioned that they don't trust their source control and therefore won't branch/merge .. I'd say if you don't trust your source control then you need new source control. You may as well not use it in that case...
We use SVN and branch/merge all the time with little or no problems. This is for a very large web project working with many types of files and at least 4 or 5 people at all times. We have many new features in dev at once all slated for different release dates, that's why we need branching.
If I need to start dev on a new feature now and it doesn't launch for three months, we have to branch. That way we can launch other features in the meantime and merge my new feature in when it's ready for release.
SVN handles all this perfectly...
Matt Nish-Lapidus on October 3, 2007 01:23 PMAt my current place of work, we use Changelogic ( http://www.changelogic.com/HomePage ), which somewhat automates branch and release management for SVN and/or CVS.
If you work with Changelogic, THERE IS NO HEAD. All the work is done in branches -- called "changes" -- and when a change is integrated, it is merged with the version (tag) it was based on, to create a new version (tag). Each developer usually works in his own branch, but sometimes 2-3 people work together in one.
The version numbers (like 2.3.1.15) are managed automatically. For example, you can do some changes based on the 2.3.x versions, but other changes in the 2.4.x versions. The changes based on older versions will also be integrated forward into the new versions. This is actually one of the few annoying bits of CL.
Usually you just create a new change from a task/issue/ticket (pretty much automatically) based on the latest version and check it out using the correct change number. After you're done, you commit it (actually it's called "end_edit"), have someone review it and then integrate it with the previous version to create a new version. If versions were added by others in the meantime, you usually have to merge those changes into your branch (pretty much automatic) before integrating.
I think this mode of development is a great relief, as it almost frees you from having to think about managing branches yourself. But it can also lead to Branch Mania if you're not at all paying attention to what you are doing. We did wander into branch mania when we started with the tool, but learned since then to keep changes/branches more coarse grained.
Villane on October 3, 2007 01:55 PMNice article -- I recently wrote about version control, which beginners may find useful(http://betterexplained.com/articles/a-visual-guide-to-version-control/).
Inside of Microsoft, each team has its own branch which is eventually merged into the mainline (reverse integrated). Changes are also propagted from the mainline to the branches (forward intergrate).
Kalid on October 3, 2007 02:37 PMJeff: merging gets hard when the merges are big. Small merges are almost always easy, because there's little there to conflict. Without history in a local checkout, you have a big pile of changes that get applied at once (at pulling from the mainline into your local checkout is essentially a merging the main lines's changes into your changes.) Most distributed version control systems take the tack of making the local checkout a full-blown repository, giving you all the revision control goodness for local changes, before they're committed to the main line. This encourages you to break all the changes you are working into individual changes, so only the actual conflicts conflict, rather than accidental conflicts. But even without this history, the merger is just a degenerate case of one change getting merged.
Aaron Denney on October 3, 2007 03:52 PMJeff,
have you ever used branching AND merging in real production projects? Ever?
I guess not. Otherwise you would mention in your post tremendous complexity associated with merging branches.
That's very rarely worth an effort to branch and merge projects.
Almost all the time it's better either to keep everything in one project. Even if branch, you'd better never merge.
And if you never merge (because it's too complex) -- why not just cut&paste new project into fully separate project?
I have two rules of thumb for branching:
1) A new branch for each version (if each client has a different version, so each one has a branch)
2) A new branch everytime more than one developer have to work in the same complex feature (simple features don't need more than one developer). The branch is used for communication between them.
If automatic merges are failing in ClearCase you are merging between branches that are not compatible or the engineers on that other branch are sabotaging you. We had this happen when we had a team who would take a merge rewrite or delete our changes and check in the change. Then when we merged back to get their changes all of our code disappeared as part of the merge.
You also have to take a look at the hyper-links--the merge arrows in the Version Tree. A merge arrow says that the merge was completed. Sometimes we will perform a merge and delete the merge arrow because we took a partial merge to fix a specific problem. When we delete the merge arrow, we can do that merge again and get what we didn't take.
The merge tool is looking for changes made in the source branch and merging these changes into the destination. Delete a line of code in the source and it will be deleted in the destination during the merge.
If the problem is with reformatting the code (tabs or spaces) you can set a property in the merge tool to ignore whitespace. Unfortunately, I don't think it is a global setting. You have to do it manually on each problem file.
The merge tool from the command line also supports taking the changes from one version of a file. This can be a lifesaver when you need to bring
asm on October 3, 2007 07:48 PMHaving been the CVS guy on 3 seperate dev teams and, along the way, having actively developed advanced environment management techniques, abusing CVS heavily, I have had my share of real-world experience.
In my experience,the best SCM techniques are discipline and communication.
The former in the form of regularity in branching/merging and the reasons why, not modifying code layout for the hell of it or (more typically) to inflict one's personal style (or belief system).
The latter on all occasions where one might create merge conflicts; beforehand (a certain amount of archtecture and planning to avoid everyone and their dog necessarily modifying the same six files every two days) and during development, at those times when well-planned interfaces prove inadequate, borken or otherwise deficient. Don't just sit on your branch and pretend the rest of the world doesn't exist; if it looks like you might be stomping on someone's toes a week from now, ask them where they think they might be putting their feet.
Note: this is no guarantee or a fail-safe method; it merely sensibly avoids creating the bulk of unnecessary conflicts.
That all said, I am turning around to see the error of my ways wrt my tendencies towards wanting to run The Department Of Centrally Controlled Source Control And Build Management For The Benefit Of All---DVCS are just sooooo much more sensible.
Schmolle on October 4, 2007 12:06 AMOn a previous project I was working on, at the site of the subcontractor we were not allowed to branch: too difficult.
Well, after a month they saw that this wasn't workable anymore, so we could branch... but only files, no directories.
Again too difficult, but again after a month we could also branch directories. (what's so difficult about merging a directory?)
So here we had 'experienced' SCMs saying that we could not branch files and directories, although we were doing this on our own site already for years. I still remember the shouting match between our SCM and theirs, ours won :-)
The branching system we used there (embedded software):
- one main branch that continues forever
- a branch per product group
- a branch (of the product group) per product (made a few months/weeks before its release)
- all branches always have to be merged to the main branch
- main branch is not merged to side branches
Never had any problems with it as long as you keep a clear overview of all branches and communicate this to everyone!
I think that is the most often overlooked thing about branches: communication! What is the branch for, who can work on it, from/to where to merge, ...
Tool we used btw, was CMSynergy/Continuus.
A very good merge tool is Beyond Compare.
Somewhat of a newbie - but here's my 2c...
As I work, I check in often via the Eclipse shell to CVS. I also update often. Several times a day sometimes.
Occasionally there are code conflicts, but these are not big - how much can a team change in a few hours, after all?
Several times a day, the Test runner sends out an email informing us whether the build passed or failed. Personally, I don't trust my own code at all unless I've written a few tests for it. These tests are also checked in, and the test runner picks them up.
This 'continuous' checkin+update+test process ensures that things work well...
Although admittedly, the team is still small. Don't know how well it would work if there were 40 of us all at the same time...
Zot on October 4, 2007 01:40 AMAnother very useful feature of CMSynergy/Continuus that I haven't seen used anywhere else is task based development.
See also http://www.cmcrossroads.com/articles/cm-journal/scm-patterns:-building-on-task%11level-commit.html
This was a big change in our organization because it's not efficient to mix task based with file based checkouts.
A task can be a normal task that you would give to a developer, but also solving a particular bug, etc.
Making a build "with that and that bug solved" is very easy then, just take a base and add the tasks you want.
I don't know which other tools support Task Level Commits.
@Jon Galloway
>Example: if you went back in time 24 hours, you'd be floating >somewhere in outer space since the earth would have moved on.
This just show how confusing parallel timelines can be and how carefully you have to think about them.
You should have said either...
Example: if you went back in time 24 hours, you'd be floating somewhere in outer space since the earth would not be there yet.
...or...
Example: if you went forward in time 24 hours, you'd be floating somewhere in outer space since the earth would have moved on.
Sorry to be the pedant, but this is typical of the kind of error found in sci-fi movies - simply because somebody had an essentially good idea but didn't think it through.
;-)
Bob Armour on October 4, 2007 02:03 AMWhat kills me about merging is that every mistake you make is a bug. And it's probably a regression, which looks the worst.
smackfu on October 4, 2007 06:05 AMgit looks pretty awesome, from the perspective of making your code look like a london tube map. ^.~
"Eclipse has had local history for years, and Netbeans 6 does as well. :)"
It helps, but it's not as helpful as a local repository (centralized or distributed). It doesn't help smooth painful merges or give anyone else the ability to view your change history.
Foxyshadis on October 4, 2007 06:34 AMRegressions are definitely one of the biggest risks when merging. Your customers will not appreciate that.
Personally I don't see what's wrong with tagging releases in SVN. I understand that it's a bit like using a sledgehammer to pound a thumbtack into your cork board, but it does the job just fine. And who knows, you might actually want to merge it someday if you discover some crippling bug that affects the 12 most recent versions and a bunch of users are still using the old one. If the tag is technically a branch, then it will save you a lot of last-minute scrambling.
Maybe some SCMs think of tagging as just slapping a label on a particular item in the history. That's fine too, but to me it's only a superficial difference. I think it's an OCD-like impulse, getting upset that there's some awful dangling branch that will probably never be merged. Who cares - I know what it's for, that's why it's in a "tags" directory.
Aaron G on October 4, 2007 06:41 AMSorry dude, you lost your argument by invoking Godwin's law! ;)
Jabosh on October 4, 2007 08:12 AMWe use Perforce at my shop, and have a semi-unique system due to the way our product (embedded software in lots of hardware) is. We've got a few configurations, all similar, all to different customers, and the customers are the type that want to know what the changes are if they get a new version, and don't want anything too different than the last one, even if it's "better".
We rely on branching to let us do our development for a specific project on a "development" branch, maintain a "release" branch for each project, and have a "mainline" where we try to collect all the latest things from every single project, so that when we start a new project, we have the sum total of all the newest stuff that we've done.
This relies on good developers communicating, as well as making sure that they're always adding features. Feature B gets added to module 1, so module 1 has Features A and B, rather than module 1 being changed to be B not A.
Our revision graphs in Perforce are truly amazing things to behold, but it works well, and we've support numerous products at varying states of release on different hardware with a small number of developers with a minumum of chaos.
Alex on October 4, 2007 12:31 PM>> "Eclipse has had local history for years, and Netbeans 6 does as
>> well. :)"
>
> It helps, but it's not as helpful as a local repository (centralized
> or distributed). It doesn't help smooth painful merges or give
> anyone else the ability to view your change history.
yes, local history is not a replacement for GIT (or other distributed version control solutions). But my point was that local code can and should be versioned in some way.
It's amazing how convenient being able to look over the diffs in your code from today can be. :)
Jess Sightler on October 4, 2007 12:52 PMI'm a first-time commenter here, but just wanted to say what a fantastic article this is.
Branching is a very much misunderstood, and mistrusted version control tool. I used to use Visual SourceSafe's branching functions when I was a developer, but there was great mistrust of it.
Some things never change!
Brian Heys on October 4, 2007 03:00 PMThat's because VSS's "branching" concept is a truly horrible thing that should be mistrusted.
Alex on October 4, 2007 03:44 PMJeff,
actually the first branching pattern you display (Branch per Release) is - as displayed - one of the worst to start with (read Appleton and related posts). Please update and replace it with a better one!
We do branch for each version, then do bug fixes in that branch and merge them back into the trunk (or development branch). It works quite well, the merging is usually done in a relatively short time and hardly causes problems. Just don't wait too long with it!
Werner on October 4, 2007 04:43 PMI have to agree with all the other fans of distributed CM systems. When branching and merging is cheap, fast, easy, and doesn't need to go thru a central chokepoint, it becomes much much more useful.
I use Mercurial myself.
Mark Atwood on October 4, 2007 05:20 PM>We do branch for each version, then do bug fixes in that branch and >merge them back into the trunk (or development branch). It works >quite well, the merging is usually done in a relatively short time >and hardly causes problems. Just don't wait too long with it!
This is exactly what we do and I find it works really well - development can continue on the trunk, and hopefully if no bugs come out in QA process we have no merging to be done backinto the trunk.
Using an automated build tool can help ease the pain of using branches. It is also important to merge often, not based on time, but on the amount of coding being done.
Mark on October 5, 2007 06:13 AMWe've previously explored the topic of branching not unfortunately have not come across a concrete strategy due to the nature of our development, but this post has certainly got us thinking, probably along the task-orientated way.
I would ask though, if anyone has any advice or experience on branching strategy for an example situation:
10+ web sites, each using common core functionality across many class libraries. When a change request comes in for WebsiteA and it is branched (let's say WebsiteA-Change1056) a task-branch works great.
Our strategy has faltered when as part of the change 5 of the common libraries also have to be updated and improved.. they need a branch also in this scenario.. but this could occur for ALL changes and could very easily get ugly.
Defining a good consistent branching strategy seems harder when you right at the start and currently using the unbroken line of development on shared common code.
biscuit on October 5, 2007 07:51 AMJeff, you forgot to include the "Mainline Model", Perforce's recommended branching practice:
http://www.perforce.com/perforce/bestpractices.html
It's like your "Branch per Release" (which Perforce calls the "Promotion Model"), except the branch you create is an archive of the released Version N. The Mainline marches ever forward to Version N+1.
And as Mike Dimmick mentioned above, you can label releases on your Mainline. You can branch retroactively only if you actually need to create a hotfix or service pack (e.g. Version 1.0.1).
Chris Peterson on October 5, 2007 11:23 AMThe Mainline Model is also nice because you never have to freeze the Mainline. When you have a Release Candidate, you can label/branch it and continue fixing bugs in the Mainline. If you find critical bugs in the Release Candidate, you can fix them in that branch.
Chris Peterson on October 5, 2007 11:29 AMRe: biscuit's question about branching multiple common libraries for a single change.
In our organization, we run into a similar issue. We follow a mainline model, and branch-per-task.
The way we have addressed it is to branch all of the libraries which might need changing into a working folder together with the main application.
Thus, we might take code from multiple trunks:
/cpp/lib/lib1/trunk
/dotnet/lib/lib2/trunk
/dotnet/app/MyApp/trunk
and branch each one to a working folder:
/dotnet/app/MyApp/branches/mybranch/lib1
/dotnet/app/MyApp/branches/mybranch/lib2
/dotnet/app/MyApp/branches/mybranch/MyApp
Thus, any work that requires changing multiple libraries can be handled in a single branch of the app, and merged back to it's multiple trunk locations when the branch work is done.
We use an ant script per app to take care of coordinating branching/merging the required libraries along with the app.
Since most changes don't require modifying the common libraries, it means we branch more than we probably need to, but it allows us to change multiple projects as part of a single change set when we need to.
Tim Keuning on October 8, 2007 08:53 AMI'd say a better term for Big Bang is Big Crunch. Here you can draw an analogy between the size of the universe and the number of branches in existance. It
Meng Mao on October 8, 2007 10:47 AMYou hate Branching And Merging
But You Can't Live Without Branching And Merging.
It's a Black Art and every software project needs it. It's very difficult to get right, because it's difficult to forecast how a project will look like in a year. The complexity increases as the number of products * the number of separate branches are needed * the number of releases that needs to be maintained * number of components shared between branches/products.
=== Branch/Merge Hell.
Kashif
Kashif Shaikh on October 8, 2007 01:21 PM> How exactly do these distributed version control systems make merging easier, though? I think merging two people's independent work is hard no matter how you slice it. This sounds awfully silver bullet-ish to me. Can someone explain this better?
@Jeff, did anyone ever explain this to you? I've watched Linus' git video, and the Mercurial google talk too. I've also used Mercurial locally, but not with other developers. I'm starting to see the benefit and paradigm shift of DVCS, but I'd like to hear an experienced person's explanation. Make me a believer!
Ben T on October 9, 2007 07:07 AMIn the company I previously worked at, we were using VSS. It was ok for small projects. And we had typically one database per project. And we had to develop so many periferal tool (Bug Tracking, Daily Building, Version Building, etc.).
Where I work know, we have been using TFS for 2 years. We tested SVN and even began the deployment before shifting reverse and going for TFS. SVN & Tortoise were not bad at all, but just not as complete & integrated than TFS. On the other hand TFS is costly (much $$$) but I guess not more than Perforce or other commercial stuff.
Also, I am now using branches with TFS. Every major version gets its own branch from dev (1.0, 1.1, 1.2, 2.0). Minor revisions go in the same branches (1.1R0+1.1R1+1.1R2, 1.2R0+1.2R1). We found this a good balance between branches & merges. It saved us a few times when the dev root had many new features not ready for any kind of release, but still needing a version NOW with a bug fix or a new small thing.
The coolest thing with TFS is that it keeps track of what was changed and not merget yet. I got complex merge sequences and still it was able to keep track of this. I should say, to be honnest, that once TFS losts its 'pointers' and lost track of my merge state... But I was really really doing scary and not-catholic things, I guess I diserved it.
For the first 30-40 merges I was paranoiac and checked what TFS did to the files. Now I am more confident. I typically review only merges that had conflicts resolution.
The other nice thing with TFS is that it integrates bug tracking & development tasks. I can find, which merges are required for which bug. This is great. And NO, TFS is not based on VSS. This is all new stuff.
philibert on October 9, 2007 07:23 AMWe use ClearCase. We have a main branch used for releases. When we start a project phase, we create a new branch for it. The phase branches are merged to the main branch for each release. Every software change (enhancement, bug fix, etc) is done in a separate development branch and merged to the phase branch.
Although we do have some merging problems (mainly when a dev branch has been in use for too long), this setup definitely works fine for us and provides enough isolation and flexibility for concurrent programming.
We manage to keep everything under control (and to keep antipatterns off) using these techniques:
-Automated branch generation. This keeps a naming convention and keeps the branches linked to the change request that originated them. We don't have misterious branches and we always know what a branch is for and who is working on it.
-Automated builds and merges. The build includes automated unit and integration tests and a mere is done at the end automatically if everything is OK. Nothing gets merged the build breaks and that prevents having those problems due to programmer mistakes.
-Automated releases. Everything is locked and tagged automatically.
-Each project has an administrator. This is not a full-tim job, I would said it take 20% of his/her time, and we rotate that responsibility around. The job is basically to serialize the tasks that require it, mainly the releases, and to move whatever cannott be automated in the life cycle of change requests.
-This is a very important point: our change requests are usually very limited in scope. We have many change requests but each is small. One of the positive consequences this has is that most merges are automatic.
I am not saying this is a perfect setup and of course I don't think it is the best for all situations. Nevertheless, it has a bunch of ideas that could be of general use because they keep the complexity in check. These ideas could be summarized as:
-Reduce the universe of options while taking advantage of source control branching.
-Have a consistent branch-merge cycle.
I was not involved in creating this setup and I understand it took a lot of effort to get it to this point.
Arturo on October 9, 2007 08:29 AMDon't let changes to accumulate before you merge.
Test on October 9, 2007 10:18 AMBrillant stuff!
Ray Kinsella on October 10, 2007 06:52 AMGood article.
I'm surprised to see only one reference to Accurev. Streams are a bit different to think about at first but well worth the effort as they make branching/merging dead simple. Subversion is imho not suited to parallel development and should be avoided if you anticipate ever needing more than one line of code.
Mike
Mike Coon on October 30, 2007 02:29 PMEven for long lived branches (for example feature branch for experimental feature which takes number of commits / revisions and long time to develop) one can avoid Bing Bang Merge at least in one of those ways:
1. By doing merge of trunk (mainline) branch into feature branch, resolving conflicts (if needed); if version control system make it possible sometimes it might be better to do trial merge, i.e. either discard merge at all, or record merge as 'mainline compatibility fixup' commit but not as a proper merge.
2. By doing trial merge of feature branch to trunk; discarding it later. SCM must make correcting and removing latest, not yet published commit possible.
3. By rebasing feature branch on top of mainline, just like it was a series of patches to apply on top of new mainline (see description of git-rebase in Git documentation for example).
If one says that branching is bad then perhaps SCM one used does not have easy branching (and merging).
Jakub Narębski on November 18, 2007 06:29 PM| Content (c) 2008 Jeff Atwood. Logo image used with permission of the author. (c) 1993 Steven C. McConnell. All Rights Reserved. |
Subscribe in a reader
Subscribe via email