What's Your Backup Strategy?

January 28, 2008

Jamie Zawinski's public service backup announcement starts off with a bang:

Option 1: Learn not to care about your data. Don't save any old email, use a film camera, and only listen to physical CDs and not MP3s. If you have no possessions, you have nothing to lose.

This is obviously meant as satire, but it's disturbingly close to reality for me. I suppose everything in my life that's worth capturing... well, let me put it this way: you're reading it. When I said make it public, I really meant it. Still, I'm fairly sure Jamie was kidding, and while Google may be a great service, it's only a so-so backup mechanism. Let's proceed to option 2, which goes something like this:

1) You have a computer. It came with a hard drive in it. Go buy two more drives of the same size or larger. If the drive in your computer is SATA2, get SATA2. If it's a 2.5" laptop drive, get two of those. Brand doesn't matter, but physical measurements and connectors should match.

2) Get external enclosures for both of them. The enclosures are under $30.

3) Put one of these drives in its enclosure on your desk. Name it something clever like "Backup". If you are using a Mac, the command you use to back up is this:

sudo rsync -vaxE --delete --ignore-errors / /Volumes/Backup/

If you're using Linux, it's something a lot like that. If you're using Windows, go f*ck yourself.

Yeah! Take that, Windows users! Hey, wait a second. I use Windows. Did I mention that Jamie is a funny guy? Moving on.

I've long been a fan of inexpensive hard drive enclosures. Jamie's advice confirms my long held opinion that multiple hard drives are the most effective and easy backup process you'll ever find. The rsync command is more than a simple copy; it actually does a block-by-block comparison, only copying the differences. So instead of backing up the entire contents of your hard drive (again), you only back up the parts that changed since your last backup. This is commonly known as incremental backup.

Incremental backups only have value if you're doing them regularly, so it's only natural to schedule this as a recurring task.

4) If you have a desktop computer, have this happen every morning at 5AM by creating a temporary text file containing this line:

0 5 * * * rsync -vaxE --delete --ignore-errors / /Volumes/Backup/

and then doing sudo crontab -u root that-file

If you have a laptop, do that before you go to bed. Really. Every night when you plug your laptop in to charge.

5) If you're on a Mac, that backup drive will be bootable. That means that when (WHEN) your internal drive scorches itself, you can just take your backup drive and put it in your computer and go. This is nice.

6) When (WHEN) your backup drive goes bad, which you will notice because your last backup failed, replace it immediately. This is your number one priority. Don't wait until the weekend when you have time, do it now, before you so much as touch your computer again. Do it before goddamned breakfast. The universe tends toward maximum irony. Don't push it.

7) That third drive? Do a backup onto it the same way, then take that to your office and lock it in a desk. Every few months, bring it home, do a backup, and immediately take it away again. This is your "my house burned down" backup.

What I like about Jamie's approach is that it's totally KISS, yet it touches all the cornerstones of a solid backup strategy:

  1. Pick a simple backup strategy you can live with.
  2. Make incremental backups a part of your daily routine.
  3. Include an off-site backup in your strategy.

And for the dissenters, although I can't imagine too many with the minimalist backup process Jamie outlined, there's this bon mot:

"OMG, three drives is so expensive! That sounds like a hassle!" Shut up. I know things. You will listen to me. Do it anyway.

I'm not sure Windows users have a direct equivalent of rsync. There is, of course, RoboCopy, and it looks like someone has ported rsync to Windows. But let's face it. I'm a Windows user. When I have a problem, I buy software. That's why, after hearing so many great things about it, I recently purchased a copy of Acronis True Image.

Acronis True Image 11 Home screen

Acronis does a lot of things, but most of all it's drive imaging software, a fancy GUI over the rsync command. With Jamie's recommended two external hard drives in tow, I can use Acronis to create a bootable mirror image of my hard drive. If anything at all goes wrong, I simply swap hard drives, and I'm back in business. I can even create those backup images incrementally and on a schedule. You don't even technically need a second or third hard drive; if you have a large enough primary drive, Acronis will allow you to create a new, hidden partition to store a complete backup image. You can restore these disk images from within Windows proper, during pre-boot, or from bootable USB or optical media. It is very cool, a logical evolution of the more primitive drive imaging products I've used for years.

Of course, as much as I am enamored of it, you don't have to spend thirty bucks on Acronis and even more for two external hard drives to have a decent backup strategy. Lots of people use completely internet based backup services, like Mozy, Carbonite, or JungleDisk, with varying degrees of success. One thing's for sure: until you have a backup strategy of some kind, you're screwed, you just don't know it yet. If backing up your data sounds like a hassle, that's because it is. Shut up. I know things. You will listen to me. Do it anyway.

Posted by Jeff Atwood
201 Comments

No love for Windows Complete PC Backup and Restore? ;)

Ex-S Woo on January 29, 2008 4:48 AM

What's that quote? "There are two kinds of people in the world: Those who have experienced catastrophic hard drive failure, and those who *will* experience catastrophic hard drive failure."

I have been using acronis for several years now, but with a simpler strategy than outlined in the article. I simply backup everything to an external usb drive every afternoon using an Acronis scheduled job (Acronis Workstation). Acronis' ability to mount a backup image as a disk has bailed me out a few times too. Like when I've changed some source code I really didn't want to change either inadvertently or with extreme stupidity. Just mount yesterday's backup and replace the damaged files.

Coach24 on January 29, 2008 4:57 AM

Honestly, I had no idea that Windows Complete PC Backup and Restore even existed:

a href="http://www.bleepingcomputer.com/tutorials/tutorial145.html"http://www.bleepingcomputer.com/tutorials/tutorial145.html/a

Vista Business, Ultimate, Enterprise, and Smoky Bacon Edition have the full feature evidently.

a href="http://www.microsoft.com/windows/products/windowsvista/features/details/backup.mspx"http://www.microsoft.com/windows/products/windowsvista/features/details/backup.mspx/a

Jeff Atwood on January 29, 2008 5:05 AM

My backup strategy? I post my important data in base64-encoded little chunks in the comments on the blogs of people who actually take the necessary steps to backup their data.

U3VpdGNhc2UgY29tYmluYXRpb246IDEgMiAzIDQgNQ==
RS1tYWlsIHBhc3N3b3JkOiBwYXNzd29yZDE=

Christopher Finke on January 29, 2008 5:05 AM

I know it's usually described as NOT to be used for backups but RAID does the trick pretty well.
No hassle nor cron job to launch rsync / rdiff-backup scripts, data always in sync.
With 3 drives if you're really concerned you can for instance get a 2 HDD raid 1 array with 1 spare that would go online if one of the main drives were to fail.
Don't know about windows or mac but on linux it's fairly easy to set up using mdadm.

10 on January 29, 2008 5:16 AM

Mine is pretty simple. I commit everything to my LOCAL SVN repository, which is on a different disk. Periodically I back up the repository. If the repository dies, I have my working copy on a different disk -- all I lose is the history of my directory reorganizations and deletions. If my working copy dies, I get a new version from the repo. I periodically move my repository directory to a portable drive which I give to my father, and he stores it at his place. Offsite backups.

Dave Markle on January 29, 2008 5:22 AM

Why, oh why, didn't MS include the image tool for Home users? It really would be nice.

I have the OS on a different drive than my data, and I only backup the data drive. First to another internal drive with SyncToy, then online with Mozy, and I feel safe. (That is, after the initial 2 week upload.) If my Raptor with the OS fails, I'm looking at reinstalling my apps which is a pain, but I can deal with it.

Lance Fisher on January 29, 2008 5:22 AM

A RAID might die if the PSU messes up, or the controller dies spectacularly. That's probably the main reason people recommend not to use it - it isn't very safe for any purpose other than protecting against mechanical failure.

LSnK on January 29, 2008 5:25 AM

My backup strategy? I have this dongle called a MacBook Pro, it runs Leopard, which in turn includes Time Machine, which in turn backs my drive up to an external drive once an hour.

Reg Braithwaite on January 29, 2008 5:27 AM

Wow. It's like putting the contents of your hard drive in a CVS repository, except easier.

Jonathan Paul Madrid Abaca on January 29, 2008 5:28 AM

I use Mozy with great success for my wife's computers. I've even tested the restore! Although time consuming, it works. I'm a Mac user, so Time Machine is a great tool for me. I can't boot off the Time Machine disk, but I have history, which is nice, and I can restore my machine almost completely from a fresh install using a Time Machine backup. I've done it from bare drive to fully functioning machine in just about 3 hours.

Mike on January 29, 2008 5:41 AM

I have used rsync to do a remote copy. Machine A running Windows 2000, cygwin, and rsyncd; machine B running Windows XP, cygwin, and rsync. I copied data from A into B before getting rid of A (and buying C :D).

So there, rsync works on Windows, and if it worked remotely with a rsync *server* both sides being Windows, it definitely works locally.

Nicolas on January 29, 2008 5:44 AM

Isn't this basically the same thing as RAID-1, but allowing up to 24 hours of data to be lost, since the drive is only mirrored once per day?

I do think the advice to keep a copy in a physically separate location is very sound.

Jake Cohen on January 29, 2008 5:46 AM

I use PathSync (http://www.cockos.com/pathsync/) to backup the important bits of 3 different computers to a little western digital USB powered drive. I do this regularly and once a week I go to my safety deposit box and swap with the other WD.

It's manual (which sucks) but it puts all the files I care about on something I know is safe.

Only problem I ever have is that PathSync doesn't play nice with certain non-ascii filenames.

Gonna have to look at Mozy, though.

Michael Kohne on January 29, 2008 5:49 AM

I have a two-tier backup system. I do a daily backup to a NAS, with weekly backups to an offsite location with Mozy. It works like a charm, the few times I've needed to replace/reformat my hard drive, I've been able to quickly get back on my feet.

Dan on January 29, 2008 5:50 AM

A local archive, while a good first step, is not a complete Backup Strategy. The data you care about needs to be moved off-site.

Get yourself a friend that has some space to share and buy a copy of CrashPlan (http://www.crashplan.com). It's multi-platform and will do compressed, encrypted, versioned, incremental backups over the wire to multiple destinations.

Disclaimer: I work on the product. Let that dissuade you if you will. Better to try it for free yourself.

Peter on January 29, 2008 5:57 AM

"But let's face it. I'm a Windows user. When I have a problem, I buy software."

True that - if you Windows users have a problem, you just chuck some $$$ at it :D

I used to be one of those people, but CLI rsync is so, so nice ;)

l33t3r on January 29, 2008 5:57 AM

my favorite backup related website:

http://www.taobackup.com

It makes some good points about backup strategies and is humorous as well.

The 7 heads of the Tao of Backup:
1. Coverage: back up ALL your data
2. Frequency: back up Frequently
3. Separation: Take some backups offsite
4. History: Keep some old backups
5. Testing: Periodically test your backups
6. Security: Secure your backups
7. Integrity: Make sure the data you are backing up is correct!

Steve Bush on January 29, 2008 5:57 AM

I typically back up to two other drives. My secret weapon? Unison - which can be found through Google or through my site. Unison is multi-platform, GUI based and it rocks. It too performs incremental backups - like a charm, letting you decide which changes to propagate. It uses multiple tools to make one application(including rsync).

I truly feel bad for those that feel that buying software is their only choice.. *cough* Jeff. :) Maybe not you... but folks, look before you pay! .. that is, unless you're growing a money tree in your backyard.

Patrick on January 29, 2008 5:58 AM

I've got software sitting in my system tray that copies any new files over to my linux home server (as a mapped network drive) every 30 minutes. Every night at 5 am, that machine uploads any new or changed files to a massive amount of FTP storage I got for cheap. Before it uploads the files, it compresses and encrypts them, for safe offsite storage. (It uses file hashes to check for changed copies).

It's nice having that setup because if I accidentally delete a file, it's right at my fingertips on my server, but yet if my house burns down I have a complete and safe offsite backup. At the same time, the FTP uploading, which is slow, is handled by my server so I can feel free to shutdown my power-guzzling desktop at night.

It's worked out really, really well. I needed a backup solution that would be complete, secure, do it's thing, and leave me alone. It's done exactly that.

Bob Somers on January 29, 2008 5:58 AM

An internet backup service such as Carbonite or Moxy is my preferred solution. I'm super-lazy, and these are pretty much install-and-forget type of programs. As long as I have an internet tube and the software is still installed, it's working.

That's unlike my previous solution, where...oh, did I forget to turn the external backup drive on? Did I temporarily unplug the backup drive to test my new camera? Etc.

And in case of a disaster (fire, hurricane, theft, etc.) my data is still safe.

The only downside is it costs money.

Michael Gebis on January 29, 2008 6:00 AM

Windows Home Server has been working a treat for me, haven't encountered much that works as simply with so little effort for automatic backups.

Chris Mayer on January 29, 2008 6:02 AM


"I'm a Windows user. When I have a problem, I buy software."

As linux user, I don't have this problem much. dd and/or rsync with a simple shell script executed as the last thing before shutdown. No fuss at all. However, I am not opposed to paying for software when needed. What I find funny is that Windows seems to lack( or have really shitty built-in versions ) basic, important stuff that you have to shell out extra for. For the license of Windows, it should include things like this.

That said, I see Vista finally has something, Windows Complete PC Backup and Restore, but of course it's in the tiered pricing lameness... Again, this kind of things should be core software across all Windows.

I have a box here with Vista Ultimate( thanks work! ) that I haven't bothered backing up( the important data is safe, yes ). I shall have to try Windows Backup and see how well it works.

The Foo on January 29, 2008 6:02 AM

Backing up my entire hard drive would be ridiculously stupid. 99.9% of my space (yes, really) is stuff I can replace with an internet connection and an hour or two to waste. A single DVD can handle all my backups for six months. Not that I would restrict myself to a single DVD, like you restrict yourself to a single HDD.

J. Stoever on January 29, 2008 6:13 AM

"I'm a Windows user. When I have a problem, I buy software."

Why? Ever heard of xcopy and task scheduler? Runs on my box at 3am daily. Works like a champ, even easy to restore from. Best of all, it has come with MS OS's since the glory days of DOS 5, if I recall correctly.

FortyPoundHead on January 29, 2008 6:17 AM

+1 for robocopy;

I've been using robocopy on Vista with a lot of success, although it isn't quite as awesome as rsync from the block level, it does a very good job of running every night for me.

Jeff on January 29, 2008 6:24 AM

I used backup my data to DVD. Then one day, my DVD was unreadable, DOH!

I used USB hard drives for a while, but it was too unstable: USB kept disconnecting then reconnecting at the slower speed.

My Backup Plan:

I built 2 identical linux machines (NAS1,NAS2). Each with 1 TB of RAID 0 drives.

I use Samba on NAS1 to share files with my Windows machine. Acronis True Images backups my Windows machine to the NAS1 Samba Share. Then, daily I rsync NAS1 to NAS2.

So I have a complete backup of my backup NAS. I have spare drives on hand to replace a failed drive.

My data could be lost if 1 drive in NAS1 and NAS2 failed within 6hrs (time to rebuild 1 NAS) of each other.

derby on January 29, 2008 6:26 AM

I used to just stick with CD/DVD backups. Does the job, if a bit inelegantly (where'd the disc from last week go, in this pile of a thousand discs!). I've since moved on, largely because my work made it rather obvious what a bad backup system can do.

Find a spare computer. They're *cheap* these days; drop by a college during their annual garage sale, and you can probably pick one up for less than ten bucks, or you can use an older one that you don't need anymore. A PCI RAID controller is optional, but it'll cost you less than twenty bucks. Three hard drives, one with an external enclosure, depending on how much space you actually need, will put you back a hundred or two. Unless you do a [i]massive[/i] amount of imagery or video editing, eighty gigabytes a drive will be more than enough. You don't need to back up your operating system and game files every day; you should have one or two DVDs worth of BartPE or your local operating system's worth of DriveXML data, and leave that so you don't have a thousand corrupted backups when a virus takes your system out or a hard drive scratch means you don't own a Mac any more.

Combine with OpenSSH (through Cygwin, for Windows users), SVN (there's a Windows version), and your operating system's favorite shell scripting and automatic timed start system, and you've got all you need to have a network-accessible, versioned, highly secure system with several points of failure.

If you really want to go nuts, a decent high-watter power-conditioning UPS will put you back less than sixty bucks, add longevity to your monitors, and make things even more secure, and combine that with 9 bucks a month worth of web storage with your web host, but chances are good even that would be unnecessary.

gattsuru on January 29, 2008 6:34 AM

OMG... I have not backed up for a while... let me do it fast.

NIYAZ PK on January 29, 2008 6:36 AM

For Macs, SuperDuper (http://www.shirt-pocket.com/SuperDuper/SuperDuperDescription.html) is a great app that makes an exact, bootable copy of your HD to an external drive. The basic version is free, and if you like it you can pay a few bucks to upgrade to the full version (which does nice things like copy only changed files on subsequent backups so the job runs faster). It runs right in OS X so you don't need downtime to reboot.

Andrew on January 29, 2008 6:42 AM

I use Time Machine for automatic, local, versioned backups and rsync to my remote server for my offsite backup. I'm also using rsync+OSXCrypt for a USB thumb drive backup of document scans.

I've played with JungleDisk to S3, but it was flaky and now with so many GBs of data, S3 is not so cheap.

David Avraamides on January 29, 2008 6:58 AM

Oh, and for those who think RAID's useless :

The average lifespan of a hard drive is, depending on manufacturer and type, five years. This is an *average*. In practice, hard drives tend to follow a "bathtub curve" of failure; new drives have a higher risk of failure because there is a known potential for manufacturing errors to creep in and result in lost data. While this is fairly rare, it is not an unknown. Moreover, these early failures tend to be catastrophic, rather than minor; large scratches rather than single bad sectors.

Nothing says fun like losing days or weeks worth of backup data (if you have a secondary backup) or, worse, losing all of it, a few months after putting your system into place. RAID set-ups are cheap, and they're quite effective against hard drive death.

gattsuru on January 29, 2008 7:00 AM

I was lucky to have made a snapshot of a large and critical code repository the day before a laid off employee deleted the entire source tree in vengeance. Backups were spotty and scarce, but I learned the lesson the easy way (and in my first week on the job right out of college!).

I've rarely had to restore from backup due to a hardware failure, but corruption of data, brain farts, and even malicious behavior have caused many a restore.

Now if I only had a backup of my 250GB external USB drive from two months ago, I wouldn't be reconstructing my data from the many smaller, dated backups on aged hard drives. :-(

Lesson learned.... again.

Eric on January 29, 2008 7:02 AM

I use FolderShare to copy important stuff (actually, whole "My Documents") to 2 other computers, one of same location and another is on different contienent.

I don't really care too much on when the data is backed up, if they are important, they should have sit on my computer for long time. They should sit long enough that FolderShare should already kicked in and do the backup for me.

I agree with J.Stoever, most of the stuff I got on my PC can be installed from Internet (except Visual Studio). If my harddisk is down, I don't see that urgent to get my PC working again within 5 minutes. Rather, I could rebuild my PC with software over the Internet within 5 hours. And what I get is the miracle of reinstallation: your PC with its best performance!

Compulim on January 29, 2008 7:05 AM

"Acronis will allow you to create a new, hidden partition to store a complete backup image"

The point being? You can lose your real data _and_ your backup all at once when your drive fails, hooray! Backing up to the same drive is little more than Windows Restore Points, it's certainly not a backup plan.

Dav on January 29, 2008 7:10 AM

Mike Rubel has a nice article on his site explaining how to do Apple Timemachine style snapshots using rsync.

http://www.mikerubel.org/computers/rsync_snapshots/

Darren on January 29, 2008 7:21 AM

This is a timely discussion. A member of my extended family phoned on Sunday saying their windows xp machine won't boot, with an error message about a disk problem. The most recent backup was 1 year old. I was able to boot into knoppix and recover most of the files they cared about (I probably could have used WinPE if I was 3133t). Now that the files are backed up I'll probably try a boot-to-xp and "repair" (but I wanted to have the files before I tried this). Makes me want to re-visit my SVN-based backup strategy to make sure it is rock-solid.

JosephCooney on January 29, 2008 7:28 AM

"Did I mention that Jamie is a funny guy?"

From http://www.jwz.org/xscreensaver/download.html:

"Microsoft killed my company, and I hold a personal grudge."

Mark Gardner on January 29, 2008 7:36 AM

I use second copy. I configure it to backup my documents directory every couple of hours. I have it configured to do a simple copy and it replaces new and changed files. It also supports more complex sync's like deleting files in the backup that are deleted in the source. I have a USB drive as my backup. My important files like Quicken I also backup into a second directory and then use Nero to do an incremental backup to a CDR.

a href="http://www.secondcopy.com/"http://www.secondcopy.com//a, $30 for a single license.

asm on January 29, 2008 7:38 AM

I really need to get into the habit of maintaining a disciplined backup routine. It's never really been my strong point. Honestly, I think there's something to be said for actually going through a catestrophic loss of data; all my friends who have done that once in their lives now have a bullet-proof backup strategy. I haven't been burned yet, and thus it's tough for me to get serious about getting a backup plan in places, even though intellectually I know that odds are I'm due for a crash. As jamie says, "the universe trends towards maximum irony".

Ethan Vizitei on January 29, 2008 7:45 AM

I'm shocked you don't use the standard Vista Imaging stuff, or Windows Home Server. There's lots of options.

Scott Hanselman on January 29, 2008 8:11 AM

Microsoft SyncToy (powertoy downloadable from their website) is also a snazzy GUI tool I use for weekly backups.

Doug Large on January 29, 2008 8:13 AM

more like an advertisement of Acronis :D
On Winodws I don't backup at all...important files are already under version control...
It always feels nice (if necessary) to wipe out everything and do a fresh install.

airfang on January 29, 2008 8:19 AM

XDelta is a CLI backup similar to rsync but with improved compression and speed, plus works across all platforms. Too bad it's not included natively in anything. ;_; That rsync gui is neat.

Mozy is my offsite. If my house burns down, I have more important things to worry about than having every single thing I ever downloaded, let alone all the ginormous working files. But a local drive does back that up, it's reasonably fast. Finally convinced myself after a few near-misses that it's insane not to.

Foxyshadis on January 29, 2008 8:24 AM

Oh yeah, the trouble with user-mode windows backups (anything CLI especially) is that you won't get the registry while it's in use, as well as any other files exclusively opened or you don't have permission to see. Offline backups are way too much hassle, so make sure you get something that uses Volume Shadow Services.

Foxyshadis on January 29, 2008 8:25 AM

"If you are using a Mac, the command you use to back up is this:

sudo rsync -vaxE --delete --ignore-errors / /Volumes/Backup/

If you're using Linux, it's something a lot like that. If you're using Windows, go f*ck yourself."

Haha! class :)

armandino on January 29, 2008 9:06 AM

Funny thing is, even after several failures, I do not regularly back up my data.

W^L+ on January 29, 2008 9:18 AM

Another Mozy user here... it works ok for me. I do not back up everything... just the data that cannot be replaced. Right now I have about 90GB of that and Mozy backs that up just fine. Just be patient when it does the initial backup.

Since Mozy will only back up one computer (technically local drives + USB connected drives on the local PC), something you can do is to sync all your files the PC that has Mozy (use Synctoy, SyncBackSE, etc.)

Chinh Do on January 29, 2008 9:28 AM

I'm in the process of storing everything I ever own (digitally, of course) on to Amazon S3, using S3 Backup for Windows. I believe there's a well written ruby script for it too if you're on *nix.

I dislike JungleDisk because it is a lock-in. JD has it own conventions for naming data on S3, so if the author ever pulls support, it'll be a hassle.

RAID-1 is actually the best insurance against hard drive failures. I'm currently running RAID-5 on an array of 4 250 GB drives right now and rather paranoid about the controller dying and not being able to find an exact replacement.

Mythokia on January 29, 2008 9:47 AM

I use Windows XP. I backup anything I wook on daily to a second HD in each PC. Weekly I backup to 2nd HDs on other PCs and external drives. Every once in a while I do an image of my C drive (operating system and installed software, all data is on D or elsewhere).

Tools:
- Beyond Compare
- Ghost
- Various source control softwares, depending on the project and client, some local, some off site
- SQL Server built-in backup

I used to back up documents and even databases to source control, but it just seemed like an extra step for something that can't be usefully compared anyway. Now I just keep the folders backed up to other drives. The exception to this is project documentation, which I keep with the source, regardless of type, and which may have useful comments tied to the check-in.

Dennis on January 29, 2008 9:56 AM

Oh, once every couple of months (I hope -- I should really make a set schedule for this) I take a hard disk (and sometimes a DVD or 2) to a safety deposit box at my local bank for off-site backup.

Dennis on January 29, 2008 9:58 AM

1. Buy a Windows Home Server machine (mine's the HP MediaSmart Server).

2. Install client app on each PC.

3. Sleep soundly knowing each PC is backed up fully every night.

Seriously, the WHS machine is the best investment I've ever made because while I can go through setting up scheduled tasks and installing various software bits and configuring various things I'm too lazy (and my time is better spent on paying projects) and setting up WHS took all of 10 minutes, plus I have a full backup of all the machines in my house with a restore so easy my wife can do it even if I'm not around to help.

Shawn Oster on January 29, 2008 10:00 AM

I can't get myself to use online data storage for backing up sensitive personal data, such as credit card info, accounts and passwords, etc. I read about breaches and leaks too often.

Dennis on January 29, 2008 10:00 AM

My backup strategy for my code at work (all automated I might add):

1. All projects are in Subversion repositories on a dedicated server.
2. The Subversion server locally makes zipped archives of all repositories daily, and keeps rolling weekly and monthly copies as well.
3. The Subversion server is backed up to tape.
4. My checkouts of those repositories on my development PC are backed up daily as zip archives.
5. Those zip archives are copied to a NAS, which is also backed up to tape.
6. Those zip archives are weekly uploaded to my home server.
7. The deployed code on the production application server is also backed up as zip archives and backed up to tape.

I never thought about it, but when I type it out, it seems paranoid... :-)

Joshua Curtiss on January 29, 2008 10:18 AM

OS X Leopard users can just use Time Machine. Plug in an external drive, click the "Ok, use this drive for time machine button" and that's about it.

Although SuperDuper is still useful as a bootable backup (I have a 10.4.10 bootable partition, a partition for the Aperture library backup and a few other things like my completed video-projects and one for the bootcamp backup image)

Aside from the laptop OS backup, I generally just copy important files to several drives, and sometimes to DVDs

dbr on January 29, 2008 10:19 AM

"5) If you're on a Mac, that backup drive will be bootable. That means that when (WHEN) your internal drive scorches itself, you can just take your backup drive and put it in your computer and go. This is nice."

Good quote. Thank you for quoting that.

"7) That third drive? Do a backup onto it the same way, then take that to your office and lock it in a desk. Every few months, bring it home, do a backup, and immediately take it away again. This is your "my house burned down" backup."

Good quote. Thank you for quoting that.

"You don't even technically need a second or third hard drive; if you have a large enough primary drive, Acronis will allow you to create a new, hidden partition to store a complete backup image."

Oh. Um. Out of curiosity, why did you quote those two previous bits of advice?

Back to the past on January 29, 2008 10:44 AM

I keep disk images of a clean OS installs that can be restored onto my MacBook Pro in a few minutes. Every major OS X version except 10.0, one Vista and an XP I probably still have my 98SE image.

Also a bootable 500GB TimeMachine disk, another spare/unused 500GB disk that could be used for backups whenever I start needing it. Nothing offsite tho, if my house burns down I have more important stuff to care about than my data.

When my harddisk fails I'm can theoretically be up and running again in 15 to 30 minutes. I learned my lesson at the last harddrive that went suicidal on me.

kris on January 29, 2008 11:33 AM

You don't even technically need a second or third hard drive; if you
have a large enough primary drive, Acronis will allow you to create a
new, hidden partition to store a complete backup image.

You must be crazy :)

Storing the backup on the same harddrive from where that backup is made? That's by far the worst backup-strategy ever and should not even be mentioned in any case (as someone might take it seriously).

Samuli on January 29, 2008 11:37 AM

Windows Home Server. No work necessary.

1) each night it does a incremental backup of all 4 computers
2) i can backup the backups to an external USB drive and take to work once a month
3) use Mozy/Carbonie for critical files (like my life would be ruined if I lost them)
4) use FolderShare to replicate C:\users\omars\documents to all my computers

Omar Shahine on January 29, 2008 11:47 AM

I created a robocopy script that copied all my data to external drive. I do this full backup every month or so, then I take to work and keep it there, so it doesn't get lost when my home burnes or computer is stolen.

For intermittent hard drive problems, I have RAID 5 built from 4 400 GB SATA II drives, so I don't care if one of them blows up.

Jakub Kocureq Anderwald on January 30, 2008 1:02 AM

This seemed a little off to me:

"You don't even technically need a second or third hard drive; if you have a large enough primary drive, Acronis will allow you to create a new, hidden partition to store a complete backup image."

Backing up your disk on the same disk just protects you from user fuckups where you delete something you shouldn't have deleted.

If your disk fails, and your backup is on that disk too, well...

I'm using 2 external disks at home with some bat files :)

David Cumps on January 30, 2008 1:24 AM

My MacBook+TimeMachine+external USB HDD
wonderful, easy, fast.

Enrique on January 30, 2008 1:30 AM

I just keep my fingers crossed.

GUI Junkie on January 30, 2008 1:32 AM

My personal backup strategy is "back up data files offsite when they change meaningfully, and locally much more often".

A live whole-drive backup is nice, but not worthy my effort.

Other people's mileage, of course varies.

(The only things I'd lose if I lost one of my machines by itself, would be game saves or recent email - and I don't care about my email nearly that much.

Now, a house fire, that would hurt more, but the important stuff, my music and longer-term files, would all be safe in off-site DVD piles.

Safe as optical media can be, at least, but there's no guarantee a hard drive will work in ten years, either, in terms both of raw function and having a computer that can physically connect to it and possibly even read the filesystem.

When was the last time you even SAW a computer with an MFM connector? Or, for the old-time Mac users, unless you have a Mac Pro, how're you going to connect an old SCSI drive to a modern machine?

Compromises all around, really. All flesh is grass, and all data is eventually toast.)

Sigivald on January 30, 2008 1:34 AM

unison and cheap NAS also does the trick. unison runs with or without gui, so use it without gui and as scheduled task in windows.

i only backup data. with wpkg you can do automated installaion of the system again and again.

offler on January 30, 2008 1:42 AM

Backup strategy:

Windows Home Server + JungleDisk for WHS.

Set it and forget it.

Andrew on January 30, 2008 1:43 AM

The week before last, some undesireable got into my house and stole my laptop and my work laptop, much to my irritation and triggering a shortlived desire to kill everything within arm's reach.

Fortunately all the data on those machines is stored in the following places:

1. The computer upstairs that they didn't get
2. My server upstairs which they wouldn't be able to carry
3. rsync.net
4. Work

So when I get a new one I just have to copy the data back again and continue where I left off.

The offsite backup strategy is so important now we're all slightly obsessed with owning laptops. You can make mirror images of your HDD to hidden partitions and use the "undo" feature of your favourite OS's filesystem... but it's not much help if your laptop is in someone else's house being erased and re-installed or sold in the local pawn shop.

So the offiste backup isn't incase your house burns down, it's incase someone steals your computer, or you lose it.

Just watch that incremental backups don't cure against "user is a moron and deleted stuff without realising for a few days" disease. Your automated backup will faithfully propagate your error.

James on January 30, 2008 1:44 AM

Windows Home Server makes a good in-house caching device for uploading to the Amazon Simple Storage Service (S3). Currently, I'm using the JungleDisk plug-in.

I have about 110 GB of data backed up remotely for about $15 a month.

Seem to work pretty well.

Timothy Lee Russell on January 30, 2008 1:47 AM

Christopeher Finke wrote: ...base64-encoded little chunks...

LOL! But where /is/ your suitcase...?


Roddy on January 30, 2008 1:51 AM

I like the irony in the fact that there is hatred for windows users in the original reference. Another spiteful mac expert (http://wilshipley.com/blog/2008/01/macbook-air-haters-suck-my-dick.html) excoriates those who don't like the fact that you can't crack open a macbook air.
Guess you can't backup that sumbitch, but you can still s*** his d***!

Card carrying 80% member on January 30, 2008 1:58 AM

Vista has "Backup and Restore" capability. Look in the control panel.

Toby on January 30, 2008 2:26 AM

Huge issue in South Africa is that we pay-for-use with bandwidth. You pay for xGb per month with DSL. This pretty much does away with offsite backups over the net - simply way to expensive ito bandwidth.

At friend of mine lives not too far away. I've setup a $200 WLAN with directional antennas and Linksys WRTs. We each have a spare (old) workstation at each other's houses which are used as offsite backup. I don't have a pressing need for a drive mirror with OS and everything, so we both have scheduled jobs which FTP new and changed files repeatedly. As soon as the job is finished, it starts again.

Works like a dream.

Chris on January 30, 2008 2:44 AM

Backup strategy? What backup strategy? OH SHI-

WurdBendur on January 30, 2008 2:52 AM

I have become very paranoid about backups in the last year, having had lost so much data to disk errors or in some cases, stupidity. The problem for me is that I work with terabytes of image data. It's actually getting to a point now where it's cost-effective to use external disks for that volume of data, but in truth I have a system which is a hybrid of the above techniques. I mirror long term stuff to an external disk using synctoy everyday. I use jungledisk for smaller crucial stuff, like project files, and this also keeps six months worth of revisions. The CVS I use (which in itself is a backup system of sorts) is backed up to LTO3 tapes once a month. And finally I use time machine, but this is purely for the system disk.

Jack on January 30, 2008 3:11 AM

This is all good, but the problems begin again once your valuable data exceeds the size of your drives. Say, you work with digital video, you will regularly run into this situation. Then you can either replace all of your drives with bigger drives, or you have this whole mess with special backup HDDs, piles of DVDs that might become unreadable over time, etc.

Sam on January 30, 2008 3:21 AM

I'm using rsync with a nice (self-written) script that at first wakes up the destination computer via wol and then backups different dirs.

It's scheduled via crontab.

Kai on January 30, 2008 3:27 AM

rsync is not like incremental backup.
It's more like efficient full backup.
I.E. after you do the rsync you only have the latest version.
Here is the rsync command I use to backup over the company LAN:
http://www.pixelbeat.org/docs/hard_disk_reliability/

One can do incremental backups with the rsync protocol using:
http://rdiff-backup.stanford.edu/duplicity.html
http://www.dirvish.org/

I find doing a full backup from my laptop to external disk
is more flexible using tools other than rsync:
http://www.pixelbeat.org/systems/usb_backup/

Pdraig Brady on January 30, 2008 3:36 AM

Regarding SuperDuper! for Macintosh, please bear in mind that it's still not Leopard Compatible (Dave is working hard to make sure it works). Alternatively you can try Carbon Copy Cloner (google for it) which will do the same (for free), yet with less features.

For Windows, I have been using SyncBack for ages and it just works. I use the freeware version: http://www.2brightsparks.com/freeware/freeware-hub.html but there's a more powerful version available.


Martin on January 30, 2008 3:38 AM

It's funny that nobody mentioned the most basic backup tool available in Windows: "ntbackup". For all intent and purpose, ntbackup just works.

NTBackup can backup and restore Windows System State, Registry Clusters, Active Directory, Exchange Server Databases, it supports Volume Shadow Copy (VSS), ASR (Automated System Recovery) and can use tapes easily. It's also easy to schedule and script via the commandline and can perform Differential, Incrementals and Full backups. NTBackup is just like the UNIX dump(backup/fbackup/mksysb etc) commands. Powerful, always there and always ignored.

The good part about ntbackup is that most systems have it (and you can get the .msi for the others), so restoring shouldn't be an issue.

If you are using Windows Vista Windows (Business, Enterprise or Ultimate), Complete PC Backup and Restore will works similar to Norton Ghost along side Automatic Backup and System Restore.

I should also note that rsync (or ROBOCOPY) are NOT backup tools. rdiff/rsnapshot/rsync with snapshots or rsync-snapshot do come closer to that concept though. Just cloning files isn't enough. You need to archive and store them (or use a snapshot capable filesystem).

Also, you shouldn't rely on just one layer of backups. You don't need something like bare metal recovery (eg: Ghost) to restore just 1 deleted file. You don't want to wait 8 hours to restore a disk image just for a failed disk. It's a combination of high availability technology (RAID) with file backup features like VSS or System Restore , your backup server / media that will backup individual files and the bare metal recovery media (stuff like Acronis, Ghost, Windows Complete PC Backup, etc).

Just a word of caution though: backups aren't important. Restoring them is what counts :-). So it's generally a good idea to test your backups, see if they actually do what they're supposed to. Create a backup policy, test it then stick to it.

Criveti Mihai on January 30, 2008 3:45 AM

One thing that pushed me towards a bootable mirror of my main disk was the time it takes to restore my system from the install disks. It took me the best part of a day to wipe and reinstall my iMac, and that was without restoring my data. I have better things to do with my time than watch progress bars all day!

I settled on SuperDuper in the end. The longer story is here:

http://www.zx81.org.uk/computing/opinion/backup.html

Now with Leopard I like the look of Time Machine, but I'll probably still keep a mirror around, perhaps off-site.

Stephen Darlington on January 30, 2008 3:55 AM

jwz said: Option 1: Learn not to care about your data. Don't save any old email, use a film camera, and only listen to physical CDs and not MP3s. If you have no possessions, you have nothing to lose.

jeff said: This is obviously meant as satire

Actually, it's not satire, it's a concept in Buddhism. Losing stuff makes you suffer, and the goal's to eliminate suffering.

Rob Janssen on January 30, 2008 3:57 AM

LOL @ Christopeher Finke

clever :)

Jibbers42 on January 30, 2008 4:00 AM

Why do so many people think RAID mirroring etc provides even remotely suitable backup? Drive failure is only a relatively infrequent cause for me having to do restores for people. It's usually due to corruptions, deletions, viruses, stupidity, dangerous-users*, etc, and a RAID will cheerfully reproduce that across the whole array.


*for example, someone cataloging all their images in Picasa/Lightroom/etc and then deleting all the jpegs off their PC

Cylindric on January 30, 2008 4:01 AM

http://www.aboutmyip.com/AboutMyXApp/DeltaCopy.jsp

JAL on January 30, 2008 4:07 AM

I agree with Samuli.

You must be crazy :)
Not using a secondary drive, you might just as well not backup at all.
Hardware will fail.

I'm a Windows user. When I have a problem, I buy software.
I'm a Windows user too. If I have a problem, I find a freeware alternative (and if I'm feeling really generous, I donate a few bugs).

Hartmut on January 30, 2008 4:16 AM

I'm a Windows user. When I have a problem, I buy software.

I’m a Mac user, and I often do the same thing. I’ll second the votes for SuperDuper. I had a go at making rsync do my back-up stuff for me, but I couldn’t figure it out enough to trust it.

SuperDuper saved my ass when my hard drive did go south.

pauldwaite on January 30, 2008 4:48 AM

I've used Acronis for years to keep 3 or 4 computers backed up. Images to an external hard drive are fast and reliable. It's a good idea to have a cloned HD handy too, because one day your external HD will also fail and you'll lose all of your backup images and data. (Of course, we all hope we just don't lose everything at once.)

Just last night, my most important software stopped loading. Reinstalling it didn't help, so I restored an Acronis image from about 6 weeks ago. Took about 15 minutes for the restoration and all updates.

Acronis True Image is without a doubt, Best In Class.

Hose on January 30, 2008 5:12 AM

cwRsync is a handy packaging of rsync/rsyncd if you want to run rsync on Windows, but don't want to have any other cygwin applications.

Also, backing up to "another partition of a single drive" is only half a backup. It protects you from basic stupidity, but not mechanical failure. The important thing to remember is that a $100 external drive from your favorite computer store is a lot less than a $5000 data recovery attempt which comes with no guarantees. If your data is worth $100, then buy another drive and back up to it.

sapphirecat on January 30, 2008 5:14 AM

IMO, Mozy FTW. It's easy to install, cheap, capacious, and pretty close to foolproof. If you're paying for massive amounts of bandwidth, why not use a smidge of it to give you secure, managed offsite backups rather than rolling your own? If there's a house fire, the guy with two extra drives on his desk is screwed, while all I have to do is go to Mozy.com with my new PC and pull down all my critical files.

Matt Wigdahl on January 30, 2008 5:24 AM

Daily backup copy of my current work folder on a thumbdrive, and an every Sunday evening backup of everything on a Western Digital My Book.

PaulG. on January 30, 2008 5:32 AM

So far I have not found a good online backup system. Some people say Mozy is good but if you do some searching there are many horror stories of people having trouble restoring from it. Also, the client software sucks.

At the moment I use a script with SyncToy (free from Microsoft) and Truescrypt to back up to an encrypted external drive. I used to use DriveImage XML for imaging my system drive, but switched to Ghost. It's old but very reliable. It doesn't pollute your system like Acronis does either.

An ideal solution would be a wifi link to a neighbour, with whom you can exchange backups. Problem is I don't know anyone living close enough who I trust enough to do it with :(

MoJo on January 30, 2008 5:32 AM

I think a backup strategy (including off-site) should be used in combination with RAID 1 to protect against hard drive failures. Of course, I figured this out after my hard drive died. You can read more about my experience at
http://www.basilv.com/psd/blog/2006/digital-disaster-preparing-for-a-hard-drive-crash

Basil Vandegriend on January 30, 2008 5:32 AM

Concerning RAID: You learn quickly how good a RAID (no matter which level) will serve for backup purposes when you discover that it will not only mirror your data in milliseconds, but also your deletions. So, after an accidental del /s *.* or rm -rf * in the wrong directory, you'll be crying for a _real_ backup, not a RAID.

I prefer TrueImage for a system drive image (always divide into at least 2 partitions - one for system programs, one for data) every some weeks (beats dd by compression), unison to keep a copy on the fileserver up to date and rsync (cwrsync, only needs one cygwin dll to be operative) to backup incrementally to three external USB disks.

I'm using a handy script called rsyncbackup.vbs from a computer magazine which will - on NTFS formatted external disks - make rsync hardlink all files which are already present. So you get a full backup for the cost of an incremental one. Cool, but only commented in german: ftp://ftp.heise.de/pub/ct/listings/0609-126.zip

Steven on January 30, 2008 5:34 AM

Backing up the windows registry:-

I was told the best way to do this is to install windows again on the machine and make it dual boot to a different windows home.

Then boot to this windows install and back everything up.

Restore becomes, create vanilla windows in the alternate home, restore backup, dual boot it back.

I've never done this but it's how a friend used to back his windows machines up.

HTH

Francis Fish on January 30, 2008 5:35 AM

Hey Now Jeff,
This was a real good PSA. I also like to make data DVD's of personal media file (pix flix) in addition to an extra drive. Some day everyone will have a home server.
Coding Horror Fan,
Catto

Catto on January 30, 2008 5:35 AM

Jumping on the dog pile toward the end, it's always good to see data backup getting more air time. Few conversations are as awkward as when your less technically inclined friends lose data and expect you to fix it.

Paraphrasing an actual conversation I've had:

"Hey, I can't access my pictures any more, it just pauses for a while and then gives me some error. Can you help?"

"Sure, has your computer done anything else unusual lately? Started making odd sounds, or having other errors?"

"No, nothings changed, I just turned it one today and nothing works!"

*Examine the computer, notice loud and painful noises coming from the area of the hard disk.

"Sounds like your hard disk is dying"

"Ohh, that noise? It can't be that, it's been making that noise for months!"

The only thing more painful than the above is when they *still* won't consider a backup solution. Because "what are the odds of it happening again? I mean, it's never happened to me before."

1) If your running Linux, there is no reason not to use RAID1. None. Hard disks are inexpensive enough to make it affordable for your primary drive, and additionally, you will get improved performance in most common tasks.

For example, application startup is usually constrained by disk seek performance. The Kernel MD driver is smart enough to interleave requests between disks. So not only do you get improved data security and less hassle when your disk(s) fail, your day-today usage will be faster as well.

2) RAID1 is not a backup strategy. Here's a hypothetical, take two HDDs manufactured at the same time on the same production line. Put them in the same operating environment, and expose them to nearly identical usage patterns. How independent do you think their failures will be?

I've seen it happen many times, to myself and others. It's not an absolute of course, but it's a lot more probable than you'd expect simply considering independent failure.

3) Offsite backup services are plentiful, give serious consideration to it. Many offer the additional convenience of remote access to your files, which can be a plus if you ever need emergency access to your data.

There are dozens, if not hundreds, of ways to do offsite backup of your data. For example, my company (Zettabyte Storage) offers what is essentially a Linux based NAS device with offsite backup of all the data stored on it. It's extremely simple to use, ranges from ~$1.50 to $0.30 per GB per month, and has capacity between 30GB and 690GB. Use it like an external HDD, point your preferred backup application to it, and you've got offsite backup with essentially zero additional work.

Whatever your needs are for backup, there is almost certainly something out there that can satisfy you. Whether it's writing your own backup solution, using a software OTS solution, or using a hardware OTS solution like ours, there is pretty much no excuse for losing the data which is meaningful to you.

Unless you plan on taking the initial suggestion made in the article, do your self a favor and find a way to backup your data. Sure it generally costs a few dollars a month, but recovering data, or taking the time to recreate it all, isn't exactly free either.

Vietor on January 30, 2008 5:36 AM

Anyone have any tips on how to make those backup drives encrypted, such that all normal operations (such as rsync) work as normal? If I was to keep a backup drive at work (or in my car, or any other place it could get lost or stolen), I'm going to want that encrypted.

Anyway, I've always just burned selected files that are truly important to CDs (which, if the surface is protected, are immune to magnetic and kinetic damage, though they will probably degrade over time). And I've never had a sudden catastrophic failure of a drive, it was always gradual or had warning signs [:knocks vigorously on wood:]. (RAID1+ helps here!)

Reed on January 30, 2008 5:37 AM

Also, is it worth it to use external SATA connections to drives? I'd be inclined to just use USB even on a desktop. Even if it's slow, the backup write will be running in the background.

Reed on January 30, 2008 5:40 AM

Instead of a complete hard drive backup (since most of the stuff on the hard drive is applications I can easily re-install) I just backup my important data every night to smaller media such as a DVD-R or USB thumb drive. I then place my backup in my fire/water proof lock box.

Billkamm on January 30, 2008 5:49 AM

Might I humbly suggest HoboCopy? I wrote it to be a very simple version of Robocopy with the added benefit that it can copy files that are in use. Useful if, like me, you can never remember to close Outlook when you're done for the night.

http://www.pluralsight.com/wiki/default.aspx/Craig/HoboCopy.html

Craig Andera on January 30, 2008 5:51 AM

More comments»

The comments to this entry are closed.