programming and human factors
by Jeff Atwood
I recently upgraded my home system with one of the 750 gigabyte Seagate perpendicular drives in order to consolidate a number of hard drives I had on my server. 750 gigabytes is a tremendous amount of storage space in a single drive-- but it doesn't quite get us across the magical terabyte threshold. It's looking more and more like the first terabyte desktop hard drive will arrive sometime in 2007.
Let's take a look back at the other magical thresholds we broke through on the way -- when were 1 gigabyte, 10 gigabyte, and 100 gigabyte drives released?
The data points are derived from this chart; the scale of the graph is logarithmic. The pace of capacity increase has dampened a bit since 2001, but perpendicular technology has gotten us (mostly) back on track.
I remember how excited I was to get my first gigabyte, 10 gigabyte, and 100 gigabyte hard drives back in the day. I've long since stopped worrying about room for applications and even games. It's all media and virtual PC storage these days. Of course, finding things on such a large drive is another matter.
Dell already has a 1TB HD listed under one of their XPS bundles, it is the high end one i think.
Thlome on October 2, 2006 3:44 AMI have a very strictly structured music directory, all done automagically based on tags and a small script I wrote. I never use it, because like Foxyshadis the search capabilities in my media players make it far easier.
[ICR] on October 2, 2006 4:28 AMA student in my Intro into computer science class has had a 1TB harddrive for 2 years. What amazed me is that he filled it already.
Mike H. on October 2, 2006 8:00 AMJeff,
What about performance and seek time? Have you looked at the difference between your perpendicular 750 Gb drive vs a striped array of the same size? I am also leary of keeping everything on just one drive, having recently lost two drives in two machines in about as many weeks. In both cases I was very grateful that the machines were an array.
Joe Palladino on October 2, 2006 8:04 AMI'm in the process of upgrading a 4x250GB RAID5 (850GB usable capacity) to 4x750GB on the same controller. The problem is that I bought 1x750G as an interim step and it's filling up faster than the price of more 750GB drives is falling.
Joop, WinXP and many programs slow down horribly if there's too many entries in a directory. With XP that number seems to be about 500 (much smaller for image software that generates "folder thumbs" and doesn't cache (like, say, windows explorer), so I've gone to YYYY/MM/YYYYMMDD-event for photos, and I'm about to go music/a-f/artist/album for music for the same reason (over 1000 artists, windows explorer is becoming unusable - if I accidentially click the "music" folder its ~10s before I get the machine back).
Moz on October 2, 2006 8:58 AMI remember when doubling my storage space meant purchasing another cassette tape (TRS 80 Baby!).
And then for my Amiga, upgrading it to a full megabyte of RAM cost me $99.00 (look who's screaming fast now!).
Haacked on October 2, 2006 10:05 AMFinding things should be really simple. Just keep things organized from the start.
./audio/artist_name/album_name/XX - track.mp3|ogg
./audio/artist_name/album_name/folder.jpg
./audio/artist_name/album_name/back.jpg
etc
./movies/movie_name (year)/movie_name (year).avi
./mobies/movie_name (year)/folder.jpg
./mobies/movie_name (year)/notes_if_any.txt
./photos/YYYY/YYYY-MM-DD_EVENT/00001.jpg
This is the basic file structure. Once you have that set up you can go OCD like me and then sort music by year, record label, style etc. All this can be done using symlinks. You can do it manually or through scripts. The same can be done for movies. Create dirs such as an actors name then symlink his/her movies in there.
Most important is to keep a directory called incoming or something to that effect and put all incoming goods in there. Only move them to the correct directory once they are complete and they are correctly tagged/labeled.
Joop on October 2, 2006 10:21 AMI remember spending $500 for a 420 MByte Hard drive in 92 (I think) so I could install Borland C++. Then everything was still easy.
Now, 100 GBytes is easy and cheap, and finding things is still not too hard. We've learned a lot about storage paradigms in the past decade, which helps a lot. Of course, if you're just dumping things in your home directory, you'll still have trouble.
1 terabyte? Bring it on!
Jim Lang on October 2, 2006 10:22 AMRE finding stuff: there's a free program online called Flexable Renamer. It's (obviously) supposed to be a batch file renamer. However, I use it for a search engine also as it is WAY faster than windows's build in search. I can search 100 gb in far under a minute. And unlike many desktop searches (Google, Copernic) Flexable Renamer doesn't need constant caching to be this good. Oh yeah, and no install necessary. Completely green.
dinah on October 2, 2006 10:44 AMoff topic -- Jeff, could you make the colored background of the posts (which is a verrrry light blue) just a little darker? It would be way easier to distinguish posts (at least on my monitor).
Steve on October 2, 2006 11:03 AMAnother data point:
Corvus Concept (IIRC) 5M and then 10M, big shoebox thing with multiple logic boards, hooked up to an Apple ][ - that'd be around 1980.
Andy Armstrong on October 2, 2006 11:18 AMI saw tb harddrives a year ago. They are not new, they are just now entering and "afforable," if you wish to call it that, range.
Mike on October 2, 2006 11:42 AMMike/Andy -- I am referring to off-the-shelf desktop hard drives.. stuff you could potentially buy from an online or physical retail vendor.
I'm sure custom and expensive 1/10/100/1000gb drives came much earlier.
Jeff Atwood on October 2, 2006 11:52 AMWhat about two 500Gb HD on RAID+0? ;)
Ava on October 2, 2006 12:01 PMI remember getting annoyed trying to purchase dedicated 40Mb SCSI drives for developer workstations in 1991. I HAD to buy larger drives because the vendors weren't making the small ones anymore.
My how things have changed!
Once paid $40K for 24Gb of Raid storage (circa mid-1990s).
Now my laptop has more space than that...
Oh the clutter I could have saved! (oh wait, squash that thought!)
-Eric
Eric on October 2, 2006 1:48 PMJoop, I used to be that nuts. Then overload an laziness led to losing lots of stuff, until winamp/foobar's libraries showed up - now the organization or lack thereof doesn't matter one bit. It's all there after typing a couple letters. (Not everything fits into winamp's view of the world, but minor changes to tagging or naming fixed that.)
My problem now is trying to evaluate corporate search solutions - since I rarely search for documents, I'm not sure what people really want, but we do need one.
Foxyshadis on October 2, 2006 1:55 PMJeff,
Interested to know your thoughts on having just one hard drive these days. My paranoia now means that I run RAID1 on all my essential computers that contain data that I cannot lose.
I'd like a whacking greating 1TB drive, but just don't feel right putting all my data on one drive - all the eggs in one basket, no?
So, 'to RAID, or not to RAID', is the question.
AndyToo on October 3, 2006 2:51 AMI am also leary of keeping everything on just one drive, having recently lost two drives in two machines in about as many weeks. In both cases I was very grateful that the machines were an array.
My recommendation right now is to get a 10k Raptor as your boot/system drive, and a larger, less expensive secondary SATA drive to store your data, applicatons, etc. If you're worried about data loss, make that two secondary SATA drives in RAID 0 mirroring.
Striping is *definitely* not worth the effort on the desktop:
http://www.codinghorror.com/blog/archives/000335.html
Dell already has a 1TB HD listed
I think you guys are thinking of two 500gb drives in a JBOD (Just a Bunch of Disks) array, so they appear as one large drive to the OS.
http://en.wikipedia.org/wiki/Redundant_array_of_independent_disks#Concatenation_.28JBOD.29
There is no such thing as a 1TB HD at the moment. Not in a single physical drive you can buy from newegg.com. It is coming, though.
Jeff Atwood on October 3, 2006 8:04 AMIt reminds me of my old 286 with it's 20MB drive.
Everytime I wanted something else on the hard drive, I had to remove something else to put it on.
20MB, amazing, in those days that went a long way! :)
Peter Bridger on October 3, 2006 10:26 AMStriping is *definitely* not worth the effort on the desktop:
Jeff, you're talking as if Clayton's RAID is the only sort of striping. RAID5 pays off it you have to buy more than about three drives - four drives plus a controller costs less than six drives but has the same available space (assuming RAID5 vs mirroring). Secondary benefits include fewer drive bays used, less power and noise and you only get one drive letter. Problems include DiskKeeper and some other defraggers choking at 512GB, the RAID card takes up a slot (I have an SLI MB with one "video" PCIe slot used by the card). But few motherboards can support three pairs of mirrored drives, you will probably only be able to get two pairs. Oh, and the three pairs of mirrored drives lets you survive having more than one drive fail if you're lucky (some combos you win, some you lose), but either way you still must have 3 working drives to avoid data loss... it's just that with mirroring you have 6 drives to start with.
Moz on October 3, 2006 11:03 AMWhen I first got into computers I was around 9 or 10.. my father bought a IBM PC XT 4.77MHz (my first love) he payed $2K+ for a Shugart(brand) 10MB hard drive upgrade. I was the first kid on my block to have a C: ! - spoiled..
ViNDi on October 3, 2006 11:06 AMI remember how stoked I was when I got a 40MB hard drive installed in my Amiga 3000. That baby was screamin'!
Now I have 280GB and am experiencing a space crisis. How far we've come.
Mike on October 3, 2006 12:29 PM... and to put this into further perspective; back in 1983 I had a summer job at Perkin Elmer where they were working on electronics for the Hubble Space Telescope. Behind me at one of the terminals where I entered commands were two 20MB (if I remember correctly) hard drives. Each was the size of an industrial washing machine. You could open the clear plastic cover and pull out the disk platters, which were the size of serving trays.
Mike on October 3, 2006 12:35 PMIn regards to Dell having a 1TB hard drive:
I myself work for Dell, and there is no option for a single physical drive with a 1TB capacity. You are most definitely referring to using a RAID striping array, available on Dell's high-end XPS systems, with two 500GB HDs, total capacity of 1TB.
I know Dell does tend to keep on top of the latest technologies (as I would assume most successful computer companies would have to) so if there was a true 1TB hard drive *economically* available, Dell would carry it. Just a matter of time!
Chris
Dell XPS CEC
I have 40GB on hdd. I don't need more!
zvonsully on October 10, 2006 2:00 AMThe problem of finding stuff has been mentioned quite a bit on this post, but nobody mentioned that as drives get bigger, the data loss when one of those drives fails gets bigger as well. Sure, companies and colleges, etc. have backups and junk, but what about average home users? We can’t all afford RAIDs, when the single 1TB disk in our system that has everything in our lives from the past several years craps out, we lose everything; the digital counterpart of losing everything in a housefire. If however one of several smaller drivers fails, we lose proportionately less (like if the firefighters put out the fire before it destroyed the whole house). Of course an array of smaller drives has its own disadvantages like higher energy consumption, larger size, and more physical electronic waste.
The comments to this entry are closed.
Subscribe in a reader
Subscribe via email
|
|
Traffic Stats |