programming and human factors
by Jeff Atwood
In Reducing Your Website's Bandwidth Usage, I concluded that my best outsourced image hosting option was Amazon's S3 or Simple Storage Service.
S3 is a popular choice for startups. For example, SmugMug uses S3 as their primary data storage source. There have been a few minor S3-related bumps at SmugMug, but overall the prognosis appears to be good. After experimenting with S3 myself, I'm sold. The costs are reasonable:
It's not exactly unlimited bandwidth, but I was planning to spend $2 a month on image hosting anyway. That buys me 10 GB per month of highly reliable, pure file transfer bandwidth through S3. Beyond that, it's straight pay-as-you-go.
Unfortunately, Amazon doesn't provide a GUI or command-line application for easily transferring files to S3; it's only a set of SOAP and REST APIs.
There is Jungle Disk, which allows S3 to show up as a virtual drive on your computer, but Jungle Disk offers no way to make files accessible through public HTTP. And as I found out later, Jungle Disk also uses a strange, proprietary file naming and storage structure on S3 when you view it directly. Jungle Disk is a fine backup and offline storage tool (particularly considering how cheap S3 disk storage costs are), but it doesn't offer the level of control that I need.
Amazon does provide a number of API code samples in various languages, along with some links to tutorial articles, but beyond that, you're basically on your own. Or so I thought.
That was before I found the S3Fox Organizer for FireFox.
S3Fox is like a dream come true after futzing around with the S3 API. Using S3Fox, I can easily experiment with the S3 service to figure out how it works instead of spending my time struggling with arcane S3 API calls in a development environment. It's a shame Amazon doesn't offer a tool like this to help people understand what the S3 service is and how it works.
At any rate, my goal is to use S3 as an image hosting service. I started by uploading an entire folder of images with S3Fox. I had a few problems where S3Fox would mysteriously fail in the middle of a transfer, forcing me to exit all the way out of Firefox. Fortunately, S3Fox also has folder synchronization support, so I simply restarted the entire transfer and told it to skip all files that were already present in S3. After a few restarts, all the files were successfully uploaded. I then granted anonymous access to the entire folder and all of its contents. This effectively exposed all the uploaded images through the public S3 site URL:
http://s3.amazonaws.com/
All S3 content has to go in what Amazon calls a "Bucket". Bucket names must be globally unique, and you can only create a maximum of 100 Buckets per account. It's easy to see why when you form the next part of the URL:
http://s3.amazonaws.com/codinghorrorimg/
Each Bucket can hold an unlimited number of "Objects" of essentially unlimited size, with as much arbitrary key-value pair metadata as you want attached. Objects default to private access, but they have explicit access control lists (for Amazon accounts only), and you can make them public. Once we've added an Object, if we grant public read permission to it, we can then access it via the complete site / Bucket / Object URL:
http://s3.amazonaws.com/codinghorrorimg/codinghorror-bandwidth-usage.png
There's no concept of folders in S3. You can only emulate folder structures by adding Objects with tricky names like subfolder/myfile.txt. And you can't rename Buckets or Objects, as far as I can tell. But at least I can control the exact filenames, which I was unable to do with any other image hosting service.
In my testing I ended up uploading the entire contents of my /images folder twice. That cost me a whopping two cents according to my real-time S3 account statement:
It's almost like micropayments in action.
S3 will probably end up costing me slightly more than the "Unlimited" $25/year accounts available on popular personal photo sharing sites. With S3, there's no illusion of unlimited bandwidth use unconstrained by cost. And personal photo and image sharing sites are often blocked by corporate networks, which makes sense if you consider their intended purpose: informally sharing personal photos. S3 is a more professional image hosting choice; it offers tighter control along with a full set of developer APIs.
...
Dreamhost (for $9.99/month) gives me 2.7 TERAbytes of bandwidth, and 233 GIGAbytes of disk space.
...
I hate to say you get what you pay for, but I'm a Dreamhost customer and their downtime has me in a bit of a funk about recommending them for anything lately.
Dale on March 7, 2007 1:09 AMThese are nice tools for S3, also:
https://jets3t.dev.java.net/cockpit.html
http://s3sync.net/wiki
FYI, S3 account page update. This is with only 4 images public (images from the last 3 posts)
3/7/2007, 3am:
$0.02 with 0.063 GB served
3/7/2007, 2pm:
$0.37 with 1.806 GB served
I think I'll be using a mixture of imageshack.us for older posts, and amazon S3 for newer posts that need more bandwidth than the 100 MB / hour limit of imageshack.us.
Jeff Atwood on March 7, 2007 2:00 AMSo you going to give him a bit of money for his obviously hard work :)
Sarkie on March 7, 2007 3:44 AMI'm sold, but you forgot to include your Amazon referrer code in the links to make some of those hosting costs back. :)
TristanK on March 7, 2007 3:55 AMBTW, I'd love to see the result of these mods on your general traffic profile after a week or two.
TristanK on March 7, 2007 4:14 AMAwesome write up, I'm inspired to move my images to Amazon S3 servers also. I didn't know Amazon.com prices were so cheap and I bet they will only get cheaper.
I'm thinking of doing a online backup of my computer also since it is so cheap. Doesn't anybody else do this already?
Dean on March 7, 2007 4:21 AMIf you use Amazon's S3 service, make sure you absolutely create a cname that points to their domain (as mentioned a few posts up). Otherwise, you'll have adblockers and other spam plugins ignore your images because they're coming from a domain other than codinghorror.com
Cody on March 7, 2007 4:21 AM"There's no concept of folders in S3. You can only emulate folder structures by adding Objects with tricky names like subfolder/myfile.txt. And you can't rename Buckets or Objects, as far as I can tell."
That doesn't sound like anything a decent frontend couldn't take care of.
HeroreV on March 7, 2007 4:37 AMJeff, we know you love components:
IBiz S3 Integrator
http://www.nsoftware.com/ibiz/s3/default.aspx
Jonny Hildo on March 7, 2007 4:46 AMhow do you make a website
jordan on March 7, 2007 4:47 AMS3 sounds good for relatively low transfer usage.
Is there something wrong with my math or am I missing something else here? It seems that once you get over 10 Gb/month (1/3 Gb/day), you're paying more than $2/month.
The Economy Plan from GoDaddy (for example) gives you 250 Gb/month for $3.19 - $3.99, depending on the length of your contract.
From you previous post [http://www.codinghorror.com/blog/archives/000807.html] I gathered that your image transfer rates are more than 2/3 Gb/day (which is the break-even point between GoDaddy and S3).
I must be missing something.
Eugene Katz on March 7, 2007 4:50 AMThis is all well and good but has anyone seen a comparison of latency times? Absolute time from browser image request to completion of download is the most important factor for many sites. You need your site to load in a snappy way, especially if it is image heavy. Infinite free to low cost image storage is of no value if we can't get better performance than hosting the images ourselves.
As we've seen from studies the past six months people have little patience for a page load.
I would expect s3 to have better latency than godaddy, but I'd like to see some numbers. :-)
Joe Pantuso on March 7, 2007 5:01 AMI second Joe's comment.. it'd be great to see a comparison of the performance of S3/regular hosts/the free image sharing services!
Dan on March 7, 2007 5:05 AMBtw, do check this out: http://www.hackszine.com/blog/archive/2007/03/howto_recover_text_from_blurre.html?CMP=OTC-7G2N43923558
Might want to color over your account number instead.
Marcus Stade on March 7, 2007 5:08 AMDo you know if there are any bandwidth usage monitoring apps for S3? It would be nice to quickly know if your usage was going to cause a big bill at the end of the month.
Rich on March 7, 2007 5:11 AMA big of googling turns up this rather damning quote:
"Aside from that, the performance is absolutely terrible outside of the US, which I really noticed since I've been in Taipei. In addition to that, it's simply not reliable. There are extended and unannounced periods of downtime, so you shouldn't be using it in production (except maybe for redundancy or backup purposes)."
http://bob.pythonmac.org/archives/2006/12/06/cachefly-vs-amazon-s3/
Good posting on this, and points out that S3 is US-centric. More than half my members are outside the US so that kind of rules out S3 right there. Cachefly sounds interesting though.
Joe Pantuso on March 7, 2007 5:17 AM@Joe, as someone who tested Cachefly i can testify that they are absolutely horrible. They are also pretty much useless outside of the states, their customer support is poor to non existing etc.
They say they distribute your content everywhere and serve it up from nearest location, in reality they monitor your use, if you have a image that gets hit frequently from the states and less so from europe, the image will always be served from the states, and they do this using 302 redirects to the states from the EU hosting farms introducing even more delay into the whole process. Only when you get a lot of hits on a image/file from the EU will they send it to their EU hosting centres, in my testing this is totally random I've hit a file 500 times from London and it just kept serving from the states, requests sent to them for clarification never got a reply.
I'd avoid cachefly at all costs.
I'd also be keen to hear about performance of S3, my dreamhost account does give me lots of storage and lots of b/w but the hosts they serve the files on are often terribly slow with huge load averages and very poor performance serving HTTP
R.I.Pienaar on March 7, 2007 5:24 AMside note for jeff:
http://dheera.net/projects/blur.php
mosaicing your account number in the image is not a good idea if you want to hide it.
If image bandwidth becomes a real problem, it might be cheaper to use SneakerNet for hosting. Just print the images out and send them by UPS to your readers.
Eam on March 7, 2007 5:44 AM@ R.I.Pienaar
Yeah and Dreamhost will suspend your account with no warning if your site starts to generate some nice traffic. They have mad CPU usage restrictions!
Murray on March 7, 2007 5:49 AMLook into "Virtual Hosting of Buckets" if you decide to stick with S3 for direct static content serving. Makes the urls nicer, more portable, etc.
http://docs.amazonwebservices.com/AmazonS3/2006-03-01/VirtualHosting.html
e.g.
From: http://s3.amazonaws.com/codinghorrorimg/codinghorror-bandwidth-usage.png
To:
http://img.codinghorror.com/codinghorror-bandwidth-usage.png
Did you look into using Akamai at all?
http://www.akamai.com/html/technology/products/http_content.html
kieran on March 7, 2007 6:05 AMAnother S3 tip, although this probably isn't very important.
You can create a CNAME record off your domain and point it to s3.amazonaws.com, creating a branded URL to your S3 bucket. Your bucket name must also match the CNAME + domain name.
So, you could create a CNAME record named "S3" and point it to s3.amazonaws.com. Then, create a bucket called "s3.codinghorror.com" in your S3 account and place all your static stuff there. Then, when you create the reference in your code you can use "s3.codinghorror.com".
Michael A. Vickers on March 7, 2007 6:25 AM@kieran
I think akamai is built for much larger clients. While I don't know their pricing plan off the top of my head this research paper shows that amazon is actually one of THEIR customers. I wonder if Amazon isn't simply buying massive bandwidth from akamai then reselling it to users.
"We focus our efforts on the Akamai CDN, which is perhaps the
most extensive distribution network in the world – claiming over
15,000 servers operating in 69 countries and 1,000 networks [4].
Without Akamai’s CDN, highly popular web enterprises such as
Yahoo, Amazon, or The New York Times would be unable to serve
the gigabytes of data per second required by the images, Flash animations,and videos embedded in their web sites."
from:
http://www.sigcomm.org/sigcomm2006/discussion/getpaper.php?paper_id=39
I can't recommend S3 for media hosting enough. I've been using it for nearly a year to host all the media files for...
http://www.trippermap.com/
All the xml files the map reads are cached to S3 as well as the swf itself and all the high resolution map data layers.
http://www.searchbots.net/
All the flash/swf files and thumbnail images.
I've never had any problems with performance and being based in New Zealand I've not noticed any international issues either.
Apart from taking the load off my server at media temple it means I can move websites around in between hosts without having to move gigs of data. I'm just left running the database and skeleton php templates. I did it not so much to save on bandwidth but to get scalability. When Searchbots was bombarded by digg users the server only had to handle the database calls and all the logic was contained in the flash files hosted at S3. The server barley showed any increase in load.
Setting up a cname record to map a subdomin is very easy so you can serve files from your own domain like...
http://map.trippermap.com/v2/map.swf
Here's the easiest php class I've found for interacting with S3. Although it is based on other PEAR classes which can be a pain to install sometimes if you don't have PEAR up and running.
http://blog.apokalyptik.com/storage3
Mark Zeman on March 7, 2007 6:47 AMAkamai is VERY expensive. You aren't worth their time and server space unless you're paying thousands in bandwidth. I used them under Gannett's Broadcast division. They have some very cool tools but you do pay for them.
If you like Micropayments, you might also look at NearlyFreeSpeech. It's a similar model to S3, in that you only pay for bandwidth/space used, but it's a full web host.
You can't compare GoDaddy and Dreamhost to something like S3 -- they are apples and oranges. For one, you shouldn't patronize GoDaddy (read: www.nodaddy.com), and two, Dreamhost sucks (they just had what, 36 hours or so of almost complete downtime?). With huge hosts you supposedly have "2.2 TB" of bandwidth, but do you honestly think they can allow customers to get anywhere near that? As soon as you start using resources, you're less profitable. S3 and NFS and other micro-payment hosts build their profit into each unit used, there is no overselling.
(This coming from a very old DreamHost customer. My site was shifted from server to server because I used a majority of my plan allowance. Each "server move" caused 24+ hours of effective downtime.)
Ceph on March 7, 2007 6:55 AMWe looked in to Akamai vs Amazon. Amazon turns out to be much cheaper. For the tiny bit of bandwidth and storage this site needs, Amazon might be cool but not the most economical solution. Its gets to be a good value when you have high and low peaks in your bandwidth say 80mb file that you ship every 3-6 months to 300,000 odd users.
Aaron on March 7, 2007 7:12 AMJeff, I would encourage you to dig a little deeper...S3 is having some growing pains because it was designed for occasionally sharing large files, not "often sharing" small files. Ping times can be over a second, sometimes two. Like Aaron said, great for 80 meg files every once in a while, but not for 15k PNGs 100,000 times a day.
Scott Hanselman on March 7, 2007 7:55 AMSounds like S3 might not be the best after reading some of those comments.
What other options are there for offloading images?
JR on March 7, 2007 8:03 AMI'm sorry for spamming your comments, but the notion of uptime concerns triggered me to post this. I've been using Pingdom GIGRIB for uptime statistics: http://uptime.pingdom.com/
It works really well, is easy to use and best of all: it's free! (as in beer)
I'd love a bit more features such as shiny graphs and what not, but that's just gravy really. It's a great tool to get quick'n'dirty stats about a given site's uptime.
for the person who said dream host gives them xxx. webhost4life gives me UNLIMITED bandwidth and 1000 megs of space, for $4.95 a month. THAT would work for photos because 1 gig is fine for lots of small photos, and you have unlimited bandwidth. nooo problem.
Darren Kopp on March 7, 2007 9:05 AMI've been using S3 to host images for a site I just launched http://www.stuffist.com. As you can see from the image load times, the latency of S3 is not so good. While S3 is fantastic for reliability, I'm looking for alternative solutions which can offer better performance.
Rishi Khaitan on March 7, 2007 9:17 AMAmazon S3 is the way to go for hosting static image content, and I'm about to roll out an implementation later this weekend. We'll see how it goes!
Elliott C. Bck on March 7, 2007 9:19 AMI was wondering what would happen if someone leeches your hosted image? Is there a way to control who access the image?
Praveen Reddy on March 7, 2007 9:30 AMS3 still seems to have some problems with speed. When serving greater than 60,000 hits per day, the images load VERY slowly. Except on days where it's so slow it's not even loading. We like the idea of s3, but we switched back to hosting our own images on likebetter.com after too many speed issues. Also, about 1 in 100 loads fail randomly.
Bryan on March 7, 2007 9:31 AMExpect all your images to be blocked by Websense.
John Vance on March 7, 2007 9:48 AMIt would be nice to turn off the "dir listing" in S3.
http://s3.amazonaws.com/codinghorror/
Col. Sanders, eh?
http://s3.amazonaws.com/codinghorror/col sanders.gif
Not sure what you use for screen shots, but I had been using Flickr as an "image hosting" service for a while and wrote a Cropper Plugin to upload the screen shot to flickr and put the URL in the clipboard for easy reference when blogging.
I have recently been using S3 in a project to host media files and like you, have been impressed with the functionality/cost ratio. I am planning on working up a similar plugin for Cropper to send screenshots to a configured Amazon S3 account:
http://www.codeplex.com/cropperplugins/WorkItem/View.aspx?WorkItemId=878
Any tips/feedback/direction on this plugin (or actually development help) would be great.
Patrick Altman on March 7, 2007 11:25 AMIf you want a more automated/command line way to upload images to s3 with public access I suggest using s3sync with the -p option.
I wrote about setting up s3sync here
http://blog.eberly.org/2006/10/09/how-automate-your-backup-to-amazon-s3-using-s3sync/
for private backups, but it would be the same process for public accessible files if you add the -p option to s3sync. I have successfully tested using s3sync for public files using a custom CNAME record - img.mydomain.com - s3.amazonaws.com.
Follow the link mentioned by John Spurlocks above for setting up Virtual hosting with S3.
John Eberly on March 7, 2007 11:45 AMS3Fox is ok, but with larger uploads, you'll want to look elsewhere. I've got over 40G at S3, and from my testing, S3Drive (free) http://www.s3drive.net/ is where you'll want to look. Tell 'em airjrdn sent you. :)
reddit's started doing the same thing with all their images, including the logo, mailbox icon, and up and down arrows, probably since your previous article made its front page. (The people behind reddit show repeated evidence of monitoring their own front page and making use of things promoted to it, usually in one-off reddit logos.)
Peter Hosey on March 7, 2007 12:10 PMBest thing about this: I no longer get big ugly pixelated WTFs instead of your images in my web-based feed reader!
steve minutillo on March 7, 2007 12:32 PMNo matter what happens, I recommend fluids, and continues used of STRONG and EM tags in all text.
doug on March 7, 2007 12:47 PMVery interesting stuff in comments ;-)
Really useful.
I'm as dreamhost customer currently is a bit dissatisfied, but not much. I have 150 GB BW/month and 100 GB storage. All works pretty well, except for upload speed from Europe and periodical downtimes.
We'll look what happens soon when I'll suck all 2.5 TB ;-)
Amazon S3 is a bit pricey for non-profit, compared to these shared megacheap (megashit?) hostings floating around.
If your after cheap storage and bandwidth, your far better off with someplace like POWWEB. (http://www.powweb.com) which currently sells 300gb of storage, and 3000Gb (100Gb a day) of bandwidth a month for only $7.77 a month.
And they're fab - I've been with them for years, and they've only been getting better.
Mike on March 8, 2007 1:19 AMAbout Amazon account numbers: this isn't really secret information, since there's not much anyone can do without unless they have your various private identifiers as well.
You might want to use S3 as a backing store for a cacheing frontend that keeps the most recently used N images locally or something. Of course, this doesn't really buy you anything in terms of bandwidth unless it's running in EC2 where you have free traffic to and from S3.
jerith on March 8, 2007 2:06 AMGah! s/without/with it/
jerith on March 8, 2007 2:44 AMSome people have gone the full monty route (as in all the way, not fully nude) and put their entire website on S3. I was browsing around the other day and after looking at the url I was redirected to I realized everything, the page, css, js and images were all coming off of S3.
While an interesting concept I can't really imagine that's going to be all that great when it comes to search rank or a nice friendly url.
Shawn Oster on March 8, 2007 7:58 AMDean:
I've been using S3 backup for backing up my PCs, and am thinking about using it for backing up a server or two. It's a little clunky in places, but I really like where it's going.
Is it me, or has the CAPTCHA here been stuck on the same word for awhile?
Michael
Ever since you started the image hosting business... I haven't been able to see any of the images.
Joshua on March 9, 2007 6:22 AMI haven't been able to see any of the images
Why? Is this URL blocked for you?
Jeff Atwood on March 9, 2007 9:43 AMHow do I create a bucket to begin with using S3Fox?
Anand Sharma on March 9, 2007 10:50 AMAgree with Joshua. WebSense blocks the images because they're coming from Amazon, and "Shopping" sites are restricted here at work.
KenW on March 9, 2007 11:11 AMDid anyone de-mosaic that account number yet? I've been screaming "ENHANCE! ZOOM! ENHANCE!" at my monitor for 30 minutes now and nothing has happened.
Maybe I should use a flashier operating system?
Trevor on March 10, 2007 3:07 AMJohnVance and KenW:
I submitted a ticket to WebSense to unblock amazonaws.com, which is CLEARLY not a shopping site. Unless you're buying bandwidth, I suppose.
If you're behind WebSense and you are seeing blocked images from amazonaws, I urge you to also file a ticket with them via this online form.
http://www.websense.com/global/en/ProductsServices/MasterDatabase/URLChange.php
Also, I set up a CNAME alias so all future images will be in the format
http://images.codinghorror.com/image.png
Hopefully that will help as well.
Jeff Atwood on March 11, 2007 4:45 AMHere is a ruby script that offers Unix like commands to access the S3 service.
http://designandcode.blogspot.com/2007/01/cli-to-amazon-s3-service-using-ruby.html
Subhachandra on March 15, 2007 7:33 AMPeople comparing amazon to generic cheap hosting are missing the point and don't understand the service.
freak on April 6, 2007 4:01 AMA nice website that you can use to manage your S3 account, including logging and setting the access control policy is a href="http://s3browse.com"http://s3browse.com/a
jvdsandt on April 8, 2007 4:25 AMAnyone come up with a reason to use this instead of the infinitely easier Flickr Pro account, with unlimited storage and bandwidth?
Andrew Swihart on April 8, 2007 7:43 AMHow to store a file with more then 5 mb size in Amazone S3 services?
Anil Desai on June 14, 2007 12:14 PMTry www.onedump.com thats much better image host than imageshack!
jack on June 27, 2007 8:15 AMa href="http://www.yourhostedimages.com"http://www.yourhostedimages.com/a
is a FREE image hosting service. It is quick and easy to upload your images. The uploader will provide codes to add your images to forums, blogs, myspace or web page!
There are many other CDN's worth a look. http://www.LocalMirror.com/ for example... http://www.speedera.com/ and others.
Jeo on July 31, 2007 1:59 PMFor all the bloggers (and podcasters) out there, I have created a plugin that will allow you to easily use Amazon S3 with your WordPress blog. File uploads are transparently saved into your Amazon S3 bucket, and you can browse and manage your bucket all from within the WordPress admin screens.
http://tantannoodles.com/toolkit/wordpress-s3/
Joe on August 7, 2007 10:13 AMAmazon s3 seems to be a great way to host image files. But I'm curious to know what measures can be taken to prevent file leeching from a publicly accessible bucket. Does anyone know of a way?
Ryan on August 11, 2007 11:30 AMHey,
This was very helpful...I scratched my head a bit trying to read through all the amazon stuff, and your post just made my day. THANKS!
Jeff, what was your verdict after using this for a few months? Looking at your current posts, I don't see any s3 hosted images.
Dave on September 6, 2007 10:57 AMThanks for the quick reply.
The site I'm considering using it on has relatively low traffic, but must store gigabytes worth of photos (many of which are never even accessed). So, transactional costs shouldn't be a problem. Paying for gigabytes of ever expanding hosted disk space is killing the site owner though.
Was the performance satisfactory? Reading through the comments here, I'm a little bit worried about that.
Dave on September 6, 2007 10:57 AMHi Dave,
I still like S3, but the change in billing to per-request in addition to per-megabyte was an expensive one for me. I've slowly been phasing it out for cost reasons, and moving towards having a dedicated host somewhere.
Jeff Atwood on September 6, 2007 11:58 AMMy read on aws and we do use it is better reliability. We use to use 1and1.com but they really screwed us several time in the past year after years of flawless service leaving us no other choice than to find a reliable solution for our customers. While the go daddy's of the year may be cheaper I think we will stick with aws, although it could be easier to use. Amazon needs to write some tools. URL mapping is confusing.
Our main company website is hosted at 1and1 and it was down for 5 days so far this year. We also have a Unix server and windows 2003 server at 1and1.com, 1and1 modified their Unix delivery screwing up our server and telling us to upgrade ... yah off to amazon. Now all we need to do is figure out ec3.
BTW - 1and1 was our 6th provider in the past 13 years. Hopefully AWS will be our last.
Although all of my family sites are on 1and1 ... who cares if you cannot see photos of the kids for a few days.
Randy H. on September 21, 2007 3:45 AMFYI I've posted "s3fs" which allows you to mount (via fuse) an Amazon S3 bucket as a local filesystem read/write... a href="http://code.google.com/p/s3fs"http://code.google.com/p/s3fs/a
Randy R. on October 28, 2007 8:29 AMYou may also review valuecdn.com their pricing does look very compelling. At 9 euro cents per gig it's almost the 1/2 of S3. I hope to test them out.... any feedback from existing clients ?
Josh on November 4, 2007 3:08 AM
I think someone mentioned speedera, which has been an akamai property for an internet age.
Hi Jeff
Do you have a follow-up to this article? It appears that you did not end up using S3 for your blog.
Tony on January 4, 2008 2:20 AMThanks for the S3Fox tip! I'm really loving the fact that most of the api is abstracted into a gem for ruby developers which lets us get on and BUILD STUFF, but sometimes, you just need something simple which works.
Chris Mowforth on January 10, 2008 5:44 AMtony, he already wrote in this discussion why S3 was dropped.
lubos on January 10, 2008 6:11 AMShared hosting accounts have limits in short they are not meant for heavy use, files like video, pictures, and audio are heavy traffic. Please don't compared the storage provided by Dreamhost to S3, the two are very different so far S3 has no limitations an does what is meant to do pretty good (store/distribute). For those of you that believe that a hosting account from other providers is a better solution then you have no idea of the limitation in CPU cycles among other limitation by those providers. By the way latency not only happens on S3 side, check your own connection.
S3 was meant for developers and people that can make the best from the service, its all up to us.
Luis on February 2, 2008 9:46 AMI didn't see any replies by anyone regarding the "leeching" question. I am building a Rail app/site that will be storing personal user images and I want to somehow "protect" them from being viewed unless viewed via my sites' application. I've been google'ing for info on this but haven't had any luck. I also can't seem to find anything about this on the S3 site. Anyone?
UptownBen on February 11, 2008 12:19 PMHi UptownBen,
SteadyOffload.com provides an innovative, subtle and convenient way to offload images. The whole mechanism there is quite different from ordinary photosharing sites and S3. Instead of permanently uploading your images to another host, their cachebot crawls your site and mirrors the content in a temporary cache on their servers. All you have to do is include a little piece of 1K JavaScript in the head of HTML pages. The image remains stored on your server while it is being delivered from the SteadyOffload cache. The URL of the cached image on their server is dynamically generated at page loading time, very scrambled and is changing often, so you don’t have to worry about hotlinking. This means that there is an almost non-existent chance that the cached content gets exposed outside of your web application.
It’s definitely worth trying because it’s not a photosharing site like Flickr or Photobucket nor a storage service like S3 but exactly a service for offloading static content.
Watch that:
http://video.google.com/videoplay?docid=-8193919167634099306 (the video shows integration with WordPress, but it is integrable with any other webpage)
http://www.steadyoffload.com
http://codex.wordpress.org/WordPress_Optimization/Offloading
Cost of bandwidth comes under $0.2 per GB - affordable, efficient and convenient. Looks like a startup but lures me very much. Definitely simpler than Amazon S3 and much more suited than Flickr.
Martijn on February 12, 2008 4:02 AMWell, it's also possible to go BANKRUPT with Amazon S3.
Consider this article: http://coding-aloud.blogspot.com/2008/01/going-bankrupt-with-amazon-s3.html
Cheers
Gil Megidish on February 13, 2008 11:06 AMAmazon S3 could be used as alternative daily backup solution, all you need is some tools to upload/download/manage your buckets. For simple S3 upload try an applet such as JS3Upload:
http://www.jfileupload.com/products/js3upload/index.html
Amazon S3 is interesting because it's really cheap.
DONT use dreamhost for anything semi-serious. I run several adult websites and once you start getting a decent flow going they hit you with every excuse possible. They'll try and will shut you down for excessive queries, high traffic per second or a myriad of other reasons to "persuade" you to move to their premium accounts, read PAY more.
Limelight is a much better option for a CDN
Is there any way to use S3 without changing the URL or setting up a subdomain? Can you mod_rewrite or something?
... or use .htconf?
Mark on April 11, 2008 6:24 AMNice way to reduce site Bandwidth Usage
Check the hosting packages in a href=http://www.totoidc.comhttp://www.totoidc.com/a">http://www.totoidc.com/a">http://www.totoidc.comhttp://www.totoidc.com/a
totonet on June 30, 2008 9:08 AMyou can also try an older site that running fat again:
Free Image Hosting Service - Pic4us
is online again and open
http://www.pic4.us
Pictures For Us is a free gallery and easy to use pictures and images hosting service. You are welcome to use our simple form to upload your photos, images, and videos and share them with other people on the Web. For uploading your photos and pictures, please use our free registration service to open an account and start creating your own images and photos gallery.
I tried out s3 storage for a day and it was a nightmare! I spend several days re-engineering my application so that I could use s3 storage.
I had everything working well after some serious 15 hr days of coding. I launched the new version and an hour later s3 storage went down for 5 hrs!
that is way too scary for me. I had worked with amazon cloud servers before and thy were pretty reliable, but this kind of outage totally killed any intentions I have of using s3 storage.
I rolled back my app to the previous version and I will never take that chance with s3 storage again. I'd rather pay the $$ for a server that won't fail for 5 hrs at a time. if all of my music files were on there and something happens to delete them, My application would be screwed and i would lose thousands of users.
way too risky for me!
mp3candy on July 22, 2008 5:36 AMnice article
sam on August 4, 2008 5:13 AMCheck out adrive. You get 50GB with no bandwidth charges for free.
http://www.adrive.com/
Thanks for the review Jeff. And lots of useful stuff in the comments! Has anyone found an easy way to use S3 to stream video into a Wordpress blog? I've been looking at the various plugins, but none seem to make it simple to use S3
Jon on October 28, 2008 6:48 AMwww.willemijns.com/backup.htm
willemijns on November 2, 2008 12:40 PMHi Guys, I am looking into hosting my FLV files on S3 for an online tv website I am building, question: - the costs for bandwidth in the UK are huge and am looking to see if this is a good option. Could be, in extreme circumstance, 1000 people looking at a 2min 2mb FLV video file. Is Amazon up to this or am I better hosting elsewhere?
Hi, there is a new solid S3/FTP GUI tool CrossFTP available for Win, Mac, and Linux. Easy to create/rename/delte/manage the buckets and objects.
Available at http://www.crossftp.com/
I use xuploadz.com
http://xuploadz.com its a great free image hosting service
You may be interested in the command-line tool to use AWS: http://timkay.com/aws/ I use it and like very much.
teleman on January 6, 2009 6:26 AMCheap and easy Ajax-to-S3 direct uploader for .NET.
http:/www.flajaxian.com
Also, free versions to do an Ajax upload to a web server.
Dave on January 21, 2009 1:15 AMoops, forgot the hyperlink
a href=http://www.flajaxian.comhttp://www.flajaxian.com/a">http://www.flajaxian.com/a">http://www.flajaxian.comhttp://www.flajaxian.com/a
ok, last try, i promise :0
sheesh, my fingers no work today
Dave on January 21, 2009 1:17 AMThe comments to this entry are closed.
Subscribe in a reader
Subscribe via email
|
|
Traffic Stats |
