March 6, 2007
In Reducing Your Website's Bandwidth Usage, I concluded that my best outsourced image hosting option was Amazon's S3 or Simple Storage Service.
S3 is a popular choice for startups. For example, SmugMug uses S3 as their primary data storage source. There have been a few minor S3-related bumps at SmugMug, but overall the prognosis appears to be good. After experimenting with S3 myself, I'm sold. The costs are reasonable:
- No start up fees, no minimum charge
- $0.15 per GB for each month of storage
- $0.20 per GB of data transferred
It's not exactly unlimited bandwidth, but I was planning to spend $2 a month on image hosting anyway. That buys me 10 GB per month of highly reliable, pure file transfer bandwidth through S3. Beyond that, it's straight pay-as-you-go.
Unfortunately, Amazon doesn't provide a GUI or command-line application for easily transferring files to S3; it's only a set of SOAP and REST APIs.
There is Jungle Disk, which allows S3 to show up as a virtual drive on your computer, but Jungle Disk offers no way to make files accessible through public HTTP. And as I found out later, Jungle Disk also uses a strange, proprietary file naming and storage structure on S3 when you view it directly. Jungle Disk is a fine backup and offline storage tool (particularly considering how cheap S3 disk storage costs are), but it doesn't offer the level of control that I need.
Amazon does provide a number of API code samples in various languages, along with some links to tutorial articles, but beyond that, you're basically on your own. Or so I thought.
That was before I found the S3Fox Organizer for FireFox.
S3Fox is like a dream come true after futzing around with the S3 API. Using S3Fox, I can easily experiment with the S3 service to figure out how it works instead of spending my time struggling with arcane S3 API calls in a development environment. It's a shame Amazon doesn't offer a tool like this to help people understand what the S3 service is and how it works.
At any rate, my goal is to use S3 as an image hosting service. I started by uploading an entire folder of images with S3Fox. I had a few problems where S3Fox would mysteriously fail in the middle of a transfer, forcing me to exit all the way out of Firefox. Fortunately, S3Fox also has folder synchronization support, so I simply restarted the entire transfer and told it to skip all files that were already present in S3. After a few restarts, all the files were successfully uploaded. I then granted anonymous access to the entire folder and all of its contents. This effectively exposed all the uploaded images through the public S3 site URL:
All S3 content has to go in what Amazon calls a "Bucket". Bucket names must be globally unique, and you can only create a maximum of 100 Buckets per account. It's easy to see why when you form the next part of the URL:
Each Bucket can hold an unlimited number of "Objects" of essentially unlimited size, with as much arbitrary key-value pair metadata as you want attached. Objects default to private access, but they have explicit access control lists (for Amazon accounts only), and you can make them public. Once we've added an Object, if we grant public read permission to it, we can then access it via the complete site / Bucket / Object URL:
There's no concept of folders in S3. You can only emulate folder structures by adding Objects with tricky names like
subfolder/myfile.txt. And you can't rename Buckets or Objects, as far as I can tell. But at least I can control the exact filenames, which I was unable to do with any other image hosting service.
In my testing I ended up uploading the entire contents of my /images folder twice. That cost me a whopping two cents according to my real-time S3 account statement:
It's almost like micropayments in action.
S3 will probably end up costing me slightly more than the "Unlimited" $25/year accounts available on popular personal photo sharing sites. With S3, there's no illusion of unlimited bandwidth use unconstrained by cost. And personal photo and image sharing sites are often blocked by corporate networks, which makes sense if you consider their intended purpose: informally sharing personal photos. S3 is a more professional image hosting choice; it offers tighter control along with a full set of developer APIs.
Posted by Jeff Atwood
If you are on Windows you can use CloudBerry Explorer for Amazon S3. With FTP like client it makes managing files in S3 EASY
http://cloudberrylab.com/ It supports most of the Amazon S3 and CloudFront features and It is a FREEWARE
I just S3 for file hosting of my mp3 blog, and I was using S3Fox for awhile and wanted to switch to JungleDisk, but there's no way to manipulate the ACLs using JD, which sucks.
S3Fox is sooo slow, it totally crushes Firefox for me, and my computer is fairly decent. Does anyone know why it does that and how to solve it? Thanks.
I use Amazon S3 for my website:
The reliability is amazing, and my monthly bills never cross 50 cents! I recommended this service to anyone.
Have a look to JS3Explorer. It's an Applet that provides S3 bucket file manager. It comes with a feature you won't see anywhere else. It can simulate folders, it's not new ? But you can see flat listing and fix filename issues generated by third party S3 applications.
I am using and i highly recommend this free image hosting service : http://xtupload.com . They are completely free moreover their offer unlimited bandwidth , disk space , a advanced online editor , image spin up , gallery etc ... so do not waste your time and money and enjoy the free service
Wow, information overload.... plenty of good views downs and ups on here! now i have what 20 tabs open from links on this page... Awsome! the search for truth on S3 continues...
why use S3 ? if you are worried about your own bandwidth , use others bandwidth, I use a image hosting and sharing service that it rocks : http://www.cerzo.com
besides that free image hosting service that I told you 2 days ago, I found another brand new , promising image host : http://www.my-image-host.com
I use S3Toolbox to move my files to Amazon S3.
It still misses some features but it is free and much more reliable than S3Fox
For those of you that keep pointing to free image hosts, what happens when those free images hosts go under or decide to stop hosting images a few years from now (or sooner) and you have like hundreds of images stored on those sites?
With Amazon S3, that's not an issue. AND IT'S FUCKING CHEAP!
Why is it not an issue? Because you're storing to a subdomain. So if for some reason Amazon were to go kaput, just find another host, use the same subdomain, and you're BACK.
Of course, for that to work you should be saving the files on your HDD in the same hierarchy but only a dummy wouldn't do that. :)
"Dear HostanyImage.com Users,
It’s with unbearable sadness I’m here to inform you all that www.hostanyimage.com was a victim of recent HyperVM hacking incident. 100, 000 of websites which were hosted on VPS servers were hacked and suffered complete data loss, and the news is the owner of LxLabs committed suicide, whom apparently has hacked (as per rumors). "
While S3 is stable and reasonably priced, I still wonder why people are willing to work with manipulating their sites to conform to the S3 structures. Conventional CDN's are much better. The best ones only require that you add/change a dns entry.
Just thought I'd toss in my 2 cents. I've just completed a full migration to the Amazon data centers...so you can call me a fanboy. :)
For dynamic web pages, I host them on an Amazon EC2 instance.
For static images/videos/etc, I host them on Amazon S3. For the one's that get hit frequently, I'm using the Amazon Cloudfront service which pulls your S3 files to a worldwide collection of edge servers.
So far, response time has been over twice as good as my last ISP. Color me happy!
Dreamhost (for $9.99/month) gives me 2.7 TERAbytes of bandwidth, and 233 GIGAbytes of disk space.
Seems like a better deal to me if I'm reading the S3 pricing correctly.
I am starting to get into S3 stuff, and have seen some slow loading from amazon, as opposed to locally hosting. The costs are nice compared to my hosting, but my latest idea is to use a local script to check various cheapo hosts and redirect to whichever host is up, thus avoiding the down-time problem of 1and1.
I am launching a site in the next week or so, so I'll know better then, but so far, the tests look really good - and since I am just using 1and1, etc. for static hosting, I shouldn't run into the shutdowns due to the minimal CPU limits. I wondered if 1and1 would complain about a constant 6Mbps connection for a couple days, and they didn't, except for my ftp script getting killed once a day or so.
If it works, I think it will be much better than Amazon's services, since the files are easy to get to ssh/ftp/rsync/etc and can run some mysql or php on the same site if you needed to.
Great post thanks
I would also like to add the there is a new Free S3 analytics application called Prism from sisense
Its on private beta but there are places on the web you can get the code
Nice article. We've just started using S3 for images/css/js to reduce bandwitdh from the main servers.
Jeff, this article was a great find. I'm on the Amazon list so when I begin doing some serious traffic, I plan on using this service for images.
I enjoyed reading other alternative solutions. It prompted me to look into another possible solution which I consider a poor man's way to host images.
After doing some website optimization techniques I learned from Yahoo, the literature described about using a Content Distribution Network. I did a Google search and came across The Coral Content Distribution Network.
I tested the service for my images and flash animation on at least three of my websites.
Since these three web sites contain more images than my other sites, it provided me with a decent testing ground. The downside to using the Coral service was that I found it wasn't consistently dependable. It seemed to work only 90% of the time; I guess I shouldn't have too high of expectations for something that is a FREE service.
Anyway, I was thinking one work around could be, I would monitor my sites uptime and if my site was ever down for whatever reason, I can automatically rewrite my apache .htaccess file (i.e., via a script) to load the images and flash using the Coral Network.
I have a shared hosting account. Does anyone know if your site is temporary down, can you still FTP into it? I appreciate any feedback.
I know this article is an old one, but it is super-helpful. Thanks!
Last year sign up for a for free and I like "MPMGC Websites"
http://mpmgc.ws very easy to understand and also has a cool
web builder. Also they gave me a free blog and 100 secured
emails I can create all types email name. I'm still going
through the site because it has a HUGE products and services
section. The customer service staff are great and very
seriously.. thanks for putting this up. i was on amazons page for about half an hour before trying to look on youtube. amazon should link to this video. i've actually made progress thanks to you.
If you want to run image hosting site Amazon S3 is the most expensive solution. My image hosting site uses over 2TB a month and i do not allow hotlinking anymore. with S3 it would cost me somewhere around $500. How many dedicated servers can you have for such an amount of money? min 3 good ones. S3 is a good solution for any other static content or site content images but not for image hosting.