June 3, 2009
Everybody loves BitTorrent. And rightfully so.
With BitTorrent, you also start by placing your large file on a central server. But once the downloading begins, something magical happens: as clients download the file, they share whatever parts of the file they have with each other. Clients can opportunistically connect with any other client to obtain multiple parts of the file at once. And it scales perfectly: as file size and audience size increases, the bandwidth of the BitTorrent distribution network also increases. Your server does less and less work with each connected client. It's an elegant, egalitarian way of sharing large files with large audiences.
BitTorrent radically shifts the economics of distribution. It's one of the most miraculous ideas ever conceived on the internet. As far as I'm concerned, there should be a Nobel prize for computing, and the inventor of BitTorrent should be its first recipient.
I've been a happy consumer of files distributed via BitTorrent for years; it was only natural that I would turn to BitTorrent to distribute our cc-wiki licensed Stack Overflow data. I figured serving a several-hundred megabyte file with BitTorrent wouldn't be much harder than downloading one. Boy, was I ever wrong. Sharing files with BitTorrent is way more complicated than downloading them! After two frustrating hours, I finally came up with a relatively straightforward way to share a file via BitTorrent, and in the interests of saving future readers a little time, I'm documenting it here.
Now, I'm going to show you an easy way, but it isn't technically the easiest way. The easiest way is to let someone else do the sharing for you. If you own content that you want to share, LegalTorrents is the obvious choice:
LegalTorrentstm is an online digital media community.
We discover and distribute high quality open-license (Creative Commons) digital media and art, and provide support to Content Creators. We host creative content in its entirety, ensure fast, reliable downloads, and enable users to directly sponsor Content Creators and their work.
We distribute content with the full permission of the rights holders and use the peer-2-peer file-sharing technology called Bittorrent.
The site is still in beta, but signup is a snap, because they support OpenID! I encourage anyone interested to check it out. If nothing else, get the furtive thrill of actually downloading legal content through BitTorrent for once! Yes, it can happen. Shocking, I know. Don't worry, you crazy kids can get right back to your regular non-copyright-respecting torrenting ways immediately afterwards.
Anyway, you can't start sharing files on LegalTorrents without some kind of special email-us-please permission, and I was in a hurry. I wanted to share files via BitTorrent right now. I did, and you can too! But you'll need a few things first:
- A copy of uTorrent (it's free!)
- Your external IP address; if you don't know what it is, use http://www.whatismyipaddress.com to find out.
- The uTorrent listen port. This is under Options | Preferences | Connection. This is typically set randomly every time uTorrent starts, so you may want to specify a more memorable value here.
- You must have port forwarding properly configured so the outside world can get to your IP address and the port specified above. A full discussion of how to do this is outside the scope of this post, but it usually starts with your firewall settings and/or router configuration. uTorrent has a fairly nice help page at Options | Speed Guide that's a good start; just click the Test if port is forwarded properly button on that dialog to begin.
Here's where I hit a major roadblock: to share files via BitTorrent, you need a tracker.
A BitTorrent tracker is a server that assists in the communication between peers using the BitTorrent protocol. It is also, in the absence of extensions to the original protocol, the only major critical point, as clients are required to communicate with the tracker to initiate downloads. Clients that have already begun downloading also communicate with the tracker periodically to negotiate with newer peers and provide statistics; however, after the initial reception of peer data, peer communication can continue without a tracker.
Without a tracker, you're sort of hosed, as clients will never be able to find your file, much less each other. Unfortunately, most of the freely open, public trackers out there are sort of.. disreputable. And the LegalTorrents tracker won't track files unless they are on its creator whitelist, which involves that manual sign-up process. You've got precious few legit options for tracking, unless you're willing to take a trip to the wrong side of town, and associate yourself and your files with that kind of .. neighborhood. I wasn't.
Fortunately, uTorrent has a solution: you can become your own tracker!
- in uTorrent, go to Options | Preferences | Advanced.
- Scroll down to
bt.enable_tracker and set it to
- Restart uTorrent.
Now, let's create the torrent for the file we want to host, which will point to our newly created tracker.
- In uTorrent, click the Create New Torrent button.
- Select the file or directory you want to share.
- Enter your tracker in this format: http://my-ip-address:my-port/announce
- That's it! Click Create and save the new *.torrent file you've created.
Now go forth and share your *.torrent file with the world. Share it with anyone and everyone! The more the merrier! Any client that opens your *.torrent file will attempt to connect to your tracker, download your file, and share it with other downloading clients in classic BitTorrent stylee. Pat yourself on the back; you just shared a file with the world using the transformative distribution power of BitTorrent!
But you do have to keep uTorrent running as a desktop application all the time, which is sort of a bummer. What if you wanted to share your file on a server, or via a silent background process? No problem. It's just a few more steps:
- Enable the uTorrent web interface under Preferences, Web UI. Note that the URL for it is, by default, http://my-ip-address:my-port/gui/, and it requires a username and password to be set here.
- Obtain a copy of the user-defined service utilities, srvany.exe and instsrv.exe. Copy them to the same folder as uTorrent.exe.
- Issue this command to make uTorrent run as a service:
instsrv uTorrent "C:\uTorrent\srvany.exe"
- Enter this registry file to set the path for the service named "uTorrent" you just created in the previous step:
Windows Registry Editor Version 5.00
- In Control Panel, Services, set the account that the uTorrent service will run under. Note that you must use the same account that you set uTorrent options with if you want the service to respect those settings, so plan accordingly.
- Start the uTorrent service.
(Obviously, replace the above paths with the actual paths that you installed uTorrent to.)
Bam -- you're sharing files with the world using BitTorrent, even when you're not logged in. You can control everything remotely, too, by navigating your browser to the WebUI URL.
Like so many things in Windows, it ain't pretty, but it gets the job done. It's ironic that BitTorrent, which is justly famous for equalizing the highly asymmetric nature of most people's internet connections, is itself so asymmetric when it comes to sharing: trivially easy to consume, but awkward and confusing to share. That's too bad, because BitTorrent is such a powerful tool for sharing. Hopefully this post demystifies the process a bit!
Posted by Jeff Atwood
I hope you didn't use utorrent to be the tracker for SO. If that's what you're running, it will allow *anyone* to use *you* as their tracker for their torrents (unless you did something special), which may include illegal distribution of files. I don't think that's something you want your business to be associated with.
I wrote a c# IIS-based tracker that you're welcome to take, modify as you'd like and use. It is very simple and only allows whitelisted torrents to be hosted. *Warning* It's only at the "proof of concept" stage, but works well enough and is dirt simple to configure and use. It basically doesn't do *anything* but act as a tracker at your announce URL, (no stats page, no listing of available torrents, etc) which is exactly what I needed.
If you're curious, feel free to contact me--you can get my email address from my SO account.
Hey Now Jeff,
Everyone loves bit torrent.
BitTorrent (original python client) works fine, but it was designed by a hacker for other hackers. It would have just been an interesting footnote, if it wasn't for all the other software authors out there writing usable software that is actually documented. Even the original command line interface is cryptic and doesn't follow standard conventions. It looks like it was designed by some freshman college student who's been using Unix for about 6 months.
If it had done properly in the first place, I believe it would be one of the primary file sharing methods today, surpassing HTTP,FTP,e-mail attachments. But really, the interface and documentation are still in a poor state.
The other problem, is that its still thought of a protocol for sharing only illegal material. I think this has to do with the fact that only seriously users are willing to put the time into getting it to work.
I predict in the next 5 years or so we'll see a 2nd-gen bittorrent protocol that will be 100X more popular, only because all the parts that make bittorrent a pain to use are removed from sight.
Your comment proves my point exactly.
1. Create .torrent file with client.
2. Upload to public tracker.
3. Download .torrent and tell it to download to the folder where you have the data.
4. (Optional) Give people a link to the .torrent.
Seriously, most people who distribute files over Bittorrent don't run their own trackers (myself included). This reads like a guide to creating basic PHP websites from someone who hasn't heard of webhosts.
Oh, and uTorrent *does* offer super-seeding; it's just called initial seeding, instead. Of course, if you're running this off of a server, then you will probably do better with rtorrent, my personal choice, which also supports super-seeding and takes an incredibly low amount of resources.
Xiong, sorry, but saying rtorrent takes low resources is ridiculous.
My experience on an old PC:
X-less Linux (tried RedHat and Debian) with rtorrent (spent weeks trying different configs, asked dozens of people, etc), text mode only, nothing running but rtorrent, 300kb/s and I was at 100% CPU, not even telnet would work anymore.
Windows XP with uTorrent, 30 seconds of config, and I run a smooth 900kb/s at 20% CPU.
@Jeff S: An even simpler site to check your IP address is http://www.whatismyip.org/
I can't vouch for its reliability, though it has been around for a while now :)
This certainly seems to be taking a fairly hard route to getting your content on a tracker.
I don't think using an existing public tracker, of which there are many of varying reputation, should be ruled out so easily. After all, is putting your content on a tracker nearby to other material any more questionable by association than putting it on the dirty, filthy internet?
Just because most trackers track (not host) both legit and "less-than-legit" content doesn't make them evil. To assert so is subscribing to a very similar notion to the one that is threatening net-neutrality.
uTorrent was the wrong choice. they don't expect their users to supserseed torrents.
if you want to do that use a full client like azureus.
Yup, µTorrent is awesome, but the heavier Vuze would be the way too go for that purpose.
I've been running my own tracker on my webserver for the last few years with PHPBTTracker, which was incredibly easy to set up. But you need a dedicated server (no shared hosting), so this isn't for everyone.
ive had bad experiences with azureus. after leaving it on for more than 12 hours, all IP ports were used up, and I crashed on a regular basis. The problem has not happened since I started using uTorrent.
Also, with Azureus, when I went to the latest version (Vuze i think?) then it was no longer just a torrent client, but it started pushing media on me I didn't want with that horrid first screen, and I had to figure out how to actually get to my torrent screen.
Plus its written in Java, and we all know Java desktop apps are eeeevil.
Heh, it always amazes me when I see things like "yea you have to have port forwarding setup." Dude, if I wanted to expose my computer to the outside world; I should've not bought a router and just stuck my Windows XP SP1 box on the outside world with the Windows firewall on. Come on man, is free music/movies/games/etc... worth the headache of some dude hacking your ass? I guess so, have fun with that one.
Don't forget about DHT aka tracker-less torrents.
Yay for teh magnet links.
Suroot: correct me if I'm wrong, but doesn't port forwarding only open access to the bittorrent client application which listens on that port? Which would mean that yes, if your bittorrent client has a bug, you're vulnerable there.
But OTOH if you don't use port forwarding but still connect to other bittorrent clients for downloading, your client is again exposed to attackers - some malicious hacker just needs to share a file you want to download, and bam! you connect to his evil bittorrent client which can then merrily exploit bugs in your client. I don't quite see how port forwarding can make this any worse :-)
"There is a Nobel Prize for Computing. It is called the Turing Award"
But that's not the Nobel Prize, it's the Turing Award. It may be a computing equivalent to the Nobel, but it's not the Nobel.
"Wat's wrong in using http://tracker.thepiratebay.org/announce?"
Actually that is a fair point, one of The Pirate Bay's defenses was that they host legitimate legal content (as well as the other stuff).
If you use Pando (pando.com), you don't even need to supply a tracker. Download their app, drag files into it, specify a recipient and off they go. You don't have to do any network-y configuration or have any idea how P2P works. Pando sends an email to the recipient and seeds your file for a limited time. Instead of a .torrent file you distribute a .pando file. The only catch is that the recipient needs the Pando app as well.
Full disclosure: I used to work for Pando.
Why would you share an archive, when you can share the files/directory directly? Unless the compression is good, it's annoying to have to keep both the archive so others can download it, and the extracted files so you can use/see/listen them.
I've never had any difficulty sharing files over BitTorrent, and I've been using µTorrent since its original beta version. Must be a Vista thing! I only had to forward the port on my wireless router, add an exception to my firewall, and find a reliable tracker. I use Demonoid's private tracker now. And with µTorrent, I can easily schedule appropriate upload times and throttle the bandwidth accordingly. Easy as pie!
I'm amazed at all the people recommend he use a _heavier_ client on the server. So there's less config? So what! That's kind of the point of a server: set it up once to be as efficient as possible.
Thanks! I've been sharing (I think) a few files for some time now. Now I can go back and ensure that they're really being shared (if they're legal [not sure], I'll use LegalTorrents.
One thing has always bugged me.. Podcasts always seemed like a perfect use for Torrents. I think quite a few people suggested it to Adam Curry when he was at the forefront of podcasting, but I'm pretty sure he resisted because he couldn't figure out a way to make money off that.
It really kind of bugs me that the initial podcasting clients (podcatchers) didn't use p2p, it would have really legitimized the whole p2p thing, saved all the podcasters from having to pay to distribute something they were giving away for free, and stopped Apple from monopolizing the entire podcast distribution system--using it to pull more people into the "Apple Store".
As a second choice, at least it was Apple and not someone who felt it necessary to find some other way to monetize it, but I still wish things had gone the other way.
By the way--cool new capta system, but I'll miss "Orange"
@Suroot: what kind of idiot are you? Opening a single port to the outside doesn't "open your computer to the outside world" completely, and definitely isn't a wide-open door for hacking. It only lets people connect to the bittorrent client.
If having an open port = being hackable, go hack google.com's port 80 and prove it.
Seriously. S3. It looks like you wasted a lot of your time.
1. Upload to S3 using any number of easy tools or browser extensions
2. Share the S3 URL with '?torrent' at the end of it
3. Step 3 not necessary.
Good grief - it *surely* can't be that hard?
Use http://checkip.dyndns.org/ as a super-lightweight way of checking your IP address. :) No ads, and can be easily parsed by scripts. It's hard wired into many dyndns clients, so that URL isn't going away soon.
I guess all of these options still throttled using DPI. BitTorrent? Must be illegal. Thanks Bell Canada!
Although people may decry it, I find that uTorrent works quite nicely with UPnP on my router - automatically forwarding whatever random port it wants when it starts up, so I don't have to worry about it.
I like it because it just sort of *works* - and isn't that how it's supposed to be??
I'd like to note that it is possible to serve trackerless torrent using distributed hash table (DHT) method. This unfortunately is not implemented in all clients...
@Keithius, UPnP may be nice if you trust all the clients on your internal network. What if friends drop by and connect their (possibly virus infected) laptop to your LAN?
Cool. I didn't know about uTorrent acting as a tracker. That thing is tiny small. Oh, and there's nothing wrong with pirate bay. It's legit. The Pirate party is the 3rd largest political party in Sweeden. http://www.piratpartiet.se/storlek
I wanted to share a 200mb file once and it was a breeze with uTorrent using The Pirate Bay's tracker.
I just selected the files, entered the tracker's URI, gave my friend the torrent file uTorrent gave me, and that's it.
LegalTorrents also offers dedicated tracking service on 3 separate networks for members. Once the torrent file is formed, the tracker needs to stay up for the sharing swarm to continue to function.
While it works slightly better, you don't have to have any incoming ports forwarded to be the initial seed on a torrent. I'm also confused about what exactly you found so awkward about the torrent creation process, as the steps you outlined are pretty simple.
Yeah, as several other comments say, your roadblock was one that most sharers never run into.
Azureus (Currently called Vuze, unless it changed its name again), as a seeder, will actively connect to other peers - no need for port forwarding, although it greatly increases the sharing speed. Using a public tracker would also eliminate the need for playing with setting up your own.
I found it challenging the first time I wanted to share a file via torrent. After the first time, through utorrent, it wasnt so bad.
funny, i hate bittorrent. If a company offers something to download the company should put a server up and no asssuming that a user installs a client only for downloading something. Linux distris are worse with bittorent.
I noticed you dismissed portforwarding with a comment about how complicated it is. But it's not! Behold the power of people with too much time:
A guide to pretty much every piece of hardware imaginable with a step by step, screenshots included guide on how to forward ports! Even a computer dummy could do it with this kind of hand-holding ;)
OpenID -> BAD
Torrent -> WORSE
OpenID+BitTorrent -> HELL
I think a valid argument can be made for not having anything to do with The Pirate Bay or Mininova or any of the various other easy public trackers, because even if the stuff you're sharing is pure as the driven snow, you're still thereby connected with a wretched hive of scum and villainy as far as dumb lawyers who would like to confiscate everything in your house that uses electricity are concerned.
But then again, I'm unconvinced that sharing ANYTHING over BitTorrent is very much less "incriminating", by the standards of idiots, who are so often the ones whose standards get to be applied. And it's, of course, completely ridiculous if you've got lots of pirated torrents running all the time anyway.
I made a torrent of the excellent, and approved-by-the-creator-for-free-distribution, "Secret Life of Machines":
...and another one of a similarly-approved issue of "The Skeptic" magazine:
...and just used the usual pirate trackers.
I would not be surprised if The Pirate Bay hosts more legal torrents than LegalTorrents :-).
Seriously, use Azureus. I needed a way to share large backups quickly and quietly with my co-admins, I sat down with the Internet and Azureus and had the entire thing running without using an external tracker in about 30 minutes, without a hitch. Azureus is bloated by default, but the bloat is easily turned off and Azureus then runs like a dream machine with an amazing built in tracker. It uses uPnP (properly) and will find all the nitty gritty details for you. All I had to do was create a .torrent through posting my file to my internal tracker via a cute little Azureus wizard, email them the .torrent, and leave my computer on all night. Everything worked beautifully and at a pretty decent speed - though the Azureus friend option will help this along too.
I wish you would have more "coding" articles rather than articles like this.
Whatever happened to posting interesting programming problems and coding horrors?
Misgivings about running a client app as a service. It might behave oddly, and there have been a few security issues in uTorrent in the past:
If the user account is a nice, low-privileged account, perhaps even with file system quotas in place, less bad happen.
Just being paranoid. And hey! What happened to 'orange'? :)
What happened to orange, I can never get this right in first try.
I actually took some CS classes at NJIT with the creator of bittorrent (Brahm Cohen)'s dad. He, of course, taught Data Structures and Algorithms. He was a good prof.
>Xiong, sorry, but saying rtorrent takes low resources is ridiculous.
>My experience on an old PC:
>X-less Linux (tried RedHat and Debian) with rtorrent (spent weeks trying different configs, asked dozens of people, etc), text mode only, nothing running but rtorrent, 300kb/s and I was at 100% CPU, not even telnet would work anymore.
>Windows XP with uTorrent, 30 seconds of config, and I run a smooth 900kb/s at 20% CPU.
J. Stoever on June 4, 2009 3:01 PM
That sounds more like a problem with how Linux is configured, to me, if those numbers are from the same machine. rtorrent uses less than 1% CPU and 3.5% RAM on my 1.8 GHz Sempron with 1 GB RAM, with a constant 80 kB/s up on ~100 torrents.
>> Plus its written in Java, and we all know Java desktop apps are eeeevil.
3 points to that guy. he hit the nail on the head. java IS evil!
And uTorrent is lighter on the resources. Iv used both, and i preffer utorrent.
Can we start downvoting non-programming-related posts? ;) I used to visit this blog every couple of days in the past, but I think from now on I will only stop by occasionally unless you return to blogging about programming. Come on, Jeff, a torrent guide? :(
How do I seed a torrent to share my files if I can only afford a shared web hosting account?
Thank-you for writing this post. This cleared up some things I did not understand about Bit torrent. It seemed like only a way to share files illegaly, but now I know it can be used for other stuff too. They really should simplify the interface so anyone can be their own tracker and share files too. I think legal uses of Bittorrent will grow with time.
I know everyone is saying Amazon S3, and that's what I thought when I started reading the article.
But I bet you did it this way for the fun of it :)
I know everyone is saying Amazon S3, and that's what I thought when I started reading the article.
Abercrombie & Fitch on Sale, Hoodies, Jeans, T-Shirts, Pants, Polos abercrombie and fitch abercrombie fitch abercrombie cheap abercrombie fitch Abercrombie Men Tee abercrombie womens polos Abercrombie & Fitch Men, women, and children's clothing
If it had done properly in the first place, I believe it would be one of the primary file sharing methods today, surpassing HTTP,FTP,e-mail attachments. But really, the interface and documentation are still in a poor state
Great post and draw. Thank you for sharing.
Thanks for your information, i have read it, very good！
Thanks for your information, i have read it, very good！
Thanks for your information, i have read it, very good！
Very cool! Congrats on the pairing.
I finally came up with a relatively straightforward way to share a file via BitTorrent, and in the interests of saving future readers a little time, I'm documenting it here.
Don't know for sure but I think that MonoTorrent has a full features torrent tracker.
Check it out! And because it is opensource you can easily create a full featured windows service from it including statistics logging.
This information is really helpful. I've been using torrent also and I can really say that it's very fun. You will surely enjoy using it for free. To check more about it just click here.