Everybody loves BitTorrent. And rightfully so.
With BitTorrent, you also start by placing your large file on a central server. But once the downloading begins, something magical happens: as clients download the file, they share whatever parts of the file they have with each other. Clients can opportunistically connect with any other client to obtain multiple parts of the file at once. And it scales perfectly: as file size and audience size increases, the bandwidth of the BitTorrent distribution network also increases. Your server does less and less work with each connected client. It's an elegant, egalitarian way of sharing large files with large audiences.BitTorrent radically shifts the economics of distribution. It's one of the most miraculous ideas ever conceived on the internet. As far as I'm concerned, there should be a Nobel prize for computing, and the inventor of BitTorrent should be its first recipient.
I've been a happy consumer of files distributed via BitTorrent for years; it was only natural that I would turn to BitTorrent to distribute our cc-wiki licensed Stack Overflow data. I figured serving a several-hundred megabyte file with BitTorrent wouldn't be much harder than downloading one. Boy, was I ever wrong. Sharing files with BitTorrent is way more complicated than downloading them! After two frustrating hours, I finally came up with a relatively straightforward way to share a file via BitTorrent, and in the interests of saving future readers a little time, I'm documenting it here.
Now, I'm going to show you an easy way, but it isn't technically the easiest way. The easiest way is to let someone else do the sharing for you. If you own content that you want to share, LegalTorrents is the obvious choice:
LegalTorrentstm is an online digital media community.We discover and distribute high quality open-license (Creative Commons) digital media and art, and provide support to Content Creators. We host creative content in its entirety, ensure fast, reliable downloads, and enable users to directly sponsor Content Creators and their work.
We distribute content with the full permission of the rights holders and use the peer-2-peer file-sharing technology called Bittorrent.
The site is still in beta, but signup is a snap, because they support OpenID! I encourage anyone interested to check it out. If nothing else, get the furtive thrill of actually downloading legal content through BitTorrent for once! Yes, it can happen. Shocking, I know. Don't worry, you crazy kids can get right back to your regular non-copyright-respecting torrenting ways immediately afterwards.
Anyway, you can't start sharing files on LegalTorrents without some kind of special email-us-please permission, and I was in a hurry. I wanted to share files via BitTorrent right now. I did, and you can too! But you'll need a few things first:
Here's where I hit a major roadblock: to share files via BitTorrent, you need a tracker.
A BitTorrent tracker is a server that assists in the communication between peers using the BitTorrent protocol. It is also, in the absence of extensions to the original protocol, the only major critical point, as clients are required to communicate with the tracker to initiate downloads. Clients that have already begun downloading also communicate with the tracker periodically to negotiate with newer peers and provide statistics; however, after the initial reception of peer data, peer communication can continue without a tracker.
Without a tracker, you're sort of hosed, as clients will never be able to find your file, much less each other. Unfortunately, most of the freely open, public trackers out there are sort of.. disreputable. And the LegalTorrents tracker won't track files unless they are on its creator whitelist, which involves that manual sign-up process. You've got precious few legit options for tracking, unless you're willing to take a trip to the wrong side of town, and associate yourself and your files with that kind of .. neighborhood. I wasn't.
Fortunately, uTorrent has a solution: you can become your own tracker!
bt.enable_tracker and set it to True
Now, let's create the torrent for the file we want to host, which will point to our newly created tracker.
Now go forth and share your *.torrent file with the world. Share it with anyone and everyone! The more the merrier! Any client that opens your *.torrent file will attempt to connect to your tracker, download your file, and share it with other downloading clients in classic BitTorrent stylee. Pat yourself on the back; you just shared a file with the world using the transformative distribution power of BitTorrent!
But you do have to keep uTorrent running as a desktop application all the time, which is sort of a bummer. What if you wanted to share your file on a server, or via a silent background process? No problem. It's just a few more steps:
instsrv uTorrent "C:\uTorrent\srvany.exe"
Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\uTorrent\Parameters] "Application"="C:\\uTorrent\\uTorrent.exe"
(Obviously, replace the above paths with the actual paths that you installed uTorrent to.)
Bam -- you're sharing files with the world using BitTorrent, even when you're not logged in. You can control everything remotely, too, by navigating your browser to the WebUI URL.
Like so many things in Windows, it ain't pretty, but it gets the job done. It's ironic that BitTorrent, which is justly famous for equalizing the highly asymmetric nature of most people's internet connections, is itself so asymmetric when it comes to sharing: trivially easy to consume, but awkward and confusing to share. That's too bad, because BitTorrent is such a powerful tool for sharing. Hopefully this post demystifies the process a bit!
Jeff,
I hope you didn't use utorrent to be the tracker for SO. If that's what you're running, it will allow *anyone* to use *you* as their tracker for their torrents (unless you did something special), which may include illegal distribution of files. I don't think that's something you want your business to be associated with.
I wrote a c# IIS-based tracker that you're welcome to take, modify as you'd like and use. It is very simple and only allows whitelisted torrents to be hosted. *Warning* It's only at the "proof of concept" stage, but works well enough and is dirt simple to configure and use. It basically doesn't do *anything* but act as a tracker at your announce URL, (no stats page, no listing of available torrents, etc) which is exactly what I needed.
If you're curious, feel free to contact me--you can get my email address from my SO account.
TorgoGuy on June 4, 2009 2:05 AMIn your case: you need to host the files to distribute anyway. Why don't you just opentracker on the same server? It's gonna eat about 0% cpu.
http://erdgeist.org/arts/software/opentracker/
Kosta on June 4, 2009 2:19 AMHey Now Jeff,
Everyone loves bit torrent.
Coding Horror,
Catto
BitTorrent (original python client) works fine, but it was designed by a hacker for other hackers. It would have just been an interesting footnote, if it wasn't for all the other software authors out there writing usable software that is actually documented. Even the original command line interface is cryptic and doesn't follow standard conventions. It looks like it was designed by some freshman college student who's been using Unix for about 6 months.
If it had done properly in the first place, I believe it would be one of the primary file sharing methods today, surpassing HTTP,FTP,e-mail attachments. But really, the interface and documentation are still in a poor state.
The other problem, is that its still thought of a protocol for sharing only illegal material. I think this has to do with the fact that only seriously users are willing to put the time into getting it to work.
I predict in the next 5 years or so we'll see a 2nd-gen bittorrent protocol that will be 100X more popular, only because all the parts that make bittorrent a pain to use are removed from sight.
Casey on June 4, 2009 3:28 AM@offler
Your comment proves my point exactly.
Casey on June 4, 2009 3:31 AM1. Create .torrent file with client.
2. Upload to public tracker.
3. Download .torrent and tell it to download to the folder where you have the data.
4. (Optional) Give people a link to the .torrent.
5. ???
6. Profit!
Seriously, most people who distribute files over Bittorrent don't run their own trackers (myself included). This reads like a guide to creating basic PHP websites from someone who hasn't heard of webhosts.
Oh, and uTorrent *does* offer super-seeding; it's just called initial seeding, instead. Of course, if you're running this off of a server, then you will probably do better with rtorrent, my personal choice, which also supports super-seeding and takes an incredibly low amount of resources.
Xiong Chiamiov on June 4, 2009 3:50 AMXiong, sorry, but saying rtorrent takes low resources is ridiculous.
My experience on an old PC:
X-less Linux (tried RedHat and Debian) with rtorrent (spent weeks trying different configs, asked dozens of people, etc), text mode only, nothing running but rtorrent, 300kb/s and I was at 100% CPU, not even telnet would work anymore.
Windows XP with uTorrent, 30 seconds of config, and I run a smooth 900kb/s at 20% CPU.
J. Stoever on June 4, 2009 4:01 AMThis certainly seems to be taking a fairly hard route to getting your content on a tracker.
I don't think using an existing public tracker, of which there are many of varying reputation, should be ruled out so easily. After all, is putting your content on a tracker nearby to other material any more questionable by association than putting it on the dirty, filthy internet?
Just because most trackers track (not host) both legit and "less-than-legit" content doesn't make them evil. To assert so is subscribing to a very similar notion to the one that is threatening net-neutrality.
Ben Garreros on June 4, 2009 5:32 AMuTorrent was the wrong choice. they don't expect their users to supserseed torrents.
if you want to do that use a full client like azureus.
simono on June 4, 2009 7:24 AMNot on MS Windows? (I'm not.)
http://ashughes.com/?q=node/85
http://www.howtoforge.com/setting-up-the-bti-tracker-bittorrent-tracker-on-fedora-centos-redhat
http://www.howtoforge.com/setting-up-a-php-mysql-powered-bittorrent-tracker-with-rivettracker
http://apcmag.com/create_your_own_torrent_tracker_in_a_few_clicks_with_snakebite.htm
Note, I've never used any of these, and have just included them for completeness.
Tom the pipers daughter on June 4, 2009 7:24 AMyou need to know your external ip + port-forwarding must work. granted.
other than that it's really easy with a fully featured client:
simono on June 4, 2009 7:26 AMYup, µTorrent is awesome, but the heavier Vuze would be the way too go for that purpose.
Lor on June 4, 2009 7:39 AMYou could also use Amazon's S3 to get started quickly. Just upload a file to a bucket, and then add ?torrent to the end of the publicly-available file.
http://docs.amazonwebservices.com/AmazonS3/2006-03-01/index.html?S3Torrent.html
WMS on June 4, 2009 7:40 AMYou could use this approach http://ayende.com/Blog/archive/2008/04/08/Hibernating-Torrent-A-simple-torrent-server-for-Windows.aspx
Anders on June 4, 2009 7:42 AMI've been running my own tracker on my webserver for the last few years with PHPBTTracker, which was incredibly easy to set up. But you need a dedicated server (no shared hosting), so this isn't for everyone.
Andy Baio on June 4, 2009 7:44 AMive had bad experiences with azureus. after leaving it on for more than 12 hours, all IP ports were used up, and I crashed on a regular basis. The problem has not happened since I started using uTorrent.
Also, with Azureus, when I went to the latest version (Vuze i think?) then it was no longer just a torrent client, but it started pushing media on me I didn't want with that horrid first screen, and I had to figure out how to actually get to my torrent screen.
Plus its written in Java, and we all know Java desktop apps are eeeevil.
Heh, it always amazes me when I see things like "yea you have to have port forwarding setup." Dude, if I wanted to expose my computer to the outside world; I should've not bought a router and just stuck my Windows XP SP1 box on the outside world with the Windows firewall on. Come on man, is free music/movies/games/etc... worth the headache of some dude hacking your ass? I guess so, have fun with that one.
Suroot on June 4, 2009 7:52 AMDon't forget about DHT aka tracker-less torrents.
Yay for teh magnet links.
Fred on June 4, 2009 8:03 AMWat's wrong in using http://tracker.thepiratebay.org/announce?
Peter on June 4, 2009 8:04 AMThere is a Nobel Prize for Computing. It is called the Turing Award, see: http://en.wikipedia.org/wiki/Turing_award.
molex333 on June 4, 2009 8:04 AMSuroot: correct me if I'm wrong, but doesn't port forwarding only open access to the bittorrent client application which listens on that port? Which would mean that yes, if your bittorrent client has a bug, you're vulnerable there.
But OTOH if you don't use port forwarding but still connect to other bittorrent clients for downloading, your client is again exposed to attackers - some malicious hacker just needs to share a file you want to download, and bam! you connect to his evil bittorrent client which can then merrily exploit bugs in your client. I don't quite see how port forwarding can make this any worse :-)
"There is a Nobel Prize for Computing. It is called the Turing Award"
But that's not the Nobel Prize, it's the Turing Award. It may be a computing equivalent to the Nobel, but it's not the Nobel.
Ian Menzies on June 4, 2009 8:13 AM@Peter
"Wat's wrong in using http://tracker.thepiratebay.org/announce?"
Actually that is a fair point, one of The Pirate Bay's defenses was that they host legitimate legal content (as well as the other stuff).
-Perros-
Perros on June 4, 2009 8:29 AMIf you use Pando (pando.com), you don't even need to supply a tracker. Download their app, drag files into it, specify a recipient and off they go. You don't have to do any network-y configuration or have any idea how P2P works. Pando sends an email to the recipient and seeds your file for a limited time. Instead of a .torrent file you distribute a .pando file. The only catch is that the recipient needs the Pando app as well.
Full disclosure: I used to work for Pando.
Andy Lee on June 4, 2009 8:31 AMWhy would you share an archive, when you can share the files/directory directly? Unless the compression is good, it's annoying to have to keep both the archive so others can download it, and the extracted files so you can use/see/listen them.
Cristian on June 4, 2009 8:34 AMI've never had any difficulty sharing files over BitTorrent, and I've been using µTorrent since its original beta version. Must be a Vista thing! I only had to forward the port on my wireless router, add an exception to my firewall, and find a reliable tracker. I use Demonoid's private tracker now. And with µTorrent, I can easily schedule appropriate upload times and throttle the bandwidth accordingly. Easy as pie!
Josh Stodola on June 4, 2009 8:35 AMI find this old article http://www.linuxdevcenter.com/pub/a/linux/2005/08/25/bittorrent.html?page=1 from O'Reilly useful for people using Linux, although I think that in can be applied on Windows, too.
[sorry for double posting]
I'm amazed at all the people recommend he use a _heavier_ client on the server. So there's less config? So what! That's kind of the point of a server: set it up once to be as efficient as possible.
Joel Coehoorn on June 4, 2009 8:53 AMThanks! I've been sharing (I think) a few files for some time now. Now I can go back and ensure that they're really being shared (if they're legal [not sure], I'll use LegalTorrents.
Brad on June 4, 2009 8:58 AMOne thing has always bugged me.. Podcasts always seemed like a perfect use for Torrents. I think quite a few people suggested it to Adam Curry when he was at the forefront of podcasting, but I'm pretty sure he resisted because he couldn't figure out a way to make money off that.
It really kind of bugs me that the initial podcasting clients (podcatchers) didn't use p2p, it would have really legitimized the whole p2p thing, saved all the podcasters from having to pay to distribute something they were giving away for free, and stopped Apple from monopolizing the entire podcast distribution system--using it to pull more people into the "Apple Store".
As a second choice, at least it was Apple and not someone who felt it necessary to find some other way to monetize it, but I still wish things had gone the other way.
By the way--cool new capta system, but I'll miss "Orange"
Bill K on June 4, 2009 10:15 AM@Suroot: what kind of idiot are you? Opening a single port to the outside doesn't "open your computer to the outside world" completely, and definitely isn't a wide-open door for hacking. It only lets people connect to the bittorrent client.
If having an open port = being hackable, go hack google.com's port 80 and prove it.
Nicolas on June 4, 2009 10:27 AMSeriously. S3. It looks like you wasted a lot of your time.
1. Upload to S3 using any number of easy tools or browser extensions
2. Share the S3 URL with '?torrent' at the end of it
3. Step 3 not necessary.
Good grief - it *surely* can't be that hard?
DavidD on June 4, 2009 10:59 AMUse http://checkip.dyndns.org/ as a super-lightweight way of checking your IP address. :) No ads, and can be easily parsed by scripts. It's hard wired into many dyndns clients, so that URL isn't going away soon.
Jeff S on June 4, 2009 11:05 AMI guess all of these options still throttled using DPI. BitTorrent? Must be illegal. Thanks Bell Canada!
SlowPeer on June 4, 2009 11:09 AMI use Ares (http://aresgalaxy.sourceforge.net/)
It also downloads files from multiple computers and easy to use, you don't mind about port forwarding and tracking.
Moosty on June 4, 2009 11:11 AMAlthough people may decry it, I find that uTorrent works quite nicely with UPnP on my router - automatically forwarding whatever random port it wants when it starts up, so I don't have to worry about it.
I like it because it just sort of *works* - and isn't that how it's supposed to be??
Keithius on June 4, 2009 11:13 AMI'd like to note that it is possible to serve trackerless torrent using distributed hash table (DHT) method. This unfortunately is not implemented in all clients...
Jakub Narębski on June 4, 2009 11:47 AM@Keithius, UPnP may be nice if you trust all the clients on your internal network. What if friends drop by and connect their (possibly virus infected) laptop to your LAN?
warpr on June 4, 2009 11:54 AMCool. I didn't know about uTorrent acting as a tracker. That thing is tiny small. Oh, and there's nothing wrong with pirate bay. It's legit. The Pirate party is the 3rd largest political party in Sweeden. http://www.piratpartiet.se/storlek
tekproxy on June 4, 2009 11:58 AMI wanted to share a 200mb file once and it was a breeze with uTorrent using The Pirate Bay's tracker.
I just selected the files, entered the tracker's URI, gave my friend the torrent file uTorrent gave me, and that's it.
mmj on June 4, 2009 12:06 PMLegalTorrents also offers dedicated tracking service on 3 separate networks for members. Once the torrent file is formed, the tracker needs to stay up for the sharing swarm to continue to function.
While it works slightly better, you don't have to have any incoming ports forwarded to be the initial seed on a torrent. I'm also confused about what exactly you found so awkward about the torrent creation process, as the steps you outlined are pretty simple.
Seis on June 4, 2009 12:13 PMYeah, as several other comments say, your roadblock was one that most sharers never run into.
Azureus (Currently called Vuze, unless it changed its name again), as a seeder, will actively connect to other peers - no need for port forwarding, although it greatly increases the sharing speed. Using a public tracker would also eliminate the need for playing with setting up your own.
Izkata on June 4, 2009 12:28 PMI found it challenging the first time I wanted to share a file via torrent. After the first time, through utorrent, it wasnt so bad.
Ryan Yockey on June 4, 2009 12:48 PMfunny, i hate bittorrent. If a company offers something to download the company should put a server up and no asssuming that a user installs a client only for downloading something. Linux distris are worse with bittorent.
offler on June 4, 2009 1:46 PMHey Jeff,
I noticed you dismissed portforwarding with a comment about how complicated it is. But it's not! Behold the power of people with too much time:
A guide to pretty much every piece of hardware imaginable with a step by step, screenshots included guide on how to forward ports! Even a computer dummy could do it with this kind of hand-holding ;)
J. Stoever on June 4, 2009 1:47 PMFunny
OpenID -> BAD
Torrent -> WORSE
OpenID+BitTorrent -> HELL
I think a valid argument can be made for not having anything to do with The Pirate Bay or Mininova or any of the various other easy public trackers, because even if the stuff you're sharing is pure as the driven snow, you're still thereby connected with a wretched hive of scum and villainy as far as dumb lawyers who would like to confiscate everything in your house that uses electricity are concerned.
But then again, I'm unconvinced that sharing ANYTHING over BitTorrent is very much less "incriminating", by the standards of idiots, who are so often the ones whose standards get to be applied. And it's, of course, completely ridiculous if you've got lots of pirated torrents running all the time anyway.
I made a torrent of the excellent, and approved-by-the-creator-for-free-distribution, "Secret Life of Machines":
http://dansdata.blogsome.com/2007/11/05/my-very-own-slom-torrent/
...and another one of a similarly-approved issue of "The Skeptic" magazine:
http://dansdata.blogsome.com/2008/04/01/free-magazine/
...and just used the usual pirate trackers.
I would not be surprised if The Pirate Bay hosts more legal torrents than LegalTorrents :-).
Daniel Rutter on June 5, 2009 5:30 AMSeriously, use Azureus. I needed a way to share large backups quickly and quietly with my co-admins, I sat down with the Internet and Azureus and had the entire thing running without using an external tracker in about 30 minutes, without a hitch. Azureus is bloated by default, but the bloat is easily turned off and Azureus then runs like a dream machine with an amazing built in tracker. It uses uPnP (properly) and will find all the nitty gritty details for you. All I had to do was create a .torrent through posting my file to my internal tracker via a cute little Azureus wizard, email them the .torrent, and leave my computer on all night. Everything worked beautifully and at a pretty decent speed - though the Azureus friend option will help this along too.
Derrick Hinkle on June 5, 2009 6:37 AMI wish you would have more "coding" articles rather than articles like this.
Whatever happened to posting interesting programming problems and coding horrors?
jlgosse on June 5, 2009 7:36 AMMisgivings about running a client app as a service. It might behave oddly, and there have been a few security issues in uTorrent in the past:
http://secunia.com/advisories/product/13466/?task=statistics
If the user account is a nice, low-privileged account, perhaps even with file system quotas in place, less bad happen.
Just being paranoid. And hey! What happened to 'orange'? :)
TristanK on June 5, 2009 7:46 AMWhat happened to orange, I can never get this right in first try.
Naunidh on June 5, 2009 10:30 AMI actually took some CS classes at NJIT with the creator of bittorrent (Brahm Cohen)'s dad. He, of course, taught Data Structures and Algorithms. He was a good prof.
Dan on June 5, 2009 11:54 AMtest
Anonymous on June 5, 2009 12:49 PM>Xiong, sorry, but saying rtorrent takes low resources is ridiculous.
>My experience on an old PC:
>X-less Linux (tried RedHat and Debian) with rtorrent (spent weeks trying different configs, asked dozens of people, etc), text mode only, nothing running but rtorrent, 300kb/s and I was at 100% CPU, not even telnet would work anymore.
>Windows XP with uTorrent, 30 seconds of config, and I run a smooth 900kb/s at 20% CPU.
J. Stoever on June 4, 2009 3:01 PM
That sounds more like a problem with how Linux is configured, to me, if those numbers are from the same machine. rtorrent uses less than 1% CPU and 3.5% RAM on my 1.8 GHz Sempron with 1 GB RAM, with a constant 80 kB/s up on ~100 torrents.
Xiong Chiamiov on June 6, 2009 8:51 AM>> Plus its written in Java, and we all know Java desktop apps are eeeevil.
3 points to that guy. he hit the nail on the head. java IS evil!
And uTorrent is lighter on the resources. Iv used both, and i preffer utorrent.
Anonymous on June 6, 2009 10:31 AMCan we start downvoting non-programming-related posts? ;) I used to visit this blog every couple of days in the past, but I think from now on I will only stop by occasionally unless you return to blogging about programming. Come on, Jeff, a torrent guide? :(
Anonymous on June 7, 2009 8:47 AMHow do I seed a torrent to share my files if I can only afford a shared web hosting account?
ConfusedJoe on June 8, 2009 4:25 AMEasiest way is to put it on s3. Check this article
http://docs.amazonwebservices.com/AmazonS3/latest/S3TorrentRetrieve.html
Thank-you for writing this post. This cleared up some things I did not understand about Bit torrent. It seemed like only a way to share files illegaly, but now I know it can be used for other stuff too. They really should simplify the interface so anyone can be their own tracker and share files too. I think legal uses of Bittorrent will grow with time.
pinkturbokitty on June 9, 2009 3:21 AMI know everyone is saying Amazon S3, and that's what I thought when I started reading the article.
But I bet you did it this way for the fun of it :)
Vaibhav on June 10, 2009 3:32 AMI know everyone is saying Amazon S3, and that's what I thought when I started reading the article.
links of london on August 26, 2009 2:20 AMAbercrombie & Fitch on Sale, Hoodies, Jeans, T-Shirts, Pants, Polos abercrombie and fitch abercrombie fitch abercrombie cheap abercrombie fitch Abercrombie Men Tee abercrombie womens polos Abercrombie & Fitch Men, women, and children's clothing
abercrombie and fitch on August 28, 2009 1:57 PMIf it had done properly in the first place, I believe it would be one of the primary file sharing methods today, surpassing HTTP,FTP,e-mail attachments. But really, the interface and documentation are still in a poor state
iç giyim on October 1, 2009 1:54 PMGreat post and draw. Thank you for sharing.
wow power leveling on October 21, 2009 7:12 AMThanks for your information, i have read it, very good!
ed hardy clothing on October 22, 2009 9:25 AMThanks for your information, i have read it, very good!
ed hardy shirts on October 23, 2009 8:07 AMThanks for your information, i have read it, very good!
Very cool! Congrats on the pairing.
street lamps on October 24, 2009 2:52 AMI finally came up with a relatively straightforward way to share a file via BitTorrent, and in the interests of saving future readers a little time, I'm documenting it here.
valves manufacturer on November 11, 2009 1:35 AMLegalTorrents recently launched a member tracking service to anyone can use our tracking service. Details:
http://www.legaltorrents.com/about/member_self_publishing
Don't know for sure but I think that MonoTorrent has a full features torrent tracker.
Check it out! And because it is opensource you can easily create a full featured windows service from it including statistics logging.
Ramon Smits on February 6, 2010 11:16 PMThis information is really helpful. I've been using torrent also and I can really say that it's very fun. You will surely enjoy using it for free. To check more about it just click here.
Edenhost12 on March 15, 2011 4:31 PMThe comments to this entry are closed.
|
|
Traffic Stats |