The Freebase G2 Cache

What is it?

The G2 cache is a php script I wrote to kickstart magnet downloads from Freebase. It gives out alt sources to the peers that connect to it, and saves the ip addresses of these, so they can in turn be used as alt sources for the next. It basically hosts a cache of anyone that downloaded a certain file recently.

Privacy implications

Due to the nature of being a cache, it remembers both your ip, and the hash of what you downloaded for a maximal time of 7 days. So it wouldn't be smart to use the cache for files that you don't want other people to know you downloaded (even though Bittorrent does the same thing, and that's not stopping some people from using that for warez).

How do I use it?

Well, it just works by appending the url of the cache ( http://cache.freebase.be/[sha1 hash of the file] ) as an external source ( &xs= ) to a magnet (or use the add source feature of your g2 client).

So a working example would be:

So when someone clicks the magnet, his client contacts the cache. The cache then does 2 things:

• It records the IP address of the client that just contacted the cache for future use
• It sends alt sources for the same file out of its cache

So this means you get the people that recently clicked the magnet as an alt source without searching the network for the file. Your download should start immediately.

Practical limits

The efficiency of the cache improves dramatically if it gets used more. It seems to do a good job for both Shareaza and Phex, who use it to distribute new versions of their install files.

Limits of this system include:

• Firewalled users don't get included into the cache
• I couldn't get it to work for all gnutella clients (Limewire, ...). They'll get sources, but their IP address doesn't get added to the cache. Patches are welcome, ofcourse.

Extra stuff

You can add a rolling download counter to your site if you use the cache. Freebase offers a png containing the number of ip addresses in the database, so you get an idea of how many people have downloaded your file in the last 7 days.

The address of the image is the following (replace <sha1> by the sha1 hash of the file):

Example:

Files

You can download the cache source code here, so you can improve it, and/or use it yourself.
Keep in mind that the cache's efficiency improves if it gets used more, so I'd encourage people to just send me code patches, instead of using it on your own. Because one full cache is better than lots of empty caches. This code doesn't work on a lot of hosts, as you need to be able to read http headers directly.

Keep in mind that there's some site-specific code in there (it also counts the number of downloads), and that you need to enter the code that connects to the database yourself.

Features that I'd like to have someday:

• gnutella support: I couldn't get it to work because I couldn't find a way to get the correct port of the connecting client. A gnutella client will be able to get the alt sources from the cache, but it won't get added to the cache itself.
  It does seem to work in some clients, though, like GTK-Gnutella and Phex. I'm willing to work with other vendors to make it work for their clients.

Thanks

A lot of people helped me to make this cache possible, so here's a list of those:

• The people at #shareaza and #limewire, because they gave me feedback and hints
• ram and Cyclonus on #gtk-gnutella for helping to support some gnutella clients better
• dcat, for essentially rewriting the code so it looks and works better
• In case I forgot you: you. Unless you didn't help me.