Coral P2P Cache Enters Public Beta

Eloquence writes "infoAnarchy reports that Coral, a peer-to-peer webcaching system, has gone into public beta. Currently the Coral node network is hosted on Planet-Lab, a large scale distributed research network of 400 servers. You can use Coral right now by appending "nyud.net:8090" to a hostname. View Slashdot through Coral. Is this the end of the Slashdot effect?"

Google

[asd-Strom]

Google cache has been a good helper to me for some time.
So this is not so new to me regarding slashdot effects.

Delay

[Anonymous Coward]

Notice that because of the caching system it isn't fully current...

Re:Slashdotted already

[Anonymous Coward]

No, it really is down, at least from here:

Windows:

C:\>ping slashdot.org.nyud.net
Ping request could not find host slashdot.org.nyud.net. Please check the name and try again.

Linux:

~$ping slashdot.org.nyud.net
ping: unknown host slashdot.org.nyud.net

~$dig slashdot.org.nyud.net
; <<>> DiG 9.2.1 <<>> slashdot.org.nyud.net
;; global options: printcmd
;; connection timed out; no servers could be reached

Seems their nameservers have some kind of problem. I am in the Midwest, going through an AT&T OC3 (everything else works fine from here; it's not a local problem). It works OK when I check from our California-based servers that peer with Mae West, however.

Re:Not too good for websites

[Coneasfast]

links should be (and usually are) relative, eg:

img src="img/logo.png"
not:
img src="http://slashdot.org/img/logo.png"

or whatever so this shouldn't be a problem

What about Freecache?

[SharpFang]

http://www.archive.org/web/freecache.php [archive.org]

It isn't P2P web proxy, it's just "big pipe"-based distributed one. Supposedly a great way to prevent slashdoting (just use http://freecache.org/http://mytinysite.com instead of http://mytinysite.com and everything goes from the cache, tiny site receiving only header requests to chceck if the document hasn't changed in the meantime) it's hardly known, way too quiet as for a project that useful. P2P may be faster and cheaper but certainly less reliable...

Re:Usefulness

[chewy_2000]

Looks like it. Haven't tried it personally yet, but I will.

Here. [nyu.edu]

Stats! Slashdot has it REALLY working!

[Danathar]

http://www.scs.cs.nyu.edu/coral/stats/

Re:Only the top page?

[mothz]

Doesn't it analyze hyperlinks?

All the links on Slashdot have the format
<a href="//slashdot.org/blahblahblah">
so that they will always link back to Slashdot. Most websites just use "blahblahblah" or "/blahblahblah" for their links. For example, links on google.com.nyud.net [nyud.net] are fully functional.

"Invalid domain name in packet"

[Anonymous Coward]

It really DOESN'T work for a lot of people.

The problem is that it doesn't seem to be compatible with Microsoft DNS severs. Below is a copy of the DNS log when I issue a query here, on my LAN which has a Microsoft DNS server running on Windows 2000, which then forwards through the University of Wisconsin. You can see that at the end it says "The DNS server encountered an invalid domain name." Perhaps someone who knows more about DNS can tell where the problem is?

Rcv 10.76.0.2 0004 Q [0001 D NOERROR] (8)slashdot(3)org(4)nyud(3)net(0)
UDP question info at 014D5A0C
Socket = 384
Remote addr 10.76.0.2, port 1263
Time Query=4338128, Queued=0, Expire=0
Buf length = 0x0200 (512)
Msg length = 0x0027 (39)
Message:
XID 0x0004
Flags 0x0100 QR 0 (question) OPCODE 0 (QUERY) AA 0 TC 0 RD 1 RA 0 Z 0 RCODE 0 (NOERROR)
QCOUNT 0x1 ACOUNT 0x0 NSCOUNT 0x0 ARCOUNT 0x0
Offset = 0x000c, RR count = 0
Name "(8)slashdot(3)org(4)nyud(3)net(0)"
QTYPE A (1)
QCLASS 1
ANSWER SECTION:
AUTHORITY SECTION:
ADDITIONAL SECTION:

Snd 144.92.254.254 39b0 Q [0001 D NOERROR] (8)slashdot(3)org(4)nyud(3)net(0)
UDP question info at 0109200C
Socket = 408
Remote addr 144.92.254.254, port 53
Time Query=0, Queued=0, Expire=0
Buf length = 0x0200 (512)
Msg length = 0x0027 (39)
Message:
XID 0x39b0
Flags 0x0100 QR 0 (question) OPCODE 0 (QUERY) AA 0 TC 0 RD 1 RA 0 Z 0 RCODE 0 (NOERROR)
QCOUNT 0x1 ACOUNT 0x0 NSCOUNT 0x0 ARCOUNT 0x0
Offset = 0x000c, RR count = 0
Name "(8)slashdot(3)org(4)nyud(3)net(0)"
QTYPE A (1)
QCLASS 1
ANSWER SECTION:
AUTHORITY SECTION:
ADDITIONAL SECTION:

Rcv 144.92.254.254 39b0 R Q [8081 DR NOERROR] (8)slashdot(3)org(4)nyud(3)net(0)
UDP response info at 012DB8AC
Socket = 408
Remote addr 144.92.254.254, port 53
Time Query=4338128, Queued=0, Expire=0
Buf length = 0x0200 (512)
Msg length = 0x00e0 (224)
Message:
XID 0x39b0
Flags 0x8180 QR 1 (response) OPCODE 0 (QUERY) AA 0 TC 0 RD 1 RA 1 Z 0 RCODE 0 (NOERROR)
QCOUNT 0x1 ACOUNT 0x4 NSCOUNT 0x2 ARCOUNT 0x2
Offset = 0x000c, RR count = 0
Name "(8)slashdot(3)org(4)nyud(3)net(0)"
QTYPE A (1)
QCLASS 1
ANSWER SECTION:
Offset = 0x0027, RR count = 0
Name "[C019](4)nyud(3)net(0)"
TYPE 39 (39) CLASS 1 TTL 1333 DLEN 25
DATA Unknown resource record type 39 at 012DBC41.
Offset = 0x004c, RR count = 1
Name "[C00C](8)slashdot(3)org(4)nyud(3)net(0)"
TYPE CNAME (5)
CLASS 1 TTL 0 DLEN 15
DATA (8)slashdot(3)org[C033](4)http(2)l2(2)l1(2)l0(5)n yucd(3)net(0)
Offset = 0x0067, RR count = 2
Name "[C058](8)slashdot(3)org[C033](4)http(2)l2(2)l1(2) l0(5)nyucd(3)net(0)"
TYPE CNAME (5)
CLASS 1 TTL 1335 DLEN 2
DATA [C033](4)http(2)l2(2)l1(2)l0(5)nyucd(3)net(0)
&am p;n bsp; Offset = 0x0075, RR count = 3
Name "[C033](4)http(2)l2(2)l1(2)l0(5)nyucd(3)net(0)"
TYPE A (1)
CLASS 1 TTL 60 DLEN 4
DATA 139.91.70.71
AUTHORITY SECTION:
Offset = 0x0085, RR count = 0
Name "[C038](2)l2(2)l1(2)l0(5)nyucd(3)net(0)"
TYPE NS (2)
CLASS 1 TTL 1991 DLEN 19
DATA (3)139(2)91(2)70(2)71(3)ip4[C041](5)nyucd(3)net(0 )
Offset = 0x00a4, RR count = 1
Name "[C038](2)l2(2)l1(2)l0(5)nyucd(3)net(0)"
TYPE NS (2)
CLASS 1 TTL 1991 DLEN 16
DATA (3)141(3)213(1)4(3)202[C09E](3)ip4[C041](5)nyucd( 3)net(0)
ADDITIONAL SECTION:
Offset = 0x00c0, RR count = 0
Name "[C091](3)139(2)91(2)70(2)71(3)ip4[C041](5)nyucd(3 )net(0)"
TYPE A (1)
CLASS 1 TTL 603196 DLEN 4
DATA

Re:me thinks not P2P

[Inominate]

It's not p2p.

It's 'distributed'.

Peer to peer implies that the users of the service are the ones supporting it's existance.

Re:Only the top page?

[SirDaShadow]

so that they will always link back to Slashdot. Most websites just use "blahblahblah" or "/blahblahblah" for their links. For example, links on

Well...all they have to do is have some modifying code like CGI-Proxy [jmarshall.com] does....

Not a good solution

[pyrrhonist]

From the FAQ:

What files are being served by FreeCache?

FreeCache can only serve files that are on a web site. If the link to a file on that web site goes away, so will the file in the FreeCaches. Also, there is a minimum size requirement. We don't bother with files smaller than 5MB, as the saved bandwidth does not outweight the protocol overhead in those cases.

Re:"Invalid domain name in packet"

[mfreed]

It appears that the Windows 2000 DNS server you are using is not aware of DNAME records (RFC 2672 [faqs.org]):

Name "[C019](4)nyud(3)net(0)"
TYPE 39 (39) CLASS 1 TTL 1333 DLEN 25
DATA Unknown resource record type 39 at 012DBC41.

We use these types of records to aid in redirecting resolvers to nearby Coral proxies (by mapping nyud.net to a "hierarchical" name http.l2.l1.l0.nyucd.net. The goal is that once you find a "nearby" server, you should remain nearby.

Given that the DNAME RFC is from 1999, it appears that some old DNS servers do not handle this record type well. We'll look into some alternatives or work-arounds. (Perhaps you can contact me directly to see if subsequent changes can fix your problem.)

Thanks for the detailed report!
--mike

Re:Hosting companies'll hate this....

[interiot]

Erm? If you mean the "5mb webspace" places, no, they don't charge for bandwidth. They just cancel your account if you post porn or anything else that will get heavily downloaded.

As far as I know, anybody in the 0.5gig/month or over (all the way up to the backbone carrierers, which have to have peering agreements as an exception to the rule of charging for bandwidth) charges per megabyte.

Re:Also a proxy...

[interiot]

There are actually a lot of sites out there that will let you access arbitrary content from elsewhere. Most corporate restricting proxies will block at least some of them (but it's impossible to get all of them). So something that could be as high-profile as Coral is less useful compared to some of the more obtuse of these:

• google cache (this has been periodically blocked at my company)
• the internet archive [archive.org]
• online translation sites (eg. if it's an english site, have the translator go from japanese to english... none of the words will be recognized as japanese, so it will pass them all as-is)
• several others I'm forgetting at the moment...

Re:Incompatible with logged in browsing

[maskedbishounen]

Sounds like we need a little lesson on How cookies work [google.com].

To summarize it, though, they're set on a per-domain basis.

www.apple.com can set a cookie.
store.apple.com can set a cookie.

The two cannot interact with each other; however, .apple.com can interact with any Apple subdomain.

microsoft.com cannot access any of your apple.com cookies.

Thus, nyud.net cannot access your /., or any other cookies.

Re:Google

[Gherald]

>Are you saying with Coral that you can?

With Coral you can get it cached just by asking for it. Of course, the Coral pcs have to connect to it at least once.

You cannot get google to cache a page at your request -- no matter how hard you try :)

Re:Is this the solution?

[focitrixilous P]

Bittorrent is your friend. It's as common as AIM or IRC these days, instead of pulling the whole file from a central server, only the first few need to use a server host, and everyone else shares with each other. Most big linux distros do it with 650 MB files, or for large video files. No reason it wouldn't work for you.

Here, I'll even link you to a good client that will give you a nice GUI for starting out. Another Bittorent Client [sourceforge.net] for all OSes.

Not quite, but here is what /. looks like!

[Tailhook]

Check out their logs...

Coral Statistics [nyu.edu]

...note the recent blip?

Re:Google

[bogie]

Google cache tip for you. There is a bookmarklet for Firefox where you simply click the bookmarklet and Google's cache of the page opens up. Its a nice feature to have at your fingertips. You can get the code at the very bottom of the following page, just drag it to your personal toolbar.

http://www.rentzsch.com/notes/googleCacheHacking

If the page won't load at all thus negating the above just use the following example to load a page.

http://google.com/search?q=cache:www.slashdot.or g

Re:Possible problems with this scheme...

[geminidomino]

I have to believe that ISPs that provide web services would find their revenue reduced since they would not see all the hits on the site.

It seems you're confusing a "cache" with a "proxy." A "cache" is only DESIGNED to work on static pages, and it doesn't hit the page more than once (barring refreshing). That's what "cache" means. The pages are stored on the cache server and fed to the clients as they get requested, cutting down on hits to the actual site.

Re:What would make it p2p

[rob101]

The above link from an anonymous coward points to a paper through some weird obfuscation that does just that. HTTP(P2P) [csiro.au]

Re:Google

[doofsmack]

Talking about bookmarklets, I just wrote a quick little bookmarklet to redirect you to the Coral cache of the current page. Here it is:

javascript:location.href=location.href.replace(/ht tp\:\/\/([a-zA-Z\.]+)\/(.*)/, "http://$1.nyud.net:8090/$2");void(0)

And if slashdot's tendency to insert spaces in long strings screws that up, try grabbing it from here [gotdoof.com]

Re:"Invalid domain name in packet"

[Anonymous Coward]

No, that doesn't work. For example:

http://www.slashdot.org.http.l2.l1.l0.nyucd.net: 8090/ [nyucd.net]

It resolves to the correct machine, yes, but the proxy doesn't understand how to extract the machine's hostname from a hostname of that form, so the HTTP request fails.