Coral P2P Cache Enters Public Beta 254
Eloquence writes "infoAnarchy reports that Coral, a peer-to-peer webcaching system, has gone into public beta. Currently the Coral node network is hosted on Planet-Lab, a large scale distributed research network of 400 servers. You can use Coral right now by appending "nyud.net:8090" to a hostname. View Slashdot through Coral. Is this the end of the Slashdot effect?"
Slashdotted already (Score:5, Funny)
Re:Slashdotted already (Score:2, Funny)
yea, but its true
Re:Slashdotted already (Score:2, Informative)
Windows:
Linux:
Seems their nameservers have some kind of problem. I am in the Midwest, going t
"Invalid domain name in packet" (Score:5, Informative)
The problem is that it doesn't seem to be compatible with Microsoft DNS severs. Below is a copy of the DNS log when I issue a query here, on my LAN which has a Microsoft DNS server running on Windows 2000, which then forwards through the University of Wisconsin. You can see that at the end it says "The DNS server encountered an invalid domain name." Perhaps someone who knows more about DNS can tell where the problem is?
Re:"Invalid domain name in packet" (Score:5, Informative)
Given that the DNAME RFC is from 1999, it appears that some old DNS servers do not handle this record type well. We'll look into some alternatives or work-arounds. (Perhaps you can contact me directly to see if subsequent changes can fix your problem.)
Thanks for the detailed report!
--mike
Re:It doesn't work with windows servers? (Score:2)
Re:It doesn't work with windows servers? (Score:2)
Jon.
Not quite, but here is what /. looks like! (Score:3, Informative)
Coral Statistics [nyu.edu]
Re:Not quite, but here is what /. looks like! (Score:2)
Re:Not quite, but here is what /. looks like! (Score:3, Insightful)
Google (Score:3, Informative)
So this is not so new to me regarding slashdot effects.
Re:Google (Score:2, Insightful)
And you can't be sure that Google has cached your page in the first place.
Re:Google (Score:3, Informative)
With Coral you can get it cached just by asking for it. Of course, the Coral pcs have to connect to it at least once.
You cannot get google to cache a page at your request -- no matter how hard you try
Re:Google (Score:2)
Google needs to start using this technology.
Google doesn't cache images (Score:2, Insightful)
Plus as others have said Google doesn't convert links.
Re:Google (Score:5, Informative)
http://www.rentzsch.com/notes/googleCacheHackin
If the page won't load at all thus negating the above just use the following example to load a page.
http://google.com/search?q=cache:www.slashdot.o
Re:Google (Score:5, Informative)
javascript:location.href=location.href.replace(/h
And if slashdot's tendency to insert spaces in long strings screws that up, try grabbing it from here [gotdoof.com]
Anyone see the irony? (Score:5, Funny)
Re:Anyone see the irony? (Score:2)
Everything goes faster if you kill leechers.
Dear Lord (Score:4, Funny)
Then again it might not be so bad....
Re:Dear Lord (Score:2)
Pff. That's my best excuse for not reading the f'n article.
Game over, slug. (Score:2)
self-referential slashdotting (Score:5, Funny)
oh, first post?
Seems to be broken (Score:2, Interesting)
It's turtles all the way down...
Re:Seems to be broken (Score:2)
Re:Seems to be broken (Score:2)
nyud.net.nyud.net.nyud.net.nyud.net.nyud.net.ny
Server CoralWebPrx/0.1 (See http://www.scs.cs.nyu.edu/coral/) at 212.192.241.154:8090
It was fast, but THAT is what I call
Re:Seems to be broken (Score:2)
files (Score:3, Interesting)
well apparently all html content, including files, will be cached. this is a great way to get around downloading from snail-pace sites, (although i will be checking md5sums)
Maybe try... (Score:2)
Not too good for websites (Score:4, Insightful)
Re:Not too good for websites (Score:5, Informative)
img src="img/logo.png"
not:
img src="http://slashdot.org/img/logo.png"
or whatever so this shouldn't be a problem
Re:Not too good for websites (Score:2)
In case Coral gets slashdotted (Score:5, Funny)
Re:In case Coral gets slashdotted (Score:3, Interesting)
I'm not a native speaker though, so ymmv.
Re:In case Coral gets slashdotted (Score:2)
Re:In case Coral gets slashdotted (Score:2)
Re:In case Coral gets slashdotted (Score:2)
Is this the end of the Slashdot effect? (Score:2)
haha no - only the lateral shifting of the slashdot effect to your local lan as some dope sets up a cache server in your office. Im sure the
sorry bud, but your bots have met their match. (Score:2)
Is this the end of the Slashdot effect?" haha no - only the lateral shifting of the slashdot effect to your local lan as some dope sets up a cache server in your office.
This is what coral says:
One of Coral's key goals is to avoid ever creating hot spots that might dissuade volunteers from running the software for fear of load spikes. It achieves this through a novel indexing abstraction we introduce called a distributed sloppy hash table (DSHT), and it creates self-organizing c
Hosting companies'll hate this.... (Score:3, Interesting)
Re:Hosting companies'll hate this.... (Score:2)
Upload bandwidth (Score:3, Insightful)
Re:Upload bandwidth (Score:2)
I'm pretty happy with my cable company. ;-) The only negative is the upload cap, so sometimes torrents are like wicked slow.
Re:Hosting companies'll hate this.... (Score:2)
Re:Hosting companies'll hate this.... (Score:2)
What kind of hosting companies are those? How long have they been in business?
Re:Hosting companies'll hate this.... (Score:3, Informative)
As far as I know, anybody in the 0.5gig/month or over (all the way up to the backbone carrierers, which have to have peering agreements as an exception to the rule of charging for bandwidth) charges per megabyte.
Re:Hosting companies'll hate this.... (Score:2)
Most large datacenters (ServerMatrix, EV1, etc) seem to charge 50 cents to a dollar per gigabyte over your monthly allotment.
Also a proxy... (Score:4, Interesting)
Re:Also a proxy... (Score:2)
x.com
and x.com.what_ever_new_p2p_producturl.com:0000
and x.com.foo.com:4352
and x.com.bar.com:4352
etc
Re:Also a proxy... (Score:5, Informative)
Re:Also a proxy... (Score:2, Interesting)
"Is this the end of the Slashdot effect?" (Score:2)
me thinks not P2P (Score:2, Interesting)
Re:me thinks not P2P (Score:5, Informative)
It's 'distributed'.
Peer to peer implies that the users of the service are the ones supporting it's existance.
Re:What would make it p2p (Score:2, Informative)
Is this the solution? (Score:2)
Re:Is this the solution? (Score:2)
An interesting question. It would seem feasible only to serve up the full page when this is requested by a cache server, in all other cases just returning a redirect.
If this actually proves possible, and no way of blocking it is found, it may kill the project stone dead.
Re:Is this the solution? (Score:3, Informative)
Here, I'll even link you to a good client that will give you a nice GUI for starting out. Another Bittorent Client [sourceforge.net] for all OSes.
What about Freecache? (Score:2, Informative)
It isn't P2P web proxy, it's just "big pipe"-based distributed one. Supposedly a great way to prevent slashdoting (just use http://freecache.org/http://mytinysite.com instead of http://mytinysite.com and everything goes from the cache, tiny site receiving only header requests to chceck if the document hasn't changed in the meantime) it's hardly known, way too quiet as for a project that useful. P2P may be faster and cheaper but certainly less reliable...
Read the freecache forums (Score:2)
Re:What about Freecache? (Score:2)
Er... no... from the site:
"Please note that you cannot submit a whole site to FreeCache as in http://freecache.org/http://www.rocklobsters.com/ This will not work as only index.html will be cached. You have to prefix every item that you want to have cached seperately."
As I understand it, Freecache refuses to cache small files anyway; I think the minimum was 5Mb.
Not a good solution (Score:3, Informative)
Only the top page? (Score:5, Interesting)
Re:Only the top page? (Score:4, Informative)
All the links on Slashdot have the format
<a href="//slashdot.org/blahblahblah">
so that they will always link back to Slashdot. Most websites just use "blahblahblah" or "/blahblahblah" for their links. For example, links on google.com.nyud.net [nyud.net] are fully functional.
Re:Only the top page? (Score:3, Informative)
Well...all they have to do is have some modifying code like CGI-Proxy [jmarshall.com] does....
Re:Only the top page? (Score:2)
Is it possible to combine this with bittorrent (Score:3, Insightful)
Re:Is it possible to combine this with bittorrent (Score:4, Insightful)
You could conceivably design a distributed tracker, but this isn't it. Anyway, there would doubtless be synchronization issues that would greatly decrease the network's overall performance.
Re:Is it possible to combine this with bittorrent (Score:2)
Only caches main server files (Score:2)
Doesn't Work. (Score:2)
Notice the random question at the bottom of the page.
Then go here: http://www.newberrycollege.net.nyud.net:8090/ [nyud.net]
The question is not randomly generating. They should have some checking for this such that if the data varies by the second, that it does not cache the page or invisible frames the HTML and filter the content it can cache.
Bleh.
It will work in certain cases, but generally I am not that happy.
Stats! Slashdot has it REALLY working! (Score:5, Informative)
Re:Stats! Slashdot has it REALLY working! (Score:2)
Work for CmdrTaco (Score:5, Interesting)
this has been possible for a long time (Score:2)
Stupid story submitter... :-) (Score:5, Funny)
http://www.nyud.net.nyud.net:8090 [nyud.net]
Re:Stupid story submitter... :-) (Score:5, Funny)
http://www.nyud.net.nyud.net.8090.nyud.net:8090/ [nyud.net]
Or the mirror-of-the-mirror-of-the-mirror:
http://www.nyud.net.nyud.net.8090.nyud.net.8090.ny ud.net:8090/ [nyud.net]
They should have posted THAT link to slashdot to see how well the system faired.
404 not found (Score:2)
Re:404 not found (Score:2)
Hackable? (Score:3, Interesting)
Regards,
--
*Art
Re:Hackable? (Score:3, Interesting)
SSH-1.99-OpenSSH_3.5p1 via: SSH-1.99-OpenSSH_3.5p1 216.165.109.81:8090 (CoralWebPrx/0.1 (See http://www.scs.cs.nyu.edu/coral/))
accept-ranges
connection: close
That definitely doesn't look too good, security-wise, when you can get access to inside services through their proxy.
Regards,
--
*Art
Re:Hackable? (Score:2)
Re:Hackable? (Score:2)
Re:Hackable? (Score:2)
Re:Hackable? (Score:3, Insightful)
That's the Microsoft way of securing things -- blocking single exploits as they are found. That doesn't solve the design problem of the proxy being able to contact any host/port, including LAN ones. Just substitute localhost with any host of choice, or even broadcast addresses.
This product needs a design change.
--
*Art
It will fail, because business will want it to. (Score:2, Insightful)
Its a noble goal, but ultimately will go the way of the video phone -- which apart from conferences planned in advance, remains a novelty dispite perfectly adaquate technology -- nobody wants a suprise video call because nobody wants to be a 50
Oooh! Graphs! (Score:3, Funny)
http://www.scs.cs.nyu.edu/coral/stats/
Doesn't give a usable time scale though; it has "HTTP requests", but not "per second" / "per minute" or anything
Re:Oooh! Graphs! (Score:2)
Looks like a reading was done every six minutes, but I could have miscounted.
Definitely interesting idea... (Score:2, Interesting)
Although I agree with others, it doesn't really compare to FreeCache. I still wonder why that never got much attention. It's an insanely great idea. Ah well. Between that, Corla, and BitTorrent, you never have to worry about /.'ing again when you submit your tiny personal site.
In other news (for the morons who continue posting and whining), you can still remove the it prefix from the /. URL, removing the fugly colour scheme. And there was much rejoicing in the land.
(-:Stephonovich:-)
Is this the end (Score:2)
No.
Safari can't open the page "http://slashdot.org.nyud.net:8090/" because it could not connect to the server "slashdot.org.nyud.net".
I believe there is a term for this.
'doh!'
Hmmm... wondering if I could use this commercially (Score:3, Interesting)
What I'm thinking is that at work I run a multi-server site that gets massively bogged down for short periods when it tries to handle upwards of 35,000 concurrent sessions. Bandwidth is not the problem, the application is, and it can't be rewritten for reasons that piss me off and I have no budget for more servers and no management support to run a static cached version of the site.
So I was wondering if it was possible to have the site automatically direct visitors to the Coralized URL when the site load gets too high. Either a manual change or an automatic one would be ok. I have some ideas on how this could be done using a failover server config on our ServerIron. Possibly a router config can also do this, though we don't run our own router since it's at a colocation facility. Worst case scenario is I can edit the home page to redirect to Coral when the load gets high.
Are there any other Slashdotters looking to use Coral in similar ways? If you have any ideas to share I'd be all ears.
Incompatible with logged in browsing (Score:2)
Re:Incompatible with logged in browsing (Score:5, Informative)
To summarize it, though, they're set on a per-domain basis.
www.apple.com can set a cookie.
store.apple.com can set a cookie.
The two cannot interact with each other; however,
microsoft.com cannot access any of your apple.com cookies.
Thus, nyud.net cannot access your
404 No Such Domain... (Score:2)
Not yet!
Does it cross country firewalls? (Score:2)
testing (Score:2)
unmentioned side effect/benefit (Score:2)
Check this out: Normal browsing [whatismyip.com] and Coral browsing [nyud.net].
Re:Usefulness (Score:2, Informative)
Here. [nyu.edu]
Re:Usefulness (Score:2)
Re:Possible problems with this scheme... (Score:3, Informative)
It seems you're confusing a "cache" with a "proxy." A "cache" is only DESIGNED to work on static pages, and it doesn't hit the page more than once (barring refreshing). That's what "cache" means. The pages are stored on the cache server and fed to the clients as they get requested, cutting down on hits to the actual site.