distributed.net Contest Setback 92
meisenst writes "distributed.net is reporting that there was a problem earlier on in the CSC project (a few weeks ago) and that about 25% of the blocks will have to be re-done. Their announcement is here." We've gotten more than a few submissions about distributed.net showing more than 100% of all packets processed, but held off posting it until we had the official word. And here it is.
CSC affected (Score:1)
Does anyone know what specifically caused the problems in the key server?
Annoying (Score:1)
I will still continue to support them, however.
Cheers!
Costyn.
annoying, but minor (Score:5)
On the plus side, with the current key rates the remaining 25% will be cracked within a week or two.
The other plus is that the key may not be at the very end of the remaining 25% of the keyspace and therefore we will finish even quicker!
It doesn´t look too bad... (Score:2)
Go distributed!
Accuracy in reporting! (Score:2)
It works like this: D.Net rechecks blocks from suspicious clients. D.Net rechecks they by handing out those blocks again. D.Net has said repeatedly that they give full credit for EVERY block handed out in this way from the keyservers. AS SUCH it's easy to have the total go over 100% - this had NOTHING to do with the server problem. The article posted does not make this clear, and should be revised!
Hey Rob, howabout that tarball!
Oops... Another 24 hours now...
Eco damage (Score:2)
Re:CSC affected (Score:1)
Mix up and other things (Score:1)
Also RC5 was not affected because it is a longterm project, but because the error was somewhere in the CSC scripts.
And apparently they only discovered the bug just yesterday orso and not some time ago.
Re:Accuracy in reporting! (Score:1)
Re:Eco damage (Score:1)
More than one problem (Score:4)
First: dbaker (Daniel Baker), released an official anno uncement [distributed.net] explaining that the same blocks were being issued to multiple clients, to attempt to detect cheaters.
Then dbaker released another anno uncement [distributed.net] in his
Second: nugget (David McNett), released an announcement [distributed.net] stating that there had been a problem with the keymaster generating invalid blocks, resulting in 25% of the keyspace being duplicated.
So, one remaining question is, are they still sending out ~10% 'verification blocks'? Or have they abandoned that to allow us to complete the project faster?
We have reached 112% due to verification blocks and could reach 140% due to 25% of the keyspace being corrupt. However, if 12% of the 25% new blocks are duplicated, then we could reach about 155%...
--
David Taylor
davidt-sd@xfiles.nildram.spam.co.uk
[To e-mail me: s/\.spam//]
Re:Eco damage (Score:1)
Re:Does that mean we'll have to do them again? (Score:2)
Re:More than one problem (Score:2)
Re:Eco damage (Score:2)
Seriously, I'd like to see a calculation on how many barrels of oil per hour/day go to distributed.net.
Yeah, I know it's a troll. (Score:2)
TomG
Re:Accuracy in reporting! (Score:2)
To be fair to everyone, everyone is given credit for the block, if it is a 'virgin' one, or a reissued one (however, not if it is a duplicate block, they are filtered by the keymaster before reaching the stats).
The way the stats server currently counts the percentage complete, is simply counting all the blocks it is told have been completed, and dividing that by the number of blocks in the keyspace.
Because people are being credited individually for duplicate blocks, the total no. of blocks done includes these duplicate blocks.
To fix it, the stats need to know if a block has been reissued, and if so, only give credit to the participant -- but not the whole effort, as doing the same block twice *doesnt* increase our keyrate.
--
David Taylor
davidt-sd@xfiles.nildram.spam.co.uk
[To e-mail me: s/\.spam//]
Re:CSC affected (Score:1)
I think that what they say in the announcement is that the RC5-64 project wasn't affected at all. The problem is only with the CSC code.
100% -- Retesting blocks (Score:1)
J. Marvin
Re:Accuracy in reporting! (Score:1)
That calculation was reasonable for contests where we did not have the potential of giving credit for a block more than once. However, with block re-verification (and now with the 25% of re-issued keyspace) there has been a lot of duplicate credit for the same blocks, so it's easy to see how the total number of credit blocks can exceed the number that were originally planned to be uniquely distributed.
Re:Eco damage (Score:3)
My monitor consumes 1055mA, or 130 watts, or $7.49 a month. Turning it off could be a big savings.
The main computer required to host web pages, etc., consumes 600mA, or 73.9 watts, or $4.26 a month. That's under full load, cracking CSC, serving MP3's, and providing limited remote control functions for the house.
My laptop with the screen off, consumes the remainder of the power.
Let's see how much power is saved by turning off the monitor, CSC cracking, MP3's, and the monitor... 1658mA, or 204 watts total. 8 watts saved? Well, there's too much going on this setup, so the savings are insignificant.
Re:Yeah, I know it's a troll. (Score:2)
Quite truthfully, releasing binary-only clients still does not completely eliminate the possibility of sabotage, since it is relatively easy for any knowledgeable person to disasemble or patch binaries. This is actually quite a trivial task, so we urge you not to try. Indeed, security through obscurity is actually not secure at all, and we do not claim it to be such.
We understand the problems that are preventing us from becoming fully open source and are working to solve them. We have already done a lot of research work in this area and are working towards an eventual solution that can be fully open source. The data re-verification introduced in CSC is part of this! You can check out some of our work in this area at opcodeauth [distributed.net]
Re:Yeah, I know it's a troll. (Score:1)
releasing the source to the proxies, which have
to attempt to communicate securely, however there
must be a large amount of non-communications
related code in the keymaster, which *could* be
reveiewed by other people, if it were open source.
What reasons are there for not releasing the source to the
keymaster? (Excluding proxy communication code)
--
David Taylor
davidt-sd@xfiles.nildram.spam.co.uk
[To e-mail me: s/\.spam//]
Re:Does that mean we'll have to do them again? (Score:1)
Participants will still receive full credit for their completed blocks, regardless of their validity. In the near future we will be supplying the stats server with the information it needs to properly discount the effects of any reverification work in the reported percentage.
--
keymaster source? (Score:1)
My Hat Is Off... (Score:2)
Now let's get cracking and finish this thing, and my applause again to nugget for getting it all fixed and admitting it.
---
Tim Wilde
Gimme 42 daemons!
Re:Yeah, I know it's a troll. (Score:1)
Re:keymaster source? (Score:2)
We've seen this before, actually, and it's still in the process of being dealt with; the release of the Quake source to the hungry open source world. Immediately, those few (it's always the few) who have suspicious morals began to use the Quake source to cheat, effectively giving themselves impressive handicaps.
The same thing could most certainly be done with distributed.net source, especially if one knows how to trick the keymaster into thinking certain things (a block is done without really being done, a client submitted 100 packets instead of 10, who knows). It is for this reason that I not only thank distributed.net for -not- releasing their source, but applaud their decision to not do so.
Sure, I'd be more than happy to know how they do what they're doing so well, and yes, the bug would probably have been noticed/squashed more quickly had there been a few thousand code monkeys jabbering away at it, but to be quite honest, the distributed.net team responded as quickly as they could, and in a very professional manner. So, there's really nothing to complain about, IMHO.
meisenst
Suspicious... (Score:1)
Seeing this note about the block corruption in CSC client, I checked my client.
I noticed that the new client had core-dumped on
my Linux machine this week, other programs are ok.
-- implying some corruption of major kind.
Has anyone else encountered this behavior ?
My client version is v2.8003-448 client for Linux
My system is SuSE 6.3 (i.e. libc6).
I run 2.3.x kernels -- but they haven't crashed ever --
so methinks this has is a problem with the
client binary.
yes, i have enabled csc and rc5 on client.
-ak
Re:annoying, but minor (Score:1)
Re:annoying, but minor (Score:1)
This is the call to d.net, stop fooling around and open your client! Even if it will take to shutdown all submission for a month or any time, we need TRUST in what we are doing. Who shall we blame when no key will be found at the end of RC5-64? Not nugget, but the closed source approach they use.
Re:Yeah, I know it's a troll. (Score:1)
Re:Eco damage (Score:1)
open source dnet and Quake (Score:1)
> We've seen this before, actually, and it's still in the process of being dealt with; the release of the Quake source to the hungry open source world.
I know this is slightly offtopic, but a week ago when I read Jeff "Bovine Lawson's .plan of Jan 6 [distributed.net] and then his treatise on operational code authentication [distributed.net], I thought I was reading a Slashdot thread on how to correct the problem of cheating in an open-source Quake.
It seems that both distributed key cracking and distributed Quake playing :) face many of the same cheating issues. It's clear Jeff and co. have thought a lot about what problems would have to be "solved" before they can go open source (which they hope to in the future). I highly recommend that anyone interested in the whole Quake cheating problem read Jeff's thoughts first. (He even mentions why Netrek-style blessed binaries won't work in general.)
I've given up on distributed.net (Score:1)
But we've had problems with bugs in the clients, (like this extremely annoying transparency bug on windows) also the fact that there isn't a way to control your team efficiently - and to top it all off, they recently had a problem over there that DROPPED ALL OF MY MEMBERS FROM MY TEAM.
That was the last straw. That's a serious problem since 2 of the email addresses on my team aren't valid anymore so we can't rejoin that email address to the team. The boxes that are running using that email address are in a location we can't go to to change the setting there.
So, after all of the hassle, it stopped being fun. Particlarly when they dumped so many people from their teams. I stopped participating, and distributed.net lost about 20 clients. Maybe I'll try SETI or some other CPU donation project that has their shit together.
Re:Suspicious... (Score:1)
Non-returned blocks (Score:1)
I understood from reading the d.net documentation that there is an issue of blocks that were never returned to the keyserver. These lost blocks were counted in the percentage but once the project "complete", the non-returned blocks would have to be redistributed assuming that the key had not already been found.
That could account for at least part of the 25%. Especially since the Windows client tends to lose the packet that it is working on when the machine crashes.
Re:Suspicious... (Score:1)
I just downloaded current client:
x86/ELF/glibc2.1/MT v2.8005.453 released on 2000-01-09.
and this one also has a problem with SIG_SEGV core dumps.
The gnu libc being used is libc-2.1.2-31
-ak
Re:Suspicious... (Score:1)
My machine is a K6-2 -- was your machine K6-II ?
since these could be specific problem with
computation-cores.
Re:annoying, but minor (Score:2)
First of all,the problem appeared on the _server_ side, not on the client. Opening the source of the client wouldn't have made any difference.
Second, this has be discussed several times before. distributed.net is well aware that their code can be reversed engineered. But it does raise a bar. And they rather have a few attempts to disrupt the contest than many. And it isn't that distributed.net hasn't tried a fully open source client. They did. And the script kiddies had their fun, so now the source is partially closed.
It's written in pretty tight C, and main crypto routines are in Asm, which make the program extremely easy to analyze
Yeah, specially since you can download the source of the crypto routines. Isn't it remarkable that the whiners about open source/closed source don't even know most of the code can be downloaded?
-- Abigail
Re:It doesn´t look too bad... (Score:2)
Uhm, no. It only effects CSC.
-- Abigail
Re:open source dnet and Quake (Score:1)
Bovine's explaination of netrek only shows his lack of understand of how "blessed binaries" work(ed). There is no "date" in the key itself. Expiration is completely artificially added by the key service agent along with other bits of data -- it was not to prevent people from recovering the secret key but more to push people to upgrade their client(s). Netrek uses (used) 128bit RSA numbers ("secret key", "private key", and "public key"). In the modern world, factoring a 128bit RSA number is not difficult. Recovering the secret key embeded in the client is _NOT_ easy. Unlike Xing, the key is not in one place; it's randomly scattered throughout a very large static binary. (The key is generated and destroyed by the build.) In ten years of working with netrek, I've never known of anyone to recover the key from the client. I have personally regenerated the secret key by factoring the public data, but that was (at the time) a very non-trivial task.
I must admit, I'm starting to truely dislike DCTI and their current attitude. The CSC clients are duplicating work. They don't tell anyone this until it's obvious -- over 100% of the blocks have been checked in. They continue to have trouble with stats -- two years and counting now.
Re:annoying, but minor (Score:1)
So, how much other shit have they not told anyone about?
Re:annoying, but minor (Score:1)
Their "security" doesn't stand up to even the simplest analysis. No debugger or disassembly is required to figure out how the blocks are being scrambled. Unfortunately, DCTI's existance depends on that scrambling remaining a secret. Once the world knows how it works, DCTI is doomed. There's no way they could prevent cheating. And there's no way they could replace the technology fast enough to keep a user base. (Plus, that would necessitate replacing every client in existance.)
The more code they release, the easier it gets to recover the block scrambler technology. That not withstanding, there's a lot of code they are sitting on that could be released. The only file(s) that must be protected are those linked with block scrambling.
PS: The "scambler technology" was written by Duncan (Adam Beberg) years ago -- long before DCTI existed. Yet, DCTI claims ownership of those few dozen lines of code.
Give them a break. (Score:1)
It is not distributed.net's problem that you cannot access your systems to update information that is no longer valid. Yes they have had some issues as of late, but they have made a real effort to correct them, maybe you should send an e-mail and demand your money back. Oh, that's right you didn't send them any money.
Good luck finding ET.
all persons, living and dead, are purely coincidental. - Kurt Vonnegut
d.net vs. the competition (Score:1)
Yeah, I've been bitten by that. CSC for the Mac and Tru64 Unix clients took a while to arrive compared to the Wintel/Linux/etc CSC clients. Of course, dcypher only has clients for the x86 architecture on 3 OSes...guess they don't want my cycles.
As for seti@Home, they haven't exactly avoided similar problems either. Complaints about duplicated blocks and un-optimized client code for their project have been posted on /. before.
Also possible with RC5? (Score:1)
Re:It doesn´t look too bad... (Score:1)
[smack]
...goes your hand on your forehead, as you re-read what you replied to.
RC-64 is losing CPU time while some dnetc clients are working CSC. Another week (or more???) of lost CPU time to CSC means fewer cycles for RC-64. Once CSC is over, the clients which continue to operate will be working RC-64 again. Everyone thought that would happen last week, but nope.
Therefore, RC-64 is hurt by the extension of the CSC project.
Re:annoying, but minor (Score:2)
Since when is my opinion gospel truth?
I wish someone would have told me, I'd have used that to my benefit a long time ago.
I'm sorry your feelings were hurt by comments made towards Seti@home....maybe you should talk to a grief counselor about it. I know that I never made any such comments.
Re: (Score:1)
Re:duplicates (Score:2)
1) If you were trying to cheat, we've noticed you and your efforts are ineffectual.
2) If you somehow have a broken client, read-only disk, or have run out of disk space and are inadvertantly submitting duplicates, you might want to check on it. You might be wasting cpu or network bandwidth that you are not aware of.
Re:Non-returned blocks (Score:1)
Re:Dysfunctionalributed.net (Score:1)
Re:annoying, but minor (Score:1)
What the hell are you talking about? distributed.net is controlled by robots. Well, at least that's what gammatron [editthispage.com] would say.
-y.
Re:I've given up on distributed.net (Score:1)
Tried mailing their help-service [mailto] to ask for the passwords for those addresses? I had a problem with two old inaccessible emails and they helped me out very quickly. I think they're doing a great job, considering the number of people out there who try and screw with the stats and break things.
Keep up the good work... Moo!
Re:Eco damage (Score:2)
Re:Accuracy in reporting! (Score:1)
If you set the number of total block to the number of blocks in the keyspace plus the number of blocks handed out twice, wouldn't the total percentage be correct again?
Re:I've given up on distributed.net (Score:1)
Why not? What's wrong with cable modems?
This statement is IMHO nothing more than a flame if you give no reasons.
Re: (Score:1)
Re:duplicates (Score:1)
In fact, the closer the project gets to 100% completion, the more quickly unreturned blocks are re-assigned (and re-re-assigned) just to try to get somebody (anybody!) to crunch them and return the results. When a project begins, a block of 10-day-old keys is still mostly useful to the project. But near the end, there's a very high chance most of those keys have been handled already.
Your computers have a finite amount of compute-power to offer the Distributed project. The best thing you can do with that power is make sure that 100% of the blocks you work on are useful to the project. The closest you can get to guaranteeing this is to process the newest blocks you can get from the keyserver.
"But what if the keys I throw away had the one!" Keys are assigned in random order from the keyserver. Nobody knows if the one is in the beginning, middle, or end of the entire sequence of keys to check. The winning key has an equal chance of being in the old block you're considering throwing away as it does in any new block you could download from the keyserver, so what the heck - just take the new keys. (In fact, there's a slightly higher chance that a new block will have the winning key - some parts of your old block may have already been processed by others, right? And nobody's won yet, right?)
Re:Open source CRAP (Score:1)