Stories
Slash Boxes
Comments
typodupeerror delete not in
  • Preferences

Book Reviews

Recent reviews from Slashdot readers:

Submitting a review for consideration is easy; please first read Slashdot's book review guidelines. Updated: 2008114 by samzenpus

Comments: 119 +-   Carnegie Mellon CAPTCHA Digitization Project Now Underway on Tuesday October 02 2007, @07:44AM

Posted by Zonk on Tuesday October 02 2007, @07:44AM
from the way-more-fun-than-the-usual-kind dept.
security
books
media
internet
technology
tomandlu writes "The BBC is reporting that Carnegie Mellon University has found a novel use for CAPTCHAs — deciphering old texts. We've discussed this project before, but it was prior to it getting off the ground. Users Entering text acts as a sort of distributed computing project. Basically, the CAPTCHA is made up of two words — one of which is known to Carnegie, and one of which isn't. If the user correctly deciphers the known word, then the unknown word is assumed to be correct. Well, almost. Two different users must give the same answer to the same unknown CAPTCHA before it is taken off the list. 'Using the reCAPTCHA system von Ahn's team is digitizing documents and manuscripts as fast as the Internet Archive can supply them, and the good news for book lovers (and bad news for spammers) is that the supply of reCAPTCHAs is not likely to dry up any time soon.'"
story

Related Stories

Submission: New Use for Old Captchas by tomandlu (977230)
This discussion has been archived. No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

Carnegie Mellon CAPTCHA Digitization Project Now Underway 50 Comments More /

 Full
 Abbreviated
 Hidden
More
Loading... please wait.

  • Fiery church? (Score:3, Funny)

    by gEvil (beta) (945888) on Tuesday October 02 2007, @07:48AM (#20821635)
    Is this proof that Carnegie Mellon (and the BBC) support religious terrorism?

  • I want to participate... (Score:4, Interesting)

    by DrWho520 (655973) on Tuesday October 02 2007, @07:50AM (#20821651) Journal
    Where can I sign up? Sounds like a great way to burn a few hours on a rainy, Saturday afternoon!
  • So, the plan is to take already hard-to-read words, make them harder to read, pair them with another hard to read word, and see how many people agree it's the same word? I've already had words like 'Alau' and '45-618' in the few I've done, and since there's an ugly line through them, I can't be close to sure it's right... They make no sense, but they look like that. I'm betting at least 1 other person agrees and puts the same thing I did, accepting that translation into the database...

    And that's not even

    • Re:I'm not so sure this is a good idea. (Score:4, Insightful)

      by necro81 (917438) on Tuesday October 02 2007, @08:26AM (#20821955) Journal

      There still needs to be a human reviewing the work before it's truly accepted, and that human might as well be doing it in the first place, with the context still there to help them.
      That is, for all intents and purposes, impractical, which was the entire point. The backlog of work was never going to get done in a reasonable timescale with dedicated humans correcting all the errors. A dedicated human, even with the context, will still make mistakes or get stumped.

      Most people, when presented with a CAPTCHA, make an honest effort to try and get it right - otherwise they can't get their precious Facebook account. The number of people who understand what's going on with this reCAPTCHA thing is probably pretty small. Finally, those who know what it is about are probably inclined to not be jackasses and purposefully screw it up. I'd say that honest errors and malicious errors are an overwhelmingly small portion of reCAPTCHA responses. While flawed, this system might still be, say, 95% correct. So, for accepting a certain amount of error, you are able to get as much character recognition done as you are able to supply. As the article says:

      Given that it takes about 10 seconds to decipher a reCAPTCHA and type in the answer, this represents the equivalent of almost three thousand man hours a day spent deciphering words that CMU's computers find illegible.
      3000 man-hours a day at 95% accuracy versus, maybe, a few dozen man-hours a day at slightly higher accuracy. You tell me which is better.
      • I tend towards optimism, so whenever I catch myself going 'Wow, that's great!' I back off and take another look. They stated only 2 people had to confirm to accept a translation... If that were more like 4 or 5, I'd be a lot happier... It's a -lot- more duplication of effort, but rules out a lot of mischief, too. If it ends up like SETI, though, they'll have so much help that they end up processing all their data many years ahead of schedule.

        I plan to use at least the mailhide recaptcha on my site. I d

    • Re:I'm not so sure this is a good idea. (Score:5, Insightful)

      by smallfries (601545) on Tuesday October 02 2007, @08:27AM (#20821967) Homepage
      Wouldn't the easy solution be to present the context as part of the reCapatcha? Rather than two single words from isolated contexts, present two "lines" with a word or two either side, and a slight colour change on the target words to indicate which ones the system is after. This would make your validation easier but wouldn't aid OCR in any way.

      For your other point, there should be a "not a word" button to hit in that case to flag up that the original OCR has screwed up the word boundary.

      I thought it was a really novel project, reminds me of the image tagging "games" that people came up with last year, but in a new problem domain.

      • For your other point, there should be a "not a word" button to hit in that case to flag up that the original OCR has screwed up the word boundary.

        That would defeat the point of the project. Words scanned from real books contain all manner of 'not a word' combinations of letters and numbers, the principle is the same. I came across several portions of words that had been hyphenated at the margin of a page. Many Capatcha type systems use random strings of characters. Any non-english words that show up should be treated as a sting of characters.

    • Re:I'm not so sure this is a good idea. (Score:5, Funny)

      by MrMr (219533) on Tuesday October 02 2007, @08:40AM (#20822143)
      've already had words like 'Alau' and '45-618' in the few I've done, and since there's an ugly line through them, I can't be close to sure it's right... They make no sense, but they look like that.

      Congratulations,
      you managed to fail the Turing test.

    • Re:I'm not so sure this is a good idea. (Score:5, Informative)

      by Falkkin (97268) on Tuesday October 02 2007, @09:07AM (#20822517) Homepage
      "And that's not even counting malice where people deliberately put wrong words in."

      We're already getting several million legitimate solutions a day. The chance that a few malicious people would happen to get the same CAPTCHA is relatively small. Also, for many of our words, the OCR's answer happens to be correct -- it just doesn't have high confidence in the word. If a single person agrees with the OCR in this case, we can mark the word as "read" with no further human confirmation. For this reason, many of the words will only ever be shown to a single human.
      • "Never underestimate the power of stupid people in large groups."

        I'm sure you've got most bases covered, but intentional malice goes way beyond 'a few malicious people'. In this case, it involves at least 1 malicious person, a captcha breaker, a few thousand anonymous free proxies, and a lot of malice. I'll admit that I find this idea trivial because I'm a programmer, but I think most (non-script-kiddie) hackers will find it trivial as well.

        I sincerely hope nobody tries to sabotage your project, but I'd f

        • Re:I'm not so sure this is a good idea. (Score:4, Informative)

          by Falkkin (97268) on Tuesday October 02 2007, @09:38AM (#20822965) Homepage
          You said "people" putting in wrong words (ala the suggestion someone said below about "everyone fill in CowboyNeal!"), which is quite different from automated attacks. For that, we have numerous scripts that notice various forms of anomalous behavior from any given IP. We manually review these to make sure the answers are reasonable. We are also working with CERT, who have a large database of botnetted machines, to detect attacks. I'm not going to give complete details of everything we check, but rest assured that we are very active in preventing attacks -- our goal is to be the best CAPTCHA in the world, and we take security threats very seriously.

          In terms of the digital output, we spot-check some of the transcribed pages every day. These spot-checks will also turn up any anomalous solutions, with high probability.
    • I got "derground". If they are getting this from digitized books, they have to work on undoing hyphenation before presenting it to the user.

      I wonder, afte this is running for a while, most of the unknown words will be nonsense (jabberwocky, snickersnee) Proper or made up names (Elric of Melnibone? I saw Benoit in the third captcha I solved, I now got one that looks like Visscher), numbers and other things people wouldn't work through.

      The other problem is with common words that OCR gets wrong. I've/me are c
      • Since this is a university project, we do actually care quite a bit about transcribing books :) In fact, that's the aspect of the system that I'm primarily responsible for. However, there is a lot of really interesting data along the lines of what you're suggesting, and I'm sure some of that data will eventually make it into papers.

        "I wonder, afte this is running for a while, most of the unknown words will be nonsense"

        It's already been running for a few months, and we're getting millions of solutions a da

  • Problems (Score:3, Interesting)

    by David_Shultz (750615) on Tuesday October 02 2007, @08:07AM (#20821777)
    Interesting idea, but here are the immediate problems as I see them...

    Captchas are now twice as annoying for the user, since you have to type two words (but maybe the fact that there is some value in it will appease the user).

    Some algorithms these days are quite literally better than humans at detecting the hidden text in captchas. Pictures, not text, are better for this purpose.

    Testing the answer against another users answer is a good idea in principle (its how they make sure no one is cheating in distributed computing projects) but giving the same answer as another user is not difficult when they are using the same algorithm. We can assume that any algorithm being applied against this captcha is trying to do loads of work (that is, after all, why you write such a program) and so it will be answering the same question multiple times.

    Am I right on these points? (I just woke up).

    • Re:Problems (Score:4, Insightful)

      by AltGrendel (175092) <ag-slashdot.exit0@us> on Tuesday October 02 2007, @08:13AM (#20821833) Homepage
      I agree, but if you think about it, it's really a win-win for Carnegie Mellon. Either way, they get the text translated.

      • Re:Problems (Score:5, Insightful)

        by jsight (8987) on Tuesday October 02 2007, @08:22AM (#20821913) Homepage
        I agree... I don't understand why people find so many silly faults with this.

        1. Its not twice as annoying. Compared to how faded and scrambled many "one-word" captchas are, this is significantly less annoying.
        2. People seem to be acting like someone will fill out one word correctly and then intentionally scramble the other to screw up the project. Not many people are crazy enough to even want to do that. But even if they were, how do they know which word is the known, and which is the unknown?
        3. Endless Supply - Each word that is correctly translated is another word that is "known" and therefore can be safely used as a known in a new captcha.
        4. Verification - Thanks to #3, they could also potentially maintain the verification % rate for various words to later determine the accuracy or inaccuracy of past translations (assuming that they ever find that to be a problem).

        Yeah, we all know that captchas are not perfect, but this project is a better idea than most. And because it is centralized, they can update the image generation scheme centrally if it is broken.

        In practice, these seem to get broken less often than people think.
      • I agree, but if you think about it, it's really a win-win for Carnegie Mellon. Either way, they get the text translated.

        I think the GP's worry is that the spammers use OCR and there are a lot of them, so the two challenges you are relying on for checking both get answered by the same OCR spambot code - so they could match even though they're wrong.
    • "Testing the answer against another users answer is a good idea in principle (its how they make sure no one is cheating in distributed computing projects) but giving the same answer as another user is not difficult when they are using the same algorithm."

      Please RTFA. How do you propose that the same bot gets the same word twice in one sitting, let alone with the same warping and strikethrough so as to guarantee the same word is typed both times?

      Check out recaptcha.net [recaptcha.net] to test it out.

    • Some algorithms these days are quite literally better than humans at detecting the hidden text in captchas.
      As the article said, by selection, these are bits of text that OCR algorithms cannot read. We can assume that CM is using the best available OCR, so even 'some algorithms' that you mention, which are better than humans at reading captchas in most ordinary cases, will be ineffective for these particular images.
    • A couple things:

      1) We've done some studies at CMU that shows that recognizing and typing 2 real English words is much easier and faster than typing 6 or 7 random letters and numbers. Would you rather type "private much" (which is what just showed up for reCAPTCHA) or "KXd2cM" (which is what showed up for Yahoo's CAPTCHA)?

      2) Any given CAPTCHA is only shown to a couple of users. We're getting millions of legitimate solutions a day, so even a relatively sophisticated bot would have little chance of seeing th
    • I can see a serious privacy problem with this, since it divulges the IP address of visitors to a third party (Carnegie Mellon). The API is fundamentally broken, since both the website visitor and the website need to contact the central server (rather than the website alone), which allows said third party to generate personalized profiles of web surfers.
  • > The test, known as a CAPTCHA (Completely Automated Turing Test To Tell Computers and Humans Apart)
    > , was originally designed at Carnegie Mellon to help to keep out automated programs known as "bots."

    Where did they get the "P" from?
  • If all slashdotters would decide to answer with CowboyNeal to the second CAPTCHAs question, there is a large chance of his name appearing in one of the deciphered old texts. CowboyNeal to the Old Testament! This points out one major disadvantage of the system: since the computer can't check whether the answer is correct, a large group of people can abuse it with a growing probability in time. Since there is no incentive to answer to the second CAPTCHA correctly, making it widely known that the second CAPTC

    • Re:`CowboyNeal' answer to all CAPTCHAs (Score:5, Informative)

      by Falkkin (97268) on Tuesday October 02 2007, @09:21AM (#20822725) Homepage
      Sorry, but we've already thought of this attack :)

      We can compute the daily frequency of each human-provided solution and automatically flag anything that suddenly jumps in popularity. It's especially suspicious if these answers always disagree with the OCR's guess (often the OCR happens to be right, but just doesn't have high confidence).

      • Is there any word on how CAPTCHA decoders, like PWNtcha, perform against the current reCAPTCHA?

        In case reCAPTCHA can be automatically deciphered efficiently, a slightly altered malevolent attack might still be feasible. Let D be a roughly complete list of English words (a dictionary), together with the relative frequencies of the words occurring in standard English texts. Generate a fixed mapping f from D to D such that words are going to be assigned to each other only in case their occurrence frequencies

        • PWNtcha does not defeat reCAPTCHA, nor are we aware of any existing OCR or CAPTCHA-breaking algorithms that do. We are working with research groups at a couple universities who are trying to break our CAPTCHA (and if they can, we'll obviously fix it). In case we do notice a break, it's trivial for us to switch to a completely different kind of CAPTCHA (using different distortions). Because our system is a web service, if there is a security breach, we can fix it for all sites at once by simply changing t

  • "Turing" test (Score:3, Informative)

    by DrLex (811382) on Tuesday October 02 2007, @08:49AM (#20822271) Homepage
    Well, this finally makes CAPTCHAs somewhat useful. I won't try to formulate it in some sugar-coated way: I personally hate CAPTCHAs. On some types (especially the ones from Digg), I fail about 50% of them, and that's getting quite annoying after a while. Especially when your code is rejected even if you believe there is no doubt about what you've read in the image.
    I believe CAPTCHAs are the wrong solution to the wrong problem. It's a bit exaggerated to call them a "Turing test", because I'm quite sure that OCR systems will be made in the near future that are better than humans in reading CAPTCHAs. A simple text-based question that requires actual intelligence is a much better Turing test, and also a much smaller nuisance for people with impaired vision. Of course, writing a foolproof system that can produce a nearly infinite amount of such questions is a challenging problem by itself.

    • A simple text-based question that requires actual intelligence is a much better Turing test... writing a foolproof system that can produce a nearly infinite amount of such questions is a challenging problem by itself.

      I think it is more than a challenge. I have introduced a system like this on a public forum that I administer. It's a phpBB mod that asks a question during the registration phase to which the registrant is required to give a correct answer.

      The problem is that I have found it very hard to come

      • If you assume english knowledge:

        What language is this in?
        What are the first five letters of the alphabet?
        What are the five vowels?

        Other stuff:
        Are you a human or a computer program?
        What is the name of this site? (see title bar)
        Pick a number, any number. (Any number is taken as correct)
        Leave the following space blank.

        Of course, the biggest problem with a limited dictionary of questions like this is that a spammer can sit through them, answer them all, or at least a portion, and then put a script to replay the

  • Peekaboom (Score:3)

    by EnsilZah (575600) <EnsilZah@ G m a il.com> on Tuesday October 02 2007, @09:08AM (#20822541) Homepage
    Sounds like what they're doing at Peekaboom [peekaboom.org] and The ESP Game [espgame.org], harnessing humans to solve problems that are difficult for computers.
    Here's an nice video [google.com] on the subject.
  • I supervise an America's Army clan website which uses phpBB for the forums. Spam bots were barely slowed down by the standard CAPTCHA registration requirement. I'd get dozens of bogus registration requests a day from bots that used OCR to get in.

    A couple of months ago I switch to recaptcha.net's plugin for phpBB and it stemmed the tide. The number of spam bots getting thru decreased greatly. Those that did, I felt slightly better when I deleted their registration requests unfulfilled. Their Evil cpu cy

  • Drupal Module makes it simple (Score:4, Interesting)

    by Slashdot Parent (995749) on Tuesday October 02 2007, @09:19AM (#20822691)
    For all of you Drupal admins out there, I just wanted to let you know that there is a reCAPTCHA module [drupal.org] that makes using reCAPTCHA a snap.

    I'm not affiliated with the project, other than as a happy, comment-spam-free user of it.
  • From their learn more page [recaptcha.net]:

    f you get email spam we have a method that will help you to reduce it. Many spammers crawl the web looking for email addresses. When they see an email address on a web page, they send spam to the address. Mailhide allows you to safely post your email address on the web. Mailhide takes an address such as jsmith@example.com and turns it into jsm...@example.com. In order to reveal the address, a user must click on the "..." and solve a reCAPTCHA. If you use the Mailhide version of

    • Any way the spammers break this involves improved OCR. Said improved OCR will be available to Carnegie Mellon too, thus in any event the stuff will be translated faster (and if they restrict reCaptcha offerings to things their OCR has in fact choked on, it will retain its effectiveness even as OCR technology improves).

  • Minor problems but good overall (Score:3, Interesting)

    by MrKevvy (85565) on Tuesday October 02 2007, @11:58AM (#20825175)
    After doing a hundred or so, several problems I can see with this that may cause problems with accuracy even if the text is human-readable:

    1) Hyphenated word fragments broken over lines. ie "vances" where you can't see the "ad-" from the previous line.
    2) Dialectic spellings of English words, ie British spelling where "s" replaces "z" in verb forms such as "categorise"
    3) Numbers with commas/decimals. Is that thirteen-thousand "13,000" or a precise thirteen "13.000" to three places?
    4) Archaic spellings and outdated words. Because these are old books being digitized (only books before 1923 are out of copyright) this is quite common.

    But it's a brilliant idea and for the majority of the text samples there was no ambiguity.

    • Re: (Score:2, Insightful)

      by cheater512 (783349)
      I've found a flaw.

      It gives you two words to enter in but you only have to get the right one correct in order to get through.

      Spammers could fill the left word with nonsense and OCR the right one and the system would crumble.
      Who cares if the OCR isnt 100% accurate. It'll be good enough to get a lot of spam through.
      • Where in TFA does it say that the one on the right is always the right one?

      • Re: (Score:3, Insightful)

        by Smidge204 (605297)
        You don't know which word is known (and checked against) and which is unknown. This makes your ORC attack less effective because you must get BOTH words right in order to guarantee success.

        Also, if the first two people to decypher the unknown word don't agree, then the word is recycled back into the system until "a lot more people" submit the same answer. This greatly reduces the threat of a "garbage attack" because any random input is unlikely to be repeated by the second person to get that word, or anyone

          • Re: (Score:3, Interesting)

            by Smidge204 (605297)
            Still won't work. It's safe to assume the distortion/noise added to the text to prevent simple OCR would be different for each instance of the image; that's the whole point, after all. Hashes of the image data are useless in that case.

            Also, storing the hashes for successfully identified images is also useless... once a word is identified by at least two parties, it is removed from circulation. That means if the attacker IDs a word correctly, chances are it won't stay in the system much longer. Even if the a
    • You can try it out at the top of this page [recaptcha.net].
      • Unfortunately I think most CAPTCHAs use JS; it's been a while since I've been to a site that didn't make me turn it on to get through login/registration. I have no idea why this is, since people have been doing login pages since before JS was around or popular, but now it seems like the way every idiot is doing it.
    • Don't worry. The system only accepts a word as correct after two people give the same answer. Hopefully the next person to get your challenge won't make the same typo you did. :)
Search
Work consists of whatever a body is obliged to do. Play consists of whatever a body is not obliged to do. -- Mark Twain

All trademarks and copyrights on this page are owned by their respective owners. Comments are owned by the Poster. The Rest © 1997-2009 Geeknet, Inc.