DuckDuckGo Insists It Didn't 'Purge' Piracy Sites From Search Results (theverge.com) 33
An anonymous reader shares a report: Users of privacy-focused search engine DuckDuckGo have been unable to site search the domains of some well-known pirated media sites recently, as reported by TorrentFreak on Friday. This follows a News Punch article last month calling out DuckDuckGo for "purging" independent media sources from search results, and naming them "Google Lite." DuckDuckGo's CEO Gabriel Weinberg called the News Punch piece "completely made up" in a Twitter thread over the weekend to respond to the public and address both issues.
To observers, it seemed as if DuckDuckGo had de-indexed searches for copyright-flouting media download sites like The Pirate Bay and Fmovies, and even a site search for the open-source tool youtube-dl came up empty. TorrentFreak later updated its report citing a company spokesperson blaming the issue on Bing search data, which DuckDuckGo relies upon. Weinberg insisted the company is not purging any results and said that site search results are not appearing due to the site operator error "Anyone can verify this by searching for an outlet and see it come up in results," Weinberg tweeted.
To observers, it seemed as if DuckDuckGo had de-indexed searches for copyright-flouting media download sites like The Pirate Bay and Fmovies, and even a site search for the open-source tool youtube-dl came up empty. TorrentFreak later updated its report citing a company spokesperson blaming the issue on Bing search data, which DuckDuckGo relies upon. Weinberg insisted the company is not purging any results and said that site search results are not appearing due to the site operator error "Anyone can verify this by searching for an outlet and see it come up in results," Weinberg tweeted.
It is True Duck Duck Go did not purge piracy sites (Score:2)
BING did it for them
Re: It is True Duck Duck Go did not purge piracy s (Score:3)
Re: It is True Duck Duck Go did not purge piracy s (Score:5, Informative)
From DuckDuckGo's own help pages [duckduckgo.com]:
To do that, DuckDuckGo gets its results from over four hundred sources. These include hundreds of vertical sources delivering niche Instant Answers, DuckDuckBot (our crawler) and crowd-sourced sites (like Wikipedia, stored in our answer indexes). We also of course have more traditional links in the search results, which we also source from multiple partners, though most commonly from Bing (and none from Google).
So yes they do use Bing results but not exclusively (apparently they use Yahoo as well for general links). Looks like they use their own crawler mostly for generating Instant Answers (the quick summary blurb results) instead webpage links.
Re: It is True Duck Duck Go did not purge piracy (Score:2)
Re: It is True Duck Duck Go did not purge piracy s (Score:5, Informative)
Re: It is True Duck Duck Go did not purge piracy (Score:1)
Re: (Score:3)
noun
noun: irony
the expression of one's meaning by using language that normally signifies the opposite, typically for humorous or emphatic effect.
"“Don't go overboard with the gratitude,” he rejoined with heavy irony"
h
Similar:
sarcasm
sardonicism
dryness
causticity
sharpness
acerbity
acid
bitterness
trenchancy
mordancy
cynicism
mockery
satire
ridicule
derision
scorn
snee
Re: (Score:2)
Re: It is True Duck Duck Go did not purge piracy s (Score:4, Insightful)
The first thing I did when I read the original article was search the banned sites and sure enough they'd showed up in DuckDuckGo.
So did I, and precisely none of them showed up. They do now though.
Things don't stay banned for very long when you're getting bad press. Also worth noting that they very much also did not show up in bing. In fact my testing on the day showed the first pages of each results was 100% identical between bing and duckduckgo, including the list of TPB proxies it showed without ever showing TPB, or the fact that I got no search results when searching "site:thepiratebay.org"
I just assume the story was Google propaganda.
Please stop participating in the re-writing of history.
"Piracy" (Score:3, Informative)
This was obviously fake (Score:2, Interesting)
Re: (Score:2)
Well, these stories aren't always fake, like the time DDG took Credulous down, and it was basically inaccessible for weeks. People had to search it in dictionary.com to access it via proxy. The links are still up there, even.
Re: (Score:2)
Now we've got two stories and a bunch of speculation, all over something which clearly didn't actually happen.
Welcome to Slashdot! I hope you enjoy your stay.
Re: (Score:3)
I immediately tested it, going to DuckDuckGo and performing searches for sites like PirateBay and YouTube-dl. Each search returned results as expected and the links worked.
So did I and no it did *not* return TPB or YouTube-dl. Heck go to the original post and you'll see plenty of people corroborating the story.
The fact that it is now fixed after it was shown (via screenshot no less in TFA which I'm sure you didn't read) to be a problem, and after the bad press it generated does not make anything "fake".
BING?! No wonder... (Score:3, Interesting)
Re: (Score:3)
I have DDG available, but usually use Startpage. It's a privacy-forward site based in the Netherlands. They use Google, but scrub the nosy parts. Here's what they do:
https://www.startpage.com/en/how-startpage-works/ [startpage.com]
Re: (Score:2)
Try using MSDN to search for an MSDN article if you want an exercise in frustration. Even Yahoo is better at finding MSDN information than Microsoft itself.
Bing seems to be the REAL problem here! (Score:2)
Why the h*ll is DuckDuckGo using Bing for anything? I thought they developed their own search crawlers and so on?
Sorry DuckDuckGo you don't get to use that defence (Score:2)
When your website proudly claims:
DuckDuckGo gets its results from over four hundred sources. These include hundreds of vertical sources delivering niche Instant Answers, DuckDuckBot (our crawler) and crowd-sourced sites (like Wikipedia, stored in our answer indexes). We also of course have more traditional links in the search results, which we also source from multiple partners, though most commonly from Bing (and none from Google).
you don't get to play the victim when your search results end up blacklisting sites.
Just come out and say it, DuckDuckGo is 99% Bing.
and DDG its right (Score:2)
If anyone bottered to try a search, puting "the pirate bay" or "youtube download" DDG shoved the sites like allways, was just the site: thing the one who not worked.
DDG should prioritize crawling banned sites (Score:3)
ok with it (Score:1)
Re: (Score:2)
Honestly, I don't really want illegal torrent results showing up when I'm casually searching. It seems weird search engines would allow them. Same thing with Amazon search returning counterfeit goods. If you really want pirate results, use a pirate search engine -- no need to nudge people to illegitimate options.
For people able to think for themselves all options should be on the table. It is hard to make informed decisions when your information is censored. That is not nudging, it is just respecting the intelligence of their users.
Re: (Score:2)
By that logic Facebook should be delisted, as way more illegal things go on their than on thepiratebay.org.
Either done by Bing or DDG, purge is real (Score:2)
All the people telling that the removal is not real, it's Google propaganda, etc. please READ the original article. What was removed were DDG "bangs" for these sites (the search feature using '!' plus a shortcut to search on a site) and the indexing of youtube-dl.org. And this is real, anyone that wants to test, just search:
site:youtube-dl.org
And you will find 0 results. Bing could be to blame for the youtube-dl.com site yielding 0 results, but how could Bing remove the "bangs", them being a DDG feature? So
More like Apple-lite (Score:2)
"you're holding the search the wrong way" - Weinberg