DuckDuckGo Insists It Didn't 'Purge' Piracy Sites From Search Results

An anonymous reader shares a report: Users of privacy-focused search engine DuckDuckGo have been unable to site search the domains of some well-known pirated media sites recently, as reported by TorrentFreak on Friday. This follows a News Punch article last month calling out DuckDuckGo for "purging" independent media sources from search results, and naming them "Google Lite." DuckDuckGo's CEO Gabriel Weinberg called the News Punch piece "completely made up" in a Twitter thread over the weekend to respond to the public and address both issues.

To observers, it seemed as if DuckDuckGo had de-indexed searches for copyright-flouting media download sites like The Pirate Bay and Fmovies, and even a site search for the open-source tool youtube-dl came up empty. TorrentFreak later updated its report citing a company spokesperson blaming the issue on Bing search data, which DuckDuckGo relies upon. Weinberg insisted the company is not purging any results and said that site search results are not appearing due to the site operator error "Anyone can verify this by searching for an outlet and see it come up in results," Weinberg tweeted.

It is True Duck Duck Go did not purge piracy sites

[the_other_one]

BING did it for them

Re: It is True Duck Duck Go did not purge piracy s

[fbobraga]

DDG really uses BING? It's not just a conspiracy theory?

Re: It is True Duck Duck Go did not purge piracy s

[Myrv]

From DuckDuckGo's own help pages [duckduckgo.com]:

To do that, DuckDuckGo gets its results from over four hundred sources. These include hundreds of vertical sources delivering niche Instant Answers, DuckDuckBot (our crawler) and crowd-sourced sites (like Wikipedia, stored in our answer indexes). We also of course have more traditional links in the search results, which we also source from multiple partners, though most commonly from Bing (and none from Google).

So yes they do use Bing results but not exclusively (apparently they use Yahoo as well for general links). Looks like they use their own crawler mostly for generating Instant Answers (the quick summary blurb results) instead webpage links.

Re: It is True Duck Duck Go did not purge piracy

[fbobraga]

Thanks

Re: It is True Duck Duck Go did not purge piracy s

[klipclop]

The first thing I did when I read the original article was search the banned sites and sure enough they'd showed up in DuckDuckGo. So if it was banned, it wasn't for very long. I just assume the story was Google propaganda.

Re: It is True Duck Duck Go did not purge piracy

[Plugh]

Ironically, GOOG spreading misinformation and propaganda to serve its own interests⦠does not seem implausible to me.

Re:

[ThurstonMoore]

noun
noun: irony

the expression of one's meaning by using language that normally signifies the opposite, typically for humorous or emphatic effect.
"“Don't go overboard with the gratitude,” he rejoined with heavy irony"
h
Similar:
sarcasm

sardonicism
dryness
causticity
sharpness
acerbity
acid
bitterness
trenchancy
mordancy
cynicism
mockery
satire
ridicule
derision
scorn
snee

Re:

[Xenx]

Maybe you should look up the meaning of irony? A search engine intentionally providing misinformation would in fact be irony. One could make an argument about it being Google specifically, but that doesn't necessarily negate the irony only weaken it.

Re: It is True Duck Duck Go did not purge piracy s

[thegarbz]

The first thing I did when I read the original article was search the banned sites and sure enough they'd showed up in DuckDuckGo.

So did I, and precisely none of them showed up. They do now though.

Things don't stay banned for very long when you're getting bad press. Also worth noting that they very much also did not show up in bing. In fact my testing on the day showed the first pages of each results was 100% identical between bing and duckduckgo, including the list of TPB proxies it showed without ever showing TPB, or the fact that I got no search results when searching "site:thepiratebay.org"

I just assume the story was Google propaganda.

Please stop participating in the re-writing of history.

"Piracy"

[fbobraga]

People steals ships today yet: it's a new trend!

This was obviously fake

[computer_tot]

I saw the SlashDot story about this when it first published. I immediately tested it, going to DuckDuckGo and performing searches for sites like PirateBay and YouTube-dl. Each search returned results as expected and the links worked. The story was obviously fake to anyone who cared to test it. Now we've got two stories and a bunch of speculation, all over something which clearly didn't actually happen.

Re:

[DigitAl56K]

Well, these stories aren't always fake, like the time DDG took Credulous down, and it was basically inaccessible for weeks. People had to search it in dictionary.com to access it via proxy. The links are still up there, even.

Re:

[cascadingstylesheet]

Now we've got two stories and a bunch of speculation, all over something which clearly didn't actually happen.

Welcome to Slashdot! I hope you enjoy your stay.

Re:

[thegarbz]

I immediately tested it, going to DuckDuckGo and performing searches for sites like PirateBay and YouTube-dl. Each search returned results as expected and the links worked.

So did I and no it did *not* return TPB or YouTube-dl. Heck go to the original post and you'll see plenty of people corroborating the story.

The fact that it is now fixed after it was shown (via screenshot no less in TFA which I'm sure you didn't read) to be a problem, and after the bad press it generated does not make anything "fake".

BING?! No wonder...

[killmenow]

I use DDG almost exclusively for internet searches. Except for the 2% - 5% of the time that I get shit results and have to go to google for accurate ones. Now at least I know why. Frankly, I am amazed it's as good as it is. Every time I try Bing I get worse results than DDG somehow. The other day I was having an issue with something specific to Microsoft (don't remember if it was an Azure or M365 issue but something Microsoft cloud-ish) and searched on Bing...where I got shit results. Same search on google and what I was looking for came up in the top 5 results. Bing literally couldn't give me good results for MICROSOFT'S OWN STUFF. But Google was like, oh yeah, here ya go... Make that make sense.

Re:

[Miles_O'Toole]

I have DDG available, but usually use Startpage. It's a privacy-forward site based in the Netherlands. They use Google, but scrub the nosy parts. Here's what they do:

https://www.startpage.com/en/how-startpage-works/ [startpage.com]

Re:

[cusco]

Try using MSDN to search for an MSDN article if you want an exercise in frustration. Even Yahoo is better at finding MSDN information than Microsoft itself.

Bing seems to be the REAL problem here!

[King_TJ]

Why the h*ll is DuckDuckGo using Bing for anything? I thought they developed their own search crawlers and so on?

Sorry DuckDuckGo you don't get to use that defence

[thegarbz]

When your website proudly claims:

DuckDuckGo gets its results from over four hundred sources. These include hundreds of vertical sources delivering niche Instant Answers, DuckDuckBot (our crawler) and crowd-sourced sites (like Wikipedia, stored in our answer indexes). We also of course have more traditional links in the search results, which we also source from multiple partners, though most commonly from Bing (and none from Google).

you don't get to play the victim when your search results end up blacklisting sites.

Just come out and say it, DuckDuckGo is 99% Bing.

and DDG its right

[gitano_dbs]

If anyone bottered to try a search, puting "the pirate bay" or "youtube download" DDG shoved the sites like allways, was just the site: thing the one who not worked.

DDG should prioritize crawling banned sites

[Khopesh]

DDG has its own crawler and presumably some (anonymized) caches of results, so it shouldn't be much of a stretch to notice when certain sites fall into oblivion and then to check on them with their own crawler. If they were on top of their game, they'd have crawled many of these sites before the news spread too far, thus healing the problem and limiting the bad press.

ok with it

[RegistrationIsDumb83]

Honestly, I don't really want illegal torrent results showing up when I'm casually searching. It seems weird search engines would allow them. Same thing with Amazon search returning counterfeit goods. If you really want pirate results, use a pirate search engine -- no need to nudge people to illegitimate options.

Re:

[Kernel Kurtz]

Honestly, I don't really want illegal torrent results showing up when I'm casually searching. It seems weird search engines would allow them. Same thing with Amazon search returning counterfeit goods. If you really want pirate results, use a pirate search engine -- no need to nudge people to illegitimate options.

For people able to think for themselves all options should be on the table. It is hard to make informed decisions when your information is censored. That is not nudging, it is just respecting the intelligence of their users.

Re:

[wisnoskij]

By that logic Facebook should be delisted, as way more illegal things go on their than on thepiratebay.org.

Either done by Bing or DDG, purge is real

[doragasu]

All the people telling that the removal is not real, it's Google propaganda, etc. please READ the original article. What was removed were DDG "bangs" for these sites (the search feature using '!' plus a shortcut to search on a site) and the indexing of youtube-dl.org. And this is real, anyone that wants to test, just search:

site:youtube-dl.org

And you will find 0 results. Bing could be to blame for the youtube-dl.com site yielding 0 results, but how could Bing remove the "bangs", them being a DDG feature? So

More like Apple-lite

[wisnoskij]

"you're holding the search the wrong way" - Weinberg