Search Engine Optimization Poisoning Way Up In '10

alphadogg writes "Cybercrooks continue to abuse the Web, boosting their ability to produce what's called search engine optimization poisoning so that individuals making use of search engines such as Google's increasingly are ending up with choices that are dangerous malware-laden URL links. Some 22.4% of Google searches done since June produced malicious URLs, typically leading to fake antivirus sites or malware-laden downloads as part of the top 100 search results, according to the Websense 2010 Threat Report published Tuesday. That's in comparison to 13.7% of Google searches having that outcome in the latter half of 2009, says Patrik Runald, Websense senior manager of security research."

Link to Actual Report and My Many Gripes

[eldavojohn]

You can find the actual Websense Threat Report in ASP-driven HTML here [websense.com]. I mention ASP because the video doesn't seem to be functioning correctly in my non-IE browser.

I thought I would find this in the NetworkWorld article. Boy was I mistaken. As I switch between the two pages of the article, I am presented with "Whitepaper" links to reports that then navigate me to a 'page1234' at accelacomm.com where it asks for all my personal information. In the middle of the article (with no indication this has nothing to do with the article) is a link to another NetworkWorld article titled 'Royal pain: British Royal Navy site hacked.' Shouldn't that go in the 'Related Content' section that is also in the article with links to how I can 'bail out my budget'? Oh look, they've hyperlinked phrases in the article that just direct me to another NetworkWorld article and at the end I get directed to their security section. Might they take a chance and link to the source of the information that they are considering an authority on SEO poisoning? So you know, I can judge for myself and further inspect the report? I mean, I'm not asking them to drive across town to get a quote from the mayor ... this is the smallest gesture of investigative reporting one could possibly do.

Sorry to rant for so long but it amuses me how a news article about SEO poisoning is obviously taking some questionable routes to up their own stats -- maybe even manipulate Google page ranks? Oh but that's just good old wholesome Search Engine Optimization -- it's those pesky cybercrooks that phish for my home address, not the "esteemed" online news sources we should criticize that ask me to enter it into accelacomm.com when I'm trying to read the news (and I'm not accusing accelacomm of being a scam, just annoyed at the principle).

Re:Link to Actual Report and My Many Gripes

[Shados]

#1: its in asp.net, not asp (big difference)
#2: asp.net doesn't have a dependency on IE. Its browser agnostic (and thus like any other environment used for web development, it works BETTER if you're not using IE)
#3: the video is in Flash using a pretty standard Flash player that has nothing to do with asp.net.
#4: it works just fine in non-IE browsers (I'm using Chrome)

Just figured I'd clear that up.

Re:

[Dalzhim]

#1 Chrome doesn't represent 100% of non-IE browsers
#2 If Chrome doesn't represent 100% of non-IE browsers, then you haven't verified that "it works just fine in non-IE browsers"
#3 The video isn't "using a pretty standard Flash player", you might be doing so, but people on an iPhone might be using a service to convert flash to HTML5 which isn't a pretty standard player.

Re:

[Shados]

Woosh!!.

My point is: if something isn't working, its not because of something that is IE-only. News Flash, even Firefox, Chrome and Safari are far from 100% compatible. Standard all you want, even those browsers disagree on some stuff. Secondly, my other point was that the video player isn't part of ASP.NET since it doesn't provide one. So whatever they're using, if it sucks, doesn't suck because of ASP.NET. Another news flash: like most web development environments, ASP.NET lets you render whatever you fre

Re:

[jojoba_oil]

When the doctype is overridden to one listed as experimental, I would expect the validator to show fewer errors; however, the source of that site clearly labels itself to be using the XHTML 1.0 Transitional doctype. That, as the GP makes reference to, has 522 errors and 7 warnings.

Re:

[E IS mC(Square)]

You mean his validator is invalid?! Mind boggles.

Re:

[Dishevel]

He is using Lynx.

Re:

[negRo_slim]

I thought I would find this in th NetworkWorld article.

Networkworld sure does seem to get linked to a lot around here lately.

That aside, the summary states 22.4% of Google Searches produced malware results. Okay so obviously 22% of searches aren't going to be for anti virus software and the like, so can we just call this one a stupidity tax and move on? I recently had to remove a virus from an acquaintance's machine (3ghz celeron w/ 248mb RAM) by the time I was done I wanted to put it back on for the gentleman assumed it must of been the government out to ge

Re:

[Tanktalus]

Can't help but wonder if these people even need a connection to the internet. Now granted that's not to say infections can't happen to everyone, because they can and they do but I think we can all agree the vast majority of infections delivered by shady sites are borne by the vast vapid masses. I mean you don't turn on your car and get on the freeway with nary a clue how it works do you? Why on earth should you get on the information superhighway when you don't even what a processor or memory is?

You had me until here. I get in to my car with nary an idea on what nearly everything in the engine (processor/memory) is or does. All I know about a vehicle is what I can reach from the driver's seat: ignition, steering wheel, gas and brake pedals, radio, climate control, spedometer, odometer. There's also a tachometer (or something) which strongly correlates with engine noise, and also tells me when my gas engine turns off while I'm stopped (hybrid) - beyond that, I have no care.

I don't see why a compu

Re:

[IBBoard]

I mean you don't turn on your car and get on the freeway with nary a clue how it works do you? Why on earth should you get on the information superhighway when you don't even what a processor or memory is?

I've only got a vague notion of what the carburetter is/does, and I probably couldn't identify it if asked to point it out. I do, however, know about petrol, oil, tire tread, lights and general driving. Does that mean that I shouldn't drive? Yes, knowledge of computers is good and people somehow assume th

Re:

[mujadaddy]

I've only got a vague notion of what the carburetter is/does

...as well as how it's spelled ;)

Re:

[IBBoard]

Granted, I did have to look it up, but you seem to have only a vague notion of what the English language is. Carburetter [wikipedia.org] is the proper (i.e. English) way to spell it ;)

Re:

[virg_mattes]

I mean you don't turn on your car and get on the freeway with nary a clue how it works do you? Why on earth should you get on the information superhighway when you don't even what a processor or memory is? Can the knowledge really get any more fundamental than that, for at some degree shouldn't we be held accountable for our own actions or lack thereof?

This isn't a good car analogy. The problem is that knowing the basics of how a computer works doesn't translate to an understanding of online threats, social engineering works on very smart people, and there's no reason why someone needs to understand the "basics" of internal cumbustion to operate a car safely so I see no reason why and understanding of processors and memory is needed to grasp online safety.

For the consequences can be just as real when you find you just sent your life savings to a scammer in Nigera, or got your dumb ass key logged while going into your PayPal.

Nigerian scams and the like worked before there was an Internet (point being that social engineeri

Re:

[mcgrew]

Your car analogy is a tad flawed. If you drive on the freeway without knowing how to drive, you risk injuring or killing innocent people. And you don't have to know how an internal combustion engine works, or how an automatic transmission works, or what a fuel injector is to drive a car, or few would be able to drive at all.

And I would say that your "Oops she's murdered" is more than just a stretch; people die on the highway every day, but you'd be hard pressed to find more than one or two murders that coul

Oblig

[Anonymous Coward]

My search engine optimization goes to '11

Re:

[Yvan256]

Enjoy your lead of 51 days while it lasts.

Malware/Spyware isn't the only problem...

[drunkennewfiemidget]

At least in my case, I've found that google's search results have gotten progressively more useless over the last 2-3 years.

I search for a linux issue I'm having, the only hits I get are ubuntu users in 2004.

I search for applications for my wife's phone, it's almost 100% adware sites, and 0% useful download links.

My google search usage is going down steadily. If I want to know about a company/famous person/whatever, it's en.wikipedia.org/wiki/.

Info on movies, actors, etc? imdb.

Looking for directions? Mapquest. Google maps has gotten me lost on countless occasions. (By doing such things as telling me to get off a highway by crossing the meridian, and exiting on the onramp for the opposite direction.)

I don't know whether it's just me, google has thinned out the effort going into their searches in favour of their (many) other endeavours, or if they're just not evolving as fast as the assholes who want to try and monetize my searches for completely unrelated shit.

Re:Malware/Spyware isn't the only problem...

[olsmeister]

By doing such things as telling me to get off a highway by crossing the meridian, and exiting on the onramp for the opposite direction.

Are you sure that it's just poor directions? Have you done anything to piss off Google lately?

Re:

[formfeed]

By doing such things as telling me to get off a highway by crossing the meridian, and exiting on the onramp for the opposite direction.

Are you sure that it's just poor directions? Have you done anything to piss off Google lately?

Don't buy a GPS systems from Atmos !

Re:

[symes]

If I want to know about a company/famous person/whatever, it's en.wikipedia.org/wiki/.

Info on movies, actors, etc? imdb.

Looking for directions? Mapquest.

I hadn't really thought about this, but on reflection I find i am doing similar... going down the domain specific route rather than the all encompassing google way. But is this the rise of places like imdb as much as failings on the part of google?

Re:Malware/Spyware isn't the only problem...

[jhigh]

It's probably a combination of the two. Google search results are definitely becoming more useless, and I think as more and more people become familiar with the Internet, their behavior patterns will evolve to reflect this. I think it's not just more specialized web sites like imdb cropping up, but user familiarity with the existence of these sites. As the Internet becomes more and more a part of our daily lives, web sites advertise on television, etc., it's only natural that average users are becoming more familiar with specific web site offerings and foregoing the extra step of typing a search into Google. The (potential) down side to this is what happens when a new, better web site crops up that may be infinitely better than the one that we're all familiar with. For example, once the world became accustomed to using Microsoft Office exclusively because that is what they were the most familiar with, it has become increasingly difficult (if not damn near impossible) for any other product to break into that space.

Is it possible that we will see similar things happening with web sites, where inferior sites are getting all of the hits simply because they are what people became familiar with early on?

Re:

[grahamm]

The (potential) down side to this is what happens when a new, better web site crops up that may be infinitely better than the one that we're all familiar with. For example, once the world became accustomed to using Microsoft Office exclusively because that is what they were the most familiar with, it has become increasingly difficult (if not damn near impossible) for any other product to break into that space.

It is perfectly possible for this to happen. Thinking back a bit, WordPerfect was just as ubiquitous as MS Word is now, but in a very short time almost everyone changed to using MS Word.

Re:

[numbski]

Actually, what I'm finding irritating at the moment is over-zealous use of robots.txt "Disallow" to prevent Google from caching. What winds up happening is that for one reason or another, the content is gone or removed, but still appears in the search results, yet you get a 404 page, and you can't pull the cached copy.

Useless Search Content

[ideonexus]

I'm seeing the exact same thing. I find that Google is becoming more and more useless for academic research. I would once type in a subject and get tons of legitimate, informative sites written by people who cared about the subjectmatter (remember ThinkQuest [thinkquest.org]? All those fantastic articles are still out there, they just aren't in Google's search results anymore), which I could use as a springboard into deeper research. Now I get Wikipedia as the first result and fifty pages of forums filled with people who have no idea what their talking about. There's still no algorithm for content quality.

mod parents ++

[Anne Honime]

Sadly, it's very, very true indeed. I made a (legitimate) site not long ago using Google sites. Submitted the url for review in both bing and google. Bing listed me the next day, without any further input. Google didn't even listed my url in its base (not talking about rank here) until I submitted a sitemap through the webmaster tools, fighting a nasty bug they made in the process but didn't cared to correct since at least a year (if you put your auth key in your DNS zone, the automated sitemap created by

Re:Useless Search Content

[Monkeedude1212]

I'm not sure if this is relevant - but perhaps you should be using google Scholar for your academic research. It's possible that they segregrated what information you're looking for into that section.

But then again, maybe not - I don't know what kind of research you do (and I've never had a problem with springboarding with a Wikipedia article...)

Re:

[interval1066]

"...perhaps you should be using google Scholar for your academic research."

Good point. And google code for code that may help that programming project. And so on (I don't know what other specific search pages google has, but there must be others.) Google.com is of course the premier page for advertisers, so using it to search is going to return some bad results skewed to whomever has paid google the most money. Fortunately has provided other tools.

Re:Useless Search Content

[TheRaven64]

Not necessarily. Google Scholar will only find peer-reviewed papers (not very competently, and omitting much of the information required to find where it was originally published), but I find reading researchers' blogs often turns up more interesting stuff. It often takes 1-2 years between doing the work and having a journal paper published (and another little while for Google Scholar to notice it), so a blog post from a decent researcher about his or her current work will tell you stuff now that won't appear in Google Scholar results for 2-3 years.

Re:

[BJ_Covert_Action]

Might I suggest trying Duck Duck Go [duckduckgo.com] as a search engine? I've been using it for a few months now and I have been consistently pleased with the relevance of the results it returns as well as the various shortcuts for specific types of searchs (for instance, if I want to use Google to find something instead, I can type "!google [search term]" and I will be redirected to a google search). It definitely has some work to do on returning more results for obscure searches, but it seems to be doing well for a relati

Re:Malware/Spyware isn't the only problem...

[TheGratefulNet]

google totally sold out and lost their mojo.

I get link farm sites from the first page that SHOULD be weeded out. I search for tech things and get mostly 'buy this!' crap sites.

google chooses to do this. they could do better (they did, once) but now they are no better than any random search engine. worse since their UI is less direct and more junk oriented. we have seen google do a lot of auto-things (animation, auto scrolling of text ads, auto complete, auto-think!) and none of it is really welcomed by the user community.

its just what we all predicted. google would be a golden child for a few years but then it will fizzle out.

its ONLY because of habit that many people still use google. but they are not any better than the rest, these days, and their search seems like a paid service for all the wrong 'content suppliers' (I use that term very loosely).

I wish altavista was back. I miss the old days.

Re:Malware/Spyware isn't the only problem...

[FudRucker]

yup, now that Google has their namebrand recognition they dont give a damn anymore as if they left the office with their servers running on autopilot while they are all out vacationing while the revenue rolls in. typical of most companies = they start out with benevolent ideals and once the ball is rolling and the money starts pouring in then it all goes to heck while the owners go out and play rich guy.

Re:

[Dishevel]

we have seen google do a lot of auto-things (animation, auto scrolling of text ads, auto complete, auto-think!) and none of it is really welcomed by the user community.

None of it is welcomed by you. Most of it can be turned off with a few clicks.

Google actually dose things fairly well compared to most companies. I can choose simple and clean or bleeding edge cool stuff. Never once have I searched for something Microsoft related and got the first 3 links pointing me to Googles competing products.

Google is not perfect and I would not trust them with my child but I trust them and like them more than Bing or Yahoo.

Re:

[16384]

Although I do agree that google seems to be on a descending path, the alternatives don't have an index as comprehensive. So, until then, you either accept sub-optimal results or you have to use google.

Re:

[TheRaven64]

I use DuckDuckGo, which is built on top of Yahoo's stuff. It returns far fewer results than Google, and at the end it has a link to try using Google instead. I've clicked on this a few times and as far as I can tell the only difference is that Google pads the results with a few thousand irrelevant pages. I've never clicked on the link and found that Google actually provides a useful response. I think I'd actually prefer a search engine to tell me it couldn't find anything than to give me 100 pages that

Re:

[delinear]

I wouldn't say Google is about to fizzle out soon, but I totally agree that a lot of people stick with them now because either it's what they've always used (if they're new to the web) or there's nothing yet that reliably does things better. I'm at the point where, if a search engine that reliably directed me to more relevant information and away from ad/malware sites (and also identified when content was just straight copy-pasted between sites, so the first five results aren't the same question and respons

Re:

[NJRoadfan]

call me old, but I remember when Alta Vista started getting banner ads. Heck, I remember when it was still part of DEC. Google's pagerank worked great... before everyone gamed the system. On top of that, webpages suffered. Ever see a SEO optimized webpage? They repeat terms needlessly hoping Google notices.

Re:

[antdude]

Ditto. I miss old Google and AV. What current search engines can match them these days? :(

Re:

[hedwards]

That's one of my top complaints about Google. The link farms and the results which require you to scroll way down to the bottom of the page to find the information. Google's approach worked well in the past when speed was more of an issue, but now that the web has adjusted to Google's stupid algorithm it's getting progressively worse.

The other annoyance with Google is that it can be a real pain searching for things if you don't know exactly what it is that you're looking for. And the seeming inability of

Re:

[labnet]

I see lots of complaining but no alternatives being offered up: Anyone.. Ferris... Anyone...

Re:

[Jugalator]

Yes, I think I've seen the same thing. And either Google is very silent about their search engine updates besides the visuals, or they're doing very little to combat the problem. All I seem to hear is efforts to let you get the results faster (the latest ideas being "Instant Search" and "Instant Previews"), although I can't say I'm having trouble with Google being sluggish. The fake blogs or forum scrapers, on the other hand...

I understand that it's hard to differentiate carefully crafted fake sites from re

Re:

[lwsimon]

As someone who is getting into SEO and Internet Marketing, I can tell you that there was a major change in the last 2-3 weeks that has lots of big names in that industry reeling.

Google makes major updates to their PageRank formula about quarterly, from what I can see.

Re:Malware/Spyware isn't the only problem...

[Trepidity]

The main thing saving Google's ass these days is that 90% of the time they can just throw up a Wikipedia result in the top-5, and usually that's good enough.

Re:

[TheRaven64]

DuckDuckGo goes a step further and quotes from the relevant Wikipedia entry in the zero-click info at the top of the results page. It also automatically uses the HTTPS version of Wikipedia if you click on it, so both the search and the page you visit are encrypted.

Re:

[bluefoxlucid]

Moreover, top 100? Who goes past page 1? If you don't see it immediately, you're searching wrong.

Re:

[nedlohs]

Looking for directions? Mapquest. Google maps has gotten me lost on countless occasions. (By doing such things as telling me to get off a highway by crossing the meridian, and exiting on the onramp for the opposite direction.)

I guess if the road is going North/South you are going to have to cross a meridian to turn off at some point...

Re:

[The Moof]

That would make interstate driving very interesting...

Re:

[NJRoadfan]

The problem is in reality, there is no break in the median to drive through. Google's mapping went downhill when they decided that they should be cartographers. The base map data they sourced from (likely the old census TIGER maps) is outdated and filled with errors.

Re:

[0123456]

When I was looking for directions to a hotel last time I went on a business trip, Google told me to follow the main highway and turn right. Which was correct, except it involved driving through a wall and then falling thirty feet into the hotel parking lot.

Re:

[tibman]

I think you are right.. the results have been going down lately. Many times when i search for something the first few links are just sites that say the phrase i searched for in the middle of an advert page.

It's like the old web again.. in a way. Its difficult to find the things you want.. but when you do find a site that works, you mentally bookmark it and just direct type it next time. When i want to buy something i go straight to amazon and ebay. Screw searching because i end up in a maze of advertise

Re:

[curveclimber]

It's not just you. I remember when I first started using google and how amazingly appropriate its results were if you knew the right search terms. Now days I'm surprised more that it does so poorly on what seems like straightforward searches.

Why is this? SEO must be part. But I also know if anything I'm looking for is even slightly related to a product, forget it, you get pages and pages of shopping results. I too, have to result on my memory and knowledge of where to look for certain things more and m

Re:Malware/Spyware isn't the only problem...

[melikamp]

IMHO, the trash in the Google search is mostly due to spammers: the people who game the page-rank. I agree with eldavojohn: everyone is doing it these days, and the "news" sites are especially notorious. The line is very blurry. I know a dude who works for gather.com, and they are doing it by inserting "keywords" into their news articles. This is not the same as using a botnet to generate traffic, but the goal is the same.

May be the future of search is Bayesian filtering? It is doable even right now: have a local program load 1000 or so Google hits and unleash on them your own personal filter. Everyone heard about spam/ham filtering, but the math and the algorithm extend naturally to any finite number of categories, so a user can create categories such as "spam", "science", "shopping", "blog", "porn", train the filter, and enjoy truly personalized search results. Google is obviously loosing to rank gamers, they are way too smart and too quick to adapt. But a personal Bayesian filter could take the raw index with 90% spam and select results relevant to YOU, while slashing the amount of spam by a couple of orders of magnitude.

My Thunderbird filter works like a charm: in the last year I've had 1 (one) false positive and what feels like less than 5% of false negatives. I think it will work just great on Web pages.

Um, I am resubmitting this, since it's not appearing. Sorry if it's a dupe.

Re:

[Arthur Grumbine]

I was hoping that Google was employing exactly that (personalized filter) when they gave me the opportunity to "star" or "trash" a given result. Before they got rid of the "trash" option I spent hours exploring the different results and returning to the original search to star or trash them based on their helpfulness/relevance. Sadly, this did not have any discernible effect on the quality or type of result.

Opportunity lost, Google...

Re:

[melikamp]

I am afraid that Google is not interested in suppressing their sponsors' sites, so that's one huge deficiency right there. On top of that, the kind of analysis I described is expensive if it is to be truly personalized, even just for people who log in. Your personal word frequency table will contain hundreds of thousands, if not millions of entries, and that's just for a single user. And if you do it wholesale (a single table for everyone), the conditional probabilities will probably average out. What am I

Re:

[Reziac]

I wonder if a return to no weighting at all, other than an obvious "filter trash" option (which might be achieved by filtering against junk sites that try to be all responses to all queries), might actually be more useful than the current SEO-gamed results.

It does seem to me that search results went downhill much faster AFTER Google started doing more weighting of results -- IOW the cure was worse than the disease.

Right now, results from other search engines aren't really any better, you just get a differen

Re:

[couchslug]

"I search for a linux issue I'm having, the only hits I get are ubuntu users in 2004."

It's not just you, things have changed for the worse, and it's disturbing.

Proficient Linux users will know to visit appropriate forums, but noobs will have a worse time.

Ah, but THAT is the problem

[SmallFurryCreature]

There is a VERY simple solution to this problem. Refine your search. On the left you can select from which period you want it. I tend to do my linux searches in the last year if they are errors and Ubuntu releases in the last half year/month.

That gives you far better results. Also adding the ubuntu version helps a lot.

But the MAIN problem is that computers still cannot understand human writing, especially chaotic human writing. If every linux article was clearly labelled with a date and a status (fixed fo

But that just begs the question then...

[Pollux]

I've found that google's search results have gotten progressively more useless over the last 2-3 years.

So, what's better than Google?

I mean, back in the late 90's, I encouraged everyone to go to Google at the time, because it was so much better than the competition: Yahoo, Excite, Lycos, AltaVista...they all paled in comparison to the accuracy of Google.

But now, is there anything better than Google? Or is it just like the airlines, where there's no "best option" because everything is terrible.

Wiki for info

[HalAtWork]

The reason you're going to Wikipedia for actual information is because the site is structured to eliminate anything that isn't factual information. You're just realizing that the web is a bunch of crappy cross-linked blogs and syndicated content behind ads/paywalls. Soon you'll be hitting podcasts for editorial content instead of the ad-laden multi-click regurgitated PR between top 10 lists that make up most sites.

Re:

[melikamp]

You're just realizing that the web is a bunch of crappy cross-linked blogs and syndicated content behind ads/paywalls.

But it's not. Well, may be it is mostly, but I still believe it's plausible to say that useful (whatever that means) content on the Web dwarfs that on just the Wikipedia. What we have here is a spam problem. It's not that Web is nothing but trash. It's not. But Google and others fall short when it comes to filtering out trash. It really is a search problem, not content or availability. And there may be a way out of it [slashdot.org].

Re:

[HalAtWork]

I guess that's true, but I would also rather use Wikipedia than an average web site. Every page is formatted and organized the same, so even after I've gotten to my destination it's a lot easier to absorb the information and also to immediately find relevant info on adjacent topics that is presented in the same way.

I do find Wikipedia a much better starting off point than typical search results, right at the bottom of the article are a bunch of non-crap links to start you off. Maybe Wikipedia is a better

Re:

[noidentity]

Same here. I have a local search page where you type your query, then click on the appropriate button, e.g. "Wikipedia", "IMDB", etc.

A while back Google had put up their old 2000-era index, for fun, and you could do a web search using it. I switched to that immediately, since the results were much higher quality, even if 10 years out of date. Unfortunately they disabled that a few weeks after.

Search engine rankings for legitimate sites

[Compaqt]

The annoying thing is when sites that have legitimate and interesting content are ranked nowhere near the spammers.

Many legitimate and useful sites are far and few between. You have to bookmark them because it's doubtful you'll find them again with Google (page 20 or something).

Re:Search engine rankings for legitimate sites

[dkleinsc]

This sounds like a very very familiar discussion. Specifically, we had this exact same problem about 10-15 years ago when search spammers had learned how to game results on Yahoo and AltaVista with stupid meta tags and repeating the same words over and over to increase their ranking.

Google figured out a way to get around that problem, which produced a massively better search engine. It sounds like the search spammers are now figuring out how to game the Google results, so in another year or two we'll be right back in the big mess that Internet search used to be.

Re:Search engine rankings for legitimate sites

[Trepidity]

I think PageRank is ultimately some of the problem, though I hear they've been de-emphasizing it (but it hasn't fixed my searches). When I search for band lyrics, I want the lovingly crafted fan site that's been accumulating information on that band for the past 10 years. When I search for reviews, I want that site too. I don't want mp3lyrics.com for lyrics or allmusic.com for reviews or whatever. But the problem is that each of the good fan sites is a separate entity (which is one reason they're good): one's at joydiv.org, another one's off some person's university webspace, another one's on free hosting somewhere, yet another one's at brainwashed.com or synthpunk.org or whatever. So they each rank lower than mp3lyrics.com or allmusic.com, which have mediocre info for every band on the planet tucked away under their single pagerank unit.

Same with non-music stuff. You're never going to find the person with a great page on blueberry pies; instead you'll get a recipe from eHow.

Re:Search engine rankings for legitimate sites

[theskipper]

Speaking of ehow (Demand Media), here's a great article about how they're junking up the SERPs. It's not just small time link farms, it's industrial strength pollution backed by hundreds of millions of dollars.

http://www.wired.com/magazine/2009/10/ff_demandmedia/ [wired.com]

Google is going to need to take a firm stand. And they most likely want to do it desperately now that there's some real competition. But it's a tough nut to crack and they certainly don't want to upset their applecart (i.e. ad revenues).

Re:

[Reziac]

Damn... all is explained. Thanks for the link. I'd wondered just why the hell all that crap was on YouTube... Now I'm wondering when Google and Demand Media will have a merger. :(

Idea for Google search and Facebook

[microbox]

I published some essays on the web, and am wondering why I cannot search for them through google. Curiously, by website jumps up straight away on bing. Grrr.

I'm thinking google search should:

• Display the results in a table -- with a few controls on each line. I generally click on a bunch of results and open them in new tabs. Ideally I want to step back to the search result page and rank the search result: {excellent, okay, poor, junk, nasty}, or something like that. I understand that this cannot be done

Re:

[Spazntwich]

I think you just clarified a lot of similar but muddy thoughts for me.

I was thinking I just romanticized the past of the internet where I was hitting new and unheard of domain names every day to find awesome in-depth websites on whatever subject I was searching for, but I don't think I'm doing so to excess.

Much of the internet's information has been centralized on these monolithic info clearinghouses, but it's just so... sterile, and Google is obviously to blame by putting far more emphasis on the generic p

Re:

[dargaud]

The annoying thing is when sites that have legitimate and interesting content are ranked nowhere near the spammers.

And there are some strangenesses: I found a site I was looking for with very unique keywords: 210th out of 231 results on Google. A friends with a different browser and location: 3rd ! It should have simply been number one since it's the reference for those specific keywords (the name of the company and its field of expertise, the others were just mentioning them).

Re:

[delinear]

Apparently ranking is now meant to take your previous searches into account, which might account for the difference, but I know what you mean, I've searched for very specific OS error messages before and found the first page that actually contained the full unique string was not always the first result returned, and was sometimes not even on the first page of results. Of course I can enclose it in quotes for an exact match, but that relies on the person who typed it up getting it exactly right, and besides,

Variable ranking considered harmful

[Compaqt]

Isn't variable ranking sort of the same as changing menus in XP, to which geeks have complained about to no end?

Basically, the problem is, everyone is seeing a different Google. I sometimes create a link as a Google search instead of a Wikipedia link. But that doesn't really work if everyone has a different ranking.

Google Can Ban Sites, So...

[BoRegardless]

If you abuse Google by deliberately manipulating to get high page results and they knock you out, then why can't Google permanently knock out the same 22.4% of the search result sites that host malware? That would END most users being able to come into contact with the criminally minded in that form of scam.

Re:Google Can Ban Sites, So...

[Antony T Curtis]

Probably because malware organisations have discovered an ancient and dark evil who would further their cause ... for a price.

They're called: Lawyers.

Re:

[IBBoard]

They also have co-conspirators: Cheap Domain Sales (previously Domain Tasting, but I think they cut down on that). You can't kill an IP because you may take innocents with it (and a domain would just shift to another IP) and you can't kill a domain because they'll just buy another domain for a few days.

Just wait until we have to play whack-a-mole with IPv6 spammers - now there's a huge range to try blocking via blacklists!

Re:

[Anpheus]

Often those sites are unwitting hosts to malware that are eventually cleaned up, so I'd hope that they don't start permanently blocking.

I have noticed this quite a bit

[WiglyWorm]

Almost always when searching for breaking news, the top results are complete spam and malware.

Re:

[lwsimon]

I think that's due to the way Google weights new stuff. I've been promoting my personal blog heavily, and got to the top 5 for a dozen of my target keywords. After a week there, suddenly I'm down to 800-1000 on all of them. WTF?

Google lets you rank highly for a few days before dropping you to the bottom. Hence, SEO experts with newsy keyword targets will outrank news sites - briefly.

Google Instant

[sottitron]

I wonder if Google Instant will soon compound this problem. Once you're apt to see a tidbit of a result and quickly click through, that would be quite the prime target for this type of attack.

Re:

[Animats]

I wonder if Google Instant will soon compound this problem. Once you're apt to see a tidbit of a result and quickly click through, that would be quite the prime target for this type of attack.

Google Suggest (the command-completion part of Google Instant) already had a major spam problem. Google Suggest isn't driven by page rankings; it's driven by Google Trends [google.com], which was updated every few minutes. So, generating a large number of search requests in a short period could push a request to the top entrie

Post search results

[Drakkenmensch]

Post contents - Trusted download - [CLICK HERE]

Post contents - Full download - [CLICK HERE]

Post contents - Key generator - [CLICK HERE]

Post contents - torrent link - [CLICK HERE]

Re:

[zwei2stein]

You know, it is actually awesome way to fight casual piracy.

After three pages of rebranded "Post contents - HD Download - [CLICK HERE], Post contents - Fast Download - [CLICK HERE]" link nests most people will simply give up before finding site with actual downloads

Re:

[ObsessiveMathsFreak]

All your Post contents FREE--Click here for free subscription.

in my experience, not as bad as Bing

[ynohoo]

Just yesterday I wanted to download VLC media player. Top link on Bing: repackaged with junk seach engine and crapware newsletters. Top link on Google: the home site which linked to the sourceforge download. Of course Microsoft could be doing that on purpose for Open Source software...

Re:

[amentajo]

Just now, I searched, and it isn't [bing.com].

Re:

[delinear]

For me, searching for VLC media player, I get www.videolan.org/vlc/ on both Bing and Google (so long as I ignore the crappy sponsored links on both sites).

Re:

[dotwhynot]

Just yesterday I wanted to download VLC media player. Top link on Bing: repackaged with junk seach engine and crapware newsletters. Top link on Google: the home site which linked to the sourceforge download. Of course Microsoft could be doing that on purpose for Open Source software...

What country are you in? It's really only US that have Bing yet (rebranding old Live Search in all the other countries to Bing without actually having the product is an amazing decision btw..) and a search for VLC on Bing US gives me a very useful and relevant top result. With direct links to download even for Mac and Ubuntu versions:

http://imgur.com/RGqtA.jpg [imgur.com]

Re:

[ynohoo]

from Ireland it gave me http://www.vlc-download.com/ [vlc-download.com], which is the crapware download.

What are they searching for?

[sudnshok]

The article is not clear what search terms produced 22% malicious URLs. That seems like a high number to me. If you search for "photoshop crack" or "keygen" you're going to get WAY more malware than searching for "fuzzy bunnies".

While I agree that more spam and malware sites have gotten into Google listings, I don't think the problem is quite as dire as the article makes it seem for the typical Google user.

Re:

[TheGratefulNet]

google thinks the ONLY valid reason for the web is to let us 'shop for things'. sorry but I do a lot of tech searches (looking for code fragments or schematics or HOWTOs) and more often than not, the first FEW pages are ads to sell me something.

we need a front-end to google to keep google honest. there have been front-ends, too, but google found out and stopped it (usually).

Re:

[whoop]

No, it's not 22% of search results, but 22% of searches made which contain a malicious URL somewhere in the top 100 search results. Like anyone goes all the way through to 100 results.

Some 22.4% of Google searches done since June produced malicious URLs, typically leading to fake antivirus sites or malware-laden downloads as part of the top 100 search results

Fear mongering. That is all.

Who browses 100 results?

[Dalzhim]

I rarely go past the first page of results (which means 10 results in my case). I don't really care about malware that makes it to the first 20-30 results.

i got a virus from youtube the other day

[alen]

clicked a real ad on youtube for a Mario Bros game because my 3 year old was interested. installed it and then Symanted popped up a warning that it was a trojan

That's funny

[countertrolling]

I've seen a couple of Slashdot journal writers who try to manipulate SEOs and page hits by getting to get you to click through their media merchandising blogs if you want to see the story they are journaling about. They should be marked as spam, because that is what they are.

Hmm... really?

[rickb928]

Really?

I rarely bother with results beyond the first 20 or so. IF I have to dig deeper, either I munged the search terms, or I'm digging for a specific item I couldn't build a specific search for. Either way, I'm wondering how what percentage of search returns in the first, say, 30, were malware.

And I wonder about the definition of 'malware'. But let's trust that.

How about a small effort, along the way, to clean up the fake links? If I search for a term that even tangentially matches a product, I get search results that invariably include Bizrate and other so-called shopping or pricing sites. And sure enough, Bizrate in particular has an actual product listing about 20% of the time for me. The rest of the time, it did the SEO thing to make it look like it had a listing, when all I get is a 'we don't have any right now, but how about these?' or 'come back later'. Argh. Abuse. Perhaps fraud. I hate them so much I ignore them even if they DO have the product.

Google doesn't care, though. They get paid anyways.

Feh.

JS:DR

[PPH]

Article requires JavaScript: Didn't read.

Need to vote or rank results...

[seanvaandering]

They really need to create a ranking system for logged in Google users so people can vote down spammy links. Could be based on the frequency of the reports. Anything in first ten results with more than 100 negative votes per hour, automatically get removed and placed into a holding queue for a Google employee to review. If it's discovered to be spam, automatically penalize the URL in all results and remove it. Hosting companies will never want to host spammers, because all their good customers will go running to the hills. Just a thought..

Manual ranking won't work.

[Animats]

They really need to create a ranking system for logged in Google users so people can vote down spammy links.

Won't work. The spammy links come and go too fast. Mean lifetime of a phishing site is a few days. Since most are created automatically, dealing with the problem manually will always be struggling to catch up.

Take a look at our list of major domains being exploited by active phishing scams. [sitetruth.com] That's from PhishTank data, which is updated manually. The list is ordered by how long the site has been

Re:

[delinear]

There have been plenty of live exploits in the last few years, but as I'm sure you're fully aware, the vast majority of people are not running FF+AdBlock+NoScript. Nor would they even if they knew what that was, since most people want all the flashy stuff that drives us geeks mad. I know. The story is still relevant, since even if you're in no danger of being infected, you're still suffering from having junk search results returned.

Re:

[phantomfive]

13 advisories for Firefox this year [secunia.com], most covering multiple vulnerabilities. Adblock doesn't do much for you because they can put the exploit in the page if they feel like it. NoScript will help you a lot more, but there have been vulnerabilities found in JPEG libraries, for example.

The problem is you don't know what vulnerability is out that that hasn't been found yet. All of the ones in that list weren't known either, even 12 months ago. There are certainly others that exist but haven't been discovered

Re:

[astar]

I have started a little project to see if I can eliminate problems with drive-by binary drops and about anything dependent on binary code. It actually looks straightforward, but only useful for someone who actually is concerned. It looks pretty doable for any of us to put together from a fpga a low end armish thing with a randomized opcode set. At the moment, I am just trying for a standard armish thing, with no randomization. As far as I can tell, this segment is just system administration skills.