AVG Fakes User Agent, Floods the Internet 928
Slimy anti-virus provider AVG is spamming the internet with deceptive traffic pretending to be Internet Explorer. Essentially, users of the software automatically pre-crawl search results, which is bad, but they do so with an intentionally generic user agent. This is flooding websites with meaningless traffic (on Slashdot, we're seeing them as like 6% of our page traffic now). Best of all, they change their UA to avoid being filtered by websites who are seeing massive increases in bandwidth from worthless robots.
How do you really feel? (Score:5, Insightful)
Why don't you tell us how you really feel about AVG?
Slimey ? (Score:5, Insightful)
if you want the definition of Slimey see Symantec/Mcafee/MicrosoftOneCare
while this doesnt excuse their behaviour, trying to protect people (a lot of them for free) is not Slimey but insulting them on the front page of Slashdot is
patheticOn the Up and UP. (Score:4, Insightful)
Smiley anti-virus provider? The integrity of Slashdot submissions just keeps going up and up! Nice example Taco.
Re:I discovered this the hard way (Score:5, Insightful)
I did the same and for the same reasons.
Not sure how this practice justified the poster calling them slimey.
I've been relatively happy with AVG. Perhaps, someone could elaborate on how they are slimey. This appears to be an attempt to protect people.
DDOS (Score:3, Insightful)
With all the readers of Slashdot, I think it would be safe to bet we will see a DDOS of AVG servers.
Re:I turned it off (Score:1, Insightful)
if you google it, you can install with command line switches to not even install this part of the program.
and thus you dont need to disable it, and thus you dont get the "somethings wrong" icon (which i just autohide anyway).
and as to AVG being slimey, get real.
The SLIMEY bastards in the anti-malware, anti-virus world are symantec and macafee.
both install horribly bloated piles of horse dung which attempt to hijack everything a user does, and prevents themselves from being disabled easily for testing purposes.
AVG provides a product that for the most part is ABSOLUTELY FREE.
thus if you dont like it, dont use it.
and as for the user agent strings, i'd be willing to agree with the poster above about them being legit looking IE strings to prevent possible redirection based on them if they used their own, by malware laden and virus laden sites.
Slimey? (Score:4, Insightful)
Re:I discovered this the hard way (Score:5, Insightful)
Perhaps, someone could elaborate on how they are slimey. This appears to be an attempt to protect people.
Ok, think of the /. effect. Now take that on almost any website who's servers aren't as strong. This is basically a huge DDoS attack on many websites by AVG that has a reason behind it. But it is still a DDoS attack.
Sorry, but no. (Score:0, Insightful)
I wish, however, they would take business needs into account before launching software that makes life even more difficult for the people trying to do the analytics
AVG is for the consumer. I want them to keep my machine free of bullshit. And we all know there's tons of bullshit out there. And with AVG, I'm becoming a little less paranoid with websites, but I'm still giving all of them bogus information because I trust no website with any of my information. Sorry webmasters and site owners, as far as this web surfer and consumer with the money that you want is concerned, you are all crooks out to plant something on my machine, grab my email address to spam me, and possibly do something I've never heard of - unless proven otherwise.
Don't like it? Tough shit! It's the cost of doing business. So suck it up and just shut up and do your jobs and work around this "problem".
Re:Sending the bills to them. (Score:4, Insightful)
You need explicit permission to access a public website now? Shit! I'd better get offline and write an apology to CmdrTaco - I've been using /. without permission for the best part of a decade!
Time to post a specific statement on all websites stating that AVG does NOT have consent to access or "visit" these websites.
That's a bit like putting up a 'No Trespassing' sign inside your cellar, and expecting it to prevent people coming over your fence.
Re:I discovered this the hard way (Score:5, Insightful)
They might be dumb instead of slimy...
Re:Sending the bills to them. (Score:5, Insightful)
no your not a lawyer, but i'm pretty sure your not smart enough to be one either.
you didn't give them permission to access your publicly available site?
really?
are you sure?
because you know, if you make something publicly available on the public internet, I'm pretty sure by definition, you've therefore given them permission to access it.
Just like everyone else "in the public".
Did you give Google permission?
how about every other search/index site?
as to the "extra bandwidth" since it is by definition, caused by your websites being found via search providers, maybe you should be sending the bill for linking to them and thus causing the "extra bandwidth" to Google/Yahoo/MS and see how far that gets you.
Re:I discovered this the hard way (Score:3, Insightful)
I could agree with it being called a DDoS attack if the intent was to actually deny service. It's not--the intent is to index pages for malware as part of their version of McAffey's SiteAdvisor.
If one wants to call such a thing a DDoS attack, then one must also condemn Google every time they index the website.
Re:I discovered this the hard way (Score:3, Insightful)
not really.
in order to "cause" the "attack" the website must come up in a search.
all this does is "pre-crawl" the pages in a search result to look for malware.
so unless everyone is searching google for the same thing, it really doesn't do a ton.
unless of course you run some pos server and have somehow gotten your result for whatever to be top ranked and of course it's a popular search string.
but then, i would blame the company, not avg, since they've gone to the trouble to probably cause this themselves.
Return the love! (Score:2, Insightful)
Re:I turned it off (Score:1, Insightful)
They're not a non-profit; they give away the base version because they sell more paid versions that way. Considering that "anti virus" software is all snake oil by definition, I think slimy fits. But then, some people still buy rabbit's feet, toss salt over their shoulder, refuse to break a chain letter, flip out if someone opens an umbrella indoors...
Re:I discovered this the hard way (Score:2, Insightful)
Perhaps, someone could elaborate on how they are slimey. This appears to be an attempt to protect people.
Yes, it's a very well-intentioned DDoS attack.
You'd have to fake the user agent (Score:5, Insightful)
When probing for sites that serve malware, wouldn't you have to make the probe look identical to a legitimate user?
Otherwise the malicious site could just serve innocuous content to the probe and malware to everyone else.
Insightful ?????? (Score:5, Insightful)
How exactly do the websites getting slammed with this bullshit traffic "not even install this part of the program" and "if you don't like it don't use it"?
Did you miss this part: (on Slashdot, we're seeing them as like 6% of our page traffic now)
So how does Slashdot "just not use" the AVG product and recover that 6% of their page traffic again?
The complaint is that they are "spamming the internet with deceptive traffic". That's a server/hosting complaint, not a user complaint about some user who can't figure out how to disable that feature.
Kudos on getting a "4 Insightful" for a ridiculously inapplicable and nonsensical response though!
Re:I discovered this the hard way (Score:5, Insightful)
So AVG is reducing your security... (Score:4, Insightful)
And with AVG, I'm becoming a little less paranoid with websites
That is, you're reducing your security because you believe AVG is providing you valid information about the reliability of websites.
Re:I discovered this the hard way (Score:3, Insightful)
I think most of the rest of your point stands, but this doesn't. The difference between google and AVG is that Google's servers coordinate so you don't end up with thousands or millions of requests from them in a short span of time (as many sites are reporting) and they obey robots.txt so you can opt out. If AVG had servers that kept track of the results of these scans so that every client didn't need to download a page but instead communicated with AVG for results, then the comparison would stand.
Slimy? (Score:5, Insightful)
I think I missed the memo - why is AVG a "Slimy anti-virus provider"? That portion of the summary BEGS for supporting links...
Re:This is not AVG itself (Score:3, Insightful)
This is not AVG doing this, it is the AVG IE toolbar. And since this is running in the IE context it is debatable if it should not use the IE user agent.
What is debatable, is why the toolbar must scan all results for each user. A site is either malicious or it's not, such site probes must be done centrally and kept in a shared database.
AVG instead decided to save themselves some effort and cost of doing this centrally, and incur the cost on their clients, and on the site owners who are unfortunate (heh...) to be present in Google's search results.
If you use Firefox or disable the toolbar it is a non issue. The issue to me is I can't figure out how to install AVG without this toolbar, or how to remove it.
So I guess CmdrTaco should just disable his toolbar and it's a non issue.. Oh, right. The issue is with the site owners as much as, if not more, than AVG's users.
Re:Slimey ? (Score:1, Insightful)
The more hits Slashdot gets, the more bandwidth they're using and the more money they're spending for that bandwidth. Increased bandwidth by bots that will never click on the adverts on the page means lots of resources wasted for absolutely no gain. It is indeed slimey and the guys running Slashdot have every right to be upset with AVG because of this.
Re:F5 IRule (Score:5, Insightful)
Re:I turned it off (Score:2, Insightful)
The issue is not with "if you want you can disable it" - because 99.9999% of users are not aware what is happening and would not care to Google this to re-install with a feature disabled.
The issue is that our servers are receiving 50% more hits than "necessary". This translates into money. Yes, bandwidth costs money and if we go over our allotment this month and are charged per gigabyte I will fully blame AVG for it.
We are a legitimate site and have taken great care to ensure our code is secure. Scanning our pages hundreds of times per day is simply a waste of resources, especially when done for users who are not even visiting the site but are only seeing links to us in search results elsewhere.
When will the webmasters grow a pair... (Score:2, Insightful)
AVG's botnet is currently 20 million strong and growing. If AVG can do this type of DDoS against websites, what is to stop any other malicious entity from doing the same?
Re:HOWTO install AVG without Search Crawling (Score:2, Insightful)
Actually I always disable Browser Helper Objects in Internet Explorer, since I've never seen a BHO that I actually wanted.
http://support.microsoft.com/default.aspx?kbid=298931 [microsoft.com]
Re:One Word (Score:0, Insightful)
There is an obvious answer to your question about why you shouldn't expect everything to be free but I wonder if you can figure it out.
Their eggs are slimy. (Score:5, Insightful)
And if that causes problems for webmasters, Thompson says, so be it. "I don't want to sound flip about this, but if you want to make omelets, you have to break some eggs."
Sounds like a "fuck off" to me.
I guess slimy is in the eye of the beholder, but the attitude reminds me of Claria.
Hanlon's razor with the save! (Score:5, Insightful)
Re:I discovered this the hard way (Score:5, Insightful)
Prefetching your search results doesn't protect you from viruses any more than just checking the pages you try to load at the time of loading.
What it does, is basically scanning the entire internet, weighted toward the pages its users search for, and I assume reporting back to AVG which websites have malware or suspected malware on them.
The problem with this theory is that malware sites can move around quickly, so learning that domain xzclqqkxzz.com tried to upload a virus to someone's computer 48 hours ago is not especially valuable information.
That's in addition to AV software being essentially impossible to keep up-to-date anyway, you can look up studies but most AV software lets a lot of malware through.
And the increased traffic annoys webmasters because the prefetches are (attempted to be) disguised as actual page fetches, and they come from all over the internet, so we think they're real clicks from real users but they're not. Plus, for some sites the increased load/bandwidth may be a problem.
Re:Hooray (Score:5, Insightful)
Hooray! Look at all the OH SHIT my server's on fire!
COMMODO (Score:3, Insightful)
Re:I discovered this the hard way (Score:5, Insightful)
Dumb is what they were BEFORE they were told about the problem. Slimy is what they are now that they are refusing to rectify the situation and behave.
I think they deserve everything they will inevitably get as a result of this.
Re:I discovered this the hard way (Score:3, Insightful)
They might be dumb instead of slimy
Hanlon's Razor [wikipedia.org] is often cited, but I don't think it applies. I rather believe in most cases the truth is the exact opposite; you can call it "mcgrew's razor" if you like.
"Never attribute to stupidity that which can be adequately explained by greedy self-interest unless proven otherwise."
I'll believe "slimy" until "stupid" is proven.
Re:I discovered this the hard way (Score:3, Insightful)
Even if it did, it wouldnt really change anything, since it's not just 1 server doing it, it's everyones' PCs. They couldnt be expected to all communicate and coordinate how often then hit servers. If they're going to coordinate, it would make more sense to just share the info about which sites were malware and which werent, which would actually be better than what they're doing now.
Re:When will the webmasters grow a pair... (Score:3, Insightful)
Because 99% of AVG installers will not have the slightest clue that they are contributing in a harmful way to Internet traffic volumes - They are just installing the latest version of their free AV product, and is is largely harmless to them.
The user is freely choosing to install a "beneficial" application, one which in many respects is a very functional capable and respected product.
This can hardly be compared to the stealth-install used by trojans and viruses which create DDoS BotNets... Can it? ;-)
OTOH, I would love to see a major ISP send AVG a bill for this traffic :)
What about advertising? (Score:5, Insightful)
Could AVG be unintentionally committing massive click fraud?
Re:F5 IRule (Score:4, Insightful)
Actually this is quite close a real solution :) what AVG should have done is cache the scan results from each page. Thus if a user tries to access a page it should first query AVG for a result. ( the result here is- OK page or not OK to visit page)
If a result exists in cache, no need to scrape the page. If there is no result in the cache, both AVG server and Client (to avoid trust issues) should query and compare results. The cache should periodically refresh and use multiple different UAs to avoid gaming. Quite a nice solution if you ask me ;) I knew I should have take up consulting instead of this damn Ph.D..
Also AVG are not slimly, the spyware/trojan/malware site operators are. Not to mention Norton/Symantec/Kaspersky et al.. The feature can easily be turned off and its purpose is to help the user at no $ cost. Besides, which self respecting /.-er needs anti virus [xkcd.com]
Re:I discovered this the hard way (Score:4, Insightful)
All AV software compaies are slimey, because AV software gives you a false sense of security. It can't detect any malware that isn't in its database, and it can't stop a luser from running a trojan. But the luser doesn't know this, and thinks it's safe to click on any damned thing.
If your OS "needs" AV, your OS, IMO, sucks badly.
Re:How do you really feel? (Score:5, Insightful)
Re:One Word (Score:5, Insightful)
Generally, you're right. 99.9% of all virus infections I see on other people's machine would have been prevented if they just followed safe computing practices:
1) Don't download useless crap. Useless crap is subjective, but programs that change your cursors, have teddy bears come out and wink at you, or other non-sense are definitely useless crap.
2) Even on Windows, try and use open source software where you can. In general, if something comes from SourceForge, I trust it a lot more than some random closed source app that I've never heard of.
3) Research your programs before you download them.
4) Use a good browser and limit what scripted content you run. Firefox with NoScript works perfectly. After you whitelist your normally visited sites you rarely have to worry about it anymore, and any new site that needs to run scripts has to be approved first. And for most sites, unless I notice something broken/not working on the site, I don't allow scripts even if the popup does appear.
Personally, I haven't ran an on-access virus scanner in 3-4 years. They're intrusive and take up valuable resources. I do currently have ClamWin installed though, and scan every so often just in case. I've always turned up clean (at least on my Windows machine - I have Linux and Mac machines as well that I don't even have to worry about scanning).
Since too many people around here seem to think that I'm a door-to-door compute repair guy and keep calling me, I've seen tons and tons of spyware infected machines. Lately I've taken to pointing them to the Apple Store to show them the Mac Mini, and telling them that I'm going to be switching over to only supporting Mac's soon . . . (hoping that if they'll just buy the damned Mac that I'll never have to visit again anyways)
Re:One Word (Score:3, Insightful)
Actually, on-access scans are resource hogs on a live workstation, and are not worth the overhead when all you need to do is exercise some common sense in the things that you open.
Remember who we are talking about now: Users. The same people who demonstrate time and again any idea of common sense and will click on any prompt that comes their way. For a sysadmin, automated scans will remove far worse headaches like the thought of a virus running free on your network.
Re:I discovered this the hard way (Score:3, Insightful)
Correction: "The important questions here are "Does the AVG spider ignore the robots.txt rules?" and "Do they try to hide/distribute their IP addresses?" If the answer to either of these questions is "Yes," then we have a problem--if not, however, we have only umbrage."
Sorry. Momentary braino.
Re:F5 IRule (Score:5, Insightful)
So doesn't this render the link scanner completely useless? I assume someone looking to dodge the AVG scanner for eeeeeevil purposes can just do the same thing, no?
Re:I turned it off (Score:1, Insightful)
Re:I discovered this the hard way (Score:3, Insightful)
Comment removed (Score:5, Insightful)
Re:One Word (Score:5, Insightful)
Anti-virus software is the one thing I would never download from a torrent site. You need to be sure it comes from a trustworthy source so you can check all your other torrents with it.
Re:Awwww, so sorry for all the webmasters (Score:4, Insightful)
I think you're missing the point: it scans links that users are not going to. It scans every result from a search, and not just the ones that you're browsing to. This significantly increases the traffic that sites have to deal with while not increasing user security at all, since the pages can just as easily be scanned while they are downloaded.
But maybe you're just trolling.
Sure AVG's not slimy... (Score:5, Insightful)
Also AVG are not slimly, the spyware/trojan/malware site operators are
However, I'd argue it's the equivalent of using a flamethrower to take out a wasp's nest - the amount of collateral damage to non-malware sites due to the spurious pulls is excessive, there are cleaner methods available.
Re:F5 IRule (Score:5, Insightful)
Not "Slimey" . . . (Score:2, Insightful)
Seriously, AVG wasn't trying to DDoS websites around the world - they were only demonstrating that they aren't very good at predicting the consequences of their software's actions.
Never attribute to malice what can readily be explained by simple ignorance.
Re:Sending the bills to them. (Score:2, Insightful)
Well, the "No Trespassing" sign in this case is presumably a robots.txt file.
AVG is choosing not to follow robots.txt. If you accept that AVG's linkscanner is, in fact, a robot, then they're basically ignoring a clear warning to keep the hell out.
What's still open to debate, in my mind anyway, is whether the AVG linkscanner really qualifies as a robot. If it is, then certainly a web browser that performs pre-fetch is as well, and ought to follow the same standards.
Re:Awwww, so sorry for all the webmasters (Score:3, Insightful)
Re:No sir, that is not the complain (Score:1, Insightful)
"What AVG is doing is illegal"
Source?
"FACT: A distributed denial of service attack is a federal offense."
So is murder, this is neither, why lie about what this is when what it really is is bad enough?
"YOU ARE AN IDIOT"
And you are a liar.
Truth said.
Uh-uh - you're putting the onus on everyone BUT (Score:2, Insightful)
Lemme get this straight - for all intents and purposes, AVG has turned their entire customer base into one huge botnet, yes? They can't instruct it to "attack server ", or to initiate campaigns to increase the size of their botnet, but a botnet it remains. Anybody with AVG software installed will accept whatever that software does (at the behest of AVG), but since it lives under a cloak of legitimacy users won't be trying to purge it from their hosts anytime soon.
So - AVG Antivirus is a trojan, it's behavior once installed is much like a worm, it has been shown to inadvertantly cause DDoS attacks on websites (hey, what's the impact on the backbone from this?). AVG Antivirus is the BitTorrent of the botnet world!
If I wrote software like that, DOJ'd have me in jail 'til my beard reached past my kneecaps.
Re:Awwww, so sorry for all the webmasters (Score:5, Insightful)
Are users not supposed to protect themselves in the interests of the website?
This isn't being done to protect users. The pages could be scanned just as easily on actual load. This is being done to prevent the users from having to suffer a small delay on loading the page by preloading it (and every other possibly link on the page since the software doesn't know what link you're going to click).
You're just putting spin on the issue because this is affecting your cost/income ratio.
You're very anti-average Joe. Most of us aren't Amazon. Most of us, in fact, make precisely zero income from our websites. And we don't have the kind of financial resources to deal with this kind of distributed attack on our bandwidth. Amazon, Yahoo, and such won't have any problem dealing with this sort of thing, but if it becomes popular, it'll force the rest of us off the web.
Since the problem of malware sites is not going to go away and since AVG is effective more antivirus software will start using these techniques. Unless you have something better to suggest?
Yes, make the user wait the extra second if the user wants to scan a page.
Frankly, as an end user, I don't give a damn about your costs and stats. I don't care about it for amazon, ebay, myspace, or paypal. I do care that if I follow a link to an unsavory site that I am protected.
If that's true, then you won't mind waiting the extra second to load a page instead of having the browser drag down the bandwidth of every site in your search ahead of time for you.
Here is another question. Do you want a userbase that is populated by malware infected computers? Is that preferable to figuring out a way to work with AVG new technique?
That's a false dilemma. Is it preferable to force everyone other than the big guys off the web so that users don't have to wait an extra second on loading a page?
Dont throw your users under the train. They have a right to their security and peace of mind.
Don't throw the majority of web page publishers under a train, just so you can save a second by preloading a page.
Re:How do you really feel? (Score:2, Insightful)
It has nothing to do with faking the user agent and everything to do with using far more bandwidth and resources than is necessary to do a job that will be minimally effective at best. 6% of slashdot's traffic is not an insignificant amount of bandwidth. As another poster put it, they are using a flamethrower to take out a hornet's nest, on the whole bloody internet.
Re:You'd have to fake the user agent (Score:3, Insightful)
Or you could, you know, just probe the site after the user clicks on the link and not before.
Re:How do you really feel? (Score:3, Insightful)
Because the websites that are blocking Opera, or non-IE browsers, or whatever are usually doing it for completely arbitrary reasons as well.
The websites that are blocking AVG are doing it because AVG doesn't obey robots.txt, it seems.
Re:Sure AVG's not slimy... (Score:5, Insightful)
No, its the equivalent of taking a flamethrower to all your neighbor's houses because you think there might be a wasp flying around.
Re:Slimey? (Score:4, Insightful)
These software and others which aren't in this list (The GIMP, Audacity) are in my highly recommended list of software that all computers should have installed by default.
Re:F5 IRule (Score:5, Insightful)
Re:F5 IRule (Score:4, Insightful)
Nice idea, except I simply won't come back to the site, as I suspect many other AVG users would not do. The novices out there will read your message as "Sorry AVG user, your antivirus is abusive and wastes our resources. Disable AVG and come back so we can infect your machine!"
I seldomly see postings so devoid of clue. (Score:3, Insightful)
I don't think I've seen a posting so completely devoid of any intelligence in a long time.
Are users not supposed to protect themselves in the interests of the website?
Sure they should. Nobody has suggested that they should not.
Since AVG is producing something that helps end-users do you really want to be seen as a promoter of the problem?
If they want to help the end-users, they should scan the content before it's given over to the webbrowser - not pre-scan all links.
Since the problem of malware sites is not going to go away and since AVG is effective more antivirus software will start using these techniques. Unless you have something better to suggest?
Why not just do the sane thing? Why not just scan the content as it's being downloaded? Why on earth be a malicious bastard costing people and companies hundreds of millions in extra bandwidth costs?
Frankly, as an end user, I don't give a damn about your costs and stats. I don't care about it for amazon, ebay, myspace, or paypal. I do care that if I follow a link to an unsavory site that I am protected.
Which you can be in any case if the software in question is anything close to sensible. In your arrogance, you've completely forgotten that there might be better ideas on how to do this. Ideas that are even simpler, and that has been implemented in a lot of products for a long, long time.
I suspect that you're either extremely dim, or you work for AVG. This thread is suspiciously full of people defending AVG, without really contributing anything but hyperbole and bullshit. You're one of those "contributors".
Here is another question. Do you want a userbase that is populated by malware infected computers? Is that preferable to figuring out a way to work with AVG new technique?
Work with them!? WORK with them!? If they pick up all the bandwidth-bill-hikes they've caused globally - then sure - I would be willing to work with them. I do suspect that they would go bankrupt if they tried, though.
And why on earth should anyone work with someone who does something as foolish as this? When much simpler, better and easier solutions has existed for a long time?
No, AVG deserves all the blame they can get.
Re:Actually, vigilantism could work. (Score:3, Insightful)
I fully expect to see someone at AVG go to jail, anyway
Hmmm. expecting someone corporate to be held properly accountable for misdeeds these days? Optimistic at best ( although not impossible ).
but hten again I'm a cynic. Maybe I've just been here too long.
Re:F5 IRule (Score:5, Insightful)
Why, what does that do?
Do you want it to pound Google's servers, bump up grisoft's search rank, or anything else I am not seing?
Re:F5 IRule (Score:2, Insightful)
I would think that a decent IT guy would explain at least the basics of this logic to his/her users.
Nobody wants to be told, "because I said so."
Re:F5 IRule (Score:1, Insightful)
*** She was more than happy to have the web a little bit slower if it meant her google results were tested & filtered for her. ***
You can't believe the results of that test. When AVG accesses the site, the site can SEE that it is AVG doing it (because of the brain-dead way in which it works). The site can then feed "good content" to AVG, and stiff you with a virus or malware when you actually click the link to visit the site.
The feature offers you no protection at all.