Slashdot Log In
AVG Fakes User Agent, Floods the Internet
Posted by
CmdrTaco
on Thu Jul 03, 2008 10:18 AM
from the way-to-go-guys dept.
from the way-to-go-guys dept.
Slimy anti-virus provider AVG is spamming the internet with deceptive traffic pretending to be Internet Explorer. Essentially, users of the software automatically pre-crawl search results, which is bad, but they do so with an intentionally generic user agent. This is flooding websites with meaningless traffic (on Slashdot, we're seeing them as like 6% of our page traffic now). Best of all, they change their UA to avoid being filtered by websites who are seeing massive increases in bandwidth from worthless robots.
Related Stories
[+]
Technology: AVG Backs Down From Flooding the Internet 297 comments
Simon Wright writes "As a website that is featured heavily in many Google Australia search results, Whirlpool (Australia's largest technology forum) has been particularly affected by AVG's LinkScanner. We've seen a traffic increase as much as 12 hits per second from these bots. So we've actively and loudly campaigned against this move by AVG, encouraging all users of AVG 8.0 to uninstall the product. The discussion starts here. And AVG's backing down is posted here."
From that URL:"'As promised, I am letting you know that the latest update for AVG Free edition has addressed and rectified the issue that [Whirlpool] have brought to our attention. This update has now been released to users and has also been built into the latest installation package for AVG Free.' — Peter Cameron, Managing Director, AVG Australia."
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
F5 IRule (Score:5, Informative)
For anyone that happens to run a site behind an F5 BigIP, here's a nice little IRule to nuke this horrible crap from orbit.
rule IRULE_block_avg-prefetch { ::avg_useragents [list \
when HTTP_REQUEST {
set
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" \
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)" \
"User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" \
"User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)" \
]
if { ![HTTP::header exists "Accept-Encoding"] } {
if { [matchclass [HTTP::header User-Agent] equals $::avg_useragents] } {
reject
}
}
}
Re:F5 IRule (Score:5, Funny)
Parent
Re:F5 IRule (Score:5, Informative)
Parent
Re:F5 IRule (Score:5, Informative)
For the record, this is a REALLY bad idea.
It will block all traffic from legitimate IE6 users, and if you have a $20K router, you probably don't want to do that.
If you read the links in the article (and some comments further down), there are things you can do to block this, including blocking requests with these UAs that also have odd or missing headers, cookies, etc.
LOL, perhaps you might want to READ the rule before replying - it is NOT blocking all IE6 users, just the ones that are missing "Accept-Encoding" header
-Em
Parent
Re:F5 IRule (Score:5, Informative)
Actually all browsers send the Accept-Encoding HTTP header, which AVG does not.. if you look at the rule you'll see that it checks for the existence of that head and only blocks if it doesn't exist.
if { ![HTTP::header exists "Accept-Encoding"] {
Parent
Re:F5 IRule (Score:5, Insightful)
Parent
Re:F5 IRule (Score:5, Informative)
Sometimes the choice of browser is beyond the user's control.
Parent
How do you really feel? (Score:5, Insightful)
Why don't you tell us how you really feel about AVG?
Re:How do you really feel? (Score:5, Funny)
Ok. It's run by Jews in a secret conspiracy to take over the World using sharks with frickin' lasers and gorgeous fembots with a penchant for evil.
Parent
Re:How do you really feel? (Score:5, Funny)
In this day and age it's sad to see that anti-sharkitism is still alive and well.
AVG = Alotta VaGina?
Parent
I discovered this the hard way (Score:5, Interesting)
A couple months ago, a random article on my company's site got around 20 times the number of hits that the top story of the day should be getting. I checked the logs, and saw legit-looking IE user agents, but they didnt look normal. None of them had any cookies, and none of them were downloading the CSS or image files that they should have been. The IP addresses were from all around the world. WTF?
I found out that Google was doing one of its things where it changes the google logo for some special occasion, and it links to a search. That article was on the first page of the results.
I did a search for the exact user agent and discovered it was AVG. When you go to a Google search, AVG downloads each result looking for malware. Hooray for falsified user agents.
Though, I suspect the reason they use a legit-looking IE user agent is because malware sites could sniff the AVG user agent and serve up an innocent page for them, and malware for everyone else.
Re:I discovered this the hard way (Score:5, Insightful)
I did the same and for the same reasons.
Not sure how this practice justified the poster calling them slimey.
I've been relatively happy with AVG. Perhaps, someone could elaborate on how they are slimey. This appears to be an attempt to protect people.
Parent
Re:I discovered this the hard way (Score:5, Insightful)
Perhaps, someone could elaborate on how they are slimey. This appears to be an attempt to protect people.
Ok, think of the /. effect. Now take that on almost any website who's servers aren't as strong. This is basically a huge DDoS attack on many websites by AVG that has a reason behind it. But it is still a DDoS attack.
Parent
Re:I discovered this the hard way (Score:5, Insightful)
They might be dumb instead of slimy...
Parent
Hanlon's razor with the save! (Score:5, Insightful)
Parent
Re:I discovered this the hard way (Score:5, Insightful)
Dumb is what they were BEFORE they were told about the problem. Slimy is what they are now that they are refusing to rectify the situation and behave.
I think they deserve everything they will inevitably get as a result of this.
Parent
Re:I discovered this the hard way (Score:5, Insightful)
Parent
Re:I discovered this the hard way (Score:5, Informative)
They are attempting to help their customers at the expense of everybody else on the Internet. If I understand the article, they're pre-scanning every possible URL on a page. In essense they're clicking every possible link before you do.
For instance I searched for "avg" on google and counted the number of "href=" appearances on the resulting page. It happened to be an even 100. AVG is visiting ALL of of those HREFs in the background. A user will click on only one.
I would assume their scanner is smart enough to remove duplicates HREFs and do some other smart things. But still, this is a terrible idea. I guess we all have to go buy more servers and bandwidth so the anti-virus people can make a living now?
Parent
Re:I discovered this the hard way (Score:5, Insightful)
Prefetching your search results doesn't protect you from viruses any more than just checking the pages you try to load at the time of loading.
What it does, is basically scanning the entire internet, weighted toward the pages its users search for, and I assume reporting back to AVG which websites have malware or suspected malware on them.
The problem with this theory is that malware sites can move around quickly, so learning that domain xzclqqkxzz.com tried to upload a virus to someone's computer 48 hours ago is not especially valuable information.
That's in addition to AV software being essentially impossible to keep up-to-date anyway, you can look up studies but most AV software lets a lot of malware through.
And the increased traffic annoys webmasters because the prefetches are (attempted to be) disguised as actual page fetches, and they come from all over the internet, so we think they're real clicks from real users but they're not. Plus, for some sites the increased load/bandwidth may be a problem.
Parent
I turned it off (Score:5, Informative)
I use AVG on a couple machines. I didn't really think about the traffic tracking piece of this when I saw it working, I just thought about it slowing me down, increasing bandwidth use, etc. and I turned it off.
I know most people don't mess with defaults - and I'm not defending them as far as the agent thing and all that - but it was easy to do.
On the negative side my avg icon in the systray has a big exclamation over it like something is really wrong - when I know it's just because I turned off a piece of functionality I don't want to use.
Re:I turned it off (Score:5, Informative)
If you are using Firefox, just disable the AVG addon within Firefox addon manager. You won't get the big exclamation mark.
Parent
Re:I turned it off (Score:5, Informative)
There is a solution to the exclamation:
http://grandstreamdreams.blogspot.com/2008/04/taming-avg-free-version-8.html [blogspot.com]
In short, run "avg_free_stf_*.exe /REMOVE_FEATURE fea_AVG_SafeSurf /REMOVE_FEATURE fea_AVG_SafeSearch" from a cmd box or the run box.
Sort of a ridiculous contortion to get to an option that should be more available, but it works.
Parent
Re:I turned it off (Score:5, Informative)
You can install AVG 8 without LinkScanner which returns AVG to it's previous functionality(just anti-virus).
From the FAQ:
If you wish to install AVG 8.0 Free Edition without the LinkScanner component, or uninstall this component from your program, please proceed as follows:
* Download the AVG 8.0 Free Edition installation package from our website. /REMOVE_FEATURE fea_AVG_SafeSurf /REMOVE_FEATURE fea_AVG_SafeSearch. One way to achieve this is to: /REMOVE_FEATURE fea_AVG_SafeSurf /REMOVE_FEATURE fea_AVG_SafeSearch
* Run the installation with the parameters
o save the AVG Free installation file directly to disk C:\
o open menu Start -> Run
o type
c:\avg_free_stf_*.exe
* The installation will be started, and AVG will be installed without the LinkScanner component.
Parent
Insightful ?????? (Score:5, Insightful)
How exactly do the websites getting slammed with this bullshit traffic "not even install this part of the program" and "if you don't like it don't use it"?
Did you miss this part: (on Slashdot, we're seeing them as like 6% of our page traffic now)
So how does Slashdot "just not use" the AVG product and recover that 6% of their page traffic again?
The complaint is that they are "spamming the internet with deceptive traffic". That's a server/hosting complaint, not a user complaint about some user who can't figure out how to disable that feature.
Kudos on getting a "4 Insightful" for a ridiculously inapplicable and nonsensical response though!
Parent
Hooray (Score:5, Funny)
Re:Hooray (Score:5, Insightful)
Hooray! Look at all the OH SHIT my server's on fire!
Parent
ACID (Score:5, Funny)
I bet AVG would score higher on ACID than IE...
Slimey ? (Score:5, Insightful)
if you want the definition of Slimey see Symantec/Mcafee/MicrosoftOneCare
while this doesnt excuse their behaviour, trying to protect people (a lot of them for free) is not Slimey but insulting them on the front page of Slashdot is
patheticTheir eggs are slimy. (Score:5, Insightful)
And if that causes problems for webmasters, Thompson says, so be it. "I don't want to sound flip about this, but if you want to make omelets, you have to break some eggs."
Sounds like a "fuck off" to me.
I guess slimy is in the eye of the beholder, but the attitude reminds me of Claria.
Parent
"as like" (Score:5, Funny)
> on Slashdot, we're seeing them as like 6% of our page traffic now
Come on Taco... proper English (or at least something seemingly like it) isn't that hard... is 6% exactly, around 6% or really just 'like 6%'
I honestly like, do not recall like the last time I like, saw someone use 'like' in that long standing improper way in like text, it's always like, been for me, like only something a person like, verbalizes.
Alternative Anti-Virus Software? (Score:5, Interesting)
So if AVG has turned to the dark side, what free/cheap non-bloatware options are out there worth trusting? I know of a few but it's a little hard to know who to trust.
Seems like every anti-malware software maker these days bloats their software into a 50+MB beast of a package that accomplishes little more than to slow your computer down. I have more trouble with their software than I do with actual mal-ware.
Re:Alternative Anti-Virus Software? (Score:5, Informative)
Avast.
It's not just for Talk-Like-A-Pirate Day any more!
Parent
Nagware alert! (Score:5, Informative)
avast! antivirus Home Edition is FREE to use but it is necessary to register before the end of the initial 60 day trial period. To register, click here. Following registration you will receive by E-mail a license key valid for a period of 1 year. After you have downloaded and installed the program, the license key must be inserted into it within 60 days. The registration process is very easy, and it will take you only a couple of minutes.
Also Avira has been getting more and more annoying over the years, it's practically adware now.
So now it looks like it's either AVG with the browser plugins removed or MoonAV (which is FOSS):
http://www.moonsecure.com/ [moonsecure.com]
(It used to have a problem where you'd need to remove the Windows service manually after uninstalling, they might have fixed it though.)
Parent
Apache Rewrite Rules! (Score:5, Interesting)
Try this on Apache servers:
#Here we assume certain MSIE 6.0 agents are from linkscanner
#redirect these requests back to avg in the hope they'll see their silliness
Rewritecond %{HTTP_USER_AGENT} ".*MSIE 6.0; Windows NT 5.1; SV1.$" [OR]
Rewritecond %{HTTP_USER_AGENT} ".*MSIE 6.0; Windows NT 5.1;1813.$"
RewriteCond %{HTTP_REFERER} ^$
RewriteCond %{HTTP:Accept-Encoding} ^$
RewriteRule ^.* http://www.avg.com/?LinkScannerSucks [R=307,L]
Brought to you by These guys [pixelbeat.org].
My ex wife.... (Score:5, Funny)
Once good (Score:5, Informative)
AVG was once a good product. Then, it got bloated and started eating up kernel memory voraciously. It was impossible to play games with it running in the background, especially Crysis (skip the jokes, my system could handle it maxed once I replaced AVG with Avast!). Now, with this development, I'll be sure to replace AVG with Avast! on all of my machines, not just my gaming one.
HOWTO install AVG without Search Crawling (Score:5, Informative)
You can actually install AVG 8 without the 'Safe Search' feature that crawls websites (it's essentially a BHO/Firefox extension). Even if you already have AVG 8, you can uninstall it and reinstall:
At a Command Prompt window, type /REMOVE_FEATURE fea_AVG_SafeSurf /REMOVE_FEATURE fea_AVG_SafeSearch
c:\downloads\avg_free_stf_xxxxxxxxxx.exe
where c:\downloads\avg_free_stf_xxxxxxxxxx.exe is the full path of your AVG 8 installer.
Re:HOWTO install AVG without Search Crawling (Score:5, Funny)
At a Command Prompt window, type /REMOVE_FEATURE fea_AVG_SafeSurf /REMOVE_FEATURE fea_AVG_SafeSearch
c:\downloads\avg_free_stf_xxxxxxxxxx.exe
where c:\downloads\avg_free_stf_xxxxxxxxxx.exe is the full path of your AVG 8 installer.
At least it is intuitive....
-Em
Parent
AVG 8 is dog slow (Score:5, Informative)
Grisoft dropped the ball with AVG v8.0 (Score:5, Informative)
I'm a longtime user of AVG. Version 7 was reasonably lightweight, effective and (most importantly to me) unobtrusive.
Unfortunately, version 8 is a different story. After Grisoft forced me to upgrade in May, suddenly AVG became a nagging resource hog. Nightly scan times rocketed from about an hour to over six hours - a scheduled scan that started at 2am would still be going at 8:30am. I have been able to reduce this time somewhat by changing the scan settings (e.g., don't scan inside compressed archives), but it's still slow.
Most annoyingly, their new "LinkScanner" and "SafeSurf" features slowed my browser to a crawl. I didn't want these, since I already use FireFox with the AdBlock and NoScript extensions. I tried to simply disable LinkScanner, but then AVG constantly bothered me with nagging warnings that my computer "was not fully protected". After a little digging, I found that it was possible to uninstall the feature entirely with the following command:
avg_free_stf_xxxx.exe /REMOVE_FEATURE fea_AVG_SafeSurf /REMOVE_FEATURE fea_AVG_SafeSearch
(Substitute "avg_free_stf_xxxx.exe" in the above command with the name of your setup file.)
This improved my browser performance, and eliminated the warnings.
I'm still (grudgingly) using AVG, but I will switch if/when I find a better alternative.
are you sure it's avg_free_stf_xxxx.exe (Score:5, Funny)
Shouldn't it be avg_free_stfu_xxxx.exe ??
Parent
You'd have to fake the user agent (Score:5, Insightful)
When probing for sites that serve malware, wouldn't you have to make the probe look identical to a legitimate user?
Otherwise the malicious site could just serve innocuous content to the probe and malware to everyone else.
I'm going to agree with the slimy assessment (Score:5, Informative)
Slimy? (Score:5, Insightful)
I think I missed the memo - why is AVG a "Slimy anti-virus provider"? That portion of the summary BEGS for supporting links...
What about advertising? (Score:5, Insightful)
Could AVG be unintentionally committing massive click fraud?
It runs in Firefox as well (Score:5, Informative)
LinkScanner, the component they're talking about, works in Firefox as well - so no, using Firefox does not 'keep you safe'.
Nor is this about the users of the thing in the first place - either they like its functionality (security theatre-advance warning blabla) and leave it on, or they don't and they switch it off.
This is about the poor, poor admins who are suddenly seeing bogus traffic and omgosh it's spoofing user agents at that! .. repeatedly*
*changes his user agent to 'cry more, Taco' in FF and hits F5
Parent
Re:Sending the bills to them. (Score:5, Insightful)
no your not a lawyer, but i'm pretty sure your not smart enough to be one either.
you didn't give them permission to access your publicly available site?
really?
are you sure?
because you know, if you make something publicly available on the public internet, I'm pretty sure by definition, you've therefore given them permission to access it.
Just like everyone else "in the public".
Did you give Google permission?
how about every other search/index site?
as to the "extra bandwidth" since it is by definition, caused by your websites being found via search providers, maybe you should be sending the bill for linking to them and thus causing the "extra bandwidth" to Google/Yahoo/MS and see how far that gets you.
Parent
Re:One Word (Score:5, Informative)
I mention this because there is a windows client that uses the same FOSS engine -- ClamWin [clamwin.com].
Parent
Re:payback (Score:5, Informative)
Parent
Re:payback (Score:5, Funny)
so we don't know what readers are actually interested in
Porn. Anime. Sometimes computers.
Hope that alleviates your concerns.
Parent