Forgot your password?
typodupeerror
Security

AVG Fakes User Agent, Floods the Internet 928

Posted by CmdrTaco
from the way-to-go-guys dept.
Slimy anti-virus provider AVG is spamming the internet with deceptive traffic pretending to be Internet Explorer. Essentially, users of the software automatically pre-crawl search results, which is bad, but they do so with an intentionally generic user agent. This is flooding websites with meaningless traffic (on Slashdot, we're seeing them as like 6% of our page traffic now). Best of all, they change their UA to avoid being filtered by websites who are seeing massive increases in bandwidth from worthless robots.
This discussion has been archived. No new comments can be posted.

AVG Fakes User Agent, Floods the Internet

Comments Filter:
  • F5 IRule (Score:5, Informative)

    by Precision (1410) * on Thursday July 03, 2008 @11:19AM (#24044623) Homepage

    For anyone that happens to run a site behind an F5 BigIP, here's a nice little IRule to nuke this horrible crap from orbit.

    rule IRULE_block_avg-prefetch {
          when HTTP_REQUEST {
            set ::avg_useragents [list \
                    "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" \
                    "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)" \
                    "User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" \
                    "User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)" \
            ]

            if { ![HTTP::header exists "Accept-Encoding"] } {
                    if { [matchclass [HTTP::header User-Agent] equals $::avg_useragents] } {
                            reject
                    }
            }
    }

    • Re:F5 IRule (Score:5, Funny)

      by rvw (755107) on Thursday July 03, 2008 @11:25AM (#24044771)
      Another suggestion I read somewhere else is to redirect all traffic to the AVG website. That will teach them!
      • Re:F5 IRule (Score:5, Informative)

        by afidel (530433) on Thursday July 03, 2008 @11:44AM (#24045209)
        I think someone did since free.grisoft.com has been down all day today! My AVG is complaining about not being able to get it's updates. Oh and the plugin REALLY freaking slows down FF on Google results so I turned the damn thing off. I guess I know why now!
      • Re:F5 IRule (Score:4, Insightful)

        by Anonymous Coward on Thursday July 03, 2008 @12:25PM (#24045957)

        Actually this is quite close a real solution :) what AVG should have done is cache the scan results from each page. Thus if a user tries to access a page it should first query AVG for a result. ( the result here is- OK page or not OK to visit page)

        If a result exists in cache, no need to scrape the page. If there is no result in the cache, both AVG server and Client (to avoid trust issues) should query and compare results. The cache should periodically refresh and use multiple different UAs to avoid gaming. Quite a nice solution if you ask me ;) I knew I should have take up consulting instead of this damn Ph.D..

        Also AVG are not slimly, the spyware/trojan/malware site operators are. Not to mention Norton/Symantec/Kaspersky et al.. The feature can easily be turned off and its purpose is to help the user at no $ cost. Besides, which self respecting /.-er needs anti virus [xkcd.com]

      • Re:F5 IRule (Score:5, Interesting)

        by Stellian (673475) on Thursday July 03, 2008 @01:12PM (#24046847)

        Another suggestion I read somewhere else is to redirect all traffic to the AVG website

        Instead of punishing the site, you could punish the users of this crappy code. Make an invisible href somewhere in you page, that triggers a script that does a temporary IP-ban. Since AVG will follow any href, when the user tries to access the site, he gets the message:
        Sorry AVG user, your antivirus is abusive and wastes our resources. Disable AVG and come back.

        If a few important sites do this AVG's user-base will drop in a week to about 100 people.

        • Re:F5 IRule (Score:5, Informative)

          by ArhcAngel (247594) on Thursday July 03, 2008 @03:01PM (#24048969)

          you could punish the users of this crappy code.

          The users of this crappy code are almost certainly happily unaware of any problem they may be causing. I have used and recommended AVG for a number of years to people I have had to reinstall Windows due to the amount of true crapware they are infected with. I upgraded to version 8 a couple of months ago and wasn't even aware of the feature until I pulled up a google search and noticed the little green check marks. I quickly located and disabled the feature because it slowed my browsing down but I could see how someone could see this as a valuable tool. You want to punish someone for using a tool that will most likely prevent them from becoming part of a botnet yet again because the tool maker has added a good feature in theory that has a negative side effect. Doesn't most medication have a long list of possible undesirable side effects? So which is worse, a horde of zombie computers controlled by malicious hackers or a bunch of unknowing PC users who's AV software pre-checks the web site they are thinking about going to and telling them whether it is safe or not? I know which I'd rather be if I were technically challenged.

          Sorry AVG user, your antivirus is abusive and wastes our resources. Disable AVG and come back.

          Actually all you need to do is uninstall [blogspot.com] the link scanner feature.

          • Re:F5 IRule (Score:5, Interesting)

            by westyvw (653833) on Thursday July 03, 2008 @05:27PM (#24051119)
            Once again: Why stop at dealing with AVG? Get rid of the whole mess. Every time I move some one from Windows to Linux the "what shall I do about spyware/adware/printer/windowsupdate" questions just go away. I used to recommend AVG about 4 years ago. Since then, I just recommend an OS without a need for antivirus software.
      • Re:F5 IRule (Score:5, Informative)

        by sjames (1099) on Thursday July 03, 2008 @03:23PM (#24049339) Homepage

        I liked the suggestion on the reader comments to add <iframe src="http://www.google.com/search?num=100&q=site:grisoft.com" width="1" height="1"></iframe> to your pages.

  • One Word (Score:4, Informative)

    by Spazztastic (814296) <spazztastic AT gmail DOT com> on Thursday July 03, 2008 @11:20AM (#24044637)

    Avira.

  • by SoupGuru (723634) on Thursday July 03, 2008 @11:20AM (#24044651)

    Why don't you tell us how you really feel about AVG?

  • by brunascle (994197) * on Thursday July 03, 2008 @11:20AM (#24044653)

    A couple months ago, a random article on my company's site got around 20 times the number of hits that the top story of the day should be getting. I checked the logs, and saw legit-looking IE user agents, but they didnt look normal. None of them had any cookies, and none of them were downloading the CSS or image files that they should have been. The IP addresses were from all around the world. WTF?

    I found out that Google was doing one of its things where it changes the google logo for some special occasion, and it links to a search. That article was on the first page of the results.

    I did a search for the exact user agent and discovered it was AVG. When you go to a Google search, AVG downloads each result looking for malware. Hooray for falsified user agents.

    Though, I suspect the reason they use a legit-looking IE user agent is because malware sites could sniff the AVG user agent and serve up an innocent page for them, and malware for everyone else.

    • by jsailor (255868) on Thursday July 03, 2008 @11:27AM (#24044795)

      I did the same and for the same reasons.
      Not sure how this practice justified the poster calling them slimey.
      I've been relatively happy with AVG. Perhaps, someone could elaborate on how they are slimey. This appears to be an attempt to protect people.

      • by Darkness404 (1287218) on Thursday July 03, 2008 @11:30AM (#24044877)

        Perhaps, someone could elaborate on how they are slimey. This appears to be an attempt to protect people.

        Ok, think of the /. effect. Now take that on almost any website who's servers aren't as strong. This is basically a huge DDoS attack on many websites by AVG that has a reason behind it. But it is still a DDoS attack.

      • by InlawBiker (1124825) on Thursday July 03, 2008 @11:54AM (#24045401)

        They are attempting to help their customers at the expense of everybody else on the Internet. If I understand the article, they're pre-scanning every possible URL on a page. In essense they're clicking every possible link before you do.

        For instance I searched for "avg" on google and counted the number of "href=" appearances on the resulting page. It happened to be an even 100. AVG is visiting ALL of of those HREFs in the background. A user will click on only one.

        I would assume their scanner is smart enough to remove duplicates HREFs and do some other smart things. But still, this is a terrible idea. I guess we all have to go buy more servers and bandwidth so the anti-virus people can make a living now?

        • by hudsucker (676767) on Thursday July 03, 2008 @12:57PM (#24046611)
          Let's say that your Google search returns some links that are NSFW, or could be considered illegal to view. As a far as anyone looking at server logs is concerned, you are choosing to view those links.

          How long before someone gets fired or arrested, and tries to explain that it was their anti-virus software that was viewing the child pr0n?

      • by jamie (78724) * Works for Slashdot <jamie@slashdot.org> on Thursday July 03, 2008 @12:02PM (#24045529) Journal

        Prefetching your search results doesn't protect you from viruses any more than just checking the pages you try to load at the time of loading.

        What it does, is basically scanning the entire internet, weighted toward the pages its users search for, and I assume reporting back to AVG which websites have malware or suspected malware on them.

        The problem with this theory is that malware sites can move around quickly, so learning that domain xzclqqkxzz.com tried to upload a virus to someone's computer 48 hours ago is not especially valuable information.

        That's in addition to AV software being essentially impossible to keep up-to-date anyway, you can look up studies but most AV software lets a lot of malware through.

        And the increased traffic annoys webmasters because the prefetches are (attempted to be) disguised as actual page fetches, and they come from all over the internet, so we think they're real clicks from real users but they're not. Plus, for some sites the increased load/bandwidth may be a problem.

      • by sm62704 (957197) on Thursday July 03, 2008 @12:25PM (#24045961) Journal

        All AV software compaies are slimey, because AV software gives you a false sense of security. It can't detect any malware that isn't in its database, and it can't stop a luser from running a trojan. But the luser doesn't know this, and thinks it's safe to click on any damned thing.

        If your OS "needs" AV, your OS, IMO, sucks badly.

  • I turned it off (Score:5, Informative)

    by stoolpigeon (454276) * <bittercode@gmail> on Thursday July 03, 2008 @11:21AM (#24044669) Homepage Journal

    I use AVG on a couple machines. I didn't really think about the traffic tracking piece of this when I saw it working, I just thought about it slowing me down, increasing bandwidth use, etc. and I turned it off.

    I know most people don't mess with defaults - and I'm not defending them as far as the agent thing and all that - but it was easy to do.

    On the negative side my avg icon in the systray has a big exclamation over it like something is really wrong - when I know it's just because I turned off a piece of functionality I don't want to use.

    • Re:I turned it off (Score:5, Informative)

      by funfail (970288) on Thursday July 03, 2008 @11:27AM (#24044803) Homepage

      If you are using Firefox, just disable the AVG addon within Firefox addon manager. You won't get the big exclamation mark.

    • Re:I turned it off (Score:5, Informative)

      by maxume (22995) on Thursday July 03, 2008 @11:27AM (#24044805)

      There is a solution to the exclamation:

      http://grandstreamdreams.blogspot.com/2008/04/taming-avg-free-version-8.html [blogspot.com]

      In short, run "avg_free_stf_*.exe /REMOVE_FEATURE fea_AVG_SafeSurf /REMOVE_FEATURE fea_AVG_SafeSearch" from a cmd box or the run box.

      Sort of a ridiculous contortion to get to an option that should be more available, but it works.

    • Re:I turned it off (Score:5, Informative)

      by thundercleese (656445) on Thursday July 03, 2008 @11:30AM (#24044881)

      You can install AVG 8 without LinkScanner which returns AVG to it's previous functionality(just anti-virus).

      From the FAQ:

      If you wish to install AVG 8.0 Free Edition without the LinkScanner component, or uninstall this component from your program, please proceed as follows:

              * Download the AVG 8.0 Free Edition installation package from our website.
              * Run the installation with the parameters /REMOVE_FEATURE fea_AVG_SafeSurf /REMOVE_FEATURE fea_AVG_SafeSearch. One way to achieve this is to:
                          o save the AVG Free installation file directly to disk C:\
                          o open menu Start -> Run
                          o type
                              c:\avg_free_stf_*.exe /REMOVE_FEATURE fea_AVG_SafeSurf /REMOVE_FEATURE fea_AVG_SafeSearch
              * The installation will be started, and AVG will be installed without the LinkScanner component.

  • Hooray (Score:5, Funny)

    by genner (694963) on Thursday July 03, 2008 @11:21AM (#24044673)
    Hooray look at all the hits I'm getting.
  • ACID (Score:5, Funny)

    by Anonymous Coward on Thursday July 03, 2008 @11:21AM (#24044681)

    I bet AVG would score higher on ACID than IE...

  • Slimey ? (Score:5, Insightful)

    by Anonymous Coward on Thursday July 03, 2008 @11:23AM (#24044707)
    please, providing millions of people with an anti-virus for free is not exactly "slimey"
    if you want the definition of Slimey see Symantec/Mcafee/MicrosoftOneCare

    while this doesnt excuse their behaviour, trying to protect people (a lot of them for free) is not Slimey but insulting them on the front page of Slashdot is

    pathetic

  • "as like" (Score:5, Funny)

    by DaHat (247651) on Thursday July 03, 2008 @11:23AM (#24044721) Homepage

    > on Slashdot, we're seeing them as like 6% of our page traffic now

    Come on Taco... proper English (or at least something seemingly like it) isn't that hard... is 6% exactly, around 6% or really just 'like 6%'

    I honestly like, do not recall like the last time I like, saw someone use 'like' in that long standing improper way in like text, it's always like, been for me, like only something a person like, verbalizes.

  • by sjbe (173966) on Thursday July 03, 2008 @11:23AM (#24044725)

    So if AVG has turned to the dark side, what free/cheap non-bloatware options are out there worth trusting? I know of a few but it's a little hard to know who to trust.

    Seems like every anti-malware software maker these days bloats their software into a 50+MB beast of a package that accomplishes little more than to slow your computer down. I have more trouble with their software than I do with actual mal-ware.

    • by LMacG (118321) on Thursday July 03, 2008 @11:26AM (#24044791) Journal

      Avast.

      It's not just for Talk-Like-A-Pirate Day any more!

      • Nagware alert! (Score:5, Informative)

        by GameboyRMH (1153867) <gameboyrmhNO@SPAMgmail.com> on Thursday July 03, 2008 @11:44AM (#24045185) Journal

        avast! antivirus Home Edition is FREE to use but it is necessary to register before the end of the initial 60 day trial period. To register, click here. Following registration you will receive by E-mail a license key valid for a period of 1 year. After you have downloaded and installed the program, the license key must be inserted into it within 60 days. The registration process is very easy, and it will take you only a couple of minutes.

        Also Avira has been getting more and more annoying over the years, it's practically adware now.

        So now it looks like it's either AVG with the browser plugins removed or MoonAV (which is FOSS):

        http://www.moonsecure.com/ [moonsecure.com]

        (It used to have a problem where you'd need to remove the Windows service manually after uninstalling, they might have fixed it though.)

      • by mapsjanhere (1130359) on Thursday July 03, 2008 @11:46AM (#24045259)
        I second Avast, it's free for home use, and has very reasonable commercial license terms. Plus it gives you one code for all machines, no need to chase 20 different keys like you do with Norton etc. And the key is good for the whole license period; before I used to loose at least 10 % of licenses to crashes or borked installs, and getting new ones from Norton was like pulling wisdom teeth on a grouchy alligator.
  • by Anonymous Coward on Thursday July 03, 2008 @11:24AM (#24044743)

    Try this on Apache servers:

    #Here we assume certain MSIE 6.0 agents are from linkscanner
    #redirect these requests back to avg in the hope they'll see their silliness
    Rewritecond %{HTTP_USER_AGENT} ".*MSIE 6.0; Windows NT 5.1; SV1.$" [OR]
    Rewritecond %{HTTP_USER_AGENT} ".*MSIE 6.0; Windows NT 5.1;1813.$"
    RewriteCond %{HTTP_REFERER} ^$
    RewriteCond %{HTTP:Accept-Encoding} ^$
    RewriteRule ^.* http://www.avg.com/?LinkScannerSucks [R=307,L]

    Brought to you by These guys [pixelbeat.org].

    • by pixelbeat (31557) <P@draigBrady.com> on Thursday July 03, 2008 @12:27PM (#24046001) Homepage

      Just to comment that this has been working flawlessly for me and others for days.
      In addition to much reduced load, AVG will be getting the combined load with an appropriate message in their logs.

      Note it's quite safe for valid IE 6.0 users as it checks for very specific user agent strings that most IE 6.0 users don't in fact have.
      In addition the referrer must be blank and the Accept-encoding header must be missing.

      Also I'm using a 307 redirect so so that potentially non linkscanner clients will keep checking the latest rules.
      This also allows you to change the redirect destination without worrying about cached old redirects.

    • by Anonymous Coward on Thursday July 03, 2008 @01:11PM (#24046825)

      I have an updated version of this redirect to AVG, based on info I've been gathering over the last 2 weeks from Webmaster World, El Reg, and of course Pixelbeat. Here is the rule set I am using now:

      RewriteEngine on
      RewriteCond %{HTTP_USER_AGENT} ".*MSIE 6.0; Windows NT 5.1; SV1\)$" [OR]
      RewriteCond %{HTTP_USER_AGENT} ".*MSIE 6.0; Windows NT 5.1;1813\)$"
      RewriteCond %{REQUEST_METHOD} ^GET$
      RewriteCond %{HTTP_REFERER} ^$
      RewriteCond %{HTTP:Accept-Encoding} ^$
      RewriteCond %{HTTP:Accept-Language} ^$
      RewriteCond %{HTTP:Accept-Charset} ^$
      RewriteRule ^.* http://www.avg.com/?LinkScannerSucks [R=301,L]

      I have the check for "GET" method in there so that the earlier "User-Agent: ..." version of linkscanner will still get redirected. See, that version does a HEAD request first, most likely to check for a redirect. So we allow that HEAD request to pass, since it is small any ways. But the GET request that follows will still get redirected. We want to redirect the maximum amount of traffic we can to AVG, to drive the point home.

      This filter is also more selective, by also checking for the non-existance of Accept-Language and Accpet-Charset we make absolutely sure we are not redirecting a valid user. No web browser out there would fail to set all 3 of these, so we can be absolutely sure this is crap coming from a linkscanner.

      I also decided to use a permanent redirect, in hopes that linkscanner caches this and it will reduce the number of repeat hits from the same user? Not sure if that is the case or not.

      Someone in this thread asked if these rules work in the main Apache config file instead od using .htaccess. I don't use .htaccess on my servers either, and these rules reside in our main Apache config file. So the answer is yes, it will work in BOTH places.

      I hope by now that AVG realizes the futility in their continuing to change how linkscanner acts to try and hide it from us. We will simply continue to work together as a community of server admins to block this crap and send it right back at them!

  • by WwWonka (545303) on Thursday July 03, 2008 @11:25AM (#24044753)
    ....used to fake user agents all the time. As a man I thought I was always properly connecting to her internet portal. guess not.
  • Once good (Score:5, Informative)

    by Rinisari (521266) on Thursday July 03, 2008 @11:25AM (#24044757) Homepage Journal

    AVG was once a good product. Then, it got bloated and started eating up kernel memory voraciously. It was impossible to play games with it running in the background, especially Crysis (skip the jokes, my system could handle it maxed once I replaced AVG with Avast!). Now, with this development, I'll be sure to replace AVG with Avast! on all of my machines, not just my gaming one.

  • On the Up and UP. (Score:4, Insightful)

    by Anonymous Coward on Thursday July 03, 2008 @11:26AM (#24044773)

    Smiley anti-virus provider? The integrity of Slashdot submissions just keeps going up and up! Nice example Taco.

  • Slimey? (Score:4, Insightful)

    by Flaystus (887453) on Thursday July 03, 2008 @11:29AM (#24044859)
    Is many years I've never heard AVG referred to as "Slimey" I don't think the toolbar is a good idea either but... slimey? AVG is awesome.
  • by bheer (633842) <`rbheer' `at' `gmail.com'> on Thursday July 03, 2008 @11:34AM (#24044961)

    You can actually install AVG 8 without the 'Safe Search' feature that crawls websites (it's essentially a BHO/Firefox extension). Even if you already have AVG 8, you can uninstall it and reinstall:

    At a Command Prompt window, type
    c:\downloads\avg_free_stf_xxxxxxxxxx.exe /REMOVE_FEATURE fea_AVG_SafeSurf /REMOVE_FEATURE fea_AVG_SafeSearch

    where c:\downloads\avg_free_stf_xxxxxxxxxx.exe is the full path of your AVG 8 installer.

  • AVG 8 is dog slow (Score:5, Informative)

    by street struttin' (1249972) on Thursday July 03, 2008 @11:35AM (#24044975)
    Has anyone else noticed that AVG 8 is also DOG SLOW on their PC? My computer is from 2001 and ran fine with 7.5, but 8.0 is unusably slow. Every time an application is opened it takes forever for AVG to scan it and let the app open. This combined with this linkscanner bullcrap has caused me to switch. I doubt I'll ever go back.
  • by GogglesPisano (199483) on Thursday July 03, 2008 @11:41AM (#24045121)

    I'm a longtime user of AVG. Version 7 was reasonably lightweight, effective and (most importantly to me) unobtrusive.

    Unfortunately, version 8 is a different story. After Grisoft forced me to upgrade in May, suddenly AVG became a nagging resource hog. Nightly scan times rocketed from about an hour to over six hours - a scheduled scan that started at 2am would still be going at 8:30am. I have been able to reduce this time somewhat by changing the scan settings (e.g., don't scan inside compressed archives), but it's still slow.

    Most annoyingly, their new "LinkScanner" and "SafeSurf" features slowed my browser to a crawl. I didn't want these, since I already use FireFox with the AdBlock and NoScript extensions. I tried to simply disable LinkScanner, but then AVG constantly bothered me with nagging warnings that my computer "was not fully protected". After a little digging, I found that it was possible to uninstall the feature entirely with the following command:

    avg_free_stf_xxxx.exe /REMOVE_FEATURE fea_AVG_SafeSurf /REMOVE_FEATURE fea_AVG_SafeSearch

    (Substitute "avg_free_stf_xxxx.exe" in the above command with the name of your setup file.)

    This improved my browser performance, and eliminated the warnings.

    I'm still (grudgingly) using AVG, but I will switch if/when I find a better alternative.

  • by StrawberryFrog (67065) on Thursday July 03, 2008 @11:43AM (#24045155) Homepage Journal

    When probing for sites that serve malware, wouldn't you have to make the probe look identical to a legitimate user?

    Otherwise the malicious site could just serve innocuous content to the probe and malware to everyone else.

  • by WarmBoota (675361) on Thursday July 03, 2008 @11:51AM (#24045345) Homepage
    I installed AVG on my mother-in-law's machine because she had an expired trial version of some other AV software. It was great for a while, but they must've had a change in direction/managment. Because all of a sudden they started with popups to get a full paid version of the software - even uninstalling the product didn't fix it. I had to surgically extract crap from the registry and program files folder to finally get rid of it. Avast or ClamWin for me - no more AVG.
  • Slimy? (Score:5, Insightful)

    by Atraxen (790188) on Thursday July 03, 2008 @11:52AM (#24045365)

    I think I missed the memo - why is AVG a "Slimy anti-virus provider"? That portion of the summary BEGS for supporting links...

  • by Panaqqa (927615) * on Thursday July 03, 2008 @12:23PM (#24045913) Homepage
    I wonder if this AVG behaviour of doing prefetch on linked sites is driving up advertising clicks at all?

    Could AVG be unintentionally committing massive click fraud?

"Someone's been mean to you! Tell me who it is, so I can punch him tastefully." -- Ralph Bakshi's Mighty Mouse

Working...