Stories
Slash Boxes
Comments
typodupeerror delete not in

Hot Comments

Comments: 203 +-   Attack On a Significant Flaw In Apache Released on Friday June 19 2009, @09:17AM

Posted by kdawson on Friday June 19 2009, @09:17AM
from the take-it-slow dept.
security
internet
Zerimar points out a significant flaw in Apache that can lead to a fairly trivial DoS attack is in the wild. Apache 1.x, 2.x, dhttpd, GoAhead WebServer, and Squid are confirmed vulnerable, while IIS6.0, IIS7.0, and lighttpd are confirmed not vulnerable. As of this writing, Apache Foundation does not have a patch available. From Rsnake's introduction to the attack tool: "In considering the ramifications of a slow denial of service attack against particular services, rather than flooding networks, a concept emerged that would allow a single machine to take down another machine's web server with minimal bandwidth and side effects on unrelated services and ports. The ideal situation for many denial of service attacks is where all other services remain intact but the webserver itself is completely inaccessible. Slowloris was born from this concept, and is therefore relatively very stealthy compared to most flooding tools."
story

Related Stories

This discussion has been archived. No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More
Loading... please wait.
  • by santax (1541065) on Friday June 19 2009, @09:22AM (#28389621)
    be prepared to feel the slashdot-effect yourself for once!
  • by Anonymous Coward
    Opera Unite?
  • by Rogerborg (306625) on Friday June 19 2009, @09:28AM (#28389703) Homepage

    It's just holding sockets open; that's the "Hello, world!" of DOS attacks.

    I'm finding it hard to believe that Apache is genuinely vulnerable to this. Did nobody see it coming? For real?

    • Re: (Score:3, Informative)

      No, it's not. It's holding an HTTP session open. That is not the same thing as a TCP socket.

    • Re: (Score:3, Interesting)

      Yes, I agree. I've seen a handful of attacks like this over the years. Maybe not exactly this one, but Apache has been vulnerable to this for years and I thought all webservers were. And I thought people just knew about it already. This one is a tough one to fix, its not like Apache can just patch something. It sounds like an architectural change is needed.

  • Why not IIS? (Score:4, Interesting)

    by MBCook (132727) <foobarsoft@foobarsoft.com> on Friday June 19 2009, @09:29AM (#28389711) Homepage

    Why isn't IIS vulnerable? Does it just assume the headers are done after some amount of time? Does it have a limit to the number of headers it accepts?

    Can this even be fixed without technically breaking the protocol (since it sounds like what's going on is correct behavior, theoretically)?

    • Re: (Score:3, Interesting)

      What I'm thinking, is this basically another time where those not vulnerable were actually not respecting the spec?
      • Re:Why not IIS? (Score:5, Insightful)

        by Malc (1751) on Friday June 19 2009, @10:08AM (#28390307)

        Does the HTTP spec say anything about the server application timing out the connection? Seems like reasonable behaviour to me. I would be surprised if this isn't a configurable option in Apache too.

        People love to hate it, but IIS has matured in to a very good web server. It's my choice over Apache.

    • by Opportunist (166417) on Friday June 19 2009, @09:51AM (#28390041)

      If the vulnerability is based on correct, standard conform behaviour of the server, I can see why IIS isn't susceptible to it.

    • Re:Why not IIS? (Score:5, Informative)

      by Amouth (879122) on Friday June 19 2009, @10:03AM (#28390209)

      unless you are using Session()'s in asp in IIS then one thread in IIS handles multiple connections.

      what this is doing is opening a connection (getting a thread to work it) and holding it open (keeping the thread busy) and just keep asking for new ones.

      it is very common (always i think) for Apache and allot of web servers to have a max thread's so that the site under heavy traffic doesn't open more connections than it can handle.

      where IIS also has a worker thread limit - there is no limit *(you can set one - but not on by default) on how many concurrent connections can be managed by a thread (and new incoming connections are passed to the thread with the lowest current work load - not always the one with less connections)..

      if you do what they are doing here i can see IIS behavior would be to slowly pile all these slow - no work connections into one thread and the others would happily go about doing actual work..

      where apache would slowly lose access to workable threads as this keeps them busy.

      this isn't an exploit on the http or tcp protocol - it is an exploit based on the behavior of the web server based on it's best practices for managing it.

              • Re:Why not IIS? (Score:4, Informative)

                by raju1kabir (251972) on Friday June 19 2009, @11:34AM (#28391487) Homepage

                if you have it set to "any avaliable ip" and the box has 2 ip's your client may request data on IPA:80 and get a reply from IPB:port

                If a client sends a SYN to 10.1.1.1:80 and gets an SYN-ACK from 10.5.5.5:80 then the client will not associate the two as being related, and will keep waiting for a response from 10.1.1.1:80 until timing out.

                You would need to have some sort of DNS arrangement that encouraged clients to make their requests to your various IPs. You can't just respond from a different IP than the client contacted.

    • by Bemopolis (698691) on Friday June 19 2009, @11:09AM (#28391121)

      Why isn't IIS vulnerable

      My guess is that the DoS attack is so slow that, by the time it would have completed, the server has already crashed for a different reason.

  • Boring (Score:5, Insightful)

    by Anonymous Coward on Friday June 19 2009, @09:29AM (#28389719)
    Talk about a boring exploit: no chance for expanding the attack into anything other than a DOS, and if it becomes widespread enough, fairly trivial to fix... (just kill the oldest waiting client that does not have a full header when the last client is taken.) I'd be embarrassed to publish something like this....
  • iptables helps (Score:5, Informative)

    by samjam (256347) on Friday June 19 2009, @09:30AM (#28389733) Homepage Journal

    You can have perlbal or any reverse proxy on the same machine but listening on a different port and then use iptables to redirect like this

    # iptables -t nat -A -PREROUTING -d ! 127.0.0.1 -p tcp -m tcp --dport 8080 -j REDIRECT --to-ports 80

    and then you don't need to change your apache configuration - and having apache listen on a different port to what users see can break some scripted sites if they read the port number from the apache config.

  • by Z00L00K (682162) on Friday June 19 2009, @09:30AM (#28389737) Homepage

    And the only resolution right now that I can see is to have a connection timeout.

    At least the problem is a denial of service problem and not a problem with intrusion so the damage is easily rectified - restart the web server. Not that you really want to restart it.

    And I suspect that other services can be vulnerable to this type of attack too, not only web servers.

    • the damage is easily rectified - restart the web server

      And you get raped again as soon as it comes back up. How does restarting it help?

    • Re: (Score:3, Interesting)

      A connection timeout should be fine. Just start the clock upon accept(). Give the client a generous but limited amount of time to send headers. If the timer expires before the empty line is received, close the connection.

      Bonus points for not getting the thread pool involved until the header is complete.

      Extra credit for a config option to send a flood of junk to the client and THEN close the socket. That could make attackers considerably more visible to their upstream provider.

      • Re: (Score:3, Interesting)

        Nothing like replyoing to yourself.

        Double extra credit if the junk you send back looks enough like downloading music that the RIAA accidentally joins the forces of good and comes down on the attacker due to ISP snooping but not enough like downloaded music to get you actually busted.

  • Could you potentially get around this if you're proxying to another web server, say lighttpd or Mongrel, or will this just blanketly affected Apache if you have it in front? I'm gathering the latter from the article:

    At the moment I'm not sure what can be done in Apache's configuration to prevent this attack - increasing MaxClients will just increase requirements for the attacker as well but will not protect the server completely. One of our readers, Tomasz Miklas said that he was able to prevent the attack

    • Having read this more this just strikes me as incredibly stupid. Did they publish this? Surely we're just talking about a timeout implementation here where the web server will say "Ahhhh, well you didn't complete that header, bye, bye"?
    • Re: (Score:2, Informative)

      I just tried it against nginx 0.6.37. The attack appears to work there as well.
  • Possible work-around (Score:4, Interesting)

    by Norsefire (1494323) * on Friday June 19 2009, @09:35AM (#28389809) Journal
    From the source:

    if ( $delay < 166 ) {
    print <<EOSUCKS2BU;
    Since the timeout ended up being so small ($delay seconds) and it generally
    takes between 200-500 threads for most servers and assuming any latency at
    all... you might have trouble using Slowloris against this target. You can
    tweak the -tcpto flag down to 1 second but it still may not build the sockets
    in time.
    EOSUCKS2BU
    }

    Lower Apache's timeout to below 166 seconds.

  • by possible (123857) on Friday June 19 2009, @09:39AM (#28389877)

    OpenBSD's pf [openbsd.org] firewall has some options that can help mitigate the "single attacker, single source IP" version of this attack. Of course if the attackers decide to spread the attack out over multiple source IPs like a DDoS, this becomes much harder to deal with until Apache has a patch.

    Filter rules that create state entries can specify various options to control the behavior of the resulting state entry. The following options are available:

    max number
    Limit the maximum number of state entries the rule can create to
    number.
    If the maximum is reached, packets that would normally create state
    fail to match this rule until the number of existing states decreases
    below the limit.
    no state
    Prevents the rule from automatically creating a state entry.
    source-track
    This option enables the tracking of number of states created per
    source IP address.

    The total number of source IP addresses tracked globally can be
    controlled via the

    src-nodes runtime option [slashdot.org].

    max-src-nodes number
    When the source-track option is used,
    max-src-nodes will limit the number of source IP addresses that
    can simultaneously create state.
    This option can only be used with source-track rule.
    max-src-states number
    When the source-track option is used,
    max-src-states will limit the number of simultaneous state
    entries that can be created per source IP address.
    The scope of this limit (i.e., states created by this rule only or
    states created by all rules that use source-track) is dependent
    on the source-track option specified.
      • Re: (Score:3, Informative)

        Something like:

        iptables -I INPUT -p tcp --dport 80 -m state --state NEW -m recent --update --seconds 60 --hitcount 5 --rttl --name SSH -j DROP

        limits to 5 new connections per 60 seconds

        • by raju1kabir (251972) on Friday June 19 2009, @10:02AM (#28390201) Homepage

          Many browsers will open more connections than this, resulting in "broken" image links on pages. I've tried all kinds of connection limits to protect against simple DOS attacks, but always have problems with corpororations whose standard desktop configs include IE/Firefox set to open way more connections than would be polite.

          It becomes politically challenging to cause those users to have a problem, especially since they don't see it with other sites. The perception is always that my server has a problem, and it doesn't matter how clearly I explain what's actually happening and how inappropriate it is to have 1000 PCs all set to open 30 connections to each web site they visit.

  • by cjb-nc (887319) on Friday June 19 2009, @09:47AM (#28389991)
    Obviously need to verify this, but we already run mod_cband [sourceforge.net] with a per-IP connection limit of 5. This is in place to stop the over-zealous "download accelerators" from taking all our connections and DOS'ing us. I expect it would stop a single attacker using this attack, but we'd still be vulnerable to a concerted attack by MaxChildren/5 IPs.
  • by moon3 (1530265) on Friday June 19 2009, @09:51AM (#28390057)
    If you keep lingerers for more then 160 seconds then no wonder this is possible.

    It should be non-issues on better designed servers that keep an eye on connections anyway. Any single IP spawning lots of unfinished connections gets flagged fast and remembered for the future, so it will get limited access and bandwidth, marked as abuser etc. This is serving 101.
  • by Anonymous Coward on Friday June 19 2009, @09:53AM (#28390091)

    http://httpd.apache.org/docs/2.2/mod/core.html#timeout

    The issue is that the default configuration waits 5 minutes for the full request, which is painfully to long a period of time. Drop that from 300 to 5, and the "attack" goes away. If you are running the default Apache config in production, you shouldn't be.

    • by ID000001 (753578) on Friday June 19 2009, @10:18AM (#28390433)

      http://httpd.apache.org/docs/2.2/mod/core.html#timeout

      The issue is that the default configuration waits 5 minutes for the full request, which is painfully to long a period of time. Drop that from 300 to 5, and the "attack" goes away. If you are running the default Apache config in production, you shouldn't be.

      seem like a potential fix, can anyone confirm?

      • Re: (Score:3, Informative)

        by Anonymous Coward
        Th work-around works fine.

        I downloaded the Slowloris and was able to take down a default apache install, however with keepalive disabled and a timeout of 5, the attack became inneffective.

        This may be a problem for sites with users that do long-running POSTs, but since we don't have any of those, all I can say is "It works here . . . "

        For more info: http://httpd.apache.org/docs/trunk/misc/security_tips.html [apache.org]
    • Re: (Score:3, Insightful)

      From the article:

      "...the server will open the connection and wait for the complete header to be received. However, the client (the DoS tool) will not send it and will instead keep sending bogus header lines which will keep the connection allocated."

      In other words.. the connection is not allowed to "timeout" as there is (bogus) traffic on the connection.
    • Re: (Score:3, Insightful)

      The problem with that is it will break nontrivial uploads using POST since they won't complete in 5 seconds. The real solution is to not count threads or connections below a certain utilization threashold towards the capped max and kill them once you hit real starvation.
  • by greed (112493) on Friday June 19 2009, @09:54AM (#28390101)

    HTTP 1.1 [rfc-editor.org] specifies a status code for "Request Timeout" (408) and "Gateway Timeout" (504).

    What is needed, therefore, is a timer running for receiving the complete header, and a second one for accepting the body. The timer for the body can be controlled by the type of request and the Content-Length header. (With, of course, a specific cap.)

    Currently, Apache 2.2 [apache.org] has a single timeout value for all types of requests, but it is interpreted differently for the different types.

    If your server only handles GETs, the obvious thing is to crank that number down. Unfortunately, for PUTs, the TimeOut value affects inter-packet time in the request, not overall request time.

    Strangely, the timeout doesn't seem to run in 2.2.10 and 2.2.11 before data is received. Oh dear. That's an even simpler DoS.

    #!/usr/bin/env perl

    use IO::Socket::INET;
    use strict;
    use constant DEFAULT_PORT => "http";

    MAIN: {
    if(@ARGV<1 or @ARGV>2) {
    die "Usage: $0 host [port]\n";
    }
    my($host)=shift;
    my($port)=@ARGV?shift:DEFAULT_PORT;

    my(@sockets);

    for(my $cnt=0;$cnt<1000;++$cnt) {
    my $socket=new IO::Socket::INET(PeerAddr=>$host,
    PeerPort=>$port,
    Proto=>"tcp");
    unless(defined($socket)) {
    die "Cannot create socket to $host:$port--$!\n";
    }
    $socket->print("\r\n");
    push(@sockets,$socket);
    print " Have ".@sockets." open.\n";
    }
    }

    Not quite as stealthy, though. At least as above.

  • by Megane (129182) on Friday June 19 2009, @12:21PM (#28392111)

    If you're going to post links to isc.sans.org, can you please post links to the specific article, and not just the main page?

    Here is the link to the specific article: http://isc.sans.org/diary.html?storyid=6601 [sans.org]

    • You may have missed the 'not' in the summary there.

    • IIS is not vulnerable, as confirmed in TFAs and the summary. Seems as though a DoS attack has affected some peoples' eyes.
        • Re: (Score:3, Interesting)

          Just tested on www.microsoft.com and I sat there in the header negotiating part for over 3 minutes before the pipe broke on remote end (I didn't do anything.) Slight change in that slowloris makes IIS* vulnerable too.

          Keeping idle sessions open on the server for some period of time is not the flaw. The flaw is that some servers (in default configurations) only handles so many of these idle sessions open before reaching a limit and refusing to accept any new sessions, and that this limit is sufficiently low,

Who will take care of the world after you're gone?