Catch up on stories from the past week (and beyond) at the Slashdot story archive

 



Forgot your password?
typodupeerror
×
Security Linux

Over 14 Million Servers May Be Vulnerable To OpenSSH's 'RegreSSHion' RCE Flaw (zdnet.com) 90

An anonymous reader quotes a report from ZDNet, written by Steven Vaughan-Nichols: Hold onto your SSH keys, folks! A critical vulnerability has just rocked OpenSSH, Linux's secure remote access foundation, causing seasoned sysadmins to break out in a cold sweat. Dubbed "regreSSHion" and tagged as CVE-2024-6387, this nasty bug allows unauthenticated remote code execution (RCE) on OpenSSH servers running on glibc-based Linux systems. We're not talking about some minor privilege escalation here -- this flaw hands over full root access on a silver platter. For those who've been around the Linux block a few times, this feels like deja vu. The vulnerability is a regression of CVE-2006-5051, a bug patched back in 2006. This old foe somehow snuck back into the code in October 2020 with OpenSSH 8.5p1. Thankfully, the Qualys Threat Research Unit uncovered this digital skeleton in OpenSSH's closet. Unfortunately, this vulnerability affects the default configuration and doesn't need any user interaction to exploit. In other words, it's a vulnerability that keeps security professionals up at night.

It's hard to overstate the potential impact of this flaw. OpenSSH is the de facto standard for secure remote access and file transfer in Unix-like systems, including Linux and macOS. It's the Swiss Army knife of secure communication for sysadmins and developers worldwide. The good news is that not all Linux distributions have the vulnerable code. Old OpenSSH versions earlier than 4.4p1 are vulnerable to this signal handler race condition unless they are patched for CVE-2006-5051 and CVE-2008-4109. Versions from 4.4p1 up to, but not including, 8.5p1 are not vulnerable. The bad news is that the vulnerability resurfaced in OpenSSH 8.5p1 up to, but not including, 9.8p1 due to the accidental removal of a critical component. Qualys has found over 14 million potentially vulnerable OpenSSH server internet instances. The company believes that approximately 700,000 of these external internet-facing instances are definitely vulnerable. A patch, OpenSSH 9.8/9.8p1 is now available. Many, but not all, Linux distributions have made it available. If you can get it, install it as soon as possible.
If for whatever reason you're not able to install a patch, Vaughan-Nichols recommends you set LoginGraceTime to 0 in the sshd configuration file and use network-based controls to restrict SSH access, while also configuring firewalls and monitoring tools to detect and block exploit attempts.
This discussion has been archived. No new comments can be posted.

Over 14 Million Servers May Be Vulnerable To OpenSSH's 'RegreSSHion' RCE Flaw

Comments Filter:
  • beyond belief (Score:5, Interesting)

    by timeOday ( 582209 ) on Tuesday July 02, 2024 @10:55PM (#64596635)
    The most infamous bug in this history of OSS was accidentally re-introduced and again unnoticed for 3 years in THE most critical piece of security software?
    • Who there? O timeOday what is the most infamous bug in this history of OSS?
    • Re:beyond belief (Score:5, Interesting)

      by gweihir ( 88907 ) on Wednesday July 03, 2024 @12:20AM (#64596707)

      Looks that way. I really expect better from the OpenSSH team. To be fair, this is the first time in a long while that they have screwed up. The last SSH security problems was clueless distro maintainer patching systemd crap into OpenSSH.

      It is not quite as bad, however. Apparently, with 100 concurrent connection attempts the attack takes about 8h on average and apparently the defaults, at least on Debian, are more like a max of 10 concurrent connections. So anybody that patches fast is probably still pretty safe. On the other hand, sshd announces its version on connection (it needs to, apparently - just telnet to port 22 on the target), so attackers can identify targets very easily and cheaply. Hence anybody stupid or incapable enough to not patch fast will probably get attacked.

      My servers, incidentally, are not vulnerable. They are either Debian/Devuan oldstable (sshd 8.4) or automatically patched from secureity.debian.org and hence already had the patch active yesterday. If this continues, I will add an sshd restart to the automatic update process though, had to do that manually on two machines yesterday, fortunately both not directly Internet-reachable.

      • Re:beyond belief (Score:4, Informative)

        by Pieroxy ( 222434 ) on Wednesday July 03, 2024 @02:15AM (#64596823) Homepage

        FYI Ubuntu patched the existing versions of OpenSSH, so you don't necessarily need 9.8p1 to be safe. For example Ubuntu 24.04 latest openSSH is OpenSSH_9.6p1 Ubuntu-3ubuntu13.3 and includes the fix.

        More info: ubuntu.com [ubuntu.com]

        • by gweihir ( 88907 )

          Interesting. At least if the Ubuntu part pf the version is not advertised by sshd (on Debian it is per default, although I have turned that off), this will make it harder for attackers to find out whether a potential target is vulnerable.

        • FYI Ubuntu patched the existing versions of OpenSSH, so you don't necessarily need 9.8p1 to be safe. For example Ubuntu 24.04 latest openSSH is OpenSSH_9.6p1 Ubuntu-3ubuntu13.3 and includes the fix.

          You'll see the same thing with AlmaLinux and the others - that's typically how LTS / Enterprise distributions of Linux work. The idea is to provide a stable, known platform that software makers can depend on for the entire life of a distribution's major release (e.g. Ubuntu 22, AlmaLinux 9). So for any packages that other software might depend on, the distro maintainers pick a particular release version at launch. Then, when any necessary fixes come up, they back-port those to that fixed release.

      • by jmccue ( 834797 )

        Looks that way. I really expect better from the OpenSSH team.

        This only affects Linux glibc systems. OpenBSD does not have this issue. My guess is due to to how complex Linux has become, I would not be surprised if there are a few more of these. From what I have read, right now, you can get in on a slow 386 Linux System after 5 or so hours of trying. No one was able to, as of yet, get into a 64 bit system.

      • by mysidia ( 191772 )

        Debian, are more like a max of 10 concurrent connections. So anybody that patches fast is probably still pretty safe.

        The other thing is This is on 32-bit systems in a lab.

        Some embedded 32-bit devices exposed to the internet may be in grave danger, but 64-bit servers Not so much.

      • On Debian (and I suspect Devuan) OpenSSH is restarted automatically. Or at least it was on my Debian machine when I updated it yesterday.
    • Re:beyond belief (Score:5, Insightful)

      by ShanghaiBill ( 739463 ) on Wednesday July 03, 2024 @01:03AM (#64596753)

      That is why everyone should have a suite of regression tests.

      Regression testing [wikipedia.org]

      • by gweihir ( 88907 )

        To be fair, this requires a very elaborate test and the old test code (if there was any) would likely not even work. I think the problem here is more in the area of code documentation, change review and security architecture.

        I hope this will lead to some introspection in the OpenSSH team. Although, again, to be fair, their native OpenBSD is not vulnerable. This is a glibc problem.

        • To be fair, this requires a very elaborate test and the old test code

          The canonical TDD way to do it is:

          1. Get a bug report.
          2. Fiddle around until you can reproduce the bug.
          3. Write code that can reliably reproduce the bug.
          4. Fix the bug.
          5. Run the code from #3 to verify the bug is fixed.
          6. Add the code from #3 to the test suite so it is run automatically.

          If you're doing it right, you shouldn't need to do much additional work because you had to write the test to fix the bug in the first place.

          • Re:beyond belief (Score:4, Informative)

            by gweihir ( 88907 ) on Wednesday July 03, 2024 @06:22AM (#64597105)

            Yes. And ordinarily this works just fine. But for a low-probability race condition that, on top, does not happen at all on the native OpenSSH platform _and_ that does likely require very specific conditions not quite the same as when it happened before, that approach does not really work.

            That is one reason why you should always be very careful to completely avoid race conditions. They typically cannot reliably be tested for and they may be very sensitive to details that usually do not matter in regular functional testing. They could well happen on one test system and not at all on a slightly different test system, that may even have exactly the same software configuration. Even things like exact server room temperature may have an influence. These types of bugs are really the worst-case for any testing approach.

          • by znrt ( 2424692 )

            the correct way to ammend a poster you believe is wrong is:

            1. carefully read the post you think contains an error
            2. carefully read at least the abstract, making sure you understand the context
            3. if in doubt, repeat #1 and #2
            4. if you still believe you know better, post your correction

            if you're doing it right, you shouldn't appear as a clueless doofus.

    • Re:beyond belief (Score:5, Informative)

      by rahmrh ( 939610 ) on Wednesday July 03, 2024 @06:17AM (#64597097)

      The current test case can hit the bug on 32-bit systems by maxing out the number of sshd connections for 6 to 8 hours. Indications are it is harder to hit on 64-bit systems. And it is not directly the sshd code but a thread-unsafe issue in a library that they are calling (syslog call).

  • by Nostalgia4Infinity ( 3752305 ) on Tuesday July 02, 2024 @10:58PM (#64596645)

    OpenSSH’s release note for version 9.8/9.8p1 indicates that successful exploitation has been demonstrated on 32-bit Linux/glibc systems with ASLR enabled. They state that the attack typically requires 6-8 hours of continuous connections to the server.

    While exploitation on 64-bit systems is considered possible, it has not yet been demonstrated. The advisory notes that these attacks will likely be improved upon.

    https://www.openssh.com/releas... [openssh.com]

    • by gweihir ( 88907 )

      Also, the 8h average seems to be for 100 concurrent connections. The defaults, at least on Debian, seem to be only 10 concurrent connections. Hence this may take a while. As the 64 bit version seems to not yet be vulnerable, anybody that patches now is probably good. Anybody that does not patch will likely get attacked, however, as sshd announces its version on a simple telnet to port 22. Apparently ssh needs that to work. Incidentally, Qualys probably just scanned the whole IPv4 range for that number. Not

    • What if my system is 36 bits? https://en.wikipedia.org/wiki/... [wikipedia.org]

      • by vbdasc ( 146051 )

        What if my system is 36 bits? https://en.wikipedia.org/wiki/... [wikipedia.org]

        Then you're out of luck, dude. I upgraded to 32-bit [wikipedia.org] from 36-bit and Linux still doesn't support that!

        You could have better success with other 36-bit machines, though. Like IBM 709/709x, GE-645 (a.k.a. Multics machine) or even the modern UNIVAC 1100/2200 .

  • by gavron ( 1300111 ) on Tuesday July 02, 2024 @11:18PM (#64596659)

    Yes, this has a high CVE rating because ***IFF*** it were to be exploited things would be bad.
    However, 1/10000 chances of exploit to get the race condition failure mitigates that CVE to
    be about a 1.0.

    This SHOULD be re-fixed immediately, no doubt. But given the odds of exploitation, it's less
    dangerous than running with scissors.

    > It's hard to overstate the potential impact of this flaw.
    Yeah, it is hard to overstate, so please stop overstating it.

    Speaking for my ITSEC friends who are tired of the wailing.

    • by Anonymous Coward
      The CVE is tagged as an 8.1, seems it already takes into account the difficulty otherwise this would be a 10.0
    • by gweihir ( 88907 ) on Wednesday July 03, 2024 @12:44AM (#64596729)

      It has an expected time-to-hack of 8h with 100 connections. As one attempt takes 2 minutes, that is 1/24000 per try, but the attack can be parallelized. As a result, _everybody_ affected should patch now. That is "patch today", not "patch sometime in the next weeks". As Linux generally has reliable updates (unlike Windows), that should not cause problems, but it needs to be done. And do not forget to restart sshd.

      There is some protection from the current attack not working on 64 bit and apparently at least Debian seems to have a (soft) limit of 10 concurrent connections. But, since sshd announces its version (just telnet to port 22, seems to be needed for ssh to work), attackers can find potential targets fast and easily.

      • You can only parallelize it up to the number of simultaneous connections allowed by the server, which defaults to 10.
        By tying up all the allowed connections, you will also cause a DoS and prevent anyone else from logging in - which is likely to get noticed.

        • by gweihir ( 88907 )

          True and that is another good thing. To nit-pick, that is the maximum number of unauthenticated connections, so if an attacker uses just 9, regular users will still get in just fine.

          Still a "patch or update now" situation. Updating ssh and restarting the service is enough though, this is Linux after all. No risky reboots or other crap packaged in with the security fix. In a situation like this it really pays to have competent security people that also understand system administration.

  • ...keeps security professionals up at night

    err.. security professionals wouldn't be using the default config...

    Thanks for telling us how to think and feel "media"

    • by gweihir ( 88907 )

      And why not? OpenSSH has a pretty good security history. Using the default config for it is typically fine. "Security professional" does not mean "insane paranoid".

      Note that this is the first bad OpenSSH vulnerability in quite some time. The last ssh issues was morinic distro maintainers parching parts of systemd into OpenSSH and not actually an OpenSSH vulnerability. It did provide another good reason to not use systemd and that may be something a security professional should at least seriously consider.

  • But, ssh may have never meant to be a layer 7 thingie...... At least not any more than information window boxes in C, for example......
  • by ichthus ( 72442 ) on Wednesday July 03, 2024 @12:27AM (#64596715) Homepage
    Subject says it all. I decided a few weeks ago to run Alma's migration script (converting from Rocky to Alma) on a couple VMs, and I'm glad I did. Alma patched it, and then submitted the patch to Centos stream. So, I guess this means, upon acceptance, RHEL and Rocky will have Alma to thank. Interesting.
    • I assume you're referring to AlmaLinux 9. AlmaLinux 8.10 is running openssh 8.0p1, which according th the description should not be vulnerable.

    • This was an embargoed CVE. Red Hat was working on it before either Rocky or Alma knew about it. The fix hasn't been published by RH because of the level of testing it requires, and the risk associated with screwing up the fix. Alma and Rocky have nothing like such commercial obligations to their users.
    • If you hang around the EL community at all you'd know the two projects work together frequently. There was even collaboration in the EL IRC channel on this. No need to try to stir stuff up between the two. Alma is ABI compatible so they can patch early; Rocky waits on Red Hat to remain 1:1. Red Hat and Rocky both patched officially today. Some people want fast; some require 1:1 - use the distro that fits your needs better, but don't try to stir up something that isn't there.

      • by ichthus ( 72442 )
        Haha. I'm just a nobody on the innerwebs. I wasn't trying to, and couldn't stir anything up. I was just sharing an amusing observation. Thanks for the insight.
  • Finally (Score:5, Funny)

    by vbdasc ( 146051 ) on Wednesday July 03, 2024 @01:08AM (#64596761)

    Now I finally know what the word "Open" in OpenSSH does mean.

  • Sigh (Score:5, Informative)

    by Artem S. Tashkinov ( 764309 ) on Wednesday July 03, 2024 @02:11AM (#64596817) Homepage
    Posted the same two days ago [slashdot.org], the /. editorial team simply ignored me.
    • by gweihir ( 88907 )

      Great. I was wondering what happened. I got the vulnerability info from some other sources two days ago.

  • Are they sure macOS is vulnerable?

    This would tend to say that some BSD-related Systems Dodged this Bullet:

    "OpenBSD systems are unaffected by this bug, as OpenBSD developed a secure mechanism in 2001 that prevents this vulnerability. "

    https://vulcan.io/blog/cve-202... [vulcan.io]

    I can't find any evidence of macOS being vulnerable to regreSSHion, other than the one statement that SSH is used for "File Transfer in Linux and macOS."

    Can some macOS/BSD Experts help clear this up? I don't think macOS is built on GNU Libraries

    • by gweihir ( 88907 )

      Does MacOS use glibc? AFAIK, this is a problem with ssh only in glibc. You could do an "ldd sshd" (does not require root) in a terminal and check.

      • Re:Et Tu, macOS? (Score:4, Informative)

        by Burdell ( 228580 ) on Wednesday July 03, 2024 @07:40AM (#64597195)

        glibc is not required, it's just that glibc is known to have the issue. The issue is that lots of long-time library routines are not re-entrant and cannot safely be called from a signal handler context. For glibc, this includes the syslog() call. OpenBSD has a re-entrant syslog_r() call instead (which OpenSSH uses there). So this issue is a security problem on any platform which OpenSSH uses syslog() (which I believe is all but OpenBSD) _and_ where syslog() is not re-entrant (which may be everywhere).

        • by gweihir ( 88907 )

          Yes. But what does MacOS use?

        • glibc is not required, it's just that glibc is known to have the issue. The issue is that lots of long-time library routines are not re-entrant and cannot safely be called from a signal handler context. For glibc, this includes the syslog() call. OpenBSD has a re-entrant syslog_r() call instead (which OpenSSH uses there). So this issue is a security problem on any platform which OpenSSH uses syslog() (which I believe is all but OpenBSD) _and_ where syslog() is not re-entrant (which may be everywhere).

          Very Informative, I guess; but is macOS actually affected?

    • macOS does not have glibc which is required for this vulnerability.

  • by Viol8 ( 599362 ) on Wednesday July 03, 2024 @04:46AM (#64597035) Homepage

    Just saying ;)

  • by Schoenlepel ( 1751646 ) on Wednesday July 03, 2024 @05:34AM (#64597067)

    Port knocking eliminates issues like these.

    You can simply have the ssh server not available on any port, until a specific packet arrives at (a) specific port(s). After that happens, you set your firewall rules to only allow ssh traffic in from the IP address where the knock came from and launch the ssh server. Then, when the work is done, the ssh server is stopped and the port is closed. Good luck making use of this vulnerability.

    • what if unprivileged employee knows about the port knocking and then does exploit to get root?

      • If your own employees are hacking you from the inside you have bigger problems.

        • Maybe, but ex-employees who leave before you change the knock-ports might be a problem. I guess if you are going to use port-knocking the port IDs should be treated like a shared password.
          • Agreed. I've posted several times here about the insanity and incompetence of companies that can't manage to lockout ex employees as they walk out the door instead of never or months later or only after a problem.

      • what if unprivileged employee knows about the port knocking and then does exploit to get root?

        You'll see something in your logs, because one of the glories of port knocking is that you don't see the rash of doorknob rattling you get without it. So literally anything sshd logs is signfiicant.

        • if you see it in logs it's too late. but thanks for showing why most SOC people and companies are pretty worthless, screaming lost horses in empty barn

    • Port knocking eliminates issues like these.

      Good thing tcpdump doesn't exist, right?

  • by v1 ( 525388 ) on Wednesday July 03, 2024 @06:48AM (#64597123) Homepage Journal

    I have interactive/password login disabled. does this affect rsa/dsa key login also?

    • by dskoll ( 99328 ) on Wednesday July 03, 2024 @07:25AM (#64597161) Homepage

      Yes. The exploit happens before any authentication is even attempted.

      • So... uh... what's it doing with the key the user supplied? Maybe searching it for secret messages or perhaps just evaluating the numbers based on their artistic merits?

        • by dskoll ( 99328 )

          The exploit works by the client not doing any authentication at all, but just sitting and waiting until the server times out. Once it has figured out the timeout, then it connects again and tries to initiate authentication right around when the SIGALRM handler would fire, hoping to cause a non-async-safe function to be called unsafely. It's a very tricky timing attack that takes thousands of attempts (on average) to succeed.

  • It's time to ask this question: Do our servers need wide-open SSH access? In many cases, it's feasible to lock down access to a set of fixed IP addresses using firewall rules. Or if that's not possible, to use a VPN so that at least the server isn't listening directly on a publicly-accessible IP address. While this trades an OpenSSH vulnerability for a potential VPN vulnerability, there are probably a lot more SSH scanners prowling the Internet than various VPN scanners.

    I have a couple of colocated ser

  • Trying to figure out if Solaris 11.4 is vulnerable to this.
  • by tabrisnet ( 722816 ) on Wednesday July 03, 2024 @07:39AM (#64597191)

    It's worth noting that the workaround listed in the summary is probably wrong. Per the man page for sshd_config
    "The server disconnects after this time if the user has not successfully logged in. If the value is 0, there is no time limit."

    • Re: (Score:2, Informative)

      by Anonymous Coward
      I think the idea is that this eliminates the possibility of the race condition needed to exploit...
  • "It's hard to overstate the potential impact of this flaw."

    LOL.

    If it takes multiple hours and many many thousands of tries against a completely known configuration on the easiest target possible in order to exploit it *once* in a lab environment, then I'd say that a great number of people have already found it very easy to overstate the potential impact of this flaw in a desperate attempt to increase their click-throughs.

    Like, sure it's very bad that sshd could be used to get a remote shell but simple measu

"It takes all sorts of in & out-door schooling to get adapted to my kind of fooling" - R. Frost

Working...