Heartbleed OpenSSL Vulnerability: A Technical Remediation 239
An anonymous reader writes "Since the announcement malicious actors have been leaking software library data and using one of the several provided PoC codes to attack the massive amount of services available on the internet. One of the more complicated issues is that the OpenSSL patches were not in-line with the upstream of large Linux flavors. We have had a opportunity to review the behavior of the exploit and have come up with the following IDS signatures to be deployed for detection."
Re:Thank you for the mess (Score:5, Informative)
Sadly, this is not the case. The evidence is that bad actors had this exploit for months: http://arstechnica.com/securit... [arstechnica.com]
Re:Thank you for the mess (Score:5, Informative)
Re:Is OpenVPN affected? (Score:5, Informative)
Some versions are. The OpenVPN appliance I was running was affected, and there were no updates for it this morning so I had to kill it.
https://security.stackexchange... [stackexchange.com]
I read somewhere that there is a TLS flag you can use in the config to disable the affected code, but for the life of me I can't find it for this post. :(
Re:What version does OpenBSD use? (Score:5, Informative)
Theo claims OpenBSD is unaffected. http://undeadly.org/cgi?action... [undeadly.org]
Theo claims OpenSSH is unaffected, because it isn't. OpenSSL, even on OpenBSD, is quite affected.
Re:Situation is a Shambles (Score:2, Informative)
You're amazingly wrong.
http://article.gmane.org/gmane.os.openbsd.misc/211963
This has nothing to do with unmanaged languages. It has to do with somebody actively sidestepping security devices that are already in place because they don't grok the way the world works outside of their test bench.
What do you think Python was written in? Here's a hint, it wasn't another managed language.
Re:Is OpenVPN affected? (Score:5, Informative)
I think if you had enabled the tls-auth option it prevents the attack.
Re:Mountain out of a molehill (Score:5, Informative)
It's not easy to fix leaked data.
You can revoke keys, change passwords, and patch the software, but you can't revoke the data that was already sent with them (and can now be decoded) no more than you can you revoke the bits of data that could have been stolen.
Re:Mountain out of a molehill (Score:5, Informative)
Re:Is there a way to tell? (Score:5, Informative)
Qualys SSL Test [ssllabs.com] is including a flag for Heartbleed vulnerability and auto-fails any domain tested that is affected.
Re:Is there a way to tell? (Score:2, Informative)
http://filippo.io/Heartbleed/ [filippo.io]
Several! (Score:5, Informative)
There have been a number of sites.
SSLLabs scanner has been updated to check for Heartbleed, and also will report when the cert validity starts (handy if you want to see whether they're using a new cert). https://www.ssllabs.com/ssltes... [ssllabs.com]
LastPass has a pretty decent scanner that just focuses on Heartbleed (without all the other info that you get from SSLLabs): https://lastpass.com/heartblee... [lastpass.com]
There are some others out there as well, of course.
There's even one for client-side testing (almost as critical):
Pacemaker is an awesome little POC script (python 2.x) for testing whether a *client* is vulnerable (many that use OpenSSL are...). https://github.com/Lekensteyn/... [github.com]
Re:Thank you for the mess (Score:5, Informative)
For people who didn't follow the link chain [seacat.mobi], it has since been updated:
Important update (10th April 2014): Original content of this blog entry stated that one of our SeaCat server detected Heartbleed bug attack prior its actual disclosure. EFF correctly pointed out that there are other tools, that can produce the same pattern in the SeaCat server log (see http://blog.erratasec.com/2014... [erratasec.com] ). I don't have any hard data evidence to support or reject this statement. Since there is a risk that our finding is false positive, I have modified this entry to neutral tone, removing any conclusions. There are real honeypots in the Internet that should provide final evidence when Heartbleed has been broadly exploited for a first time.
Reality Check. The sky is not falling. (Score:4, Informative)
1. The issue only exposes 64k at a time. Let's assume that the average enterprise application has at least a 1G footprint (and that's actually on the low end of most applications I work with). That's 1,048,576K. At best, this means that this exploit can access 0.006% of memory of an applications memory at one time.
Ahh you say, I will simple make 16,667 requests and I will retrieve all the memory used by the application.
2. The entire basis of this issue is that programs reuse memory blocks. The function loadAllSecrects may allocate a 64k block, free it and then that same block is used by the heartbeat code in question. However, this code will also release this same block which means that the block is free for use again. Chances are very good (with well optimized code), that the heartbeat will be issued the same 64k block of memory on the next call. Multi-threaded/multi-client apps perturb this but the upshot is that it's NOT possible to directly walk all of the memory used by an application with this exploit. You can make a bazillion calls and you will never get the entire memory space back. (You're thinking of arguments to contrary, your wrong... you wont.)
Congratulations, much success... you have 64k internet.
3. Can you please tell me where the passwords are in this memory dump:
k/IsZAEZFgZueWNuZXQxFzAVBgNVBAMTDk5ZQ05FVC1ST09ULUNBMB4XDTEwMDMw
MzIyNTUyOFoXDTIwMDMwMzIyMTAwNVowMDEWMBQGCgmSJomT8ixkARkWBm55Y25l
There will be contextual clues (obvious email addresses, usernames, etc) but unless you know the structure of the data, a lot of time will be spent with brute force deciphering. Even if you knew for a fact that they were using Java 7 build 51 and Bouncy Castle 1.50, you still don't know if the data you pulled down is using a BC data structure or a custom defined one and you aren't sure where the boundaries start and end. The fact that data structures may or may not be contiguous complicates matters. A Java List does not have to store all members consecutively or on set boundaries (by design, this is what distinguishes it from a Vector).
Long story short. Yes, there is a weakness here. However, it's very hard to _practically_ exploit... especially on a large scale (no one is going to use this to walk away with the passwords for every gmail account... they'd be very, very lucky to pull a few dozen).
This doesn't excuse developers from proper programming practices. It's just putting "Heartbleed" in perspective.
Re:Mountain out of a molehill (Score:5, Informative)
I work for a large financial organization. While fixing the hole itself was easy- having to tell a bunch (I can't even legally give you a ballpark, but its a lot) of customers to change their passwords (or forcing them to change) is very bad PR. Plus we don't know if any financial data was accessed. The data could literally bankrupt very large companies or my own company. This is no small problem!
Re:what? (Score:5, Informative)
Was this badly translated from another language, or have I been out of system administration too long?
Allow me to translate from buzz-ard to sysopian:
SSL-Ping Data Exfiltration Exploit: Detection and mitigation even a flaming lamer that can't patch OpenSSL can use
"Since this 0-day vuln was published skiddies have been exploiting it to leak data available to OpenSSL 64KB at a time via running one of the pre-written exploit proof-of-concept sources (as skiddies are wont to do) against a bunch of affected Internet facing services. This SNAFU is particularly FUBAR since all the distros that noobs use are building an ancient OpenSSL ver so they can't even push out a simple patch, obviously. We fingered the exploit in use and have a signature so your punk-buster scripts can detect the crackers and ATH0 before your cipher keys get the five-finger discount."
Re:Thank you for the mess (Score:3, Informative)
Re:what? (Score:2, Informative)
If someone is using an ancient openssl library (0.9.8) they have nothing to worry about. The problem was introduced in 1.0.0
Re:Thank you for the mess (Score:4, Informative)
Re:Situation is a Shambles (Score:4, Informative)
That sounds like a Mint thing. Seriously, Debian (the great grandparent of Mint) had the patch as fast as anybody. Heck, by the time I logged into my Mac at work, MacPorts had pushed the patch.
I wouldn't make such a sweeping statement about the "situation" when you've hitched your wagon to a project that's pulling from a project that's pulling from a project that's (etc).
Re:Reality Check. The sky is not falling. (Score:5, Informative)
Can you please tell me where the passwords are in this memory dump ...
Have you ever seen a real exploited piece of data?
These are taken from Yahoo production servers, a day or two ago:
http://cdn.arstechnica.net/wp-... [arstechnica.net]
http://cdn.arstechnica.net/wp-... [arstechnica.net]
Can you guess where the password is, now? (And those didn't even take that many tries)
I have not seen actual SSL private keys floating around just yet, but given that the original researchers said [heartbleed.com] they managed to get private keys from their own servers, I think it is reasonable to assume that some production servers must have already leaked them.
Re:Situation is a Shambles (Score:3, Informative)
This is not a memory management issue per se, and has nothing to do with mmap or malloc.
But what the grandparent post said still applies. It's how C treats memory via pointers. The issue, from looking at the code you posted, is that memcpy() copies from beyond the length of rec_p. In a sane language that doesn't treat memory as free-for-all, this isn't possible.
No, that's not the issue, in fact there really isn't any significant pointer arithmetic used here. Yeah, it does use a bit to pull the size field out of the incoming request, but there's nothing wrong with that part of the code.
The issue is that the code allocates a buffer of a size specified by the user, without validating it, and doesn't zero the allocated memory. Yes, many languages automatically zero heap-allocated arrays, which is good, but it's also a performance cost which is often unnecessary and there's nothing wrong (IMO) with not doing it by default. There is, however, plenty wrong with just accepting a user-provided value without validation, and with not completely overwriting heap-allocated memory before sending it.
Re:Thank you for the mess (Score:2, Informative)
I usually agree, but this is a vulnerability where an extreme amount of damage can be avoided by taking down SSL servers as quickly as possible.
Due to the nature of the bug, recorded traffic from up to two years ago can be decrypted if the site doesn't have PFS enabled, which most sites haven't. If you have packet dumps from the coffee shop wifi downstairs, all it takes is a little luck while running the attack on a couple of big mail providers to get the servers' private keys: Then you can decrypt everybody's mail sessions, web shopping and online banking, including user names, passwords and content, from up to two years ago! The exact nature of the bug should not have been revealed before telling everybody who's running OpenSSL 1.0.1 onward to shut down the server.
Nobody should have kept a vulnerable SSL server running for even just a minute after the announcement. The right reaction would have been to halt the server immediately upon being told of the bug. Patching as quickly as possible is not quick enough.
IDS patterns are pointless now: The attack is already being integrated into end user software to alert users of vulnerable sites. The servers will be hammered with this attack from thousands of IPs and the users who run these "attacks" won't even know they're doing it.
I think most people have yet to realize the magnitude of this fuckup.