Duplicate SSH Keys Put Tens of Thousands of Home Routers At Risk 114
alphadogg (971356) writes A setup mistake has apparently left hundreds of thousands of home routers running the SSH (Secure Shell) remote access tool with identical private and public keys. John Matherly, founder of a specialized search engine company whose technology is used for querying Internet-connected devices, found more than 250,000 devices that appear to be deployed by Telefónica de España sharing the same public SSH key. A different search found another 150,000 devices, mostly in China and Taiwan, that have the same problem. Matherly said in a phone interview on Wednesday it is possible the manufacturers copied the same operating system image to all of the routers.
No surprise... (Score:5, Informative)
Most embedded guys are batting out of their league and don't have a clue when it comes to security... and I say that as an embedded guy who often has to do exactly that to get the product out.
Most embedded development I've done is far from 'software engineering' - it's whack and hack until the tests pass(often because you loosened the testing requirements).
Re:No surprise... (Score:5, Insightful)
Re: (Score:1)
1) "Gubmint Bad", or ...
2) General ignorance, or
3) Billions of lobbying dollars from the manufacturers ensuring that the cost of each unit doesn't go up $1, or
4)
5) PROFIT
UL (Underwriters) is a private, for-profit company (Score:5, Informative)
Most electrical equipment mass-marketed in the US is tested by UL (Underwriters Laboratories). Many consumers and most large purchasers recognize the UL mark as indicating a degree of safety. Contracts can specify that products an components meet various UL standards. That's why your router's power suppy wall-wart probably has the UL mark, and doesn't generally catch fire.
The "Gubmint" doesn't force UL certification or listing, purchasers choose UL listed products. There's no "billions of lobbying dollars", in fact companies PAY to have their products tested, because if they are recognized by UL they don't sell nearly as well. Not only do individual consumers recognize the UL logo, but purchasing agents for Walmart and Target know they'd rather buy and sell UL tested products, so if you want Walmart to order 500,000 fire safes from you, you better have UL test it.
So no, it doesn't have anything to do with "gubmint" or "lobbying" - UL or another organization could check the firmware in the router just like they already check the power supply circuit.
Re: (Score:2)
Re: (Score:2)
After the recent crypto fiasco, a NIST certification might be seen as a mark of shame.
Re: (Score:2)
There are already programs in place. One example, NIST certifies private security testing laboratories to test according to FIPS standards. It just nobody asking for certified products outside of the government procurement.
FIPS 140 certification, which I assume is what you're referring to, is almost worthless in terms of determining how resistant to real-world attack a product really is. It would have done nothing to prevent the problem discussed here. Its main use is as a measure of how desperate a vendor is to get government contracts, which is also why no-one asks for it outside government procurement.
Re: (Score:2)
Kinda Apples and Oranges. UL testing is fairly straight-forward. The quick explanation - they stress the device in various ways and see if it catches on fire. Checking a crypto setup to a reasonable level of satisfaction can't be done externally. The code for the entire system must be examined, and that is relatively difficult to do.
-Matt
Re: (Score:2)
Not only do individual consumers recognize the UL logo, but purchasing agents for Walmart and Target know they'd rather buy and sell UL tested products, so if you want Walmart to order 500,000 fire safes from you, you better have UL test it.
So no, it doesn't have anything to do with "gubmint" or "lobbying" - UL or another organization could check the firmware in the router just like they already check the power supply circuit.
You're assuming perfect certification and a lack of counterfeiting. The running joke at the moment is that CE stands for Chinese Engineering rather than being the European equivalent of UL. We see products in Australia a lot carrying both the UL and CE logos, because frankly the same products are sold all over the world. It quickly becomes apparent when dismantling them that either the certification is fake or that the certification isn't worth the paper it's printed on. In fact early last year if I recall
not perfect, just better than nothing (Score:2)
>. You're assuming perfect certification and a lack of counterfeiting
No, I'm pointing out that it's better than NO testing or certification. If 5% of the products are counterfeit, that means 95% aren't. Compare the safety of what's one the shelves at Walmart vs what street vendors sell in Mexico or China. It does in fact work.
> For a certification scheme like this to work you need perfect certification
There's no "would need". UL has been testing products for over a hundred years, so it's not theo
Re: (Score:2)
Compare the safety of what's one the shelves at Walmart vs what street vendors sell in Mexico or China.
Well that's kind of my point. These stories while sensational are actually quite rare. I think you were being generous to my argument about 5% being counterfeit, in the western world it would be lower. But equally low are the number of products we have major security issues with. Yes it seems like every week there's another story about some small embedded device with a problem, but every week 20 new such devices hit the market. 10s of thousands of home routers? I have about 5 routers in my home, so the effe
All router makers have bad security, none have fir (Score:2)
> . I think you were being generous to my argument about 5% being counterfeit, in the western world it would be lower. But equally low are the number of products we have major security issues with.
Being in this industry, it seems to me that ALL major router manufactures have had multiple major security problems. NONE of them have had major "catch on fire" problems to my recollection. So the assertion that the number of devices with security problems is the same as the number that have fire problems is f
* or counterfeit, but UL tests for fire (Score:2)
My reply didn't exactly match your comment, but I'd say it's true for counterfeiting too. Pick a random electronic device at a random big-box store. It's probably NOT counterfeit. It probably DOES have lax security.
Even more, I'm talking about testing like UL does. UL focuses primarily on fire safety, and it works - our electronic devices rarely catch fire. Fire safety is a success. Data safety is a miserable failure - I can personally hack most devices.
Re: (Score:2)
I don't think the point was to actually have UL look into it, but rather create a security certification scheme.
But while your comments are true in the general sense I think there is more to it than that. I don't think I've been to a random big-box store in years. Now that could be our local market where the big-box stores do their best to screw users with ridiculous prices, but one thing that is very popular in Australia for instance is drop shipping. In many cases I have no idea at all where the device co
Sounds good, but shelves full of UL say otherwise (Score:4, Insightful)
> Back then you could justify the increased costs associated with getting the UL stamp of approval as a benefit to the consumer's safety.
> Today, if you tried this, you'd get absolutely buried.
That "sounds good", especially if it plays well with your personal political feelings. However, go pick ten random electrical products at your local big box store. Notice that at least nine, if not all ten, do in fact have the UL mark. The actual fact is that today almost all manufacturers do indeed "justify the increased costs associated with getting the UL stamp of approval". You can be surprised that they do, but you can see with your own eyes that they do. If that doesn't fit your current ideas, your ideas must be mistaken.
Re: (Score:2)
You're seeing an agenda where none exists. Yes, most electrical devices found in the home today have the UL mark on them; I never said otherwise. My point is that currently that is inertia from a decades-old system. If you tried to implement a brand new UL-type company TODAY, you would never succeed. Anything that increases costs, even if it adds significant value, is seen as evil and "unnecessary regulation", and therefore to be avoided at all costs. Especially when the value added is something that t
My typos are not UL approved (Score:2)
Apparently my post was not tested for typos.
Re: (Score:2)
They could do that, but then Telefonica wouldn't be able to buy the routers from China for $15 each (non wholesale price for the exact model Telefonica had in my house when I lived in Spain).
Re: (Score:3)
FIPS is a joke and doesn't really do much of anything to ensure real security. Its just a compliance checkbox. An appliance running a piece of software can be 100% insecure from the ground up and be FIPS certified quite easily.
Re:No surprise... (Score:4, Insightful)
Re:No surprise... (Score:4, Insightful)
It is a total joke.
FIPS 140-2 ensures your algorithm is part of a standard set - big deal. It does no investigation at all as to how you use that algorithm or why you use it. If you are using AES with a FIPS-certified library, you get the checkbox. Nevermind the fact that the private key you are using is sitting in plain text on the disk.
Its the same as all federal standards - FedRAMP, FIPS, FISMA, ISO 27K. They all do *SOME* things, but none of these standards, or any intersection of them, actually do anything with regards to real secure engineering. Note, I am not even sure it would be close to practical to do this. I am just raising awareness that saying that an application is "FIPS certified" is next to useless.
Source: I have dealt with getting applications certified in all of these umbrellas, and more. It is an extremely time consuming and expensive process for all involved to get certified - but in the end does very little in terms of real application security. But it sure makes the auditors a lot of money!
Re: (Score:3)
In case of NIST CAVP (part of FIPS testing most people are familiar with), the risk they are mitigating is that cryptographic algorithm you are using is flawed in some way. This certification program is hugely successful, there are robust standards and specs, and hardly anyone these days end up with bad algorithms
Re: (Score:2)
FIPS level 3 has nothing to do with software, that is the level which requires safeguards against physical tampering - tamper-evident seals etc. Again, nothing to do with the actual operation of the software. Level 4 takes Level 3 up a notch requiring even more hardening around "the module"... but AGAIN, nothing about how your software actually USES the module. Such a thing is totally outside the scope of FIPS.
FIPS is an outdated standard. It made sense when it was created and crypto was not well understood
Re: (Score:3)
FIPS may not be a joke, but most government networks are, especially, but not limited to, those outside of the DOD and IC. They are (in large part), administered by people who follow proscribed procedures, not people who understand what they're doing or why. While some "rogue" administrators will implement best practices beyond those they're required to do, they are the exception, not the rule -- especially admins who actually understand what they're doing rather than overestimating their own competence,
Re: (Score:2)
IMHO, we're trying to be optimistic that no one would be interested in our computing resources.
I think in general, most computer users think that if they can drive a car, they should be able to use a computer. I've been starting to wonder if we shouldn't have a similar licensing program for users? I know, computer users can't physically damage anyone until you take into account cyber bullying and those help desk calls that seem so idiotic, you wonder where their head is. Tech Support probably wouldn't get nearly as much volume if there was enough testing to say, sure this person has a fundamental grasp of technology.
This works out so well for driving. No, the cat is out of the bag. If it is a 'utility' then it has to be so simple that anything with opposable thumbs can use it.
Re: (Score:3)
Yes please, require 10 certifications until you can connect a router to the internet. Best is the build is signed, and the user's access to the system is restricted, otherwise they can install malware because of infected computers. Of course every single of those 10 certifications needs money, but everybody can pay 1.000.000$ for releasing a build, so thats no problem. With government certifications, it would be ensured that insecure and uncertified router firmwares like dd-wrt are forbidden, which is good
Re: (Score:2)
Sure, we can start doing product certification. You won't be able to buy $15 routers anymore though, they'll go back up to $150.
Re: (Score:2)
And they'll still be insecure.
Re: (Score:2)
Only in ways that the certification fails to cover.
Re: (Score:2)
That depends. If the certification process is highly regulated, they'll find new and more spectacular ways to fail. It the certification itself is left for the market to figure out, they'll vote with their dollar and select for the rubber stamps and drive the people who actually know the standard and expect it to be met into another line of work.
Re:No surprise... (Score:4, Informative)
Re: (Score:2)
Most embedded guys are batting out of their league and don't have a clue when it comes to security... and I say that as an embedded guy who often has to do exactly that to get the product out.
Most embedded development I've done is far from 'software engineering' - it's whack and hack until the tests pass(often because you loosened the testing requirements).
Gee man, get a spine and at least raise a stink about the problem next time. I understand that sometimes you gota just deliver what you got for business reasons but if you keep doing that without making a point about the quality and security of the solution, eventually they will *rightly* hold you responsible for the failures. Such stuff can get hard to hide on a resume. Do the *right* thing as often as you can and when someone forces you into doing the wrong thing, document the issue, if for nothing els
Re: (Score:3)
On the same note, Most CS guys could not survive 10 minutes in an embedded systems world. And it's not because those of us in Embedded systems don't have the skills, it's because the Idiots in upper management wont let us have the resources to do it right. If I can give you 900% more security by raising the cost of the box by $0.10 each then you freaking do it, not the "it will probably go unnoticed, so it's not that important" crap I hear.
Re: (Score:2)
Are you kidding?
Yes, some, usually large, companies have very competent engineering teams doing embedded work. Often, they're some of the smartest guys around(although IMHO the Verilog guys are smarter on average...YMMV). Then there are hoardes of engineers around them who do most of the integration, grunt work, bug fixing, etc. Their work is usually not thoroughly inspected by the smart guys.
Then you have the thousands of small companies doing embedded development. They usually have one guy who is some
WTF? (Score:3, Insightful)
Isn't TFS supposed to explain what it's talking about?
1. Why does a router have public-facing SSH? The reason to use SSH on your router is to configure it, over a wired connection from your PC, innit?
2. Why does a router come with SSH keys already installed? Don't you generate your own SSH keys?
Re: (Score:2)
Isn't TFS supposed to explain what it's talking about?
1. Why does a router have public-facing SSH? The reason to use SSH on your router is to configure it, over a wired connection from your PC, innit?
2. Why does a router come with SSH keys already installed? Don't you generate your own SSH keys?
Given that they were deployed by one particular provider (Telefónica de España in this case) they probably requested a special firmware from the vendor for their CPE to allow remote management. And then did a bad job of keeping the master key safe (by putting a copy of it on 250,000+ devices). And then the vendor used it elsewhere, too.
Honestly, after the Carna botnet, does anyone think the internet isn't a raging sea of completely compromised devices?
Re: (Score:2)
Isn't TFS supposed to explain what it's talking about?
1. Why does a router have public-facing SSH? The reason to use SSH on your router is to configure it, over a wired connection from your PC, innit?
2. Why does a router come with SSH keys already installed? Don't you generate your own SSH keys?
Given that they were deployed by one particular provider (Telefónica de España in this case) they probably requested a special firmware from the vendor for their CPE to allow remote management. And then did a bad job of keeping the master key safe (by putting a copy of it on 250,000+ devices). And then the vendor used it elsewhere, too.
Honestly, after the Carna botnet, does anyone think the internet isn't a raging sea of completely compromised devices?
I don't think so. The pubic and private keys are only good for outgoing connections and not incoming.
Re: (Score:2)
I don't think so. The pubic and private keys are only good for outgoing connections and not incoming.
Not funny. Anyway, having the private key is a valid way to authenticate inbound (if so configured) and its often enabled by default as it is thought to be a pretty good way to do it (until you put the same keys on a quarter of a million devices).
Re: (Score:2)
The host key pairs are NOT used to authenticate the incoming user.
They're used to prevent MITM attacks (by uniquely identifying the endpoint), so this statement
"It’s hard to say if the key errors means that a remote attacker could log into all of the devices, as it would depend on how the routers are configured for remote authentication."
It's complete bull; the article is written by a clueless moron.
Attackers would have to use the keypairs to setup MITM attacks for EVERY machine they wish to compromis
Re: (Score:2)
Are you speaking with specific knowledge of the device in the article? Because, in a general sense, keys can be, and are used for ssh authentication.
The device at hand is said to run Dropbear, which is "Compatible with OpenSSH
Re: (Score:2)
Why in the world would you add a device's public host key to the authorized key file?
Re: (Score:2)
Which authorized key file are you asking about?
One can picture an ISP, who has to securely support many installed devices, going to a manufacturer and saying: "Here, put this private key in the devices, so when we connect we know it's one of our devices. And put this public key in the authorized_key file, so we can can connect easily without every support agent needing to know a password which works on all those devices.
Re: (Score:2)
Right, you would put Telefonica's pubic key into the authorized_keys on each device file but never the local device public key. The simple fact is that unless they are far more stupid than the article suggests this cannot be used to break into the routers.
Re: (Score:2)
On the other side, having a public key in authorized_keys increases security. As I pointed out, it eliminates a need for a whole bunch of support personnel to know
Re: (Score:2)
Again, why would you use the host key for this purpose? Most likely the client would generate the key (no relation to the host key) they would want preloaded. The manufacturer has no reason to use the host key as both a host key AND a key in the authorized_key file. That is simply stupid.
Re: (Score:2)
Re: (Score:2)
Honestly, after the Carna botnet, does anyone think the internet isn't a raging sea of completely compromised devices?
It's the World Wide Web and the Wild Wild West all at the same time.
Re: (Score:1)
I could be wrong, but Telefónica (Actually Movistar) uses a Internet portal to manage your router: Portal Alejandra [movistar.es].
In fact, some routers had their configuration pages crippled so you have to configure them throug the portal.
Re: (Score:2)
The answer to both those questions (and more!) is the same: users are stupid, so make the router as easy as possible to configure even at the cost of security (assuming the programmer even notice that the security got fucked up).
I'm surprised the damn things don't have public-facing telnet or UPnP [wikipedia.org] (and for all I know, they might).
Re: (Score:2)
The reason to use SSH on your router is to configure it, over a wired connection from your PC, innit?
What is this 2014? The PC I bought this year doesn't have a wired connection.
2. Why does a router come with SSH keys already installed? Don't you generate your own SSH keys?
Users are never trusted with such things. Also I've never seen one where the SSH key was configurable. I believe something like this should automatically happen on first powerup without any visible indication to the user that it is or has happened.
"possible" (Score:1)
FTFY (Score:2, Informative)
John Matherly, founder of a specialized search engine company whose technology is used for querying Internet-connected devices
Translation -
John Matherly, founder of a company who randomly portscanned over 350,000 internet-connected devices whithout their user's consect, for the sole purpose of enriching his company's bottom line.
Re: (Score:3)
/sarcasm
Re: (Score:1)
This is the fingerprint, not the root user's key.
They are identified as having the same fingerprint (which is derived from the same source as the private key). So, someone with the private key for one device (ahem, anyone who has a copy of the firmware and knows how to use binutils) can authenticate to all devices.
Re: (Score:3)
Anyone use it? (Score:1)
Do the majority of users ever log into their device via ssh? So if everyone has the key you can decrypt other people's traffic, but the leaked keys in question probably don't allow for login.
Re: (Score:2)
Re: (Score:1)
"But it is questionable whether SSH should be running on a home router anyway"
Use SSH they said. There is no winning with this crowed.
not really "at risk" (Score:1)
Re: (Score:2)
what most people don't think of doing (Score:2)
...when the router comes out of the box it was shipped in, is power it up with the only network connection being a wired from port 1 to a pc, through which the router is locked to accept administrator connections from the currently conencted IP and machine ID ONLY (and the IP reserved for that machine), and ONLY via the wired LAN interface - from which point, you then change the wireless SSID and all the passwords FROM the defaults, and all that before you even physically connect it to the WAN cable. That w
Re: (Score:2)
Most people don't think of doing it that way because it's not a good way of doing it.
Re: (Score:2)
What's this "machine ID" you speak of? The MAC address? It's pretty easy to configure the same MAC on a different machine.
Most people don't think of doing it that way because it's not a good way of doing it.
That part was a little bizarre (if you are physically connected directly from PC to router it doesnt matter much what ID the PC might have) but as a whole it creates a pretty tight way of ensuring integrity of the router configuration. In particular, not allowing any inbound access from the WAN until a hard password is set, and not allowing any association to the WLAN until a hard encryption key/passphrase is set. However, these are two steps most users don't want to bother with (and more importantly, wil
Re: (Score:2)
If your WIRED Admin system is plugged in to your LOCKED DOWN router how is someone OUTSIDE the LAN going to see it?? All they're going to see is an IP which accepts TCP packets. That could be a computer or it could be a data centre. Or it could be an NSA honeypot. Or a backbone connection via a rotary dial telephone in a small room a hundred feet below Kings Cross Station.
Can you tell if I'm directly connected to the Internet or through a router or through a tethered mobile phone through a proxy? On a wired
Re: (Score:2)
Take the TP-LINK TL-WR841N, currently shown as Amazon's "#1 Best Seller in Computer Routers":
Similarly with the #2 Asus RT-N66, but they don't even tell you how to do it in
Re: (Score:2)
the Virgin Media Netgear routers don't. Neither did their old Terayon modems. They'll both allow admin connections from anywhere by default.
This is a general problem (Score:2)
It does concern ssh-keys, disk-encryption keys, etc. If the people doing this do not understand how encryption works, or are botching initial entropy gathering, the same or pretty close keys can end up on a lot of deployed devices.
In general, this has to do with developer inexperience and (for older ones) incompetence.
Re: (Score:2)
I do not dispute that. But if you are inexperienced, being incompetent is not really your fault IMO, and hence "inexperienced" is the main characteristic.
Re: (Score:2)
/rant
Re: (Score:2)
You are right about software developer. They routinely do not understand crypto, but they also routinely do not understand non-trivial algorithms, data structures and any other CS concept as soon as it is a bit more advanced.
I'm a security person, and generally there is nothing with /dev/urandom. You have to make sure it is properly seeded though, otherwise you are screwed. On the other hand, using /dev/random in an entropy-starved situation is also bad, because it may effectively cause a lock-up. (Yes, thi
Be A Good Geek (Score:3)
Every geek should be a good netNeighbor or netRelative and suggest or guide anyone we care about or don't wish to be fucked over by .gov/corporations/prick wardriving kids and suggest something along the lines of DD WRT or TomatoUSB on their routers that may be ripe targets...maybe even offer to help them secure them, I'm getting pissed at all this crap that is going to get all of them and us reamed. I also like the idea of sticking it to all the evil and/or stupid bastards that let this stuff happen. I let an elderly neighbor of mine know a few months ago by naming my SSID something that might alert their kids or whomever is their "IT expert" (who happened to her daughter) to email a disposable account I set up so I could explain what was up and I spent 20 minutes getting it going for them. I told her to let me know if it needed any fixing with another one-time email anytime remotely. But really, Toastman's TomatoUSB is very stable and needs hardly any tweaking or fixing...probably less than stock firmware. Especially for the crappy Cisco FW that was on it - disaster waiting to happen. They are even on 5ghz N band now, the only other one than me (I'm on both 2.4 & 5). PS InSSIDer is a great wireless app.
Re: (Score:2)
SCRAGGLY NECKBEARD GEEK: "Hi neighbor, I'm a geek and I'd like to help you configure your router so you aren't at risk for man-in-the-middle attacks and other security risks."
NEIGHBOR: "Wut? A guy called on the phone from Microsoft the other day and helped me through all of that."
SCRAGGLY NECKBEARD GEEK: "No no no. That's one of the problems I need to help you with."
NEIGHBOR: "But he already fixed it. Besides, you're weird looking and I don't want you messing with my computer."
SCRAGGLY NECKBEARD GEEK:
Re: (Score:2)
Take a valium, If you read my post I let them email me if they needed help and we aren't strangers.
Re: (Score:2)
I guess I left out a lot of info, but I get the point. The gist of this whole scenario is that my neighbor's daughter is pretty hot so it ended up being all good.
Fantastic! (Score:2, Insightful)
In the meantime, I can't even get my Canon wi-fi printer to connect to my router, but some script kiddy can see all my porn.
What the hell is wrong with software these days?
Re: (Score:2)
Blame the executives. They are 100% the fault of crap products, crap firmware, and crap software.
Why isn't port 22 blocked by default? (Score:2)
What are the actual risks to your network? (Score:2)
OK, this is clearly a bad thing, but I don't think it means that your private LAN is immediately accessible to people all over the world does it? Multiple routers using the same keys means you could be tricked into logging in to someone else's router without knowing, but that would still require some way of directing your traffic to the imposter's device to begin with, such as DNS hijacking.
Knowing someone's keys would also allow you to encrypt/decrypt traffic as that device, facilitating a man-in-the-middl
Re: (Score:2)
No, in this case, knowing the host key would let you pose as the host.
Then again, you don't even generally need the host key to post as the host because 9 times out of 10 nobody actually verifies that the presented host key matches the expected host key.
If the host is unknown, generally they simply assume the key is correct.
If the last stored key and doesn't match the one presented, they generally ignore the error that ssh spews telling you of a potential MITM attack.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:1)
OK, this is clearly a bad thing, but I don't think it means that your private LAN is immediately accessible to people all over the world does it? Multiple routers using the same keys means you could be tricked into logging in to someone else's router without knowing, but that would still require some way of directing your traffic to the impostor's device to begin with, such as DNS hijacking.
Finally, a breath of sanity... Thank you, nuckfuts! A shame this is the bottom thread in the post.. at least when I got here.
/etc/ssh/ssh_host_rsa_key for those with access to a Linux shell, and is never encrypted or password protected. How do I kno
There is a huge difference between a host key and a user key. These consumer devices all share the same host key, which is only used by the client to verify that the host you're connecting to is the host you think you're connecting to. This is the key in
Purchasing a WRT (Score:2)
It's rare (Score:1)
Where can I buy this?