Invisible Hardware Hacks Allowing Full Remote Access Cost Pennies (wired.com) 84
Long-time Slashdot reader Artem S. Tashkinov quotes Wired:
More than a year has passed since Bloomberg Businessweek grabbed the lapels of the cybersecurity world with a bombshell claim: that Supermicro motherboards in servers used by major tech firms, including Apple and Amazon, had been stealthily implanted with a chip the size of a rice grain that allowed Chinese hackers to spy deep into those networks. Apple, Amazon, and Supermicro all vehemently denied the report. The NSA dismissed it as a false alarm. The Defcon hacker conference awarded it two Pwnie Awards, for "most overhyped bug" and "most epic fail." And no follow-up reporting has yet affirmed its central premise.
But even as the facts of that story remain unconfirmed, the security community has warned that the possibility of the supply chain attacks it describes is all too real. The NSA, after all, has been doing something like it for years, according to the leaks of whistle-blower Edward Snowden. Now researchers have gone further, showing just how easily and cheaply a tiny, tough-to-detect spy chip could be planted in a company's hardware supply chain. And one of them has demonstrated that it doesn't even require a state-sponsored spy agency to pull it off -- just a motivated hardware hacker with the right access and as little as $200 worth of equipment.
But even as the facts of that story remain unconfirmed, the security community has warned that the possibility of the supply chain attacks it describes is all too real. The NSA, after all, has been doing something like it for years, according to the leaks of whistle-blower Edward Snowden. Now researchers have gone further, showing just how easily and cheaply a tiny, tough-to-detect spy chip could be planted in a company's hardware supply chain. And one of them has demonstrated that it doesn't even require a state-sponsored spy agency to pull it off -- just a motivated hardware hacker with the right access and as little as $200 worth of equipment.
Watchdogs was first. (Score:2)
" And one of them has demonstrated that it doesn't even require a state-sponsored spy agency to pull it off—just a motivated hardware hacker with the right access and as little as $200 worth of equipment."
Reminded of that latter mission in Watchdogs 2 where you basically bug a satellite by installing a spy chip.
Maybe not too far-fetched (Score:5, Interesting)
I've seen some weird stuff with contract manufacturers. They want to switch to cheaper parts all the time. If an NSA or Chinese agent offered them a substitute part that cost half as much they would use it as long as the documents looked like they matched.
The feasability of such attack was never questione (Score:5, Interesting)
Tampering with equipment on the supply chain is a standard criminal enterprise for years now. Nothing about state involvement, but simple normal greed. Cash terminals for payment with credit cards are made and assembled in China. There have been many many news stories about those where criminals inserted skimmers and mobile phones to steal credit card data, even in big nationwide chain stores, or especially in them.
There is/was even an all out tech war on both sides: criminals put in skimmers. Recipients weigh the terminals before deploying them and sort out the ones that have the additional weight of the skimmer. They cannot simply open ther terminals since they are sealed for security reasons and if unsealed they aren't usable for transactions anymore due to regulations and anti tampering mechanisms. The criminals react by removing not needed plastic inside the cases so the weight matches again. Then the recipients measure angular momentum ie weight distribution inside the case. etc.
https://twitter.com/arbedout/s... [twitter.com]
Re: (Score:3)
Indeed. The whole thing is an arms race. "Sealed" hardware that the end-customer cannot verify makes the situation worse. The latter was thought out by lawyers, no doubt.
Re: (Score:3)
Even if the hardware weren't sealed you can't tell what a chip is by looking at it. You'd have to test each suspect chip. And which chips you'd need to test would depend on what you suspected them of doing.
Re: (Score:2)
Re: (Score:2)
Indeed. It does raise the bar for the attacker though. Those GSM modules hidden in some terminals are easy to do and pretty cheap as an attack.
The arms race already ended. (Score:2)
Tell me, how would you test for dopant-level hardware backdoors?
Without destroying the chip. As proof that this chip was good, it not proof that the same one from that source will be.
And yes, that is an actual thing now.
So unless you have your own fab, under a mountain, hand-made from scratch by yours truly, and then used to make the chip and entire computer, while you stand guard in a way that does not allow anyone to sneak in, you are fucked.
Oh, and the first time you look away from the resulting computer
Re: (Score:2)
>Tell me, how would you test for dopant-level hardware backdoors?
Tell me you know how to do one. You can read the paper and think "Oh, that looks easy" but the paper wasn't about that and no one actually pulled off a dopant attack. Making masks is not trivial. The paper was on the state space and linear properties of the BIST and how IF you could do such an attack, you could find an entropy reduction attack that BIST wouldn't notice.
Trojan detection has improved in the intervening years and the hypothesi
Re: (Score:2)
Without destroying the chip. As proof that this chip was good, it not proof that the same one from that source will be.
You can always x-ray suspected chips, and compare against known-good ones. We had an issue on some ethernet modules where a specific batch of the cards wouldn't stay connected during testing. Lot number on the specific IC that was causing the issue was shared across the batch. We had some of the parts x-rayed, come to find out the whole lot was counterfeit, and it was plainly obvious from the pics. You'd never tell the difference between the real part and the fake part from a visual inspection. (These
Re: (Score:2)
There is no tool to just "test each suspect chip."
You'd have to X-ray it a bunch of times, and reverse engineer the whole thing. And that only tells you what it is, not what it does. Understanding all possible behaviors of the circuit is a ridiculous task.
To "know what it does" is substantially more work than, for example, pulling an encryption key build into the circuit out.
Re: (Score:2)
Trying to trust the hardware is going about it all wrong.
Security in depth is the only way to be safe. If you are completely reliant on one device being secure you are not secure. Even if the hardware is perfect the software probably isn't.
Re: (Score:2)
Re:The feasability of such attack was never questi (Score:5, Interesting)
It is possible, but improbable. Anyone in a position to execute the attack and with the ability to do so can achieve the same by altering the factory bios/firmware. Cheaper, more reliable and much, much easier.
I've been looking for a good place to comment, this seems like as good a place as any. I've done some work in the contract manufacturing industry, not physically on on the pick-and-place line that assembled the boards, but close enough to know how things worked. (I was testing the boards that came off the line. I drank (probably too much) with the guys who ran the line.) These guys made ~12/hour, nothing beyond a basic background check to get hired, and most didn't have any education beyond high school. Find the right guy, slip him a couple grand and a spool of parts that you want worked into the assembly line, and were done.
To your point, though, altering bios/firmware would be slightly more difficult. As a test technician, I didn't have access to even the binary bootloaders/etc that shipped in our product, and the means to re-program the parts wasn't part of my toolkit. All of the EEPROMS came from the supplier pre-loaded with whatever they needed to have on them. If I suspected it was code that was causing a no-boot (or whatever) I had to requisition a new part, replace it, and disposition the old part. Would have been FAR easier to get a reel of "bad" parts on to the build line (and dispose of the "good" parts than try and re-program them off the line.
Re: (Score:1)
Re: (Score:2)
Who, in their right mind, would plug a computer into their network without first re-imaging it?
Re: (Score:2)
"Who, in their right mind, would plug a computer into their network without first re-imaging it?"
Anyone who doesn't understand the risk, which is 99% of the population. I guarantee every single Fortune 100 company has devices plugged into them daily that are not authorized. They all have policies against that, but that doesn't stop Tim in the warehouse or Suzy in accounting from bringing their laptop in so they can listen to music or whatever reason they think is a good idea.
Now if you'll hold my beer while
Re: (Score:2)
I guarantee every single Fortune 100 company has devices plugged into them daily that are not authorized.
The company I work for certainly goes to great efforts to project itself from unauthorized devices. All company issued devices have a VPN client to access company resources, including the gateway to the Internet. Each Ethernet jack, in every office and cube, is isolated and all non-VPN packets are blocked. If a non-authorized device is connected, it is detected and an IT tech (and a security guard) is immediately dispatched to remove it. There is a guest WiFi, but it requires devices be registered before th
Re:The feasability of such attack was never questi (Score:4, Interesting)
It is possible, but improbable.
Anyone in a position to execute the attack and with the ability to do so can achieve the same by altering the factory bios/firmware. Cheaper, more reliable and much, much easier.
Possible, cheap, can be done by bribing a small number of low-level workers, and gives you the keys to the kingdom.
The idea that it even might not be being done is absurd.
Altering the firmware is a much bigger task, because it has a bunch of security baked in, and firmware is easily verified by the manufacturer. An extra chip that accesses analog signals on the motherboard is nearly impossible to detect, and there is not even any security process by which a motherboard manufacturer could detect or prevent it.
And why would a bios hack be more reliable? It would be version dependent, in addition to the other problems. And you'd have to hire a whole team of really talented engineers. More expensive, less robust, more difficult.
Re: (Score:2)
It might be less realiable by being too obvious and thus people already having defenses against it in place:
From an article (https://trmm.net/modchips) linked from the article this is about:
"Many "high assurance sites" replace their flash chips with their self-programmed versions."
Re: (Score:1)
No one doubted that such an attack on servers would be impossible.
Quite a few people doubted the claims outright.... https://www.servethehome.com/i... [servethehome.com] NSA did similar interceptions years ago, so I cant see how China doing it more recently isn't plausible from the outset.
What can be done to protect against that? (Score:2)
Supply chain attack seem basically impossible to protect against. If you can't trust common parts, what can you trust? It seems obvious that if someone gets access to the machine physically for a long period of time, there is very little you can actually do.
If you can plug yourself on the PCI-e bus, you see all and can intercept all network and GPU traffic.
If you can plug on the DRAM bus, you can see all memory as it goes through.
If you can plug on the sata bus, you see all disks.
It seems that if you can't
Re: (Score:2)
Supply chain attack seem basically impossible to protect against.
It is exceedingly difficult to verify what electronics actually do so your best option is mitigating risks.
If you can't trust common parts, what can you trust?
You almost certainly trust hardware that is made in-house and only uses components sourced from companies located within your own nation while avoiding mailing components.
It seems that if you can't trust the supply chain, the range of attacks that you become vulnerable to is way to large to be able to protect against much.
This is true if you are a small company that doesn't make much money. However, nation states aren't interested in such companies... unless they are used to get to larger companies.
Re:What can be done to protect against that? (Score:4, Insightful)
It really comes down to risk mitigation though; assume people can do anything they want to your systems and data. Define how loss of each system or function would be addressed, and the time required to implement.
I am not sure what recovery strategy actually reduces your risks though— all new equipment is a great opportunity for a new vector.
Right now my biggest worry is the spreadsheets my admin and accountant keep with username/passwords for all the different accounts they need. Need to force a better solution on them.
Re: (Score:2)
Right now my biggest worry is the spreadsheets my admin and accountant keep with username/passwords for all the different accounts they need. Need to force a better solution on them.
Do your due diligence and warn them and keep on warning them but make sure your warnings are in the corporate records system in multiple ways if possible to avoid them being "lost" so you're not used as a scapegoat when a breach occurs. CYA
Strat
Re: (Score:2)
Focus holistically and not on CYA. That is the only way to address the threats. You have to create a honest dialogue if you want a chance at preparing an organization. This is one of the things that can bring IT back into the forefront of the discussion and not as aback-bencher.
Unfortunately (for me), I understand the issues and can communicate impact, but I need other people to be effective at determining solutions. There is too much required to have an effective solution that all players really need but-
Re: (Score:2)
Plugging into the PCI-E bus in that way is nontrivial due to the number of conductors involved. DRAM is even harder for the same reason. SATA is relatively accessible, but even that's not easy to do invisibly. The real danger to those buses (or in general, actually) is during production.
Re: (Score:2)
OTOH, ethernet, USB, wifi, are really easy.
And if OSes store some data in predictable memory locations, you might need less conductors.
Re: (Score:2)
Those are all easy, but all those adapters have to communicate through the bus, and all modern systems have an IOMMU. The attacks that bypass such protections are the really frightening ones. If you could listen in on all traffic on the PCIE or DRAM bus you'd really have something. Just having a link into a device on the bus, much less so.
I don't want to dismiss them as useless, only say that they don't cause the same kind of puckering.
Re: (Score:2)
Supply chain attack seem basically impossible to protect against. If you can't trust common parts, what can you trust?
I'd trust a domestic supplier like Texas Instruments who can provide almost all the ICs, but you'd have to buy large quantities, prepay, and arrange the manufacturing.
You'll never be able to "trust" things you buy off the shelf, and you never should have thought you could.
If you have something really important, maybe it is sometimes better to use an embedded system that is smaller and needs less parts, instead of only buying fancy servers for everything important?
People ask me why I use embedded systems bas
Re: What can be done to protect against that? (Score:2)
If it is a small order, off the shelf parts are more secure. No one is going to track down a random digikey order going to a random destination.
Order 100,000 parts direct from a manufacturer, and everyone in the supply chain knows where the batch is going.
I always interpreted the Bloomberg article as a "not technically accurate" piece of journalism. Nowadays, you can slip exploit code into any random hardware device, including the efi bios, and only the most experienced security people will figure it out.
Re: (Score:2)
No one is going to track down a random digikey order going to a random destination.
That isn't the only attack surface, and it doesn't actually even have to be targeted to be a problem.
They might infect 100,000 parts without knowing who ends up with them.
You're adding additional complications that don't need to be there. That is poor analysis.
Freedom isn't free! (Score:2)
https://raptorcs.com/content/T... [raptorcs.com]
Insert lame joke about putting POWER back into the hands of the end users.
Not a surprise (Score:2)
If he has hardware access, he can also patch the firmware to add a backdoor. More effort, but no soldering required. Of course, if you do this attack before the firmware gets flashed by the manufacturer, then it becomes valid.
He is right about the difficulty-level of this attack: It is not very hard to do. I could do this as well and so could a lot of other people.
Re: (Score:2)
Re: (Score:2)
The uC is so small it is barely visible to the naked eye in existing designs. Even the smallest surface mount packages, most of the volume of the IC is the plastic needed to hold the wires in place and run them to the external connections. There is plenty of room. ESD protection inside an IC is easy to add, it is just a pair of diodes. uCs already have IO protection, you just beef it up a little bit. It is not hard, it is not expensive, and it leaves you in the same situation as you started, where most of t
Re: (Score:2)
While microcontrollers are indeed small (e.g. the Padauk PMS150 die is just a quarter of a square millimetre and nearly half of that is space for the 8 pads), I wonder how they would avoid being found out when the chip is decapped.
Re: (Score:2)
Every firmware flashing process I use involves automatic verification at multiple stages, it would be very very difficult. You'd have to infiltrate not only the installation system used to flash the chips, but also every single development workstation, and the continuous integration system. Otherwise there would be hash problems that would be known.
People who don't understand the technical details keep trying to wave their epeen about firmware hacks being easy, it is really pathetic. When there is a securit
Re: (Score:2)
I am talking about in-circuit reflashing when the thing is finished, obviously. All your automatic verification is entirely worthless against that because the only code that will check any hashes (and BTW, you actually need _signatures_ here, not hashes) is part of the firmware as well.
I think you are pretty clueless as to the subject matter.
Headline vs summary (Score:2)
Cost pennies or as much as $200
Re: (Score:1)
The $200 is for the tools necessary to make the spy device. The device itself can cost pennies.
Re: (Score:2)
It is technically the same, since 200 dollars is 20000 pennies and "pennies" can be any quantity of pennies except zero or one.
Re: (Score:2)
I guess it could be zero... you would still say "zero pennies".
Anything that is "technically" right ... (Score:2)
... is *actually* wrong.
Otherwise it would not have to be only technically right.
Now there's a rule for all you nitpickers and autists out there to memorize!
Re: (Score:1)
That's wrong. It is literally wrong, and it is technically wrong.
Re: (Score:2)
Not to mention time involved, which apparently every one of those people seem to value at exactly zero.
Do these companies not have QA processes? (Score:1)
Anti-China paranoia (Score:1)
Now this new report adds the information that, by soldering a
Re: (Score:1)
Re: Anti-China paranoia (Score:1)
Re: (Score:1)
JavaScript write to specific memory chips (Score:2)
> I'd be still interested, though, in the details of how one could get, from the outside, such a low-level access on a production Amazon server.
High-leve and low-level are representations of the same thing.
Any high-level access is immediately translated into a low-level access.
Remember the rowhammer exploit in JAVASCRIPT?
You can actually address specific rows of memory (offset) via JavaScript, which is running in a sort of virtual machine.
See also any write buffer overflow ever - you can arbitrary code e
Re: (Score:1)
Re: (Score:1)
Re: (Score:1)
Re: (Score:2)
Re: (Score:2)
A grain of quinoa.
Re:Anti-China paranoia (Score:5, Informative)
The reported PoC isn't great but a prior exposition showed a BMC could easily be backdoored using a chip that's in place of a 202 pull-down resistor [trmm.net] which would make it undetectable (much smaller then a grain of rice).
Re: (Score:2)
That story is almost certainly false. If a device the size of an 0202 resistor with that capability existed and had been discovered then they could easily have produced x-rays or a die image to prove it. It would have been an impressive work of engineering too, given that a resistor is a two pin device and somehow with just two pins it was able to power itself and access data busses it wasn't even connected to.
It's also a fairly pointless attack. Why go to all that bother when you can just find some flaws i
Re: (Score:2)
Rice has been farmed in Europe since at least about 344-324 BCE.
without knowing anything
The one true thing you said.
Re: (Score:1)
Given apples subservience to China (Score:1)
Why not just move product to America? (Score:2)
Listen, I love a good conspiracy theory. I really, really do. But it's just highly unlikely this is actually possible to do, at least not without detection. It's one thing to replace this capacitor with that slightly-cheaper capacitor without anyone noticing. It's another thing to create a special secret hardware back door.
Consider it this way, the moment any new product hits the market, hackers around the world -white and black hat- try to tear it apart from every direction. If their was a secret Chinese b
Re: (Score:2)
Listen, I love a good conspiracy theory. I really, really do. But it's just highly unlikely this is actually possible to do, at least not without detection. It's one thing to replace this capacitor with that slightly-cheaper capacitor without anyone noticing. It's another thing to create a special secret hardware back door.
Consider it this way, the moment any new product hits the market, hackers around the world -white and black hat- try to tear it apart from every direction. If their was a secret Chinese back door installed on servers around the world, it would have been found and published by now. Even IF the NSA and the US government and literally EVERY manufacturer were all in on the cover-up AND every professional hacker that identified it was paid off enough to sign NDAs, it would still be found and widely published by amateur hackers.
This is true up to a point. For an iPhone sure. But a $5M storage frame from EMC is not likely to get that level of scrutiny.
Re: (Score:2)
I think it's MORE likely to be closely checked. I would be surprised if nobody would check its boards against documentation. Plus, you can't just put the chip anywhere on the board, it has to be located in a place which allows it to tap into a certain data stream, so the checks should focus on those areas.
Re: (Score:3)
I think it's MORE likely to be closely checked. I would be surprised if nobody would check its boards against documentation. Plus, you can't just put the chip anywhere on the board, it has to be located in a place which allows it to tap into a certain data stream, so the checks should focus on those areas.
That wasn't my point. OP said millions of amateur hackers would find the chip. In a consumer product thats true. Millions of amateur hackers won't be looking at $5M industrial compute.
Re: Why not just move product to America? (Score:2)
The larger point was that it's really hard to believe that it wouldn't be caught. Even if the device was only snuck into 1:1000 devices, it would be very hard to implement without detection. Most notably, it would very likely break something, which would cause the company to have to open the device up and figure it what's going wrong. Again, it's almost impossible to imagine this happening without being detected.
On the other hand IF the design company themselves was Chinese (like a certain designer of 5G eq
Yeah, cause no agent ever infiltrated a foreign co (Score:1)
Sorry, as always, nationalism is a logical dead end.
If a uneducated barefoot Mexican can get to the US and get a job despite not even speaking English ... (Obviously not saying "all Mexicans".) ... then a highly trained experienced operative with a whole state behind him that can fake passports like they were his parents' signatures sure as hell gets wherever the fuck he wants. Hell, even the NSA needs janitors and cleaners and outside contractors with shitty sub-contractors, etc.
Re: (Score:2)
People may do tear-downs commonly, but X-raying and reverse engineering every chip on the board? That isn't frequently done. Even when people do xray an IC, they don't spend to time to audit every possible function, they're just looking for something they expect to be there, and they ignore everything else.
Re: (Score:2)
People may do tear-downs commonly, but X-raying and reverse engineering every chip on the board? That isn't frequently done. Even when people do xray an IC, they don't spend to time to audit every possible function, they're just looking for something they expect to be there, and they ignore everything else.
I don't think the average joe really gets how complicated these chips and devices are. A billion transistors. There are giant portions of the chip that one engineer might even not understand how it works.
Re: (Score:2)
But it's exactly the complication as why this kind of attack is so unlikely in my opinion. You can't just drop in a chip somewhere and expect everything to work alright. Computers today are incredibly complicated. You throw in a little backdoor chip, and you are probably going to break something else. I'm just saying, someone is eventually going to notice and start asking questions.
It might have been nice (Score:3)
It might have been nice if they had mentioned that the Bloomberg allegations were never backed with evidence even though unambiguous evidence should have been easy to come by, and that the pictures in that article were "artistic conceptions" before the very last paragraph after the reader's eyes glazed over.
The new report is true enough, attacks in the form demonstrated are obviously possible, but as others have pointed out, firmware hacks are more fool-proof at this point and certainly more deniable if they're found.
Beyond that, exploits are still available, especially when you add in mis-configurations up to and including connecting management interfaces to the public net with default passwords still in place.
Intel Management Engine (Score:3)
Why bother with a chip when you can build it directly into the CPU
Re: Intel Management Engine (Score:1)
Re: (Score:2)
Point is, most can't.
Building a backdoor into a CPU is something very few can do. Finding out about and exploiting a backdoor in a CPU might be open to a few more, but sitll not that many.
Still, what would you do if you are a minor government agency of some small country or some medium sized NGO, e.g. some organized crime organization? Asking the NSA or the Taiwanese secret service or the KGB about their CPU backdoors, and how you can use them is unlikely to get you what you want. Placing an agent somewhere
If it's hardware (Score:2)
...it's not invisible.
Unless it's part of a legitimate chip. (Score:2)
Good luck grinding open all your chips, *before* considering them trustwurthy and usable, but destroyed.
That is not even the worst kind! (Score:2)
The worst kind is a hardware backdoor created solely via modifications on the dopant level! Impossible to detect, even with a microscope and going layer by layer.
There have been papers about this.
A state agency like the NSA, FSB, Mossad, or the Chinese equivalent, can absolutely pull that off. (Remember, any good spying ageny has to have their own fab, to create trustworthy chips. But those chips are only for them. Not for e.g. infrastructure businesses.)
BIOS did it years ago (Score:2)