Bug Bounties Don't Help If Bugs Never Run Out 235
In 2007 I wrote:
It's virtually certain that if a company like Microsoft offered $1,000 for a new IE exploit, someone would find at least one and report it to them. So the question facing Microsoft when they choose whether to make that offer, is: Would they rather have the $1,000, or the exploit? What responsible company could possibly choose "the $1,000"? Especially considering that if they don't offer the prize, and as a result that particular exploit doesn't get found by a white-hat researcher, someone else will probably find it and sell it on the black market instead?
Well, I still believe that part's true. You can visualize it even more starkly this way: A stranger approaches a company like Microsoft holding two envelopes, one containing $1,000 cash, and the other containing an IE security vulnerability which hasn't yet been discovered in the wild, and asks Microsoft to pick one envelope. It would sound short-sighted and irresponsible for Microsoft to pick the envelope containing the cash — but when Microsoft declines to offer a $1,000 cash prize for vulnerabilities, it's exactly like choosing the envelope with the $1,000. You might argue that it's "not exactly the same" because Microsoft's hypothetical $1,000 prize program would be on offer for bugs which haven't been found yet, but I'd argue that's a distinction without a difference. If Microsoft did offer a $1,000 prize program, it's virtually certain that someone would come forward with a qualifying exploit (and if nobody did, then the program would be moot anyway) — so both scenarios simply describe a choice between $1,000 and finding a new security vulnerability.
But I would argue that there are certain assumptions under which it would make sense not to offer a cash prize program — and, in keeping with my claim that this is equivalent to the envelope-choice problem, under those assumptions it actually would make sense for Microsoft to turn down the envelope containing the vulnerability, and take the cash instead. (When I say it would "make sense", I mean both from a profit-motive standpoint, and for the purposes of protecting the security of their users' computers.)
On Monday night I saw a presentation put on by Seattle's Pacific Science Center "Science Cafe" program, in which Professor Tadayoshi Kohno described how he and his team were able to defeat the security protocols of a car's embedded computer system by finding and exploiting a buffer overflow. That's scary enough, but it was more interesting how his description of the task made it sound like a foregone conclusion that they would find one — you simply sink this many person-hours into the task of looking for a buffer overflow, and eventually you'll find one that can enable a complete takeover of the car. (He confirmed to me afterwards that in his estimation, once the manufacturer had fixed that vulnerability, he figured his same team could have found another one with the same amount of effort.)
More generally, I think it's reasonable to assume that for a given product, there is a certain threshold amount of money/effort/person-hours such that if you throw that much effort at finding a new security vulnerability, you will always find a new one. Suppose you call this the "infinite bug threshold." Obviously the amount of vulnerabilities is not really infinite — you can only do finitely many things to a product in a finite amount of time, after all — but suppose it's so close to infinite as to make no difference, because the manufacturer would never be able to fix all the vulnerabilities that could be found for that amount of effort. I'm sure that $10 million worth of effort, paid to the right people, will always find you a new security vulnerability in the Apache web server; the same is probably true for some dollar number much lower than that, and you could call that the "infinite bug threshold". On the other hand, by definition of that threshold, that means that the amount of vulnerabilities that can be found for any amount of money below that, will be finite and manageable.
(I'm hand-waving over some details here, such as the disputes over whether two different bugs are really considered "distinct," or the fact that once you've found one vulnerability, the cost of finding other closely related vulnerabilities in the same area of the product, often goes way down. But I don't think these complications negate the argument.)
Meanwhile, you have the black-market value of a given type of vulnerability in a given product. This may be the value that you could actually sell it for on the black market, or it may be the maximum amount of effort that a cyber-criminal would invest in finding a new vulnerability. If a cyber-criminal will only start looking for a particular type of vulnerability if they estimate they can find one for less than $50,000 worth of effort, then $50,000 is how much that type of vulnerability is worth to them.
Now consider the case where
infinite bug threshold > black-market value
This is the good case. It means that if the manufacturer offered a prize equal to the black-market value of an exploit, any rational security researcher who found a vulnerability, could sell it to the manufacturer rather than offering it on the black market (assuming they would find the manufacturer more reliable and pleasant to deal with than the Russian cyber-mafia). And we're below the infinite bug threshold, so by definition the manufacturer only has to pay out a finite and manageable number of those prizes, before all such vulnerabilities have been found and fixed. I've made a couple of optimistic assumptions here, such as that the manufacturer would be willing to pay prizes in the first place, and that they could correctly estimate what the black-market value of a bug would be. But at least there's hope.
On other hand, if
infinite bug threshold < black market value
everything gets much worse. This means that no matter how many vulnerabilities you find and fix, by the definition of the infinite bug threshold there will always be another vulnerability that a black-hat will find it worthwhile to discover and exploit.
And that's the pessimistic scenario where it doesn't really matter whether Microsoft chooses the envelope with the vulnerability or the envelope with the $1,000, if the infinite-bug-threshold happens to be below $1,000. (Let's hope it's not that low in practice! But the same analysis would apply to any higher number.) If the black-market-value of a bug is at least $1,000, so that's what the attacker is willing to spend to find one, and if that's above the infinite-bug-threshold, then you might as well not bother fixing any particular bug at that level, because the attacker can always just find another one. It doesn't even matter whether you have a prize program or not; the product is in a permanent state of unfixable vulnerability.
At that point, the only ways to flip the direction of the inequality, to reach the state where "infinite bug threshold > black-market value", would be to decrease the black market value of the vulnerability, or increase the infinite bug threshold for your product. To decrease the black market value, you could implement more severe punishments for cyber-criminals, which makes them less willing to commit risky crimes using a security exploit. Or you could implement greater checks and balances to prevent financial fraud, which decreases the incentives for exploits. But these are society-wide changes that would not be under the control of the software manufacturer. (I'm not sure if there's anything a software company could do by themselves to lower the black-market value of a vulnerability in their product, other than voluntarily decreasing their own market share so that there are fewer computers that can be compromised using their software! Can you think of any other way?)
Raising the infinite bug threshold for the product, on the other hand, may require re-writing the software from scratch, or at least the most vulnerable components, paying stricter attention to security-conscious programming standards. Professor Kohno said after his talk that he believed that if the programmers of the car's embedded systems had followed better security coding practices, such as the principle of least privilege, then his team would not have found vulnerabilities so easily.
I still believe that cash prizes have the potential to achieve security utopia, at least with regard to the particular programs the prizes are offered for — but only where the "infinite bug threshold > black-market value" inequality holds, and only if the company is willing to offer the prizes. If the software is written in a security-conscious manner such that the infinite bug threshold is likely to be higher than the black-market value, and the manufacturer offers a vulnerability prize at least equal to the black-market value, then virtually all vulnerabilities which can be found for less than that much effort, will be reported to the manufacturer and fixed. Once that nirvana has been achieved, for an attacker to find a new exploit, the attacker would have to be (1) irrational (spending an estimated $70,000 to find a vulnerability that is only worth $50,000), and (2) evil beyond merely profit motive (using the bug for $50,000 of ill-gotten gain, instead of simply turning it in to the manufacturer for the same amount of money!). That's not logically impossible, but we would expect it to be rare.
On the other hand, for programs and classes of vulnerabilities where "infinite bug threshold < black-market value", there is literally nothing that can be done to make them secure against an attacker who has time to find the next exploit. You can have multiple lines of defense, like installing anti-virus software on your PC in case a website uses a vulnerability in Internet Explorer to try and infect your computer with a virus. But Kaspersky doesn't make anything for cars.
Bennett Haselton (Score:5, Insightful)
Bennett's Ego (Score:5, Insightful)
" I was an early advocate of companies offering cash prizes to researchers who found security holes in their products, so that the vulnerabilities can be fixed before the bad guys exploited them. I still believe that prize programs can make a product safer under certain conditions. But I had naively overlooked that under an alternate set of assumptions, you might find that not only do cash prizes not make the product any safer, but that nothing makes the product any safer — you might as well not bother fixing certain security holes at all, whether they were found through a prize program or not."
Is the whole premise of this article Bennett having a conversation with himself, talking about his previous points that no one also cared about? I understand slashdot is trying to start doing op-eds by having this guy write. But everything he writes is this long-winded, blowhard, arrogant, ego-massaging nonsense that no one but him cares about. Here he's writing about his previous writings and how his thoughts have changed..in a poorly-written article with no sense of a conclusion..just rambling.
Bennett is also not an information security expert ..he's just a blowhard..can we have someone really involved in information security, like Bruce Schneier, write articles for Slashdot instead of this nonsense?
By this logic... (Score:2, Insightful)
You should be ashamed of your apathy.
Wrong (Score:5, Insightful)
There is no such thing as a "black market value" of a security vulnerability. Both the demand and supply have curves. I.E there are security researchers who would demand say 1 million bucks before selling the bug to the CIA (because they view that action as unethical, illegal and risky careerwise) while they would gladly accept 10.000$ in a responsible disclosure offer. Other color hats would go to the highest bidder. Similarly, there are large transaction costs and information asymmetries, it's not necessarily true that the demand and supply meet or that they can trust each other. A spy agency might rather develop in house (at a much larger cost) then shop around and rise suspicion.
In short, offering a non-trivial sum of money will always increase the costs of the average attacker and might completely shut off the low impact attacks like spam zombification, email harvesting etc., the developers of which can't invest millions in an exploit but would gladly use the free zero day+exploit just made public.
Re:There aren't infinite bugs (Score:5, Insightful)
tldr (Score:5, Insightful)
I think you're working from a few false assumption (Score:5, Insightful)
First, bugs in a given program are not infinite in number. By definition. Because the code itself is finite. Finite code cannot have infinite bugs. Also, due to the nature of code and how it is created, patching one bug usually also takes care of many others. If you have a buffer overflow problem in your input routine, you need only patch it once, in the routine. Not everywhere that routine is being called.
I have spent a few years (closer to decades now) in IT security with a strong focus on code security. In my experience, the effort necessary to find bugs is not linear. Unless the code changes, bug hunting becomes increasingly time consuming. It would be interesting to actually do an analysis of it in depth, but from a gut feeling I would say it's closer to a logarithmic curve. You find a lot of security issues early in development (you have a lot of quick wins easily), issues that can easily even be found in a static analysis (like the mentioned overflow bugs, like unsanitized SQL input and the like), whereas it takes increasingly more time to hunt down elusive security bugs that rely on timing issues or race conditions, especially when interacting with specific other software.
Following this I cannot agree that you cannot "buy away" your bug problems. A sensible approach (ok, I call it sensible 'cause it's mine) is to get the static/easy bugs done in house (good devs can and will actually avoid them altogether), then hire a security analyst or two and THEN offer bug hunting rewards. You will usually only get a few to deal with before it gets quiet.
Exploiting bugs follow the same rules that the rest of the market follows: Finding the bug and developing an exploit for it has to be cheaper than what you hope to reap from exploiting it. If you now offer a reward that's level with the expected gain (adjusted by considerations like the legality of reporting vs. using it and the fact that you needn't actually develop the exploit), you will find someone to squeal. Because there's one more thing working in your favor: Only the first one to squeal gets the money, and unless you know about a bug that I don't know about, chances are that I have a patch done and rolled out before you got your exploit deployed. Your interest to tell me is proportional to how quickly I react to knowing about it. Because the smaller I can make the window in which you can use the bug, the smaller your window gets to make money with the exploit, and the more interesting my offer to pay you to report the bug gets.
Re:Bennett's Ego (Score:5, Insightful)
While I don't share the AC's animosity towards you, the premise of your argument is entirely wrong.
The number of bugs are not limitless, they are very much a finite thing.
The benefit to the company is not limited to closing that single bug. When someone reports one bug, you likely are learning a new method and/or way of thinking in regards to the procedure/module/whatever is involved. One "reported" bug could likely make many dozens or more other bugs readily apparent in your code.
It also teaches your organization how to avoid that bug in the future. How many bugs were in the wild, being used by blackhats for YEARS through multiple iterations of a software package before being caught?
Also, you get to find the mistake in the code and, if you're managing your code correctly, you will know who made the mistake. So you can coach if it was something that should have been caught.
And lastly, it solidifies your place in the market as a leader. People study your code intently, use it more, get more involved. The more people involved, the bigger your talent pool, the more industry respect you have, and as a result the more people will look to you as a company that cares about the stability and long term viability of your product.
Re:Metaphor (Score:5, Insightful)
The notion that you can't have code without these flaws (buffer overruns, dangling pointers, etc) is just asinine. I've worked on significant codebases without any such flaws. You just have to adopt a programming style that doesn't rely on being mistake-free to avoid the issues.
Want to end the danger of buffer overruns? Stop using types where it's even possible.
Want to end the danger of dangling pointers? Managed code doesn't do anything to solve this problem, and is often the worst offender since coders often stop thinking about how memory is recycled, and well-formed objects can hang around in memory for quite some time waiting on the garbage man. So you have to write code where every time you use an object you check that it hasn't been freed, and importantly hasn't been freed and then re-used for the same object! (That happens on purpose in appliance code, where slab allocation is common.)
Heck, for embedded code I simply wouldn't use dynamic allocation at all. All objects created at boot, nothing malloced, nothing freed. Everything fixed sized and only written to with macros that ensure no overruns. I wrote code that way for 5 years - we didn't even use a stack, which is just one more thing that can overflow. That style is too costly for most work, but it's possible, and for life-safety applications it's irresponsible to cheap out.