Study Finds Bug Bounty Programs Extremely Cost-Effective

itwbennett writes "U.C. Berkeley researchers have determined that crowdsourcing bug-finding is a far better investment than hiring employees to do the job. Here's the math: Over the last three years, Google has paid $580,000 and Mozilla has paid $570,000 for bugs found in their Chrome and Firefox browsers — and hundreds of vulnerabilities have been fixed. Compare that to the average annual cost of a single North American developer (about $100,000, plus 50% overhead), 'we see that the cost of either of these VRPs (vulnerability reward programs) is comparable to the cost of just one member of the browser security team,' the researchers wrote (PDF). And the crowdsourcing also uncovered more bugs than a single full-time developer could find."

VRPs are the new sweatshops

[OleMoudi]

This is indeed true specially for popular companies with rather mature SecOps that pay minimum wages for vulnerabilities that are indeed hard to find or require a pretty darn good skill level to discover. Some of them even only offer swag in exchange of finding serious threats such as persistent XSS or authentication bypass. They maybe feature the researcher in some blog post to publicly thank him and attract the wannabe crowds.

Having said that, I myself have participated in several of these programs (with varying success) and come to realize that probably Google and Facebook are the only VRPs currently paying reasonable wages for bugs in terms of cost efficiency for the researcher.

On the other hand, some of us just enjoy from time to time trying to find security bugs for fun (maybe because we are huge nerds) so these programs offer a great opportunity to test things and not risking ending up in jail.

Re:dilbert

[CastrTroy]

I wonder if anything like this is going on internally. Let's say a developer at Google knows about a problem. He could either fix it, and get his regular pay, or he could tell his friend about the bug, and split the bounty with his friend who "discovered" the bug. Either way the bug gets fixed. And it probably get's fixed faster this way, since it's now an externally known vulnerability.

Why employees don't find these bugs

[ulatekh]

Because the sort of programmer that's good at finding/fixing these bugs...is not the sort of programmer that the interview process determines would be a "good fit" for the organization.

Ineffective, unfortunately

[gweihir]

This is effective for the low-hanging fruit, i.e. the easy (relatively) to find security-related bugs. For things that require advanced techniques or expensive tools (like Fortify), it fails. Unfortunately, the harder to find bugs are still well within reach of spy agencies of all kind, including a number that is allowed to do industrial espionage (like the US or France).

So while this looks good on the surface, it is really just making the problem worse. The only exception is software that has very low security needs.

For reliability, it is about as ineffective, as only easy to identify bugs will be tracked down.