Forgot your password?
typodupeerror
Security Bug Software

Thinking of Security Vulnerabilities As Defects 158

Posted by timothy
from the doesn't-everyone-already-think-that dept.
SecureThroughObscure writes "ZDNet Zero-Day blogger Nate McFeters has asked the question, 'Should vulnerabilities be treated as defects?' McFeters claims that if vulnerabilities were treated as product defects, companies would have an effective way of forcing developers and business units to focus on security issue. McFeters suggests providing bonuses for good developers, and taking away from bonuses for those that can't keep up. It's an interesting approach that if used, might force companies to take a stronger stance on security related issues."
This discussion has been archived. No new comments can be posted.

Thinking of Security Vulnerabilities As Defects

Comments Filter:
  • by ardle (523599) on Saturday June 28, 2008 @04:53PM (#23984733)
    If they weren't, they would be in the program design.
    • by Anonymous Coward on Saturday June 28, 2008 @04:55PM (#23984747)
      Thread over on the first post. Well done.
    • by kesuki (321456) on Saturday June 28, 2008 @05:13PM (#23984953) Journal

      but then what do you call design features like windows networking telling you if you got the first letter of your password right, even without the rest of the password, and then letting you do that for the next letter, and so on and so on.

      it was a feature of early windows networking, to do just that! like people might 'forget' their password, so they would 'need' a feature that would tell them letter by letter, if they were getting warmer on remembering the password! hackers had a FIELD day with various 'features' of Microsoft products.

      • Re: (Score:3, Insightful)

        by spidr_mnky (1236668)
        If the code works as designed, and that's how it's designed, then that's a mental defect.
        • ... or legal telling you it's a good idea to include flaws in the design, so people can't sue.

          No jokes about legal department being a mental defect, ok?

      • Re: (Score:3, Insightful)

        by lucifuge31337 (529072)

        but then what do you call design features like windows networking telling you if you got the first letter of your password right, even without the rest of the password, and then letting you do that for the next letter, and so on and so on.

        Seem to me that is still a defect. Not in the software itself, but in the software design.

    • Re: (Score:3, Insightful)

      by magisterx (865326)
      Sometimes they are. Remember that there are some times when a vulnerability is a technical exploit of something subtle, and those possibilities are always bugs and defects and should be treated as such. But there are many other times when there is a trade off to get that additional security. There is very often a balancing act between usability and security.

      This example is certainly not ideal as it does not involve software design, but it is analogous and one that I have personally seen happen. Consid
      • Re: (Score:3, Insightful)

        by turbidostato (878842)

        "Was the lack of security and the potential vulnerabilities a defect or a design flaw for the small company?"

        How can somebody twist a simple concept into such a contorted one?

        Defect is nothing more and nothing less than something not working as expected. If something is there by a concious decision is a feature; if something is misdoing, it's a defect. It's as simple as that. Really.

        Now, on defects: if something works as designed, but the designers didn't thought in advance of a given (misdoing) situatio

        • Re: (Score:3, Interesting)

          by Kjella (173770)

          If something is working as designed and the designer doesn't feel some behaviour to be misdoing, then it's a systemic defect (either an unethical seller or an idiot/uniformed buyer).

          Or the designer is an idiot. Been there, had to fight long and hard because first I struggled with coders that said the code is according to spec. Second I had to struggle with the designer that clearly didn't understand what the system was supposed to do, because logically and businesswise it made no sense but somehow he thought the design was right. It was close but he had mixed up two columns of data with similar content and used the wrong one. I finally got through and got it categorized as a defect and

      • Re: (Score:3, Insightful)

        by KGIII (973947)

        As the company grows, this will become unacceptable. But once security is laid on, now you have to make sure that everyone has the right permissions to read the documents they need. It adds layers of overhead and usability. It increases security, but that security comes at the price of tremendous man hours for a select few domain admins and often forcing users to wait for a designated admin when they need basic things like software installed.

        Two things... The first is that therein lies the rub. Permissions are a flaked out aspect. Why? See the second point. The unfortunate reality is that when there are permissions there are also methods to enforce them. Second? You're describing, to the letter, DRM and we can't have a discussion about DRM as a viable tool. The reality is that CHMOD is a basic form of DRM.

        So security, as you state, is a matter of usability vs. security. No, not any, online computer is secure. To deny that is just blind. If it

        • Re: (Score:3, Informative)

          by Nullav (1053766)

          The reality is that CHMOD is a basic form of DRM.

          I must say, my keyboard would be absolutely soaked if I was drinking something right now. My maildirs and private keys? Not yours and I'm not setting public read. httpd.conf? Sure, I'll set public read and owner write so you can look at it and even copy it for non-disruptive editing for whatever reasons. It's also worth noting that a lot of 'users' don't even have people behind them on a typical home system.
          It would be DRM if RO access meant you couldn't edit the copy. Moreover, all current incarnations of

          • Re: (Score:3, Insightful)

            by KGIII (973947)
            In very basic form one could call passwords DRM, albeit an aspect of digital rights management and not a whole solution. DRM and "copy protection" overlap and the term DRM has some really awful connotations. There is more to DRM than just copy protection however.

            The current restrictions (and methods) of the widely used copy protection are barbaric and disturb me greatly. The way that the term DRM has been used has pretty much tarnished its reputation forever and some people have come to the conclusion th
    • It's the other way round Dammit!

      The vast majority of security vulnerabilities are merely exploits of defects!

      How do you hack a system? Find a bug, that's usually pretty easy....

      Then you have the system operating already, "not as the designer intended" and you're more than halfway there...just add a bit of creativity and malice aforethought.

    • Re: (Score:3, Insightful)

      by TubeSteak (669689)

      Of course vulnerabilities are defects

      If they were defects in the eyes of USA law, they'd be considered a material defect or design defect under existing contract or product liability law respectively.

      There are a few possible outcomes from such a scenario
      A) Nobody writes software anymore because they'd be sued into oblivion
      B) Prices go up because coders & companies have to buy software defect insurance
      C) Prices go up because companies spend more in labor to produce defect free code
      D) The EULA lists every possible failure scenario (plausible

      • Re: (Score:3, Insightful)

        by turbidostato (878842)

        If problems on cars were defects in the eyes of USA law, they'd be considered a material defect or design defect under existing contract or product liability law respectively.
        There are a few possible outcomes from such a scenario
        A) Nobody builds cars anymore because they'd be sued into oblivion
        B) Car prices go up because builders & sellers have to buy car defect insurance
        C) Prices go up because companies spend more in labor to produce defect free cars
        D) The EULA lists every possible failure scenario (pl

        • Re: (Score:3, Insightful)

          by tchuladdiass (174342)

          Except, when I bought a new car, there was a small defect in the paint job -- a nearly unnoticeable paint bubble. I'm sure that every car that comes off the lot has a blemish somewhere. Doesn't cause the car to crash, and life goes on. Same thing ain't true with software -- that same "blemish" could easily be turned around and allow someone to break into the software.

          The only way we could get comparable results with software v.s. physical objects is if computer systems develop the ability to withstand a c

      • by jc42 (318812)

        D) The EULA lists every possible failure scenario (plausible or not) in the interests of full disclosure and business continues as usual

        Actually, IBM discovered all this decades ago. One of the ongoing jokes in IBM systems, dating at least from the 1960s, is about a customer documenting and reporting a bug, only to be pointed at page 485 in volume 17 of the documentation, where exactly that behavior is documented. And since it's documented in the manual, "It's not a bug; it's a feature" and won't be fixed

    • by slarrg (931336)

      Sadly, most manages only recognize a defect if it i something likely to be noticed by their customers. Unless your customers are hackers, they'll never discover the problem so there's no need to test for them.

      The biggest problem, in my opinion, is that management as a whole is generally "success based" which means failures a swept under the carpet or otherwise ignored. This means the company as a whole never leans from its mistakes or avoids them in the future. Everyone runs around talking about how great e

    • If they weren't, they would be in the program design.

      "It's not a bug; it's a feature."

      That old joke isn't always a joke. Some vulnerabilities are built in, because they were put there intentionally by the designers and/or developers.

      This is one of the primary arguments behind Open Source: If you can't get at the code and study it, you don't have any idea what "special features" might be hidden in there. The people who built it could have provided all sorts of back doors for exploitation by themselves or b

  • No (Score:3, Funny)

    by blargfellow (948805) on Saturday June 28, 2008 @04:56PM (#23984773)
    Of course they aren't defects, they should be treated as features!
    • Since we already call "bugs" defects ... umm ,,, on second thought ...

      Seriously, security faults, "bugs", "features" that aren't, etc., are defects. They're mistakes. Errors. They didn't just crawl in there accidently, no matter how much we strive to give them independent life by calling them "bugs" or "random behaviour". Not on deterministic systems like computers.

    • by devloop (983641) on Saturday June 28, 2008 @08:00PM (#23986059)

      The elephant in the room is that primitive, unsafe tools endlessly perpetuate these problems. Buffer over/under flows are not difficult problems to solve at language design level, but the common tools We currently use to create applications make diagnosing them and fixing them rocket science. C and C++ (and other lesser used languages) are notorious for being hostile to catching these problems at compile time or debugging them when they happen later. In most cases, the problem goes "unnoticed" affecting unrelated functions in the application downstream and incorrect behavior or crashes happen at a later time when they can no longer be traced back to the original cause.

      For kicks check http://en.wikipedia.org/wiki/Buffer_overflow#Choice_of_programming_language [wikipedia.org]
      Google search on http://www.google.com/search?hl=en&q=%2Bbuffer+%2B%22overflow%7Coverrun%7Cunderrun%22&btnG=Search [google.com]

      • by mrsbrisby (60242)

        What a load of crap. FORTH is as primitive and unsafe as they come and they don't have to deal with over/under flows the way incompetent C++ programmers have to.

        If users knew that there was a corrolation between competence and bug-free and problem-free code, they'd stop accepting crap. Instead, there are a lot of programmers- some good, and some second rate, defending bugs and security problems as mere accidents at worst- the kind everyone makes.

        Instead, we have this culture that has convinced the user to a

        • by TheLink (130905)
          "What a load of crap. FORTH is as primitive and unsafe as they come and they don't have to deal with over/under flows the way incompetent C++ programmers have to."

          One reason is because people hardly ever bother to exploit FORTH programs or if they do they don't usually make so much noise about it. I crashed a forth webserver on my first try. But guess what, who cares. If you could crash Apache or IIS now at your first try, it's worth $$$.

          But yeah, C++ is overused. Crappy programmers like me should stay away
  • They'd get sued out of existence for shipping defective products. I can't see any company agreeing to label its products in such a way.

    • by nine-times (778537) <nine.times@gmail.com> on Saturday June 28, 2008 @05:32PM (#23985109) Homepage

      The article (at least in my reading) isn't saying that they should be held legally accountable as selling a defective product. Instead it's about how companies should approach a bug report of a vulnerability. He's saying, when someone reports a vulnerability, consider it something that you're obligated to fix, not as a feature request.

      But then, I think most people do. It seems like he hit a bad support person.

      I ran into a similar problem once with Citrix, actually. Their software was relying on some library that it assumed was installed, even though recent Linux releases (at the time) had stopped using that library. The result was that the software didn't work until you tracked down that library, dropped it in the right place, and then it worked fine.

      So I went to their website to give feedback, just to let them know. I mean, I'm sure they would have figured it out, but I thought, "may as well give them a heads up" because it was happening on major linux distros almost a year after their release. Citrix had released several updates to their software, and never fixed this problem. I couldn't find anyplace on their website to provide feedback, except for a form to give feedback about the website itself.

      So I wrote up a little feedback, trying to explain the situation briefly (i.e. "I wanted to drop some feedback to your development team letting them know there's a problem, how to fix it, but I can't find any contact information on your website. Is there any way to submit this sort of feedback). The response came back quickly, "If you want support, you'll have to pay for a support contract."

      I wrote back again, trying to explain, "No, see, I'm not looking for help, I'm trying to be helpful. I'm letting you know that there's a problem I already know how to fix. I was just wondering if there was a place to submit this sort of feedback."

      Again, the response came in, "I'm sorry sir, but if you want us to help you with this problem, you'll need to buy our support contract."

      At that point, I gave up.

      • "when someone reports a vulnerability, consider it something that you're obligated to fix, not as a feature request."

        Why any company in the world would do something like that!!!???

        Oh, yes, only if they are legally or financially forced, that's how.

        Or do you think any company in the world would rise their production costs for no benefit?

        • What about customer satisfaction, and the financial detriment of losing your customers?
          • "What about customer satisfaction"

            bollocks

            "and the financial detriment of losing your customers"

            Which part of "legally or financially forced" didn't you understand?

      • by dcam (615646)

        Wow. You'd never expect a significant software company to do anything like that.

        Wait... That process is *exactly* the Microsoft process.

    • by arth1 (260657)

      Very true. Also, the bonus system won't work for two reasons:
      1: Security bugs tend to be discovered years down the road.
      2: A lot of companies would have to start paying bonuses to programmers, and not just to S&M. That'd never float.

    • It's not so much a legal question as a business problem. Vendors are obligated to fix their bug and security holes because they are problems for customers.

      I would just leave the courts out of it and let the free market decide. In this particular case, a good response would be: (A) I have reported this to management and we will seek other vendors to suit our needs. (B) I am obligated to report this security hole to the usual vulnerability lists, because since I found it obviously somebody else could too.

      I

  • There are companies that DON'T treat security vulnerabilities as defects??

  • by EdwinFreed (1084059) on Saturday June 28, 2008 @05:07PM (#23984883)
    We've treated potential vulnerabilities in our products, even extremely minor ones, as defects for over two decades now. And we have always given them very high priority.

    To the best of our knowledge we've never had a remote exploit vulnerability, but even so we've gone so far as to scrap thousands of freshly pressed CDs a day before releasing them because I spotted a way to get root access through a tricky bit of business with shared libraries. (And that was for something spotted internally - no customer ever reported it.)

    The real question isn't whether to treat security vulnerabilities as a defect - of course you do - but - somewhat paradoxically - whether or not to treat them as security vulnerabilities. We were acquired some time ago and have now adopted (and adapted to) various more complex procedures typical of a large company. There's this little box you're supposed to check in our current bug reporting system that says "this is a security vulnerability". The problem is that checking that box fires up a whole lot of extra process that rarely helps and can actually hinder prompt resolution of the problem and getting the fix into customer's hands.
    • by HalAtWork (926717)
      Why is this even a question?

      Because it leads to the line of thought that one might hold a company liable for these defects, the repercussions of which would imply large changes both in the approach of software development, in the support of existing software, and in regard of the current state of widely used software.

      The real question isn't whether to treat security vulnerabilities as a defect - of course you do - but - somewhat paradoxically - whether or not to treat them as security vulnerabilities.
    • by Anonymous Coward on Saturday June 28, 2008 @07:02PM (#23985679)

      We've treated potential vulnerabilities in our products, even extremely minor ones, as defects for over two decades now. And we have always given them very high priority.

      Go ask your corporate legal counsel what would happen if the law treated software vulnerabilities as design defects.

      • by Kjella (173770)

        Go ask your corporate legal counsel what would happen if the law treated software vulnerabilities as design defects.

        Umm... nothing? Last I checked pretty much every EULA disclaimed any liability for defective design along with everything else.

  • Here's a summary of the article:

    Vendors should make their developers work more on security (via money)

    Meh. How often are the developers free to choose which parts and aspects of their companies' software they want to work on? As long as the companies tell their developers what to work on, here's the easy way: tell the devs to work on security testing and fixing. Letting the developers manage themselves is not going to sit well with management types, so you almost always have developers who are told what to work on.

    Also, if anything external to the way you work (i.e. the pr

    • Everybody, please laugh at the subject of my post which has no relation to its contents ;)

      What I meant to write when I wrote the subject is that, from the point of view external to the organization developing insecure software, you are, according to the wisdom of the /. masses, supposed to vote with your wallet.

      Yet, how's that expected to take place? To apply some of Schneier's observations, you have multiple parties, each with their own security agenda; the sysadmin might want the most secure option becau

    • Re: (Score:3, Insightful)

      Also, if anything external to the way you work (i.e. the promise of more money) can make you work better, you're slacking off in your daily work: why don't you deliver peak performance without the extra money?

      There's two ways to look at performance vs. compensation. Employees, ideally (at least from the employer's viewpoint) will look at it the way you do: you're being paid to do your best, so you should need no extra incentive to do so. Project management, on the other hand, should be pragmatic about it. Sure, employees SHOULD do their best no matter what, but maybe cash incentives can add motivation. If that is found to be the case, a good manager will choose results over principles.

      • If that is found to be the case

        I'm waiting for you to google up someone to cite on a verdict ;)

        Seriously, that would be interesting to know: how well does cash make people work better, and how does it depend on the "price structure"...

  • by maz2331 (1104901)

    Yes, they are defects.

    That said, there's one caveat to it: WHO'S defect is it? If the defect comes from a licensed library, OS issue, or other hidden cause, then that defect belongs to the source author/vendor.

    Sometimes you end up having to work around someone else's crap.

    • by Sique (173459)

      But that's no different to any other product. Buy defective capacitors from a manufacturer for your product, and your product blows up because of them: It's your responsibility to recall the product and replace the capacitos.

  • Intentional misuse (Score:3, Insightful)

    by asc99c (938635) on Saturday June 28, 2008 @05:13PM (#23984955) Homepage

    If a user was intentionally mis-using software I had written, I wouldn't consider it a bug. Although a vulnerability is generally mis-use by someone other than the owner of that piece of software, I'd still have to conclude it's not a bug. If I'd built a car, I would be more than a little annoyed if I got the blame that someone had broken into it and run someone else over with it.

    I think it needs to be left to the market to decide what is acceptably secure software. Many Ford cars from the early 90s had locks that were far too easy to break - just stick a screwdriver in and it opens - even did it myself when I locked the keys in the car once. They got a bad reputation, and Ford improved the security to a level the market was happier with.

    The market in software doesn't work quite as well as for cars unfortunately, but that's another issue.

    • Re: (Score:3, Insightful)

      by John Hasler (414242)
      You might want to read up on merchantability [findlaw.com], implied warranty [wikipedia.org], and fitness for use. These legal concepts apply to cars and other tangible goods but not to software. They should.
      • These legal concepts apply to cars and other tangible goods but not to software. They should.

        Would that not destroy hobby software, and much of OSS and Free Software along with it?

        • No. The UCC is about commerce, and consequential damages can be and usually are disclaimed. If I make a free gift to you of a copy of my software and it turns out to be buggy how can you sue me for selling something defective? On the other hand if you sell a copy of my (GPL) software to someone else they should be able to sue you if it proves to be defective.
      • by asc99c (938635)

        They do apply to software. EULAs in most places at least are pretty much unenforceable nonsense. If software doesn't do it's job, you can return it for a refund.

        Most software vulnerabilities are caused by inputting in random or intentionally malicious data. Continuing the car analogy, I can go a pour sugar (or worse) into a car's fuel tank and it will break the car (or explode and kill someone depending what I used). That isn't a bug. If I do this to my own car I'm completely to blame for my actions, a

        • > They do apply to software. EULAs in most places at least are pretty much unenforceable
          > nonsense. If software doesn't do it's job, you can return it for a refund.

          I'm not talking about EULAs. I agree that they are largely unenforceable. As far as I know in the US what is warranted is the tangible goods: the physical copy. If the CD is scratched you can insist that tney make you whole by replacing it with a good one or refunding your money but the software can be buggy as hell as long as the copy a

    • > Although a vulnerability is generally mis-use by someone other than the owner of that piece of software, I'd still have to conclude it's not a bug

      So if you would write e.g. an FTP-server and user would setup that anonymous users have no write permission (read only). And then someone would as anonymous user send invalid command, which would cause a buffer overflow and corrupt the whole file system. You would not consider that a bug, because someone had just misused it by using a non-standard command?

      • by asc99c (938635)

        I suggest leaving it to the market to decide precisely because of this sort of issue - an FTP server designed to be openly accessible shouldn't have this kind of vulnerability. Users wouldn't accept this sort of problem, so yes it should be treated as a bug.

        Even so, sending a few Kb of malicious data down to the server is hacking and the legal 'blame' for this should be with the hacker not the developer.

        But not all software is subject to this requirement. Probably the majority of applications developed ru

        • by mrsbrisby (60242)

          I suggest leaving it to the market to decide precisely because of this sort of issue

          Justify that.

          This issue is presently a public-relations problem with public-relations solution. So far, the strategy is to convince the user that they did something wrong and need more software. This works because the user tends to be under-educated, and because both competent and incompetent programmers are lock-step on message: programming is hard, you can't do it, and everyone makes mistakes.

          I reject this premise, so I re

  • Bonuses based on code "quality" and or quantity has been tried before. It simply does not work. Counting the number of security issues some guy makes does not necessarily mean he or she is a good developer. It depends on several factors:
    - difficult or simple code
    - amount of code written
    - where in the application does the developer code? Some code is more likely to result in security issues.
    - how well that person knows the code
    • by dave420 (699308)
      Unfortunately, from my experience of bosses, that will quickly change to "Penalties for Bad Developers", which will destroy morale and make most of their developers, including their good ones, walk. Many bosses seem to prefer to punish undesired behaviour than reward the positive.
  • Absolutely (Score:4, Interesting)

    by cryptoluddite (658517) on Saturday June 28, 2008 @05:20PM (#23985009)

    As a software developer I spend about a quarter of my time rewriting code that one of our other developers writes. His code is like a rhesus monkey came in and started flinging shit all around. He 'keeps up' with the other developers because he does the absolute minimum, never ever rewrites code to fix problems, cuts and pastes, etc. One time he cut and paste a second copy of a 200 line function so he could change one loop constant.

    There's lots of developers like him, and they and/or their company should get sued over that code. At least when it is from negligence. Or there should be a licensing requirement.... something so that the people who are irresponsible or incompetent are held responsible for it.

    Pretty much the only thing that makes programming not worth while is that people can hack out a 80% working code, get credit for it, then move on and leave all the crap for competent developers to fix. I would gladly pay a malpractice insurance fee if it means less having to deal with bullshit code.

    • by Dahamma (304068)

      Tell me you are not so desperate for a job that you would spend 25% of your time fixing a coworker's mistakes? Bring it up with your manager, have his faults explained to him, document it over a month or so, and fire him if he doesn't get better. If the company doesn't agree to hold your coworkers accountable for their work, leave. There are plenty of other companies that do.

      In fact, if you haven't already tried the above suggestions, the problem is almost as much your fault as anyone else's.

      • Tell me you are not so desperate for a job that you would spend 25% of your time fixing a coworker's mistakes? Bring it up with your manager, have his faults explained to him, document it over a month or so, and fire him if he doesn't get better.

        Because the other 75% of the time I get to code awesome stuff. He's friends with the CEO and went to college with a good portion of the company, so licensing or liability would help -- but your suggestions would not, Mr. Know-It-All.

  • No matter what tricks you try to use to get your developers and others to focus on security issues, it is going to cost money. Denying bonuses won't help because your developers can always leave and work for a competitor who doesn't play that game. And you'll still have to fix those vulnerabilities.

    The solution is to ask your customers. Given the choice between a more secure, more expensive product and a less secure, less expensive product, which will your customers choose? Once you have the answer to that,

  • I dare say that MOST developers do NOT know how to combat the most heinous, or even the most common hAx0Rz tricks. I've never triggered a buffer overflow in my life (on purpose).

    To make software bullet proof requires the developers to have the skills of the hackers and malware writers, and the resources, the secret handshakes, the underground culture.

    Yer talking one of two scenarios here:
    1. PhD in Software security *by the time you got it, it'd be obsolete.
    2. sending "nice" developers to infiltrate the blac
    • by dvice_null (981029) on Saturday June 28, 2008 @05:52PM (#23985247)

      Actually you really need just one person in the company with "haxor" skills to test the security of the products that others make. A single person can very quickly find a lot of common holes. That person doesn't need to a developer. He/She can be there just for testing or even just for supervising others that make the testing, to make sure that they test for security vulnerabilities also.

      • by Nerdfest (867930)
        Security is the responsibility of all developers and all _designers_. A lot of problems creep in very early, so it's something people just need to learn. Luckily, there are courses.

        There are also tools, both OSS and commercial to analyze source code. All of these things help, and everyone should be using at least some of the measures available.
  • by awitod (453754) on Saturday June 28, 2008 @05:25PM (#23985059)

    "The problem of course is I'm saying how the companies should handle them, and I have no authority at any of these places, save people actually valuing my ideas. Personally, I've done some development in the past, and there was the concept of defects. Your bonus would depend on how many defects were in your application at delivery time. These were feature-based defects, but shouldn't vulnerabilities be considered defects as well?"

    So, the author freely admits he is neither a developer or a manager. If he was a developer he'd know that these are defects and everyone treats them as such.

    If he was a manager, he'd know that one of the surest ways to wreck a good shop is to start doing comp based on defects. Here is what invariably (in my experience) happens when a shop includes defect counts in there comp plans.

    1. Relationships between Dev, QA, Product Management and Operations get worse because the terms 'defect' and 'bug' become toxic. In reality these things always exist in software. The last thing you want to do is create barriers to dealing with them. Making the acknowledgment of a defect cost someone money means you will have arguments over every one of them unless they cause an out right crash.

    2. Culture becomes overly risk-averse - No one wants to take on difficult problems or blaze new territory. The smartest people will naturally pick the easiest work to minimize the risk of defects.

    3. Over-dependence on consultants - More CYA behavior. If it's too complex people will outsource to keep the defects away. This is a very bad thing if the nasty problems are because of business and not technical challenges. Now the people who know enough about the problem domain to understand the risk are hiring proxies who know nothing to avoid responsibility for 'defects'.

    • by jesterzog (189797)

      Here is what invariably (in my experience) happens when a shop includes defect counts in there comp plans.

      So far I've been fortunate enough not to have worked in such an environment. If I did, though, I could imagine that there would be a lot of motivation to fix the minor, easy-to-address defects such as "text is a slightly incorrect shade of gray", and ignoring the larger, more complicated defects that take longer to solve and might be orders of magnitude more serious for the end product. I guess this i

  • by SamP2 (1097897) on Saturday June 28, 2008 @05:29PM (#23985087)

    ...the nature of the security issue.

    A defect, by definition, is an unintended behavior of a program. Something was designed to work, but for whatever reason, doesn't. Compare this to a lack of a feature, which means that something doesn't work because there was never the intention for it to work in the first place.

    A buffer overflow or SQL injection related issue is almost definitely a defect, since there is a dedicated, designed parsing mechanism to process input, and if some types of inputs are not processed as intended, it is a defect of the software.

    On the other hand, for example, a security issue arising from plaintext transmission of sensitive data over the net, is not necessarily a defect. If the site in question was never designed to use SSL or another encryption mechanism, then it's a lack of a feature. If the site in question is an online banking site, then it is a blatantly poor and inexcusable design shortcoming, but nontheless, not a defect. (Of course, if the site DID intend SSL to work properly, but for whatever reason there is a hole allowing to crack or circumvent the encryption, then it IS a defect).

    Besides, assigning a "defect" status to a security issue is not necessarily useful for it's own sake. The understanding is that a responsible company should treat a security issue with much higher priority than a non-security related one, defect or not (compare "we released an emergency hotfix to download" to a "we'll ship the patch in the next release cycle"). Saying a security issue is a defect, is like saying that a cardiac arrest is "organ numbness" - true, but not very useful.

    • You are correct. A security vulnerability is a defect if and only if the defect represents a failure with respect to a requirement. In fact, security is but one dimension of reliability, and so if the vendor is responsible for reliability, then the vendor should also be responsible for security. If there is no requirement for security, then it is not a defect. One of the posters mentioned that software that is sold for the intended purpose of general use on the Internet carries with it some implied requi
  • The EULA's exempt holding the software maker for defects in their products anyway. Be them security holes or total meltdowns.

    • by Khyber (864651)

      EULAs are unenforcable in California, which is why VMware, among other companies, has in section 8 of their EULA "This EULA will be governed by California law."

      So us Californians are safe. No EULA can go against a law.

      • by nurb432 (527695)

        In that case, id would forbid sales of my software in that state without written pre-sale contract.

        Just to protect myself.

        • by Khyber (864651)

          Too bad most major software companies have HQ here in California. They'd be SOL.

  • by stephanruby (542433) on Saturday June 28, 2008 @05:32PM (#23985107)

    ZDNet Zero-Day blogger Nate McFeters has asked the question, 'Should vulnerabilities be treated as defects?' McFeters claims that if vulnerabilities were treated as product defects, companies would have an effective way of forcing developers and business units to focus on security issue. McFeters suggests providing bonuses for good developers, and taking away from bonuses for those that can't keep up. It's an interesting approach that if used, might force companies to take a stronger stance on security related issues.

    When I think of defects and total quality management, I think of Edward Demings [wikipedia.org].

    Edward Demings saw the problem of defects as a systems issue, not an individual performance issue. And his theory was that paying someone based on performance would have the unintended consequence of increasing the number of defects, not decrease them (Here is the list of Deming's 14 principles with my emphasis added in bold).

    Deming offered fourteen key principles for management for transforming business effectiveness. The points were first presented in his book Out of the Crisis (p. 23-24)[19].

    1. Create constancy of purpose toward improvement of product and service, with the aim to become competitive and stay in business, and to provide jobs.
    2. Adopt the new philosophy. We are in a new economic age. Western management must awaken to the challenge, must learn their responsibilities, and take on leadership for change.
    3. Cease dependence on inspection to achieve quality. Eliminate the need for inspection on a mass basis by building quality into the product in the first place.
    4. End the practice of awarding business on the basis of price tag. Instead, minimize total cost. Move towards a single supplier for any one item, on a long-term relationship of loyalty and trust.
    5. Improve constantly and forever the system of production and service, to improve quality and productivity, and thus constantly decrease cost.
    6. Institute training on the job.
    7. Institute leadership (see Point 12 and Ch. 8 of "Out of the Crisis"). The aim of supervision should be to help people and machines and gadgets to do a better job. Supervision of management is in need of overhaul, as well as supervision of production workers.
    8. Drive out fear, so that everyone may work effectively for the company. (See Ch. 3 of "Out of the Crisis")
    9. Break down barriers between departments. People in research, design, sales, and production must work as a team, to foresee problems of production and in use that may be encountered with the product or service.
    10. Eliminate slogans, exhortations, and targets for the work force asking for zero defects and new levels of productivity. Such exhortations only create adversarial relationships, as the bulk of the causes of low quality and low productivity belong to the system and thus lie beyond the power of the work force.
    11. a. Eliminate work standards (quotas) on the factory floor. Substitute leadership.
      b. Eliminate management by objective. Eliminate management by numbers, numerical goals. Substitute leadership.
    12. a. Remove barriers that rob the hourly worker of his right to pride of workmanship. The responsibility of supervisors must be changed from sheer numbers to quality.
      b. Remove barriers that rob people in management and in engineering of their right to pride of workmanship. This means, inter alia, abolishment of the annual or merit rating and of management by objective (See CH. 3 of "Out of the Crisis").
    13. Institute a vigorous program of education and self-improvement.
    14. Put everybody in the company to work to accomplish the transformation. The transformation is everybody's work.
    • by tuomoks (246421)

      As would I, not that it matters.. You take, point by point, Edward Demings list and compare it to any software company practices! What do you get - a total mismatch, because it would break the management structure (and lower the next quarter profits - maybe?)

      Part of blame goes to the consumer! There was a time when I was able to get an answer and often a fix in 12 hours from vendor. And they better - otherwise our ($1 million in 70's) monthly license payments somehow created the same problems, got lost, wha

  • Security Tracker [securitytracker.com], best source of information on security vulnerabilities that I know of.
  • companies would have an effective way of forcing developers and business units to focus on security issue

    I don't think developers need to be 'forced' because generally they understand the importance of making their software secure and if they don't do it then it's usually due to external pressures such as unreasonable deadlines and management not wanting them to spend time on something that does not have tangible results.

    In short, the problem with a lot of companies is that management doesn't value secu

    • Management are trying to maximise profit, and typically don't care anywhere near as much as developers whether the job is done 'right'.

      The problem is most buyers of software are way more interested in shiny bits and pieces than the security. If (more) people weren't willing to put up with insecure software, managers would be asking the developers to work more on the security aspects of the application.

  • "McFeters suggests providing bonuses for good developers, and taking away from bonuses for those that can't keep up. It's an interesting approach that if used, might force companies to take a stronger stance on security related issues."

    They already are considered defects. The only thing that might change is the prioritization of defects. This topic is ridiculous.

    Bonuses? Are you suggesting bonuses for finding defects? In 20 years I've worked at only once place that awarded bonuses for resolving defect
  • by darkPHi3er (215047) on Saturday June 28, 2008 @05:49PM (#23985223) Homepage

    In the RW, i'd suggset that we should consider the following;

    You are Programmer Sian (notice the trendily androgynous name), you work for a gigantic software company, or conglomerate or industrial that does all its own major development inside, you are potentially confronted with;

    1. Antiquated Developer Tools -- in general, the larger the development environment, unless you're Disgesting Your Own Pet's Nutrition, you are very likely to be using multi-year and/or multi-generation old development platforms and tools.

    The question here is then, how can you effectively hold poor Sean accountable for vulnerablities, that are intrinsic to many older tools?

    Who's more accounatable here? Sian or the managers who make the procurement decisions?

    2. "Science Fiction" Application Programming Interfaces - depending on whether you are programming on a well-established product or not, if you are, Poor Sian is probably stuck with API's that were developed many years before and have been the victim of Design Creep, and its, Lunatic Cousin, Design Implosion.

    In many instances the APIs, while they may once have had a large degree of Paradigmatic and Philosophic Design Integrity, as their initial Designers and Implementers have moved on to other; products, companies or, Worst Case, Inpatient Mental Health Facilities. Many New Designers have come in to add "Their Own Programming Uniqueness" to the APIs, frequently rendering the API's a jumble of radically different approaches to similar algorithms.

    Should Sian be subjected to having their pay docked because 9/10 Functions implement a Library Call one way, and some "Johnny-Come-Lately" API function implements a similar looking, but substantially different in output function?

    Shouldn't the API Designers/Architects be held more responsible for this one?

    3. PHB Stupidity - As QC forwards endless application/OS defect notices to the Development/Maintenance Team, these defects are reviewed by the Team Managers and Supervisors. It's understandable, given the 11 hours per day of Absolutely Vital Meetings that most PHBs love to, i mean are forced to attend, that Defect Prioritization will suffer.

    Sian can't choose what defects to repair, and in what order to repair them.

    This is a management function, and one, in my experience, that Mgt usually jealously and zealously guards.

    SOOOO, it's been the case in every Development project that i've worked on and know about, that PHB's have a well-understood tendency to prioritize Defect repair, according to external pressures, especially from Sales and Marketing.

    Sales and Marketing organizations are usually likely to priortize according to their immediate impact on quarterly projections.

    Vulnerablities are only likely to affect quarterly results when they are CATASTROPHIC defects, i.e. App or OS Killers. Otherwise, the majority of vulnerablities, which are usually well submerged in the Defect numbers, tend to get shoved aside for the higher priority defects that S&M believe impact immediate sales.

    There are numerous other considerations here; including Contract Programmers, Legacy Compatability (ask the Vista Team about that one), Vendor Driver Teams that don't even know what to do with a new code base, etc, etc..

    But it seems to me, that, while financial incentives CAN BE, useful as a Mgt tool for improving product quality, they should, to be even-handed, applied across the entire product team, with specific ***POSITIVE*** incentives used to take care limited, high priority problems across the product line.

    There's already a tendency to "blame the programmer", and my Best Guess is, that any attempt to lay the responsiblity for vulnerabillites, THAT AREN'T CLEARLY THE RESULT OF SLOPPY/POOR/INCOMPETENT CODE PRODUCTION, at the feet of the programmer, will merely increase the employee turnover in the Development Team. Something that already is a problem most places.

    from my experience: "The Fault, Horatio, Usually Lies Not In Our Code, But In Our Process"

  • This reminded me of a funny/ typical/ stupid/ aggravating thing at work a few weeks ago. I pointed out a security vulnerability in one of our intranet apps during a meeting to discuss the next release. Despite exasperating efforts to educate --and a heated argument over the correct term-- a project manager insisted on spreading the word to upper management that we had a "security breach." But in the war with management (and those who THINK they're above us on the org. chart), I guess it's all about the p

  • by Heembo (916647) on Saturday June 28, 2008 @06:25PM (#23985469) Journal
    Functionality tests are easy to prove through unit and integration testing. Normal users spot functionality bugs quickly during normal product cycles.

    However, security bugs are not easy to test or discover. In fact, it's very expensive to do testing to uncover even some easy classes of security vulnerabilities. Normal users do not stumble on security problems like they do with functionality issues.

    Also, none of your developers were ever taught anything about application security in college. They professors are clueless. Even Michael Howard from MS who is hiring out of the best universities in the world cannot find a new grad who has any clue how to build secure software.

    Functionality bugs and Security bugs are apples and oranges and deserve very different consideration. (Like measurement of Risk, etc)

    Last, you can make a piece of software work. But you an never make a piece of software secure, only reduce risk to an acceptable level.
  • McFeters is nuts! (Score:3, Interesting)

    by SwashbucklingCowboy (727629) on Saturday June 28, 2008 @06:35PM (#23985529)

    I work with the vulnerability management team and product security team at a large software company, and trust me vulnerabilities are treated as product defects. The cost of addressing vulnerabilities in the field is huge, and not addressing them is simply not feasible - customers would never tolerate it.

  • Redefining something rarely changes anyone's behavior.

    Vunerabilities aren't defects unless the behavior of the application is inconsistent with it's requirements.

    In any case, the solution isn't likely to be found in rewarding or punishing developers, but rather making security part of the requirements and providing enough development and testing time to insure that the software is secure.

    Generally the market drives the process rather than software quality.

  • That's too broad of a question; it depends on the vulnerability. If my car's door lock can be bypassed by pulling slightly sideways on the door handle, that's a defect. If I can hold a cell phone over my car's windshield and it triggers the remote door unlock, that's a defect. But if I can smash a car window and climb in through it that's not a defect. If I can use an advanced lock pick set and have 2 hours alone with a car and get in that's not a defect.

    The same thing applies in software - no piece
  • Well of course vulnerabilities should be considered defects. Even more so, DRM should be seen as defects as that is even in the program's planning. A security hole is a small cut, something that should be patched, DRM is like a gaping wound that will never be patched.
  • Security vulnerabilities are a backwards way of describing the issue. There are "security issues" and there are "vulnerabilities". If you throw user passwords into the cloud, it's a security issue. If there's a way for someone to hack into your database of passwords, it's a vulnerability.

    Vulnerabilities don't get solved -- virtually ever. Vulterabilities get "handled". By that I mean that every "solution" costs something -- resources, performance, interactivity, flexibility, time, something.

    For example

  • There is no bug free code. You are dependent on SO many variables, most of them not under your control (your program runs on an OS that you didn't write, was compiled with a compiler you didn't write, runs on hardware you didn't design which uses a BIOS that's not under your control...), that you can never credibly claim your program runs flawlessly. Even any given Hello World has the potential to be buggy, not even due to you. Worse yet, any (but the compiler, I give you that) can change on the target mach

We can predict everything, except the future.

Working...