Become a fan of Slashdot on Facebook

 



Forgot your password?
typodupeerror
×
AI Microsoft Security

AI Spots Critical Microsoft Security Bugs 97% of the Time (venturebeat.com) 41

Microsoft claims to have developed a system that correctly distinguishes between security and non-security software bugs 99% of the time, and that accurately identifies the critical, high-priority security bugs on average 97% of the time. From a report: In the coming months, it plans to open-source the methodology on GitHub, along with example models and other resources. Their work suggests that such a system, which was trained on a data set of 13 million work items and bugs from 47,000 developers at Microsoft stored across AzureDevOps and GitHub repositories, could be used to support human experts. It's estimated that developers create 70 bugs per 1,000 lines of code and that fixing a bug takes 30 times longer than writing a line of code, and that in the U.S., $113 billion is spent annually on identifying and fixing product defects. In the course of architecting the model, Microsoft says that security experts approved the training data and that statistical sampling was used to provide those experts a manageable amount of data to review. The data was then encoded into representations called feature vectors and Microsoft researchers designed the system using a two-step process, in which the model first learned to classify security and non-security bugs and then to apply severity labels -- critical, important, low-impact -- to the security bugs.
This discussion has been archived. No new comments can be posted.

AI Spots Critical Microsoft Security Bugs 97% of the Time

Comments Filter:
  • by Anonymous Coward
    ftfy
    • ftfy

      Boy, that's the truth!

      Microsoft products are the most consistently-buggy, inconsistently-improved software products on the planet.

      I have to Develop around their many giant turds on a daily basis; and the sheer mind-numbing quantity of their bugs, both big and small, both in the products and the tools for working with them, is both appalling, and a source of daily frustration for me,

    • Easiest way is just to always say "YES YOUR PRODUCT HAS A BUG". Then in the off chance it doesn't have a bug, hopefully that's less than 3% of the time...

      This can be applied to the healthcare industry as well. "YES THE PATIENT WILL DIE". (Eventually. Maybe even 40 years later.)
    • AI picks at random and 97% of the time it's a bug.
  • by BAReFO0t ( 6240524 ) on Thursday April 16, 2020 @02:14PM (#59955740)

    "You were using C or C++ and did not intend to write optomized low-level routines."

    The others will mostly be:
    "You are a script kiddie. Go back to your homepage!" and
    "You wrote an enterprisey abomination. Are you perchance a consultant?" :)

    • If you're using C you should be switching to C++.

      But ... one robo-detected bug every 14 lines of code? How low did they stoop to find those code samples? Did they just download the whole of GitHub for analysis or something?

      • by znrt ( 2424692 )

        from the article, this thing doesn't seem to be detecting bugs, but categorizing text descriptions of known ones, so as to prioritize them.

        • by znrt ( 2424692 )

          sorry, responded to wrong post. ignore.

        • from the article, this thing doesn't seem to be detecting bugs, but categorizing text descriptions of known ones, so as to prioritize them.

          If this tool could actually detect 97% of bugs (or even 10% of bugs) it would be an astounding advance toward strong-AI.

          What it actually does is rather trivial and uninteresting.

    • You missed a wonderful opportunity to give it a Clippy UI and dialogues.
    • I'm just waiting for the flood of "But if you'd just used Rust, then..." posts.

      • by Tailhook ( 98486 )

        Hour and a half later you're the only one to have chimed in about Rust... perhaps the insufferable Rust advocates you were hoping for aren't really the problem you think they are. It may even be that people bitching about Rust advocates are the actual problem now.

        It has been half a decade since Rust 1.0. The initial hype has died away and the posers and evangelists are off hyping something else now. What we're left with is a maturing language and serious people making stuff, and they don't spend a lot

  • If (author == "Microsoft") { is_critial_bug = true;}

  • Not too suprising. (Score:4, Interesting)

    by jellomizer ( 103300 ) on Thursday April 16, 2020 @02:37PM (#59955834)

    Many bugs and security glitches are from often simple mistakes, made when the developer is tired (either short term, from physical exhaustion, or long term from just doing that big project for such a long time) shortcuts are made, Proof of concepts are put in and forgotten to harden. Specs changed mid-project so half-written code is still there, debugging backdoors are not closed...

    Even the best developer isn't perfect. AI isn't as good as a person is, but it is tireless and will catch a lot of the simple things people do.

     

    • by znrt ( 2424692 )

      AI isn't as good as a person is, but it is tireless and will catch a lot of the simple things people do.

      from the article, this thing doesn't seem to be detecting bugs, but categorizing text descriptions of known ones, so as to prioritize them.

    • by Anonymous Coward
      many more are from maintaining other peoples code on large projects. much easier to make critical errors when fixing or modifying other peoples work.
  • 97% of Microsoft code has critical security flaws?
    We knew that since Windows 2.03, no AI needed for that.

  • Maybe (Score:3, Insightful)

    by jwymanm ( 627857 ) on Thursday April 16, 2020 @02:56PM (#59955924) Homepage
    After so many decades of writing code stop doing feature enhancements or icon graphics changes and fix your god damn broken processes? Like maybe make it impossible to remotely exploit shit that doesn't need to be remotely exploitable by um not even fucking inventing it in the first place? How hard is it to just say no to doing things that constantly have proven to break? Why does each update you release fix 4 vulnerabilities and add 40? Why are you not redesigning your architecture so it can't be such a fine example of swiss cheese?
    • I know. Why doesn't Microsoft just hire perfect people. They only seen to hire crap coders who produce buggy software.

      In case you can't tell I'm mocking you for how you think programming and people work.

  • Look, the only kinds of articles about security bugs I want to hear about are articles about Zoom security holes. It's been almost 24 hours since the last Zoom security hole article. WTF is up with the Editors? We need to kill that company's stock ASAP, so it should wall to wall "Zoom Sucks!" articles until the company goes broke.

  • Did anyone notice the comment about MS developers generate 70 bugs per 1000 lines of code? That comes to one bug per 14-15 lines of code. That is absolutely terrible! Over the course of my life I've developed systems that were thousands of lines long and had maybe one or two bugs! One was for a system deployed in a nuclear reactor system, it had 15 thousand lines of code and had only one issue (a 16 bit integer wrapped around!). If I had a developer that generated 70 bugs per 1000 lines I would fire him or
    • Over the course of my life I've developed systems that were thousands of lines long and had maybe one or two bugs!

      So what you're saying is you've worked for places without proper QA/testing and all your projects kept getting cancelled before they were deployed?

    • Oh, it gets even better if you keep reading and run the numbers:

      It's estimated that developers create 70 bugs per 1,000 lines of code and that fixing a bug takes 30 times longer than writing a line of code...

      Note that 70 * 30 = 2100 > 1000. Apparently one must allocate 3.1 times as much development time as expected from the code alone in order to also fix the bugs; this also assumes that finding bugs is a zero cost operation.

  • Itâ(TM)s estimated that developers create 70 bugs per 1,000 lines of code and that fixing a bug takes 30 times longer than writing a line of code

    Who estimated this? Microsoft? If so, that explains quite a bit.

    I'm not trying to troll here, but that number is about 5 times what it should be. Without naming names, I know for a fact that it is possible for large organizations to achieve less than 15 defects per kloc through formal testing and code reviews. If you bring in standards-based approa

    • If you read TFA, the number isn't from microsoft. It's just sourced from this random article (https://coralogix.com/log-analytics-blog/this-is-what-your-developers-are-doing-75-of-the-time-and-this-is-the-cost-you-pay/) that claims (totally unsourced) that 70 bugs/1000 lines is before QA, and 15 bugs/1000 lines is what get to customers

    • by Kaenneth ( 82978 )

      Really going to depend on the complexity of the lines.

    • Itâ(TM)s estimated that developers create 70 bugs per 1,000 lines of code and that fixing a bug takes 30 times longer than writing a line of code

      Who estimated this? Microsoft? If so, that explains quite a bit.

      I'm not trying to troll here, but that number is about 5 times what it should be.

      If you read the article, you'd find that those numbers come from Ariel Assaraf writing for coralogix.com [coralogix.com], and the next line reads:

      15 bugs per 1,000 lines of code find their way to the customers

      • by znrt ( 2424692 )

        15 bugs per 1,000 lines of code find their way to the customers

        that's orders of magnitude worse! :'-) sounds like a really shitty sw shop to me, and not just the junior programmers ...

        that said, measuring bugs per lines of code is kind of nonsense, unless used only in a closed and very specific environment (language, field, expertise, ...) over time to read 'trends'. even so ...

      • Something doesn't add up: if they spend 75% of their time fixing bugs, that means the cost of fixing the bug is only 3x the cost of writing the code, not 30x. To get a 30x number, they'd have to spend only 2.5% of their time writing code, 22.5% on something else.

        The 15/1000 number is in an article linked from the original. I suppose I should have looked at that, but the linked article appears to be a marketing pitch. Granted, that's not a bad number, but it's still on the high side of my experience.

    • The number of bugs in code is highly dependant upon the developer producing the code. Additionally, some developers have bad habits that are too ingrained to fix. I once worked with a "consultant" who just could not understand the concept of freeing a resource at the same level of acquiring it in a C project. What he would consistently do is malloc() a piece of memory in a function in the middle of the call tree, and then free() that memory in all the leaf functions that could potentially be reached from th

  • As much as I dislike Microsoft, this is an excellent development. A reduction in programming errors is always a good thing, no matter who is to credit because bad code needs fixing. However, I do wonder if this will catch too many non-bugs that are really just shitty/non-deterministic code. Too many false positives and it's worthless.

    • Re:Great! (Score:4, Informative)

      by znrt ( 2424692 ) on Thursday April 16, 2020 @05:09PM (#59956308)

      read again. this is not about bug detection. this is about evaluating bug reports, likely raw user input or first level report, and highlighting those that might raise security concerns, which are "must fix. asap". it can probably do so in real time, so you'd expect an alarm bell going off when one of those is comes in.

    • by bobby ( 109046 )

      This is Microsoft we're talking about. I doubt there will be many false positives.

  • bool microsoft_code_contains_security_bug() { return true; }

    My invoice is in the mail.

  • And the state planners and computer scientists of a certain very large communist dictatorship with massive investment in AI were shocked, SHOCKED, upon learning this.

"You know, we've won awards for this crap." -- David Letterman

Working...