Please create an account to participate in the Slashdot moderation system

 


Forgot your password?
Close
typodupeerror
×
Security Bug Microsoft IT

Security Patch Creation at Microsoft 274

Posted by CowboyNeal from the inside-looks dept.
devonshire writes "Officials at the Microsoft Security Response Center have provided a detailed look at the process used to create security patches. From the time the first vulnerability data is received from grey hats to the time a bulletin is shipped, it's a pretty interesting look at how they handle the information flow and patch testing and why it takes so darn long to release an IE update."
This discussion has been archived. No new comments can be posted.

Security Patch Creation at Microsoft More

Security Patch Creation at Microsoft

Comments Filter:

  • Next week's headline: (Score:5, Funny)

    by Anonymous Coward on Friday June 10, 2005 @03:22AM (#12777878)
    New Windows worm circumvents Microsoft patching process
    • Microsoft makes security patches? And tests them too?

      • Right (Score:2, Insightful)

        by soloport ( 312487 )
        It's closed source; Closed architecture; Closed development processes. They could be throwing code together like monkeys and making all this stuff up for the PR value. Who knows?

        I'm not trying to flame-bait here, either. These are the simple facts. Flame-baiting would be saying something like: Haven't they always boosted their value via PR and under-delivered? Or: Doesn't Microsoft lie like sacks enough for you to notice?

        That would be flame-baiting. But I'm not flame-baiting.

        • Re:Right (Score:3, Insightful)

          by DogDude ( 805747 )
          It's closed source; Closed architecture; Closed development processes. They could be throwing code together like monkeys and making all this stuff up for the PR value. Who knows?

          And how, exactly, am I to be any better re-assured with Open Source? I can't read the code. I don't know anybody who can. And if I do find somebody who says, "There's a bug in application X", how do I know I can believe them? This whole "everybody can check out the code thing" is really just idealistic fluff to make people f

          • Open source leads to accountability (Score:5, Informative)

            by SirCrashALot ( 614498 ) <{jason} {at} {compnski.com}> on Friday June 10, 2005 @10:35AM (#12779712)
            Maybe you can't but others certainly can, and if you are so inclined, you can learn.
            Also the fact that code is open makes the authors more careful, i would think. If I am going to publish code with my name on it, I would hope it doesn't suck.
            Closed source could have terrible code style and use all sorts of hacks, and no one would know. Or it could be perfectly written using Hungarian notation. With OSS, if you have bad code, people who can read it, will, and tell others.
            Besides, if you want, just do a search for printf and gets in the code -- you might find some bugs w/o having to write a thing.

    • Obligatory Dilbert quote (Score:4, Funny)

      by goombah99 ( 560566 ) on Friday June 10, 2005 @10:10AM (#12779503)
      Dilbert: As part of your ISO 9000 certification do you have a defined pathing process and what is it?

      Elbonian Balmer: We hold a village meeting where we boast of skill and curse the devil spawned enduser.

      Elbonian Gates: Sometimes we Juggle.

      Elbonian Balmer: The at the last second we slam out some code and go roller skating.

      What always cracked me up about that joke is that it is defined process and therefor meets the requirements of iso 9000 certification.

  • Typical corporated programming (Score:5, Interesting)

    by guruevi ( 827432 ) on Friday June 10, 2005 @03:28AM (#12777895)
    Instead of just believing the people that there is a problem, they have to test it out and develop a plan and then reprogram the piece. I hate that. In my company they have implemented such system too and if you have a problem you have to wait a month before it is planned in (if it is accepted by a group of non-technical managers) and then another month before it is fixed making a problem sometimes last for over 6 months and after an endless amount of pointless meetings there is finally some kind of fix. Programmers in corporation are under a lot of (time) pressure and that is not good as it makes them make mistakes. But they have to be able to make quick fixes (as is with most Linux projects) without any corporate meetings or managers.

    • Re:Typical corporated programming (Score:5, Interesting)

      by Atrax ( 249401 ) on Friday June 10, 2005 @04:03AM (#12778039) Homepage Journal
      Your company just seems to have a problem of balance. Your company may have a slow process, but equally they'd be insane to lean too much the other way and just let the techies spin out patches willy-nilly without fear or favour.

      Striking a balance is the trick, and non-technical managers will tend towards the extremely cautious end of the scale without their caution being necessarily grounded in a realistic appraisal of the problem. They don't realy understand it, so they go slowly and have accountability at every step.

      Sounds like you might want a shorter chain of command, with technically knowledgable managers making the calls.

      How you get that to happen, well, I really don't know. A new CEO might be a start (it's worked at my old company)

    • Re:Typical corporated programming (Score:5, Interesting)

      by Tune ( 17738 ) on Friday June 10, 2005 @04:34AM (#12778126)
      Either you have no idea about how (software) project management works or you have seen some worst-in-class examples at your company. Testing and reproducing a bug is *very* important. Bypassing that step is a guarantee to waste valuable programmer's time on non-issues. In a healthy organization with averagely skilled testers, this part of testing takes a couple of hours at most.

      Next is bug fixing. This is by far the most variable and unpredictible part, requiring the best of any programmer. It may take minutes or it may take weeks. Besides good programmers, good process can be of great help here.

      Finally comes the release testing, which is what the article is talking about. This phase is essential: *never* trust a programmer if he says its "fixed and I tested it". Generally, programmers are simply incapable of testing their own stuff. I know as a programmer. Release-testing takes a considerable, but predictable amount of time, assuming the programmer did a good job. Skipping this phase will sooner or later lead to disasters like the recent Netscape 8 release.

      Now I agree with your complaint on workload and lack of tech-savvy managers, but it's nonsense to say that the process as a whole sucks.
    • If you don't test the error, how do you know that (a) there really is an error and (b) where/what the error actually is?

      Sure, the process should be streamlined so that you don't take months to do that, but then the process described in the article _doesn't_ take months if it's handled properly.
    • I don't quite understand your objection... How are you supposed to fix an issue if you don't repro it first? And you object to them making a plan for the development, testing and deployment of the patch? Are you a developer?

  • UDP Floods (Score:4, Interesting)

    by Anonymous Coward on Friday June 10, 2005 @03:29AM (#12777898)
    I don't think there's a single service on a windows box that can withstand a UDP flood. This has been known to be an effective DoS method for years...roommate using all the bandwidth with bittorrent? Playing Doom3 in the middle of the night with the volume jacked up?

    Send a UDP flood to ANY of the services which are actively listening by default, problem solved. Where's the triage team on that one? I guess 99.9% resource consumption isn't a vulnerability in their eyes.

    • I believe their solution is that, since SP2, there are no services listening by default through the firewall. Windows filesharing maybe, though, although that's only subnet-accessible.

      • Not even filesharing is listening. It takes all the fun out of connecting to open Access Points in my apartment building and seeing what naughty pictures they have

  • Nice to know that... (Score:2, Insightful)

    by Anonymous Coward
    Microsoft's non-security is well organised. :-)

  • From the article: (Score:3, Interesting)

    by guruevi ( 827432 ) on Friday June 10, 2005 @03:33AM (#12777918)
    It's not easy to test an IE update. There are six or seven supported versions and then we're dealing with all the different languages. Our commitment is to protect all customers in all languages on all supported products at the same time, so it becomes a huge undertaking. 1: languages shouldn't be a problem, that is (hopefully) not completely split up throughout the source code is it? aargh!!!! 2: I know only a 3 SUPPORTED IE versions (IE 5, IE 6 and IE 7)

    • Re:From the article: (Score:3, Insightful)

      by XanC ( 644172 )
      I would imagine that the IE version that runs on each OS (2K, XP, 2K3, etc) is probably unique enough to warrant a full battery of tests.

    • Re:From the article: (Score:5, Funny)

      by N3Roaster ( 888781 ) <{nealw} {at} {acm.org}> on Friday June 10, 2005 @03:38AM (#12777934) Homepage Journal
      You missed the funniest bit:

      This is exactly why it can take a long time to ship an IE patch. [snip] We have to make sure it doesn't break the Internet.

      So, the next time someone tells you, "The Internet is broken," you can just blame Microsoft for putting out an IE patch too quickly.

    • Re:From the article: (Score:2, Insightful)

      by timmarhy ( 659436 )
      they are a fucking multi BILLION DOLLAR company, dont' they DARE try and cry about being short on man hours.
    • Well, there are major sub-versions, too, like IE5.5SP2, etc.

      Several times over the years I've discovered multiple code paths in Windows which apparently perform the same function. I discover them because performing what is ostensibly the same act via more than one of the typically myriad interface controls to initiate the given desired action sometimes differ ever so slightly (note the sarcasm in my voice) in result. I've seen these sorts of artifacts all the way up through Windows 2000. This problem e

    • Re:From the article: (Score:3, Interesting)

      by Vo0k ( 760020 )
      1: languages shouldn't be a problem, that is (hopefully) not completely split up throughout the source code is it?

      You'd be surprised. Very surprised.
      Things are far more screwed up than you'd think. An article on development of a new OS release would come in handy, but putting things shortly, somewhere between 60 and 80% down the way with the development of the new OS, the code is branched into "local versions" which are independently developed by corresponding local Microsoft divisions. Bugfixes, features

  • IE is the internet? (Score:5, Funny)

    by gd2shoe ( 747932 ) on Friday June 10, 2005 @03:35AM (#12777921) Journal
    "This is exactly why it can take a long time to ship an IE patch. We're dealing with about 440 different updates that have to be tested [for different versions]. We have to test thoroughly to make sure it doesn't introduce a new problem. We have to make sure it doesn't break the Internet."

    ? ? ? ? ? ?

    • Re:IE is the internet? (Score:3, Insightful)

      by Atrax ( 249401 )
      To the consumer, yes. IE is 'the internet'. Besides which, a patch which had a regression flaw and opened something exploitable by a major worm could cause mayhem beyond just breaking windows clients. A massive DDOS caused by a hole in IE? that would be nice, eh?
    • They should check with Al Gore before they do anything that could break his internet...
    • We have to make sure it doesn't break the Internet [web access provided by IE, which as far as our customers go means breaking the Internet]

      The Internet wouldn't be broken as such, but I doubt the users would see it that way. To them, it doesn't matter if it is the browser, the connection or the servers (massive worm?) that is broken. They can't do what they want, hence it is broken. It is as simple as that.

      Kjella

    • Yes (Score:3, Funny)

      by samael ( 12612 )
      For 90% of people, the web is the internet.

      For 88% of them, the internet is IE.

      Which means that 79.2% of people think that the internet is IE.

      • Since your variables are not independent I would say that the total of people that think that internet is IE is 88% (assuming that your numbers are in fact correct). This would be due the fact that the set of people who think that internet is IE and don't think the internet is the web is most likely empty.
    • I'm a reasonably literate computer user, even for slashdot. All of my machines run linux, happily, even my linksys router. I modify my systems to be useful and I have no stupid blue lights on anything.

      With that said at work, I've said things like "the internet is broken". Why? Because I'm there to handle medicines, and not beat my head against the wall trying to explain to my PHB which exploit is in the wild and how to patch it. Forget it.

      No one at work knows that I know a damn thing about compu

  • Pick me, pick me! (Score:5, Funny)

    by Infinityis ( 807294 ) on Friday June 10, 2005 @03:37AM (#12777928) Homepage
    I know the process!

    1. Identify holes in current software
    2. Release patches that only fix some of the holes
    3. Start charging for tools to take care of the rest of the holes
    4. Profit!

    (If you're from Indonesia, no problem, the software will only cost $1 anyways)

    • Re:Pick me, pick me!...Alternate Patch Process (Score:5, Funny)

      by darkPHi3er ( 215047 ) on Friday June 10, 2005 @04:24AM (#12778104) Homepage
      Customers Complain About a New Security Hole. The number of complaints reaches Management's "Action Threshold". The Patch Process is started.

      1. First, blame the customers' other software packages for the insecurity.

      2. Then, blame the customers' for failing to apply services and hot fixes in a timely fashion.

      3. Security Focus (or another of the Sec/Priv sites) calls up and threatens to "out" the hole if it isn't fixed.

      4. Accuse the complaining entity of having a "partisan" agenda against your company, initiate "Four Corners" Stall -- while you try to figure out if you actually CAN patch the damn thing

      5. As the news of the new exploit makes it into IRC and the UGs' Forums you issue an indignant press release stating that it has never been proven that the new exploit has even been used and is principly "theoretical".

      6. As your Patch Team frantically works to get the patch out, explain that even though the chances of this exploit being used WERE previously slender or non-existent, now that some details of the exploit have been malicously leaked, HEAVY SIGH, you'll now go ahead and fix it.

      7. Issue the patch, take credit for being "Right on Top" of security issues, explain how much money and time you are spending to counter the effects of the "Few Bad People" on the Internet.

      8. News start to come in that your patch has broken a number of somewhat older apps -- explain that Users have a responsiblity to use "current" software products and refer them to Sales.

      9. News of another exploit comes in --GOTO 1

      BTW, this is pretty much AN INDUSTRY STANDARD APPROACH

      In Commerical Software, it's the FEW companies that DON'T do some version of this that are the (delightful) and unfortunately RARE exception.

    • Mozilla process:

      1) JFDI

      2) er..that's it
    • (If you're from Indonesia, no problem, the software will only cost $1 anyways)

      I think the concept of $1 Indonesian software just got filed between Korean old people and Soviet Russia.

  • Real world equivalent (Score:4, Funny)

    by Anonymous Coward on Friday June 10, 2005 @03:38AM (#12777939)
    Microsoft is adding a patch to a pair of jeans, but it's difficult because after all the previous patches the pair of jeans looks like a spherical ball of patches 10 feet in diameter.
  • why it takes so long to issue a patch is because it takes 8 days a week for them to get off their ass .

  • Hahaha. (Score:3, Funny)

    by BJH ( 11355 ) on Friday June 10, 2005 @03:42AM (#12777953)
    We have to make sure it doesn't break the Internet.

    Don't worry, guys, no matter how badly you screw up it won't hurt the Internet - because the Internet doesn't run on Microsoft boxes. Hard to imagine, I know, but true.
    • Yeah because all that matters is you keep them Linux webserver farms up serving next to no requests because 80-90% of clients are dead in the water.

      I'm sure the likes of Amazon.com would appreciate that so much. Thanks.

    • Re:Hahaha. (Score:3, Interesting)

      by multi io ( 640409 )
      If they accidentally deliver a patch to IE that makes the browser send 256 requests per second to randomly chosen servers, something that's indistinguishable from "breaking the Internet" will happen.

  • The Big Blue E (Score:5, Funny)

    by value_added ( 719364 ) on Friday June 10, 2005 @04:06AM (#12778048)
    "It's not easy to test an IE update .... We have to make sure it doesn't break the Internet."

    Sometime a joke doesn't need a punch line.

  • Ha! (Score:5, Funny)

    by KenFury ( 55827 ) <kenfuryNO@SPAMhotmail.com> on Friday June 10, 2005 @04:21AM (#12778097) Journal
    "It's not easy to test an IE update .... We have to make sure it doesn't break the Internet."

    Here I fixed it for you.

    "It's not easy to test an IE update .... We have to make sure it breaks Firefox and Opera."

    Better

    • "It's not easy to test an IE update .... We have to make sure it breaks Firefox and Opera."

      You didn't get the entire quote.

      "It's not easy to test an IE update .... We have to make sure it breaks Firefox and Opera. We don't really have to worry about Netscape because it broke itself, so we were able to get this patch to you in a much more timely fashion."

  • Liars (Score:5, Informative)

    by cperciva ( 102828 ) on Friday June 10, 2005 @04:26AM (#12778110) Homepage
    Quoth the article:
    We respond immediately to the initial vulnerability report and provide the researcher with contact names, e-mail addresses and phone numbers. We make it clear we want to work closely with the researcher to pinpoint the problem and get it fixed. We commit to providing [researchers] with a progress report on the Microsoft investigation every time they ask for one

    My experience directly contradicts this on all points.

    When I reported the hyperthreading security flaw [daemonology.net] to Microsoft, I was provided with the first name of the person who was responsible for dealing with it ("Christopher"), but I was not provided with his last name, phone number, or any e-mail address (apart from the generic secure@microsoft.com address which I used to report the problem). Later the issue was transferred to "Brian" -- again, no last name, no email address, and no phone number.

    Over the following two months, I heard from three independent third parties that Microsoft was "very concerned" about this issue, and had "several people" looking at it; but they never made it clear that they wanted to work closely with me -- in fact, they ignored all my attempts at co-operation.

    Finally, prior to releasing my paper, I sent several emails to Microsoft asking about their progress and asking for a vendor statement for my web site; again, they did not respond.

  • I'd say the difference is... (Score:5, Insightful)

    by Kjella ( 173770 ) on Friday June 10, 2005 @04:35AM (#12778132) Homepage
    ...purely political.

    Microsoft wants to give you one "bad news" per month. Predictable, patch time is "low" meaning the time between release and installation is low. It is easy for IT staff to work that way, you can schedule it.

    OSS will give you a patch per issue, patch time is near instant, but they keep coming at you all the time, whenever you can't afford to waste time installing them. That is why you need a distro to keep you patched at all times.

    The rest? Bullcrap. The security patches for Linux don't cause more regression issues than Windows. Like Microsoft, they do audits but instead of one "catch 'em all" release, they do several. In short, it is to make Windows look good.

    Kjella

  • Do you think if Bill Gates... (Score:4, Funny)

    by Your Average Joe ( 303066 ) on Friday June 10, 2005 @07:33AM (#12778580)
    got laid in high school, do you think there'd be a Microsoft?

    Of course not.

    You got to spend a long time stuffed in your own locker with your underwear wedged up your ass before you start thinkin "I'm going to take of the world with computers! you'll see, I'll show them."

  • code != bloat (Score:3, Insightful)

    by tomstdenis ( 446163 ) <tomstdenis@gma[ ]com ['il.' in gap]> on Friday June 10, 2005 @08:09AM (#12778676) Homepage
    This is why concise, clear and well documented modular programming is a winner. Even Firefox suffers this. It's a huge mess of code that a handful of people could even be bothered to read...

    In microsofts case everything has to be implemented upon layers of undocumented C++ classes to which the average microsoft employee [let alone third party developer] can't decode.

    Tom

  • Grey hats?!? WTF (Score:2, Interesting)

    by thomasj ( 36355 )
    Some people submit a vulnerbility report to the brickwall called Microsoft Support. Then after 6 months they release a security opdate. And now they call the submitters "Grey hats"? What do they call themselves? The "Pink hats"?
  • Let me summarize what the MS people in the article said.

    Our software is so hopelessly intermingled due to the manner in which we tried to get around the anti-trust laws, that simple updates take far longer than they should.

  • From TFA:

    "In theory, we can release an update with a patch very quickly, but that's a big mistake. One of the things customers demand is quality patches. They don't want to deal with faulty patches that break their applications and they don't want to deal with all the associated trouble"

    He's close, but not spot on; customers demand quality software, but are forced to deal with faulty programming and broken applications. Customers wait for 'quality' patches, and deal with the associated trouble of a syst

Slashdot Top Deals

THEGODDESSOFTHENETHASTWISTINGFINGERSANDHERVOICEISLIKEAJAVELININTHENIGHTDUDE

Close