Please create an account to participate in the Slashdot moderation system


Forgot your password?
Government United States IT Your Rights Online

Secret Service Runs At "Six Sixes" Availability 248

PCM2 writes "ABC News is reporting that the US Secret Service is in dire need of server upgrades. 'Currently, 42 mission-oriented applications run on a 1980s IBM mainframe with a 68 percent performance reliability rating,' says one leaked memo. That finding was the result of an NSA study commissioned by the Secret Service to evaluate the severity of their computer problems. Curiously, upgrades to the Service's computers are being championed by Senator Joe Lieberman of Connecticut, who says he's had 'concern for a while' about the issue."
This discussion has been archived. No new comments can be posted.

Secret Service Runs At "Six Sixes" Availability

Comments Filter:
  • Wow. (Score:5, Funny)

    by moogied ( 1175879 ) on Friday February 26, 2010 @08:59PM (#31293134)
    They should just flip the availability numbers over and get rid of the decimal. "Sir, its not 66.. its 99! You have it upside down!" -- Fixed.
    • and get rid of the decimal.

      What, don't you realize they could have 999% reliability?? This could be revolutionary!! Think what you are saying, you nearly threw that away.

      • I think that means that it logs into itself at times to do things you might do. Think of how revolutionary that really is. Perhaps next we can get a server that plays solitaire for you while you're away from the server room.
  • Upgrade... (Score:5, Funny)

    by ak_hepcat ( 468765 ) <leif@de n a l i . net> on Friday February 26, 2010 @09:00PM (#31293140) Homepage Journal

    To windows, and get 73% uptime!

    Or.. that other OS that you don't have to license per seat, and get in the solid 90+% uptime.

    • by Anpheus ( 908711 )

      If you can't manage two nines on a basic windows server, you're doing it wrong. If your service depends on a single server, you're still doing it wrong.

      Lastly, is a performance reliability rating the same thing as uptime? I doubt it. If their server is down eight hours a day, they'd swap it immediately.

      • Re: (Score:3, Insightful)

        by scubamage ( 727538 )

        If their server is down eight hours a day, they'd swap it immediately.

        As soon as they fill out all of the paperwork, and find a way to blame the downtime on someone with we don't like.

        • by Anpheus ( 908711 )

          I'll pre-whoosh myself here:


          There, with that out of the way. Actually, it's probably going to take until they can resurrect their last COBOL programmer or find someone who they can train on a thirty year old system in less than a year.

      • by TapeCutter ( 624760 ) * on Friday February 26, 2010 @09:37PM (#31293478) Journal
        "If your service depends on a single server, you're still doing it wrong."

        666 666 -> Devilishly clever redundancy.
      • Re:Upgrade... (Score:4, Informative)

        by peragrin ( 659227 ) on Friday February 26, 2010 @10:20PM (#31293834)

        if you can't manage 2 nines on a an IBM mainframe your doing it wrong to begin with what makes you think they can do with something vastly more complicated as a massive windows deployment.

        • by Anpheus ( 908711 )

          I made that point in my second sentence. I can't imagine a mainframe with an active support contract has less than 99% uptime. I'm pretty sure that "performance reliability rating" is not a euphemism for "service level availability."

        • It's never the hardware failures that get you on any platform. From what I've seen, 99% of all outages are caused by idiots.
        • Re: (Score:3, Informative)

          by wwphx ( 225607 )
          The bias against mainframes is sad, but as old as their equipment is, the whole thing does need to be scrapped. That said, I don't think the photo with the article is of their actual system. Our mainframe is about to be retired, and the only time it was restarted was twice a year to adjust for DST because it wasn't properly maintained and a DST patch was never installed. Our other downtime with it was mainly because the building UPS couldn't support it during power failures. Otherwise, 99% plus was not
      • Two nines? What, 9.9% ?

        Even a monkey could manage that.

    • Re: (Score:3, Insightful)

      by timmarhy ( 659436 )
      90+ uptime for free is useless if the OS can't fill your requirements.
      • by tsm_sf ( 545316 )
        you're confusing "work" with "business related activities". One of these is handled by every OS, the other requires Windows.
    • 90%? (Score:3, Insightful)

      by hedronist ( 233240 ) *
      If you are only getting 90% from any OS you really should be shopping for a new OS. I've got flaky machines in my garage running Linux that regularly are up for 6 months or more at a time, and that includes dodgy power in my area.
      • by smash ( 1351 )
        If you're only getting 90% uptime from any of the common OSes you should be shopping for a new admin and/or new hardware as appropriate. 90% is 2.4 hrs of downtime per day. Even windows can do better than that, by a few orders of magnitude - quite easily.
  • by WrongSizeGlass ( 838941 ) on Friday February 26, 2010 @09:02PM (#31293162)
    ... I have several old P4 1.6Ghz w/ 256MB RAM & 100Mhz FSB in a store room at a client site. They originally shipped with Win 98 but they've since been upgraded to XP. The Secret Service can have them fro free if they just come and pick them up. I would have put them on Craig's List but I don't trust a web site where they let just anybody post things.
    • Re: (Score:3, Insightful)

      by fm6 ( 162816 )

      I could point out that the cost of replacing this mainframe would mostly involve rewriting its applications to run on modern hardware. But then you'd be deprived of your joke, even if it is a pretty lame one.

      • by Anonymous Coward on Friday February 26, 2010 @09:31PM (#31293406)
        it's an IBM mainframe. They can replace it with another (modern) IBM mainframe, no code change necessary. Posting anonymously, so you can believe it or not, but I do have a clue about the specifics. It's not a technical problem, it's not a financial problem, it's a bureaucratic problem. Government at it's finest.
        • by Khyber ( 864651 )

          AC isn't lying, I'll back it up. Even if you wrote your stuff on an old System32, you can run it on most any IBM machine today.

        • Re: (Score:3, Interesting)

          by fm6 ( 162816 )

          Should have RTFA. If we're talking an IBM mainframe, then you're certainly right. I cut my programming teeth on those 40 years ago (and haven't been near one in 35), and the basic architecure is still around, though many details have changed. In fact, I've long been convinced that this kind of need for backward compatibility is the only reason people still buy mainframes. Though there are those who are convinced they're fundamentally kewler.

          The problem is probably as much political as bureaucratic, if not m

      • by jamesh ( 87723 )

        But then you'd be deprived of your joke

        Are you sure? I assumed that the joke was that he was posting the ad on Slashdot because he didn't like Craigslist because anyone can post anything there (unlike Slashdot, where anyone can also post anything there too).

        But then I guess it's not funny if you have to explain it.

      • You could probably write an emulator for the hardware a lot easier than you could rewrite all the software that runs on this thing
        • by fm6 ( 162816 )

          Good point. I think even the oldest IBM mainframes are still around in emulator form.

  • by Anonymous Coward on Friday February 26, 2010 @09:05PM (#31293186)

    Mainframes of yore had a hell of a lot of moving parts: a large system might have dozens of tape drives and disk drives. Tape drives in particular broke down all the time and were taken offline until the maintenance guy came for his weekly or monthly visit and tightened the belts or whatever the hell they did. Knuth remarked on that situation in his magnum opus TAOCP vol 3 on sorting and searching. In the part about sorting with tape drives, he remarked that he'd never seen a large computer installation where all the tape drives were working. You'd have a computer with ten tape drives, two of them would be down pending repairs, and you'd use the other eight. In other words your computer was operational but not FULLY operational.

    There is a similar situation in today's data centers. Even at the wimpy little shop I worked in last year (about 2000 computers) some were always down. We were doing pretty good if the number down at any moment was less than a few dozen. I don't think we ever had a single day of being fully operational (every single computer up at the same time). That was fine, it wasn't a requirement, it was a distributed system and the data and functions were all sufficiently replicated that we kept running, by design, even with parts of the system unavailable.

  • I mean why settle for five nines when you can have... NINE FIVES! :D

  • Color me skeptical (Score:5, Insightful)

    by belthize ( 990217 ) on Friday February 26, 2010 @09:21PM (#31293326)

    There's something about this whole thing that simply doesn't ring true. I believe parts, I believe they have a 1980's main frame, I believe it's not terribly reliable but something about the whole: leaked memo according to Joe Leiberman, we need more money, they won't give us more money' spiel sounds off. I suspect they have huge chunks of computing that's much newer and reliable, I'd be shocked if that IBM serves any significant purpose.

    If nothing else I predict a large percentage of the umpteen million dollar final cost somehow going to Connecticut, but I'm probably just incredibly jaded.

    • If nothing else I predict a large percentage of the umpteen million dollar final cost somehow going to Connecticut, but I'm probably just incredibly jaded.

      What's a few million? Connecticut is one of the top haulers, thanks to Electric Boat, where many nuclear subs (and a number of other ships) are made.

      Every time the Pentagon tries to cut its budget, congrescritters get all up in arms about "jobs", so the Pentagon has all these useless projects (congress forces the programs it wants.) It's the primar

  • 1980's mainframes did not use reel-to-reel tape. They used tape cartridges, often managed by automatic tape libraries.

    • The picture looks an awful lot like late 60's drives for an IBM 360 model 44.

      I guess they needed a picture that screamed 'main frame'.

    • TG Daily claims [] that the Secret Service uses a IBM 704 []

    • Re: (Score:3, Informative)

      by Eric Smith ( 4379 )
      Plenty of nine-track tape was still in use on mainframes in the 1980s.
      • Plenty of nine-track tape was still in use on mainframes in the 1980s.

        Yes, I did backups at work on 9 track well into the 1990s. Admittedly we were late for an upgrade then. The coating used to come off on the heads on the old tapes. Cleaning was a chore.

        At my current job I rescued an old 9 track tape which was going to be left behind for the cleaners when we moved offices. Its at my desk right now. Maybe somebody will come looking for it one day.

    • 1980's mainframes did not use reel-to-reel tape. They used tape cartridges, often managed by automatic tape libraries.

      Nine track drives like that were still in use the 1980s. Into the 1990s, even.

  • Misleading photo (Score:5, Informative)

    by Jeremy Erwin ( 2054 ) on Friday February 26, 2010 @09:33PM (#31293444) Journal

    The story uses a stock photo [] captioned "Obsolete mainframe super computers in [Computer History] museum". I don't think the Secret Service uses IBM 2401 magnetic tape units []

    • The media uses stock photos whenever they don't have real photos of something. This is normal. I've even seen stock photos of Bumble Bee tuna used in contamination stories for another brand. (I forget which one.) Talk about misleading...

      • by socsoc ( 1116769 )
        I guess it worked though, you can't remember the other "bad "brand. The brand which happened to be an advertiser.
  • what more is there to be said aboutit?

  • by gmuslera ( 3436 ) on Friday February 26, 2010 @10:08PM (#31293732) Homepage Journal
    At last a computer that can be safe even in a cyberwar, no modern hacker would be able to enter there, or at least, do anything dangerous. Even the Morris worm would scream and run facing that technology. Leave that multivac running enough time and will eventually make light.
  • Curiously, upgrades to the Service's computers are being championed by Senator Joe Lieberman of Connecticut

    Its not curious. Don't confuse his desire to censor, restrict, or otherwise hinder the people's access to free information(the internet).

    Doesn't mean he won't allow every resource into that same tech if security/administration needs it...especially if it achieves the former.
  • That's better than our goal of Nine 5's...up a little over half the time!

  • $187 million? (Score:5, Insightful)

    by RightSaidFred99 ( 874576 ) on Friday February 26, 2010 @10:37PM (#31293956)

    They're claiming it will cost $187 million to replace. Bullshit. If the hardware is more than 15 years old, which it sounds like it is, it's impossible to conceive how they could spend more than $100k on hardware to replace it and still give 100x the performance and capacity. OK, let's splurge - spend 5 million on hardware.

    These jackoffs would have us believe it's going to cost $180 million to replace some bullshit law enforcement database software that's 20 years old? Complete bullshit. Instead of the mythical $500 government hammer, now we've got the $180 million dollar software package that should cost

    • These jackoffs would have us believe it's going to cost $180 million to replace some bullshit law enforcement database software that's 20 years old?

      We don't know what that software does. Thats why its called the Secret Service. My guess is that nothing will be delivered for less than 300 million USD. And yeah I do work on large civil/military projects, though on the European side.

    • Don't forget labor costs. Someone has to port that old old old code nobody understands anymore.
      • Don't forget labor costs.

        Labor with sufficient security clearance, proper technical certification, and adequate competence. It might be a "pick any two" situation...

    • Because they would spend 37k on hardware and hire 100 contractors at 1m/year to port everything over.

    • Re: (Score:3, Interesting)

      These jackoffs would have us believe it's going to cost $180 million to replace some bullshit law enforcement database software that's 20 years old?

      The rated you funny, but it's true. FBI CASE system also vintage 1980's mainframe system. They have tried and failed twice to rewrite 20 year old law enforcement database software at over half a billion spent so far. First time they said they didn't even have anything salvageable to show for it and threw out the entire project which aos hap

      • by dzfoo ( 772245 )

        >> People blame the government bureaucrats, but the failed work is done by the huge consulting companies.

        Billions of dollars sunk into failed projects, and yet the government bureaucrats keep hiring these consulting companies for software work. Therefore they share a large part of the blame: at the least it shows incompetence and negligence, at worse it implies collusion and fraud.


    • by DingerX ( 847589 )
      Okay, let's do the numbers (in millions) NSA consultancy to determine the need for new hardware: $7 External auditors of NSA consultancy: $3 Environmental impact study: $5 Feasibility study: $2 Legislative Task Force (aka "Marketing"): $10 Publication and administration of call for outside bids: $5 Hardware: $.1 Comprehensive installation and migration package: $10 Ten years support: $20 Audit of tender and winning bid: $5 Annual support audits: $5 Verification and Validation (Internal): $3 V&V (extern
  • "Curiously"? (Score:5, Insightful)

    by DesScorp ( 410532 ) <> on Friday February 26, 2010 @11:03PM (#31294130) Homepage Journal

    "Curiously, upgrades to the Service's computers are being championed by Senator Joe Lieberman of Connecticut "

    What's curious about that? It's not like the guy is a Luddite or something. The Secret Service, at the forefront of protecting POTUS, is a national security issue, and Lieberman is very involved in those issues. If the author threw that in because he doesn't like Lieberman's politics, then that's kind of lame. One would think that issues like keeping government IT systems up to date would transcend party politics.

    • by Zorque ( 894011 )

      They could mean it's curious that Joe Lieberman is doing something that isn't a waste of time.

    • by socsoc ( 1116769 )

      then that's kind of lame

      No, that's the quality of reporters these days when everyone has a blag, the true folks have forgotten what separated them.

  • by Animats ( 122034 ) on Saturday February 27, 2010 @01:36AM (#31294906) Homepage

    The article is bogus, but the problem is real. Computer support systems for investigators are hard to build. The FBI has struggled with this, taking about a decade to deploy their "Field Office Automation" system. They're hard for many of the same reasons medical systems are hard - much of the incoming data is unstructured, and many people enter data relevant to the same case. It's even harder than in the medical world, because links between various individuals and events are important, but unreliable. The "customers" aren't cooperative, they usually don't have unique identifiers, and a sizable fraction of the information is bogus. The security problems are tough to even define - exactly who's allowed to see what is a big issue.

    The older law enforcement systems didn't offer much searchability. Unless you had a hard search key, like a driver's license number or a full name, you couldn't retrieve much. Now, everybody expects Google-like searchability, and the older systems just didn't have the machinery for that.

    • by AHuxley ( 892839 )
      Layers and layers.
      The telco wire tap system is out sourced to telco billing systems outside the USA and needs paper work.
      If you in good with the billing system, you have real time insight into most legal wiretaps in the USA.
      The NSA looks at all packets, but could not talk about its findings.
      What would the Secret Service have? A card system like hyper card.
      One person, one big virtual card, a pic, list of past issues ect.
      Lieberman also got interested in a p2p hunting system to tag p2p shared media with
  • They should make an effort to get at least NINE sixes. Or more.

  • Layoff your data center and software staff. Cancel the contractors & consultants. Get rid of the hardware. Setup a remote data center in India or China. Hire a couple thousand locals to rewrite the legacy apps. You'll be fine.

    Security problem? Loss of jobs?!? In your mind senator.

e-credibility: the non-guaranteeable likelihood that the electronic data you're seeing is genuine rather than somebody's made-up crap. - Karl Lehenbauer