Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
Security News

Power Grid Insecurities Examined 248

Posted by CowboyNeal from the critical-looks-at-critical-services dept.
Joe Barr writes "Chris Gulker has taken a long and careful look at the infrastructure of our power grids and has come to some rather unsettling conclusions." A good read that outlines where the current power grid is at, and suggests some paths for the future that may help avoid future blackouts.
This discussion has been archived. No new comments can be posted.

Power Grid Insecurities Examined More

Power Grid Insecurities Examined

Comments Filter:

  • Shocking (Score:2, Funny)

    by Neppy ( 673459 )
    The insecurities in our power grid are quite shocking.

    • Re:Shocking (Score:3, Interesting)

      by SpaceLifeForm ( 228190 )
      Or insecurities in computers.
      Recent grid failure in the U.S. and Ontario may (likely?) be related to computer problems [theinquirer.net].
    • While the article was right when it comes to internal networks to the control stations (such as ISOs) the extent of insecurity in the energy bussiness is far greater that most people can think of. The fact of the mater is the reason most of the grid is immune to hacker attacks these days are the devices that control power transmission at the lowest level (relays, they control the circuit breakers) are all vt100/rs-232 terminal devices hooked up to aging modems 19.2 is the fastest I know of. Theses relays fo

  • Scared yet? (Score:5, Insightful)

    by krray ( 605395 ) * on Thursday September 04, 2003 @08:35PM (#6875348)
    Wonderful -- as I read the article, plastered in the center of the page is the ad:

    "Microsoft - Big business ambition. Small business resources. Get your FREE 6-month trial now. Windows Small Business Server 2003".

    The very fact that the power grid, atm's, so on and so forth -- hell, I worked on the power supply to a embedded PC today for a newspaper printing press that had NT on it ... it frankly scares the hell out of me.

    There I'll be sitting there in front of my OS X or Linux box. Can't be too smug I suppose with no power. No telephone. No gas. No cash to buy bread. Hell, the auto-checkout lanes (which I refuse to use on principle) at Jewel are Mickey-MouseSoft based. Certainly no Internet.

    For my business' I absolutely refused to allow a Windows server of any type in the datacenter. I still say, "are you nuts?". Yet people still did it. Once again, Bill Gates will get a chance to screw us I guess.

    So, when is the next worm due to hit? At least my TiVo will still work... :)

    • Re:Scared yet? (Score:2, Interesting)

      by randyest ( 589159 )
      Best part of the article, and hilarious:

      While legacy control systems are often UNIX-based ("Control-Alt-Delete scares power plant operators," Ahern said) and thus immune to MS worms and virii, their 10-megabit networking technologies can easily be overwhelmed. "Even the load from leading intrusion detection and monitoring systems can create a denial of service and shut these plants down," Ahern said.

    • Re:Scared yet? (Score:5, Insightful)

      by BWJones ( 18351 ) on Thursday September 04, 2003 @09:16PM (#6875553) Homepage Journal
      ... it frankly scares the hell out of me.

      Hey, it's not just the power grid and atm's. There are command and control systems used by the department of defense that folks have migrated to Windows. Our Dept of Homeland security has standardized on Windows. Certain FAA traffic control systems are running on Windows. The Army's Landwarrior program is using Windows. Traffic control for trains and shipping are running on Windows. etc...etc...etc...

      This should scare the hell out of a lot of people.

      • Personally, it would scare the hell out of me if they were using Linux, or OSX, or pretty much any desktop operating system for life-and-death tasks. Modern OS's like these are just too big to trust with your life -- they've all got bugs, and none of them have had the kind of scrutiny that they would need. If your system absolutely cannot go down, you can't trust code written by dozens of relatively unsupervised people.

        • Re:Scared yet? (Score:3, Insightful)

          by digitalunity ( 19107 )
          I highly recommend QNX [qnx.com] real-time OS. It is top notch. We have embedded devices here where I work that have *never* failed and some of them are running QNX. Just amazing stuff.
        • I can't agree more. OS X is my personal GUI of choice these days -- and yeah, since the beta release I've seen this thing go down maybe 4 times (not the "server" edition, not that it matters much). I was, each time, completely beating the hell out of the system -- and one of the times I had successfully mounted the core _live_ OS X file system (/) in a Linux based VirtualPC running on said file system. It didn't last too long... :)

          I've run Linux for years upon years without interruption and my record keepe
      • Actually while the display terminals are often running Windows, a lot of the traffic control for trains runs on a combination of big iron for the back end, true embedded systems for data collection and control modules and finally OS/2 (sometimes as old as v1.0!) for signals aggregation and distribution between the two extremes.

    • Re:Scared yet? (Score:4, Informative)

      by itwerx ( 165526 ) on Thursday September 04, 2003 @10:07PM (#6875815) Homepage
      The auto-checkout lanes at QFC and Safeway here in WA state are Linux. :)
      Now for those who read that article, here's a reality check.
      I worked on one of the Y2K project teams that did high-level analysis for a number of midwestern power plants.
      I can tell you that NONE of their control and monitoring systems were in any way connected to the Internet or even, usually, to any other networks internally.
      The reason cited in every case was security.
      The folks I worked with are called EPRI (Electric Power Research Institute) [epri.com] and they are widely regarded as the world's leading authority on national and international power generation and distribution systems.
      Check out their website, they often have some interesting white-papers available for public perusal.

    • Re:Scared yet? (Score:4, Interesting)

      by Anonymous Coward on Thursday September 04, 2003 @10:35PM (#6876073)
      I work at a company where we sell grid control sofware (SCADA software for in-market lingo)

      We had a product which used a particular UNIX, not a BSD or Linux, but the real high dollar, blessed by AT&T stuff. It hardly mattered because so many of our customers are not computer people, they are power engineers. They're not interested in event the user/group/everyone security model, they are interested in which breakers to open or close in the event a thunderstorm takes out this power line.

      As a result, many of the UNIX systems were set up for conveinence, not security, and anything that reduced conveinence created cries of frustration from our customers (and developers). Eventually we succumbed to pressure from our customer base, and now large portions of our system have been replaced with MS Windows systems. The customers (our power companies) love it.

      You can't sell security to those who don't want to buy it, but you can always complain when it's not there.

  • We should all generate power (Score:5, Interesting)

    by Anonymous Coward on Thursday September 04, 2003 @08:35PM (#6875350)
    In most states, if you generate your own power (ie solar), you can feed it back to the grid, and the electric companies are required to credit you! Any excess power you have can make you money. Sure, it's an investment up front to move to solar, but it is doable, and some states even offer tax credits.

    • Re:We should all generate power (Score:5, Interesting)

      by segment ( 695309 ) <sil@@@politrix...org> on Thursday September 04, 2003 @08:56PM (#6875452) Homepage Journal
      Sure, it's an investment up front to move to solar, but it is doable, and some states even offer tax credits.


      It's a nice thought but unless you live somewhere country-like, it's unfeasible to most people. Here's why, now firstly sure it is expensive to set up, but you would have to live in a geographically correct place as well. Say Florida, California, Arizona, Texas. States where it is rather sunny as opposed to say Seattle.

      You could use alternatives such as windmills, but again you would need massive space. When I was in Sweden, the government there was trying to limit where windmills could be used, as they often killed birds, some of which may have been rare, or on the verge of existence.

      I wish I wasn't too lazy and tired to offer links to prove my Swedish claims, but I'm sure anyone can find it on Google.

  • heh (Score:5, Insightful)

    by Comsn ( 686413 ) on Thursday September 04, 2003 @08:36PM (#6875360)
    Says Skroch: "If you have too much security [i.e., no network connections], then the power plant probably won't work."

    power plants worked long before the internet was created. no important computer controlling very important things should ever be put on the internet.

    • Re:heh (Score:5, Insightful)

      by Steinfiend ( 700505 ) on Thursday September 04, 2003 @08:47PM (#6875413)
      I cannot agree more with this, it amazes me every time I hear of some important computer system being affected by an internet based infection or an internet routed hack.

      Surely the only people who need to control a power plant (or dam release valves, or weapons sytems or whatever) are the people in the facility working at that time? So why have any type of network access to the system other than what is required within the grounds of the facility?

      Of course I might be being naive, but I don't think so.

      • The case for remote control (Score:4, Insightful)

        by Beryllium Sphere(tm) ( 193358 ) on Thursday September 04, 2003 @09:02PM (#6875484) Journal
        The valve at a dam probably doesn't need to be turned very often, so it's economically tempting to save the cost of 24/7 onsite coverage and have one central operations center.

        Remote monitoring is all but imperative. The plants are already in a cooperative network sharing their power. Everyone on the grid needs at least basic information about what's going on.

        None of which is ANY excuse for a direct or indirect connection to the public Internet. This is a job for a private network, and I don't mean a VPN that can be DOS'ed when a worm spreads through the public network.

      • Re:heh (Score:3, Insightful)

        by segment ( 695309 )

        So why have any type of network access to the system other than what is required within the grounds of the facility?

        It is a matter of convenience to be able to access offices from other offices, as we as people have become so lazy due to the boom in computer usage. It is much easier to be able to perform tasks using computers rather than doing things manually, and depending on what job duties you have, it can actually be a bit safer for the worker. However, in my opinion, people have just become lazy as

        • Re:heh (Score:3, Informative)

          by canadian_right ( 410687 )
          I live in BC Canada and we are 90% Hydro power, and most of the dams are in the middle of no where. All sites have people locally, but actual "production and control" is centrally managed to optimise system utilization and profits. Remote control and monitoring is done on private networks (much of it microwave) - there is NO connection to the internet and the control networks. All critical systems are multiply redundant (opening the wrong gate full open could flood a town). But there is a mix of old, new, a

      • Re:heh (Score:5, Interesting)

        by delcielo ( 217760 ) on Thursday September 04, 2003 @09:59PM (#6875786) Journal
        Economics come in to play here a bit as well.

        The market for buying and selling excess power is VERY active and exists primarily on the internet. Multi-million dollar deals are made quickly, and while they can be made in advance, they may also be made at the whim of mother nature (excessive heat causing a company to purchase power, or a drop in temp making excess power available).

        Implementing the deal means interacting with control systems. I will admit to ignorance of how this happens exactly; but I suspect that the traders aren't driving to the power plant or transmission control centers and doing it themselves.

        For a company that has efficient generation, they can make a great deal of money selling excess power. This means their customers don't have to pay quite as much.

        Here is the real issue: Everybody wants better security; but just tell anyone that you're going to have to up their rates to provide it and see what the reaction is.

        • Re:heh (Score:4, Interesting)

          by ebuck ( 585470 ) on Thursday September 04, 2003 @11:45PM (#6876520)
          Wish I had some mod points to add an insightful your way, because you're right.

          I've seen some of these "isolated" power-grid lans compromised because it was "critical" that the data be fed into the marketing department or server appliations which determined optimal generation schedules based on the ability to sell "excess" power when it's most profitable.

          The days of assuming you can secure via isolation are gone in the power market, but the debugging and testing cycles are so complete that it takes at least a year to implement a new anything. So despite CNN making this the "story of the year", a solution won't be available until well after the media decides that a particularly brutal murder is much much more newsworthy (or something to that efect).

          Meanwhile thousands of developers that have always assumed their code was safe from attack because of physical (ie isolation) security are now scratching their heads on how to refactor these systems while trying not to be sidetracked by the security rabble-rousers who are asking if the system will withstand the latest exotic attack X (which requires someone to duplicate almost valid messages via a morris code trainer attached to an ethernet cable).

          Unfortunately the most dangerous of these rabble-rousers come in two forms, lobbists and consultants. Although they complain the loudest about the problem, secretly they are in favor of keeping the problem around as long as possible because they only make money while it is still a problem. These people are rarely die-hard techs, but they know how to play the media like a violin.
      • A friend of mine works on fligh simulators. The fligh simulators are based on specialised computers doing the motion and the graphics - but they have no file store as such. They rely on 4 XP-Pro Pcs to serve data files that are used by the real time computers.

        He was upgrading a simulator belonging to a well known German airline company and this meant pulling interface cards. As the XP systems came from the simulator company, they were not running corporate edition so they came up asking for a new keys in

    • Re:heh (Score:4, Informative)

      by Jordy ( 440 ) <jordan@NOSPam.snocap.com> on Thursday September 04, 2003 @09:23PM (#6875591) Homepage
      power plants worked long before the internet was created. no important computer controlling very important things should ever be put on the internet.

      Network connections != internet connections. Current power systems have network connections since it is kind of nice to be able to monitor it from time to time. They typically run over fiber rings independent from the power grid itself.
    • power plants worked long before the internet was created. no important computer controlling very important things should ever be put on the internet.

      That's like saying that people lived just fine without electricity 200 years ago, so we should all stop using it now so we don't have to worry about blackouts.

  • Security Vs Usability (Score:3, Insightful)

    by Admiral Justin ( 628358 ) on Thursday September 04, 2003 @08:43PM (#6875393) Homepage Journal
    The article does bring up a valid point. Many times, when large systems attempt are forced into security by fear, they overdo it, and the system becomes nearly unusable to the users, who have to run around in circles with security measures.

    The lesson? Security is nice, but lets not go biometrics and 30 different passwords just to check the email.

  • Stock up on booze and smokes (Score:4, Insightful)

    by soupforare ( 542403 ) on Thursday September 04, 2003 @08:45PM (#6875400)
    "The situation is so bad, experts say, that bored script kiddies could soon be knocking out power stations as easily as they concoct viruses from toolkits available on the Web."

    Is it any easier now then it has ever been? It always seemed pretty simple to me. Go down to your local, unmanned, power station and blow it up. Get your buddies and some trucks and knock down some high tension wires. wheeeeee.

    Why do people get excited by this? It might be my misanthropic nihilism talking, but shit happens. Every day. Deal with it.

    You might lose power, you might lose running water, you might get hit by a bus.
    Even if you hole up in a shack to protect yourself from the script kiddies, psychopaths, terrorists and/or government... you're still gonna die!

    Have fun! :)
    • no, that's always been booze and guns and ammo. the survivalists always tuck away hooch and hoglegs... whiskey for trading, bangsticks for defense... with their six-month dry food kits.

      now, now many valved gel-cell batteries should you stash to keep the MP3 server running when society collapses?
      • If we're really talking end of civilization disruption, and not one due to a transient riot/earthquake/brownout, then you don't want VRLA batteries. You want plain old flooded batteries stored dry that you can add water to, top off when too much water gets boiled off.

        VRLA batteries are better for unattended operation, but if you want to play the survivalist's game, you need batteries designed for extreme long-term operation, and a charging/generation system to back it up.

        The best set up would be a wate
    • You might lose power, you might lose running water, you might get hit by a bus.

      Indeed. I often find myself reminding people of this.

      It's a weird and messy world: water falls from the sky.

  • Well, what did they spend all my payments on... (Score:5, Insightful)

    by BSOD from above ( 625268 ) on Thursday September 04, 2003 @08:45PM (#6875403) Homepage
    The power industry needs to be reinvesting profits in infrastructure (powerlines), not stock dividends. The same companies should have been upgrading their command and control systems to prevent chain reaction blackouts. Am I expected to believe the computer systems that manage the cooling rods in the nearest nuke plant are secure?

    Seriously consider the economic impact of the grid failure compared to the recent worm problems. Then think about a nasty combination of the two.

    • Spent on Enron-style energy trading companies (Score:5, Insightful)

      by swb ( 14022 ) on Thursday September 04, 2003 @09:01PM (#6875480)
      It used to be that the utilities were highly regulated entities that had their profit margins basically regulated by the states they were in. They had to provide a given amount of reliability, and rate increases (and occasionally refunds!) were carefully scrutinized as to where the money went. You couldn't raise rates without showing some meaningful improvement that resulted from it.

      Then along came degregulation, where the power seller and the power generator became two different things (which makes even less sense than the deregulated-but-shared local phone loop). Utility companies wanted out of the power generation arena -- too expensive, too many regulations, it was better to be in the new "commodity" end of the business, arbitraging power. So they split themselves into trading companies and generation companies, taking all the cash into the trading companies, who were deregulated and could spend it freely.

      And then 10 years later, Enron and the whole deregulated power "market" has collapsed, and we wonder why we're 15-20 years behind the curve on power grid and other key infrastructure elements. All the money got spent on speculating in the newly deregulated power markets, and its all gone.

      Nobody really pays any less for electricity, I don't have a bunch of people knocking on my door offering me their window electricity or biodiesel electricity or their pig shit methane electricity for that matter.

      I only have the sheepish looking local utility trying to explain to me how they're trying to fix the power infrastructure built in the 1970s with the cash made in the 1980s which was spent in the 1990s on the promise of getting rich in the new millenium. When in fact, they actually need me to pay the prices of the next millenium for the service delivered in the 1990s, and, oh, would I please only use as much power as I did in the 1970s?
      • For purposes of this discussion, in the industry there are two things: generators and high voltage lines.

        Now, once upon a time in the good old US of A, an official of a steel plant woe'd the outrageous slings suffered at being forced to buy energy from a utility due to that fact that his plant was located in said utility's fiefdom. In the industry, this is urban-lore explanation of how deregulation started.

        Guv'ment steps in. There'll be no Ma-Bell style bust-ups; rather, generators will be managed
    • Well the current setup is very bad for investment. First of all there is enough capasity for 95% of the year, which means that anything new you build is only going to get used on a minimal basis at first. But still has to be paid for year round.

      Second in the Area hit by the blackout you have a regulatory patchwork mess. You have two sets of federal regulations (USA + Canada) Plus a large number of state and provintial regulations.

      If some power company were to go to wall st to raise money to build a power
    • Am I expected to believe the computer systems that manage the cooling rods in the nearest nuke plant are secure?


      maybe not?
      there was a story after the blackouts that back in Febuary 2003 a nuke power plant in Ohio somewhere lost it's safety systems for over 5 hours because of a worm/virus that took down the M$ system they were running. The story was on the news the same day they were reporting the Blaster worm messing up the switches in a Baltimore train yard. yikes!

  • Is Linux the latest "silver bullet"? (Score:5, Interesting)

    by KNicolson ( 147698 ) on Thursday September 04, 2003 @08:45PM (#6875404) Homepage
    That article read a bit like an advertorial for Verano (some Linux SCADA security company), with the "Oh, if we only had Linux all this wouldn't have happened!" conclusion.

    However, reading the text, the problem seemed more that the plant operators had indiscriminately attached critical systems to the Internet without proper firewall security in place, which seems to me to be a human, not a computer or OS, flaw.

    • It's too bad that people still connect systems to the internet without any kind of firewall.
      Let's be serious though...joe schmoe with his cable modem should be able to plug his computer directly into the internet without having to worry about it getting compromised. In the perfect let's-hold-hands-in-a-giant-circle-jerk world, this would be true. Linux may not be the silver bullet, but it certainly gets exploited a lot less that its friends. On the other hand, that could be attributed to the sheer volume o
      • On the other hand, that could be attributed to the sheer volume of Windows machines out there.

        This is almost certainly the primary reason. How many people actually keep up on all the security problems in commonly-installed Linux software? (It's harder than windows security flaws, because generally the Linux problems don't get posted to the /. front page a dozen times.) Now, imagine that everyone who uses Windows today was using Linux. Do you really think that the patched/vulnerable ratio would be any
    • The only firewall fit for the link between the Internet and a SCADA system is an AIR GAP ffs.
    • seems to me to be a human, not a computer or OS, flaw

      Actually it seems to be the MS mantra at the moment. "Darn those pesky humans, without them our OS would be perfect."
    • I liked the bit about their product creating an air-gap as soon as an intrusion is detected.

      An air-gap is the only perfectly secure way of preventing outside access. However, an air gap only exists when two networks have no connection between them. A super-safe-hyper-firewall-box which connects the two does not create an air-gap - it is a firewall. Now, a firewall can be a good thing, and this product may very well be the best firewall which has ever been designed, but it isn't an air-gap. If the devic

  • Leave Power Grid alone you big meanies! (Score:5, Funny)

    by GuyMannDude ( 574364 ) on Thursday September 04, 2003 @08:47PM (#6875409) Journal

    Well of course Power Grid is feeling particularly insecure right now. I mean it's old and weak and obsolete and just got caught with it's pants down a few weeks ago. That kind of spectacular failure is bound to make anything or anyone feel pretty insecure. I doubt the last thing Power Grid wants is to have its insecurities examined publically! C'mon, people, let's not kick it while it's down!

  • Very nice commercial (Score:3, Interesting)

    by cspenn ( 689387 ) <financialaidpodcastNO@SPAMgmail.com> on Thursday September 04, 2003 @08:58PM (#6875464) Homepage Journal
    ... for Verano.

    And if you connect ANY critical operating system to the Internet, frankly, you're insane. There's no sensible reason to do so. Monitoring your systems is fine, that's what a management network is for... but the actual core of the critical system should be as close to that powered-down concrete encased computer as possible.

  • Garbage (Score:5, Informative)

    by Anonymous Coward on Thursday September 04, 2003 @09:01PM (#6875479)
    Did anyone actually read this garbage before they posted it. This is absolute nonsense. The blackout had _nothing_ to do with computers, much less internet security. The blackout happened because a half-rate utility (First Energy) tried to squeak through an emergency without buying expensive power or shedding load. Period. They operated lines until the sagged into brush. Some small subtransmission and distribution lines had twice rated load. Do the math. That's four times the temperature or over 400C. That had zippo to do with M$ or any bleepin' computer.

  • Legacy = Semi Safe, Microsoft = Unsafe (Score:5, Interesting)

    by Bruha ( 412869 ) on Thursday September 04, 2003 @09:04PM (#6875498) Homepage Journal
    Legacy systems will provide more resistance to viruses than any MS based system mainly due to the lack of coders with the knowhow to write viruses for such systems. Though when paried next to and on networks containing Microsoft based systems a MSVirus could cause havoc just by crippling the network that those systems rely on.

    In any case a system using NFS/NIS would be especially vulnerable to traffic floods by MSVirii due to the lockups that can happen when high traffic causes such file/security systems to fail.

    I've seen flapping interfaces on certain cisco equipment that have made messes of NFS and NIS based systems requireing a total reboot of the entire network from the top down. And the flapping can be caused by recent MSBlaster virii that has recently seen action.

    As a safety precaution the legacy networks should be extremely firewalled, and not allowed to work on any shared media that also caters to any Microsoft systems. Such seperation of the network would prevent either from spamming the other to death. Also in many critical areas private networks with private loops vs being carried over the internet should be considered with backups such a MicroWave or Sattelite communications to critical centers in case of any large infrastructure outages in your carriers network.

  • I'm sure the government will step in and ensure changes are made before we have a massive blackout.

  • Power Grid (Score:4, Informative)

    by hardburlyboogerman ( 161244 ) <kwsmith41747@windstream.net> on Thursday September 04, 2003 @09:04PM (#6875504) Homepage Journal
    I have taken myself off the grid years ago,using Solar,Wind,Hydro power(tapped into the abandoned Hardburly Deep mine and using the water to generate power) and have a 20kw diesel generator for backup.
    Most of the power grid problem stems from the fact that very little maintainence is being done.The Power lines out here have been here since the late 1950s or early 1960. Every time it rains,you can watch an electricial light show less than 50 ft from my home.(Phone calls to the power co.does no good,so I informed the Public Service Comission about it,sending a video tape of the light show.AEP now has 10 days to change the lines out or get fined to the tune of $50k/day!)
    Greedy utilities have brought this on themselves.Cutting jobs for the maintainence personell,doing nothing about aging lines, and then asking "WHY is this happening?

    "We call ourselves Homo Sapiens Spaiens.Our true name should be Home Stupidus"

    • Re:Power Grid (Score:3, Insightful)

      by Tailhook ( 98486 )
      "Most of the power grid problem stems from the fact that very little maintainence is being done."

      "Greedy utilities have brought this on themselves.Cutting jobs for the maintainence personell,doing nothing about aging lines, and then asking "WHY is this happening?"

      There is nothing wrong with the "old" lines. The distribution grid carries some rated voltage and does it without much complaint. The problem is that there simply isn't enough of it, so most of the system is running at design capacity, and a

  • MS Blaster is NOT at fault!! (Score:3, Interesting)

    by edison490 ( 551402 ) on Thursday September 04, 2003 @09:16PM (#6875556)
    I work for a utility in protection and process engineering and we do not have any remote ability to change settings. As stated in the comment section of the article control and protection systems do not normally have any remote access even to on-site network operators. This philosophy protects everyone from the utility (employees/technicians) to the customer.
    One key issue that seems to be on everyone's mind is the latest MS Blaster virus, could it have caused the outage? Not likely. As stated above our protection and control systems send data via leased phone lines and/or private fiber and do not have any connection to the Internet. Thus no possible way of receiving a virus.
    Finally, to all of you who are dying and just can't understand why the investigation is taking such a long time...hang on! Part of my job is to study disturbances on the grid (ie why did the lights go out?). The studies take anywhere from a day to months to explain what happened. And remember the 1965 blackout study took over a year to finish.
    • As far as I'm concerned, this is the big news to come out of the whole incident, and it's apparently being suppressed by the mainstream media - the MS Blaster Worm could have caused the blackout. Here's what the article has to say on the subject, referencing problems with an earlier worm at a nuclear plant:

      ----

      The Slammer worm penetrated the plant's internal network and lodged in an unpatched Windows server. The worm's scanning slowed the internal network to a crawl, eventually crashing the plant's Safet
    • MS Blaster may not, as you state, be the fault. My initial reaction was this outage was likely a cascade from a physical failure as was the case in 1965. In all probability, this will be a complicated set of problems that led to a systemic failure.

      However, this is not 1965 and our systems should have improved since then. If the monitoring systems were reliant on MS operating systems, Internet-exposed and compromised by Blaster, that could be a problem. The suspect utility had a nuclear plant that got h
      • The machines doing the monitoring wouldn't even need to be running a Microsoft operating system in order to be taken offline by a worm that exploits one of the NUMEROUS security holes in Windows. The non-Microsoft machines - or monitoring devices they rely upon - could be knocked offline by the flood of packets generated by compromised Windows boxes on the same network.

  • Finally... (Score:2, Funny)

    by rune2 ( 547599 )
    A vurnerability that isn't Microsoft's fault. I suppose that we could blame them anyways though.... just for the fun of it.

  • "Virii" (Score:3, Informative)

    by jemfinch ( 94833 ) on Thursday September 04, 2003 @09:41PM (#6875687) Homepage
    Maybe I'm just being an anal-retentive grammar Nazi, but I simply can't respect an author who uses the non-word "virii" in his works.

    Sorry. It's simply not a word [perl.com]. He might as well be writing in l33tspeak.

    Jeremy
    • It IS a word because it is in common usage and understood by the majority of the audience. It may be slang but it is inarguably a word.
  • The software and management side don't tell the whole story. Combine that with the power grid physical security and infrastructure issues and then you have a glimmer of how thin the electric thread we depend on really is. That's not being paranoid, that's being practical. It's a challenge from a cost position to be completely grid independent, no matter where you live. But it is feasible, at least technically, to be less grid dependent. The best cost/benefit balance I've found is to have enough wattage to
  • From the article:

    The worm's scanning slowed the internal network to a crawl, eventually crashing the plant's Safety Parameter Display System, according to reports.

    [snip]

    Control systems operate in real time, where processes, availability, and reliability are paramount.

    So they are imposing realtime requirements onto a shared medium (a computer network)? That's like not putting lights or sirens on emergency vehicles, and then complaining about not being able to get to the scene in time during heavy tr

    • Re:Data Networks & Realtime Requirements (Score:4, Interesting)

      by ebuck ( 585470 ) on Friday September 05, 2003 @12:36AM (#6876772)
      Actually, they were wonderfully designed.

      Read the research documentation that came out in the 80's, the pinnacle of SCADA system research.

      Oh, and then that pesky TCP/IP became available, so people moved from tons of serial cables to cheaper CAT3/5. If you didn't migrate your system, you went out of business. Problem is, who could afford to re-design their software from the ground up to use a non-realtime network in a manner resembling realtime?

      So SCADA has long moved from "real-time" to "really fast". Or they isolate the real-time requirements to parts of the system where it can still be achived.

  • Bull. (Score:5, Interesting)

    by Telecommando ( 513768 ) on Thursday September 04, 2003 @10:12PM (#6875876)
    Hackers controlling the power grid? Utter and total bull.

    I work in IT for a major power company. Our control systems have never been hooked to our own network, let alone the Internet, and never will be. How stupid does this guy think we are?

    We've been running computerized control systems in nuclear and other types of generation plants for years. We've had computers in substations and control stations monitoring, controlling and reporting status before most industries even knew what to do with them. I saw my first Z-80 processor in a SCADA system shortly after the Z-80 came out. It could talk any of 5 different control protocols and replaced 2 seven-foot racks of hot, high-current RTL and DTL control logic. It was a thing of beauty.

    We're not newbs at this. And no way do any of our control systems run Windows. Get real.

    Why would we even want to hook up a generating plant or substation to a network just so it can be controlled from anywhere in the world, BY ANYBODY? No way. No how. Nuh-uh. Ain't gonna happen.

    We can't even monitor what's happening on the system from the company's own computer network. It's all totally seperate. And for good reason. Who wants a disgruntled employee or just some joker who's bored messing with the system? The only people who can make operational changes to the system are the people actually present at the secured control center or at the generation plants.

    We run quarterly modem audits, company-wide, looking for unauthorized lines with modem. We even restrict who gets an analog phone line and whether they can receive calls on that line. Computers attached to the control systems get NO modems. Never ever.

    Even our remote monitoring terminals at regional work centers require dedicated connections to the control center and are receive only. The control computers think the remote monitors are printers and only send data, not receive so they can't be hacked from there either.

    It's impossible to get to our control system through the Internet. It could probably be done to some degree (perhaps sending a 'breaker open' command to a key substation, if you know which one), but only by hijacking an existing dedicated connection undetected, which is getting harder as we connect stations via fiber optic.

    (Often we connect stations by installing the fiber near the high voltage lines on our towers, a security measure in and of itself. Imagine splicing a broken fiber hanging off a helicopter platform while the line 12 feet below you is energized to 350 thousand volts. No, I haven't done it, but I watched it being done and the crew earned every penny.)

    If any utility out there has their control systems connected to computers that can be reached via the Internet (or modem for that matter), the persons responsible should be taken out and shot. Then taken to a doctor, stitched back up and shot again. Same for their bosses all the way up to the CEO.

    Sorry if I seen a bit testy on this subject, the subject of keeping the control system secure has been drilled into me for more years than I care to remember. Now it's just automatic.

    However, on the subject of aging infrastructure, I totally agree. I blame deregulation. Every utility is now trying to cut each other's throat trying to grab customers away from each other. To cut costs (and thus lower their prices to better compete), most if not all utilities have cut their expenses by eliminting maintenance, lengthening replacement schedules and cutting staff, specifically skilled line workers). It's a race to the bottom to see who can provide the cheapest service. And it will probably go on until the whole thing blows up on them. And unfortunately, us as well.

  • Bored script kiddies would never do this... (Score:4, Insightful)

    by thepacketmaster ( 574632 ) on Thursday September 04, 2003 @10:34PM (#6876068) Homepage Journal
    A script kiddy would never bring down the power grid...If they did, they'd be bored out of their Internet-dependent minds. Can you imagine these types of kids playing scrabble or cards?!? Or worse yet, being forced to take the opportunity of a black-out to spend quality time with their families. The Horror!
    • I guess causing a power outage is the hackerish way to test a UPS for proper operation. Its "better" than pusghing the test button or pulling the plug on the UPS itself because it ensures that you did not forget to plug the wall wart for the router into the UPS. It also simultaneously tests all the UPSes (UPI???) in the house/office. It will also tell you if your local internet connection (be it modem, DSL, cable, someone else's Wifi net, etc.) is dependent on the local grid.
  • I believe the protocol used to manage this stuff is SCATA or something like that. I also believe that it's security model is quite like telnet.

    I have heard they were going to beef up their security but I wonder if they ever did.

  • Like a lot of "market failures", this one has arisen because of boundary issues between private and public spheres of ownership and control.

    This article from the Von Mises Institute [mises.org] explains it far better than I ever could.

  • You all think too high tech... (Score:3, Interesting)

    by Anonymous Coward on Thursday September 04, 2003 @11:53PM (#6876573)
    If there's anything that 9/11 taught me (and should have taught the rest of us), it's that sometimes, the "best" attack is a low-tech one...

    We can have high-tech biochemical sniffers looking for anthrax and C4, etc., but who really would have thought of stealing a plane or two and flying it into a building? Really - think about it. It's pretty low tech, but extremely effective...

    Same thing with the power infrastructure - why worry about hacking in? Figuring out passwords and all that nonsense when the FUCKING INFRASTRUCTURE IS OUT IN THE OPEN!?!?!

    Drive down any road - and you're likely to see a power line, a transformer, etc... I'm sure we ALL know where at least one substation or transmission line is located. AND they're out in the open...

    Have the brains engaged yet? Think about it folks - dig out the old graph theory notes from your data structures classes and then plot out the national power grid -- just the big ole transmission lines...

    What happens if you make some cuts in that graph? Wanna bet that about 7 pieces of wire would do it?

    You don't even need explosives... some wire, maybe a bicycle chain or two and a modified potato launcher would do the trick... and blamo - lots of chaos and commotion... (and yes, I DO know someone who was a complete moron when he was 14 yrs old and tossed a bicycle chain into a transformer at a local substation.... but I digress).

    How are you planning to protect the entire infrastructure against attack? Even if it's redundant, and resiliant - a bit of thought and you're right back where you started....

    I don't have solution to this intractable problem - Do You?

  • Canada Who??? (Score:3, Interesting)

    by magical22 ( 664542 ) on Friday September 05, 2003 @02:00AM (#6877086)
    I am sick of control, this might not be the right place to talk out about canada's problems in general but lets say the US already has control of our power, as proven with the california state vs bc hydro, they also control our lumber industry (softwood trade agreement), our wheat industry, our cattle industry (thanks to mad cow), we might aswell give it up or get invaded at this point. No one cares about us and we are so small that we get bullied into everything anyways. I say divert all the rivers leading into the states into the lower half of Alberta and Saskachewan (to those not familiar with canada its the 2nd and 3rd most western provinces) cut the power lines (thus fixing the grid problem), stop all exports and imports to the states, and give them the middle finger.

  • Poor analysis, but there are real problems (Score:5, Insightful)

    by Animats ( 122034 ) on Friday September 05, 2003 @02:08AM (#6877118) Homepage
    That's not a "long and careful look". It's more like "general mouthing off".

    We're starting to see a few problems appear more than once, though.

    • Telecom vulnerability to power failure.

      AT&T was determinedly independent of the power grid in the days of Ma Bell. Every central office ran on 48VDC storage batteries, with backup generators. The backup generators were started once a week, and run for several hours once a month. Once a year, each central office ran for 24 hours cut off from external power.

      That was a long time ago, back when AT&T was a regulated monopoly common carrier. In the new, competitive era, that depth of backup can no longer be assumed. Carriers in trouble (WorldCom, Adelphia) tend to cut things like that.

      The details aren't in yet, but it's beginning to look as if, during the recent big blackout, some comm links went down very early, so that the fault information that's supposed to divide the grid cleanly into islands didn't get through. Once all the logs have been correlated, it will be clear what happened.

    • "Non-critical" systems that aren't.

      A few weeks ago, CSX, the railroad, had a shutdown due to a virus. Railroad signalling has used "code lines" for decades, for remote control of switches and signals. These are basically serial links over which commands and responses are sent. The safety logic is local, but if you lose a code line, the dispatcher can't throw switches and route trains.

      The tendency to centralize train control has resulted in a need to transmit code line signals hundreds or thousands of miles. So they tend to be multiplexed over telecom-like facilities. CSX apparently routed theirs over their in-house general purpose network. The routers in that network were managed by a network management system that ran on Windows. When the Windows machines went down, system management of the routers stopped, and, after a while, this apparently took some key routers down. So a "non-critical" system actually stopped train movements.

    • Cross-connection between business systems and control systems

      It's really convenient to be able to see what the plant is doing from your desktop. Order processing is more efficient if the sales network connects to the factory network. Energy traders need to be able to see what the power plants are doing, and give directions to power dispatchers. These things all create vulnerable paths.

    That's a more realistic picture of what's going on.
    • Telephone companies are the only real carriers in the US. (ATT, SBC, Sprint, MCI, etc..) By Federal law telco's must stay up in the event of disaster. There is a direct relationship between communication and death in the event of a disaster. When the WTC fell in NY the ATT telco switch in the basement was still up. Comm links went down because telcos and businesses are trying to save a buck or two. So they sign contracts and pass communications through "wanna-be" carriers like Verio, Cogent, Level3, etc...
  • YOu'd think that nuclear power station control systems would be connected to the net. THey should be stand alone. Whats doing with that.

    'Its pronounced New-cu-ler, Honey' - Homer

  • Ethernet is a bad choice in this environment (Score:4, Interesting)

    by RevMike ( 632002 ) <{moc.liamg} {ta} {ekiMver}> on Friday September 05, 2003 @07:30AM (#6878189) Journal
    While legacy control systems are often UNIX-based ... and thus immune to MS worms and virii, their 10-megabit networking technologies can easily be overwhelmed.

    ...corporate firewalls tend to focus on protecting data integrity and are not suitable for protecting control systems. Control systems operate in real time, where processes, availability, and reliability are paramount.

    I'm assuming whenthey say 10 megabit they mean 10 megabit ethernet.

    Repeat after me: "Ethernet is not an appropriate networking technology for industrial control systems!"

    This is exactly the type of environment that tokenbus (IEEE 802.4) was designed to handle. Tokenbus can guarantee QoS and does not require a "master" node, so it is immune to that kind of single point of failure. Tokenbus was designed with factory automation in mind - IIRC the major auto manufacturers in the US were big players in the committee - so it is optimized for the industrial environment.

    FYI, tokenring is similar, but not identical. Tokenring is a simpler standard that requires a master node. A ring can be locked up if the master node goes into a strange state. Rings are fit for applications where a network failure would be inconvenient, not tragic.

  • It seems to me the that real problem is inadequate transmission capacity coupled with sky-rocketing demand. Everyone wants to turn on their air conditioner and power-hungry PCs, but nobody wants to have a power line in their backyard. Throw in parochial state utility regulatory boards, half-hearted attempts at deregulation, clueless execs at utilities, and Enron and you have the makings of a bigger mess than even Microsoft can create.

    Its too bad we can't just double the clock rate on the power line an

  • Real problems with the grid (Score:3, Informative)

    by tjstork ( 137384 ) <todd@bandrowsky.gmail@com> on Friday September 05, 2003 @08:08AM (#6878506) Homepage Journal

    The real problem with the grid is that the midwest and the south have not modernized their --people- systems. The PJM grid and to some extent NEPOOL have been moving to a more RTO model that allows for a good balance between a clear market and the command and control necessary to avert disasters.

    First Energy made the wrong decisions during the blackout. Let us recall the sequence of events.

    a) High voltage lines from Canton to Cleveland drop off line
    b) Cleveland begins pulling power from the rest of the grid
    c) Normally outbound power from the midwest begins to "flow" back to the midwest.
    d) This causes power plants in Michigan to trip off line... by this time the regional disaster was largely guaranteed.

    The correct move for First Energy would have been to disconnect Cleveland from the grid off line, immediately.

    Even better, had First Energy had a decent vegetation removal program, the transmission line would not failed in the first place.

    So basically, had First Energy kept the lines clean and been willing to bounce Cleveland from the grid, their would have been no wider blackout.

    But they didn't. They are a utility, not a regional grid operator.

    Had this happened to say some power lines from some place to Philadelphia, PJM would have yanked Philly from the grid, told the utility to fix the lines, and there would be no wider blackout.

    And, by the way, PJM has a more transparent networking market. Just look at the whose got the better web site, PJM or Midwest ISO?

Slashdot Top Deals

It's a naive, domestic operating system without any breeding, but I think you'll be amused by its presumption.

Close