Power Grid Insecurities Examined

Joe Barr writes "Chris Gulker has taken a long and careful look at the infrastructure of our power grids and has come to some rather unsettling conclusions." A good read that outlines where the current power grid is at, and suggests some paths for the future that may help avoid future blackouts.

Garbage

[Anonymous Coward]

Did anyone actually read this garbage before they posted it. This is absolute nonsense. The blackout had _nothing_ to do with computers, much less internet security. The blackout happened because a half-rate utility (First Energy) tried to squeak through an emergency without buying expensive power or shedding load. Period. They operated lines until the sagged into brush. Some small subtransmission and distribution lines had twice rated load. Do the math. That's four times the temperature or over 400C. That had zippo to do with M$ or any bleepin' computer.

Power Grid

[hardburlyboogerman]

I have taken myself off the grid years ago,using Solar,Wind,Hydro power(tapped into the abandoned Hardburly Deep mine and using the water to generate power) and have a 20kw diesel generator for backup.
Most of the power grid problem stems from the fact that very little maintainence is being done.The Power lines out here have been here since the late 1950s or early 1960. Every time it rains,you can watch an electricial light show less than 50 ft from my home.(Phone calls to the power co.does no good,so I informed the Public Service Comission about it,sending a video tape of the light show.AEP now has 10 days to change the lines out or get fined to the tune of $50k/day!)
Greedy utilities have brought this on themselves.Cutting jobs for the maintainence personell,doing nothing about aging lines, and then asking "WHY is this happening?

"We call ourselves Homo Sapiens Spaiens.Our true name should be Home Stupidus"

Re:heh

[Jordy]

power plants worked long before the internet was created. no important computer controlling very important things should ever be put on the internet.

Network connections != internet connections. Current power systems have network connections since it is kind of nice to be able to monitor it from time to time. They typically run over fiber rings independent from the power grid itself.

"Virii"

[jemfinch]

Maybe I'm just being an anal-retentive grammar Nazi, but I simply can't respect an author who uses the non-word "virii" in his works.

Sorry. It's simply not a word [perl.com]. He might as well be writing in l33tspeak.

Jeremy

Re:The grid is over centralised

[Angry White Guy]

Covering your roof with solar cells is not a practical solution at this point. Covering your NEW house is. Retro-fitting tiles onto your house is generally not the easiest, cheapest, or safest way to generate power. The amount of tiles needed alone to power your house would be staggering, let alone the storage batteries, the need for the proper exposure (I believe that they reccomend a large southern exposure for most of the U.S. and Canada, for best results), or the power inverter which needs to be retro-fit into existing wiring.

New construction and large office buildings are where solar should be targeted. Also, these buildings should be routinely inspected to make certain that they are not feeding power back into the grid in blackout conditions. Hydro workers have a hard enough time during a blackout without worrying whether some good samaratin numbnuts has just energized the segment he's working on or not.

To feed the grid via private enterprise without safety precautions, well thought out implementation plans and regulation would be at best ill conceived, and at worst, homicidal.

If you want power during the next blackout, buy a generator, and for the love of god, shut off the main!

Re:Scared yet?

[itwerx]

The auto-checkout lanes at QFC and Safeway here in WA state are Linux. :)
Now for those who read that article, here's a reality check.
I worked on one of the Y2K project teams that did high-level analysis for a number of midwestern power plants.
I can tell you that NONE of their control and monitoring systems were in any way connected to the Internet or even, usually, to any other networks internally.
The reason cited in every case was security.
The folks I worked with are called EPRI (Electric Power Research Institute) [epri.com] and they are widely regarded as the world's leading authority on national and international power generation and distribution systems.
Check out their website, they often have some interesting white-papers available for public perusal.

Re:heh

[canadian_right]

I live in BC Canada and we are 90% Hydro power, and most of the dams are in the middle of no where. All sites have people locally, but actual "production and control" is centrally managed to optimise system utilization and profits. Remote control and monitoring is done on private networks (much of it microwave) - there is NO connection to the internet and the control networks. All critical systems are multiply redundant (opening the wrong gate full open could flood a town). But there is a mix of old, new, and inbewteen systems. Most systems are proprietary and it is hard to get information about them that would be useful to hackers. We do run some non-critical monitoring over our normal private intranet.

I'm sure many other utilities are similar, but I think the real problem is that with deregulation there is zero incentive to build new reliable infratructure like transmission lines. Why would you spend a penny on a new transmission line when the current one is only 87% utilized 75% of the year? The old monopolies did do this kind of long range planning and upgrading. It still gets done, but not until the last minute.

Re:Bull back at you

[Anonymous Coward]

Not a troll.

I also work installing SCADA control centers, and yes this does happen. However, usually there's a extreme lack of windows hosts on our control systems so virii are not much of a problem.

The parent works in a company where they're doing things right. Audits, checks, and a lot of hard work to ensure that the system stays secure.

But sometimes I install a software upgrade, only to notice a new host on the system... Well, someone was only trying to leverage the "extra" ports on the switch. Or marketing needs access to the historical records for analysis. Big companies which are prepared to take security seriously have no problem, but there are others...

Horrible others, which have personnel connecting homebrew "proxy" boxes so they can view the web after hours. Systems where every operator has the same dictionary password. Systems where the security camera video feeds get "rerouted" to allow the viewing of Sienfeld. Systems where the SYSTEM ADMINSTRATOR can't remember how to change directories in UNIX or the difference between a command, and that command's argument.

These dark corners are usually cash strapped companies, so yes they scheduled to replace X five years ago, but hey, it sill works, so let's get our money out of it.

Unfortunately I have to post anonymous, as I still intend to make my living scrambling to refactor for security

Re:The grid is over centralised

[sbryant]

It's hardly something I'd want to rely on as a primary source of power, but it would definately help on those hot sunny days when everyone is running an air-conditioner.

On a hot sunny day, a solar panel will help you much less than you think. Their efficiency decreases when it gets hot (ie: direct sunlight). On the other side of the scale, solar panels are still quite effective on overcast autumn days.

A normal set of panels on a house roof will generate enough electricity for 3-4 houses during the day. You still have the main grid as your backup, and you can often sell your excess back to the power company. A lot of setups have battery installations, which can run your house at night.

-- Steve

Re:Poor analysis, but there are real problems

[joe_cisco_was_here]

Telephone companies are the only real carriers in the US. (ATT, SBC, Sprint, MCI, etc..) By Federal law telco's must stay up in the event of disaster. There is a direct relationship between communication and death in the event of a disaster. When the WTC fell in NY the ATT telco switch in the basement was still up. Comm links went down because telcos and businesses are trying to save a buck or two. So they sign contracts and pass communications through "wanna-be" carriers like Verio, Cogent, Level3, etc... these guys are not phone companies people, wake up. Also, UPS systems must link to generators. If a faliure in this chain then power problems happen. 79% of most power outages are caused by failed UPS systems, generators no kept warm and tested. Comm links also went because the general power infrastructure of the facilities they use sucks. Comm facilities or CO's should be using this power system: "Hitec CPS (Continuous Power Systems) units on-site, identical to power backup systems utilized by the U.S. Department of Treasury, NATO Radar Silo installations, Intel, IBM and the air forces of Israel and Brazil. 60,000 gallons of fuel stored on-site for 72 hour full-load power capacity." This power system is also provided in the Internet Data Center I use via Pacific Business Solutions. You should check out Sfcolocation at www.sfcolocation.com or pb-solutions.com (pretty pictures of HITEC power systems and more details)

Real problems with the grid

[tjstork]

The real problem with the grid is that the midwest and the south have not modernized their --people- systems. The PJM grid and to some extent NEPOOL have been moving to a more RTO model that allows for a good balance between a clear market and the command and control necessary to avert disasters.

First Energy made the wrong decisions during the blackout. Let us recall the sequence of events.

a) High voltage lines from Canton to Cleveland drop off line
b) Cleveland begins pulling power from the rest of the grid
c) Normally outbound power from the midwest begins to "flow" back to the midwest.
d) This causes power plants in Michigan to trip off line... by this time the regional disaster was largely guaranteed.

The correct move for First Energy would have been to disconnect Cleveland from the grid off line, immediately.

Even better, had First Energy had a decent vegetation removal program, the transmission line would not failed in the first place.

So basically, had First Energy kept the lines clean and been willing to bounce Cleveland from the grid, their would have been no wider blackout.

But they didn't. They are a utility, not a regional grid operator.

Had this happened to say some power lines from some place to Philadelphia, PJM would have yanked Philly from the grid, told the utility to fix the lines, and there would be no wider blackout.

And, by the way, PJM has a more transparent networking market. Just look at the whose got the better web site, PJM or Midwest ISO?