Smart-Grid Control Software Maker Hacked

tsu doh nimh writes "Telvent, a multinational company whose software and services are used to remotely administer and monitor large sections of the energy and gas industries, began warning customers last week that it is investigating a sophisticated hacker attack spanning its operations in the United States, Canada and Spain. Brian Krebs reports that the attacker(s) installed malicious software and stole project files related to one of Telvent's core offerings — OASyS SCADA — a product that helps energy firms mesh older IT assets with more advanced 'smart grid' technologies. A follow-up story from Wired.com got confirmation from Telvent, and includes speculation from experts that the 'project files' could be used to sabotage systems. 'Some project files contain the "recipe" for the operations of a customer, describing calculations and frequencies at which systems run or when they should be turned on or off. If you're going to do a sophisticated attack, you get the project file and study it and decide how you want to modify the pieces of the operation. Then you modify the project file and load it, and they're not running what they think they're running.'"

Re:

[TWX]

I am tired of the use of characteristics that don't seem to apply being applied by marketing staff.

Clearly "smart" doesn't apply.

Re:

[Anonymous Coward]

I don't understand why it doesn't apply. Are you implying that OASyS SCADA isn't "smart" because Telvent was hacked into? Or that the "smart grid" isn't because it's assets can be misused? Or...?

Re:

[Dr Max]

So if something can be hacked it isn't smart? So smart phones should have been called slightly more powerful feature phones. Carrying on with that, you can hack a human with magnetic fields http://www.livescience.com/438-remote-controlled-human-sensation.html [livescience.com] so none of mankind could be considered smart either.

smart grid, stupid access and control sw

[swschrad]

YOU. DO. NOT. CONNECT. VITAL. INFRASTRUCTURE. TO. THE. INTERNET.

fucking idiots.

guess we better learn to live in the dark again, because these fools and the power companies they blather money out of will put us there yet.

Re:smart grid, stupid access and control sw

[Shoten]

YOU. HAVE. NO. CHOICE.

Telvent is the world's leader in what's known as "ADMS" systems. Advanced Distribution Management Systems. This is, for lack of a better way to put it, the "Smart" in "Smart Grid." By definition, it requires broad and extensive connectivity with many other systems.

In the old days, power plants...a few big ones...made power. And that power kind of spread outwards in straight lines to substations, and then to homes/businesses/etc. Well, now, smart grid is going into place. So you get more information from the homes/businesses/etc about what power they are using, and you will have more sources...small sources...of power all over the place. The power grid will look more like the Internet...interlaced, routable, managed. But you need a monolithic "God System" to keep track of what's going on, and control the changes that need to be made. Examples of systems that ADMS ties into are AMI, where the connectivity indirectly extends out to literally millions of collectors and meters attached to homes, to wind farms, to solar farms, to hydroelectric turbines, to coal-powered generation facility, and to CT (combustion turbine) generators. Oh, also...substations, protective relay systems...I think I'm forgetting some. Oh! I forgot...your local Balancing Authority, who is responsible for the stability of the larger power grid.

So yeah...this whole "Oh, you just need to air gap it because it's a control system" is ignorant. That hasn't been realistic in the power industry for about a decade now. Before you call a whole industry "fools," maybe you should first learn about how the industry functions, hm?

Re:

[sjames]

YES. YOU. DO.

All of that calls for a network, but none of it requires the INTERnet. It is a CHOICE (a bad one) To expose that network to the internet.

Re:smart grid, stupid access and control sw

[Shoten]

Actually, it does require the Internet.

Balancing Authority interconnectivity, for example...that's a whole other organization. You think people run dedicated lines that are, in some cases, hundreds of miles long? When you're talking about the really big ones, like WECC, you could be talking about a thousand miles of distance between the ADMS/EMS systems and the Balancing Authority. And the link needs to be reliable. So nope, not an option. If the utility is in a market that permits energy trading, then you also need other interconnections..again, over long distances, and that means the Internet all over again. I do security in the power industry for a living...these systems are never put just on the Internet at a power company, but it's always just a couple of hops away. And nation-state attackers have little trouble hopscotching their way through to the target. The problem isn't the connectivity, it's the lack of good patch management/antimalware/security monitoring systems and processes. And that's pretty much what the problem is when it comes to most breaches.

Look into the following acronyms, and keep digging. After a week of it, you might understand this better.

NERC
ERCOT
PJM
WECC
ERO
NERC-BAL
NERC-CIP
NERC-PRC
NERC-EOP
ISA99

Re:

[sjames]

We had a power grid before the internet existed, therefor we can have a power grid without connecting it to the internet.

Re:

[Dr Max]

Sure you could, but it wouldn't be very efficient, it would cost a lot more to run (for expanding and maintenance you don't need to over compensate as much when you have good data to work with) and it's probably not capable of accepting power back into the grid (from say a residential solar panel); unless of course you built a new network covering the entire country (BIG MONEY $).

Re:

[sjames]

Backfeeding has been done for a long time in some areas. We would be better off if we would actually overbuild a bit more. There is a great deal of dark fiber out there. You can gather data FROM a control net to the internet using a one way serial connection. The danger lies in allowing commands to flow from the internet to the control network.

You make it sound like the entire grid has been on the internet for all of living memory. The fact is it has run just fine without it until very recently. It isn't e

Re:

[Dr Max]

You could do a little bit of back feeding, but hook up to many solar panels and you'll start blowing up peoples electronics, because to backfeed you need to have a higher voltage than the network and without some one watching the network voltage (lowering it when there is a lot of backfeed) it would easily get too high. Also Substations aren't cheap if you can check the data and see what you have now is fine, then you don't have to build a new one twice the size just because you saw a high max watts reading

Re:

[sjames]

Those are all good reasons why networked controls can improve on the old un-networked system, but there isn't a single objection there that would require connecting that control network to the internet, even indirectly.

I strongly agree that they should use a control network. My objection is to connecting it to the internet. There is no good reason someone in wherethehellisthisistan needs to be able to shed load in the Bronx.

Re:

[Dr Max]

and run a separate network to every house, one that runs right next to the internet phone lines? Or maybe we are on a different page, the substations aren't on the internet (maybe the data recording) but all the control is done by a separate (expensive) fibre network connecting them to control; It's not like a smart meter can operate hv switching, your lucky to get relay control on the whole current meters (anything using current transformers the meters have zero control over).

Re:

[sjames]

SCADA systems are somewhat distinct from smart meters. SCADA does do HV switching.

The 'smart meters' don't currently have access to the internet. The lines that would do that belong to the homeowners. It is unlikely that the homeowner has the legal right to let the power company borrow some bits, even if they are willing.

Smart meters also shouldn't be controllable from the internet.

Re:

[Dr Max]

No one is getting access to the scada system without going through private fibre or being onsite. About half of our smart meters have an ip address the rest are using gprs, but most of them don't control anything, and a couple don't do anything audio frequency load control wasn't already doing for a long time (the tech behind how they turn your hot water system on and off, which i think is the 'frequencies' the hacker has stolen in the story).

Re:

[sjames]

The post I replied to claimed otherwise.

Re:

[sjames]

If a thing has existed, it is possible. That's perfectly solid logic unless you can point out any existent but impossible things.

Re:

[sumdumass]

I had 3 t1 connections connecting 3 buildings in 2 different states together in 2000 and none of it provided internet.

If you can get the internet, you can get communications not on the internet. It doesn't solve all the problems you listed, but isolating control systems and using a VPN to access their networks means you can at least have a front end that has good patch management/antimalware/security monitoring systems and processes. It will just cost more.

Still no reason for putting idiots on the job

[Casandro]

I mean look at SCADA. The whole field seems to be staffed by idiots.
They think that OPC (OLE for Process Control) is a good idea, they still use that, even though the networking component works via DCOM, and it's all Windows only.

I mean a sane person would go and have sensors spit out text. That text can then be easily processed and archived easily. You can even batch process it, if you want.
You can of course, also pour it into some SQL database if you prefer to, but having your primary data as text means t

Re:

[ThreeKelvin]

Sensors that spit out text? Who in their right mind would want that?

SCADA grabs sensor readings from the underlying control system, most likely running on some PLCs, where you have to do calculations on the data in order to feed back control values to the process being controlled.

Now, a PLC is, admittedly, sort of like the general purpose CPU's dumb brother, and the instructions it accepts are rather limited. But, for a number of reasons, they're immensely suited for their task. The single most important on

Re:

[optimus2861]

Sensors that spit out text? Who in their right mind would want that?

Ignorant IT people who think they know better than control engineers how to design & operate control systems. The rant about OPC was beautiful, in its own ignorant way, and completely exposes the GP as someone who's probably never seen a control system in his life, probably never will see one, and wouldn't have the first clue how to program, troubleshoot, or maintain it.

There's historically been a certain amount of tension between control engineers and IT folks for that very reason; the smartest IT folk

Re:

[Casandro]

If you cannot get IT working in such critical infrastructures, don't blame the people who are telling you what you are doing wrong.
Besides where is the problem building a PLC sending its output variables as text?
Furthermore, in the example of the company mentioned in the article, the smart meters probably didn't have a PLC connected to them. They were probably small devices with a micro controller.

Again, it's all fine and dandy if you connect your PLCs via a ProfiBus or whatever, but once it involves actual

Re:

[optimus2861]

Besides where is the problem building a PLC sending its output variables as text?

Limit switches, solenoids, pushbuttons, motor contactors, relays, pilot lights, current transmitters, modulating control valves, even robotic motion controls -- good luck replacing all that gear with 'smart' versions that now have to understand text instead of simple electrical signals and not losing so much as a millisecond of response time.

I probably ought to give you some benefit of the doubt in that you meant "the PLC's data that passes to/from the SCADA" but you used the word "output" and that has a me

Re:

[Casandro]

I'm not talking about connecting end switches via text-based protocols. And I never have. I have been talking about "sensors" as in "smart meters". There it makes sense. There you transmit data over some non-closed network. Again, I'm not talking about valves and servos.

Please stop acknowledging my prejudices.

Re:

[GameboyRMH]

the smartest IT folks are the ones who ask the control engineers what they need to do their jobs, provide it, then stay the hell out of our way.

Like unsecured access from the Internet or dial-up systems I'm sure.

Re:

[Darinbob]

Much of it is IPv6 but it does not necessarily have to be directly connected to the internet. Maybe at some points there is the grid network and the internet on the same computer or device, and it's those devices that need to be extra secured. The devices on the grid of course need solid security. But a lot of DA and SCADA infrastructure devices are older and may not be designed with security in mind.

Re:Yep, better be the last nail in the coffin..

[TWX]

If they come out to change the meter housing you really won't have a choice. You realize this, right?

It's either smart meter or else no service.

Re:Yep, better be the last nail in the coffin..

[Beardo the Bearded]

I love my smart meter. My electric bill is half what it used to be.

Of course, that was after I installed my own software on it, but hey, fuck em they're a power company.

Re:

[GameboyRMH]

You're right, but you could firewall off your appliances from communicating with the grid using some kind of line filter and/or or battery bank. You'd lose efficiency but gain security and privacy.

Obvious what this is.

[Anonymous Coward]

The attackers will produce a cascading failure in the electrical grid that brings down the entire North American power grid. A few additional well timed physical attacks, and we're back to the bronze age for the foreseeable future. Food stocks will quickly run down, as will supplies of petrol. The government will attempt to exert control, but without food and as the situation deteriorates, most of the soldiers will go AWOL to try to get home to help family. Soon, the dying begins. Roving bands of robbers gradually coalesce into gangs ruled by small time warlords, and eventually regional rulers who hoard the remaining food, fuel, and ammo. The few isolated people who planned ahead and who have escaped into their countryside shelters are systematically hunted down, plundered, and given the option to swear fealty to the new regime or be dispatched. Huge fires sweep through most large cities and pollute the atmosphere with soot. Winter soon sets in early due to the reduced sunlight penetrating the atmosphere, and is the harshest one in generations. Eventually, as the winter ends and spring sets in, over 75% of the population is either dead or close to it. Suddenly, armies of foreign soldiers appear at our shores, and before long all of the remaining Americans are conscripted and forced to farm the still fertile fields of America's breadbasket for meager rations, which is still better than starvation and death.

Re:Obvious what this is.

[localman57]

The attackers will produce a cascading failure in the electrical grid that brings down the entire North American power grid.

Frankly, I'm surprised we haven't had this happen already. It always blows my mind when there's some massive cascading power failure across mulitple states, and people are somehow relieved that it wasn't terrorism. Just a normal failure. How the fuck is a system that just collapses all by itself better than one that has to be pushed to collapse?

It seems to me that instead of fucking around with underware bombs and shit, our enemies might get a lot better cost return with some iron spikes, aluminum wire, and some helium filled weather balloons. Giant transmission lines in the middle of the desert are virtually impossible to defend, and are already stressed to the breaking point when it's hot across the nation. All they need is a little push...no complicated cyber-hacker-shit required.

Re:

[snspdaarf]

Have to be hydrogen. We have a helium shortage, remember?

Re:

[cavreader]

"Frankly, I'm surprised we haven't had this happen already"

It hasn't happened yet because believe it or not it is not that easy to do. The U.S. power grid is segmented into three zones with safeguards that prevent an outage in one zone from tripping a blackout in another zone and this makes causing a nation wide blackout extremely unlikely.

Re:

[CanHasDIY]

Dude - you should totally make that into a short story; 'twould be a good one.

Re:Obvious what this is.

[Columcille]

And someone will make some lame tv show about the whole thing.

Re:

[Anonymous Coward]

I really have only two questions.

1. Will raiding parties sent out by the regional rulers be stuck with muzzleloaders (except badass officers; badass officers always get teashades and autopistols)?

2. Will cute-but-surprisingly-asskicking-and-unsurprisingly-obnoxious girls cut the buttstocks off crossbows to make one-handed, but implausibly imbalanced, weapons? And maybe lose the stirrup, too; that serves no useful purpose...

Re:

[chill]

Dan Brown, is that you?

Re:

[YrWrstNtmr]

Soon, the dying begins. Roving bands of robbers gradually coalesce into gangs ruled by small time warlords, and eventually regional rulers who hoard the remaining food, fuel, and ammo.

Given a large enough and complete enough failure, 'soon' = 48-72 hours.

Thursday at 9 on SCTV

[swschrad]

I know how this ends. Chunky (tm) soup warms you up and fills you up. and then Weatherbreak.

Re:

[Neil Boekend]

That'd be us when there is no power to make coffee.

Re:

[rwv]

Just to nit pick... you don't need power to make coffee. Water+pot+fire, coffeebeans+mortar/pestle, and a frenchpress will do the trick quite effectively.

Re:

[Dr Max]

That's a great story i really enjoyed it. If you want to know the scary truth, a couple of guys with rifles could take the major power stations offline a lot longer than hackers, by simply shooting the insulators on the phases coming out of power stations.

Are 'smart' meters mandatory?

[fustakrakich]

Sure hope not. I mean, does every goddamn thing need to be computerized?

Re:Are 'smart' meters mandatory?

[brianhaddock]

Lots of utilities are rolling them out right now and big companies who want to keep an eye on their usage patterns demand them. Remember when they used to hand read meters? The guy would open the gate, dodge your barking dog, and write down the meter reading in his little book. Then they moved some to radio transmitting meters where a utility truck simply drove down your street and recorded the readings that were transmitted from each meter. Now they have meters that communicate wirelessly and send the readings to the utility company in at intervals.

Re:

[northerner]

Smart meters are a lot less safety critical than the SCADA control systems. Networking smart meters makes a lot of sense. It's desirable to do it right, but if the billing is messed up, it can be corrected. The ability to change the operation of the system needs to very secure in comparison.

Re:Are 'smart' meters mandatory?

[Spy Handler]

Yes because computerizing stuff increases efficiency. Look under the hood of your car, all those chips and sensors are helping your engine make a lot more horsepower for the same amount of fuel than engines from 30 years ago. (Or, same amount of power for less fuel consumption)

What we should really be asking is, does everything need to connect to the internet? And is enabling USB ports on critical systems so that workers can bring infected USB stick from home to bridge an air gap a good idea?

Re:Are 'smart' meters mandatory?

[Dunbal]

Tell me how efficient they are when the whole grid goes down.

Re:

[Ichijo]

Tell me how efficient they are when the whole grid goes down.

100%.

Re:

[flaming error]

Ok, but I'll probably need your snail mail address.

Re:

[GameboyRMH]

HAHAHAHA you know very little about cars don't you?

Plastic radiators? (end tanks sure, but plastic is shit at conducting heat). Cars lighter now than 30 years ago? (Maybe today's subcompacts vs. old American luxo-barges, but otherwise NO).

Engine technology has advanced greatly but it's had to fight increasing safety regulations which greatly increase the size and weight of vehicles. In the '80s you could buy compact sedans that got 40MPG just like today, but they weighed less. Vehicle weight has been forced

Re:

[localman57]

Sure hope not. I mean, does every goddamn thing need to be computerized?

Computers make things more efficient. Which means you can consume more for the same price. If it weren't for the computers, people would have to get by on less. And if there's one thing people hate more than computers, it's getting by on less. So you get what we got right here. Which is the way we want it. Well, we get it. And I don't like it any more than you men.

Re:Are 'smart' meters mandatory?

[TWX]

Computers only make things more efficient when the systems architects know how to do their jobs effectively and don't rely on vendors and consultants to do it for them. It's not in the interests of vendors or consultants to save their customer money. It's in their interests to make as much money from the customer as practical, and that can mean everything from selling them equipment that's overspec to selling far more equipment than necessary to excessive costs for setup and configuration that are difficult to determine at the outset of the project.

As problematic as our telephone system has been at times, at least from a bureaucracy standpoint, that Bell did basic research and development in-house and for a long time owned almost everything internally, advances were made and the system functioned very well. The Baby Bells have inherited this legacy, and the biggest cracks have only manifested as they've each independently implemented technologies post-Ma-Bell, like DSL.

If you've had to work with vendors extensively you'd realize what a bane it can be to actually achieving, especially when non-technical persons have the ultimate decision in your organization.

Re:

[localman57]

X only make things more Y when the Z know how to do their jobs effectively

That tend to be true for just about any professional combination of X,Y, and Z. Don't you think?

I worked in the power industry while our state was in the throws of deregulation. I know what you're talking about. But this thread isn't about that. And Ma-Bell suffered from the same flaws that the power companies do. Security is not a primary issue to them. During the glory days you're talking about, you could phreak the phone system with a Captin Crunch whistle. That was the state of their security.

Re:

[Anonymous Coward]

There's a downside to smart meters though. It's known that old power meters tend to get increased drag as they age. So say you use 1200kw this month, the meter may read as if you had only used 1125kw or so (that might be a bit of a large gap for a lower power use, but you get the idea).

So here's the trade off. Our power bill, three months ago was an actual reading, which was around 900kw or so (guessing). Two months ago was an Estimate, and it was listed as 1600kw. Then last month was another actual reading

Re:

[sumdumass]

On demand gas or propane is the best solution for a water heater. The smart grid turning your water heater off is problematic if you are not a 9-5er or had a couple of kids. I can easily run out of hot water on certain days because the damn thing is off and need to take a shower in the cold to save making the kids do the same. God forbid someone starts a load of laundry at the wrong time either.

Well, when I say can, I meant before I change the heater out. Put in an on demand gas system and never run out of

Re:

[tlhIngan]

There's a downside to smart meters though. It's known that old power meters tend to get increased drag as they age. So say you use 1200kw this month, the meter may read as if you had only used 1125kw or so (that might be a bit of a large gap for a lower power use, but you get the idea).

True, because it's mechanical and subject ot all sorts of enviornmental conditions of cold and heat, so they do read slower as they lose calibratoin. Though the power company does swap out meters on a regular basis - usually

Re:

[Mashiki]

They're mandatory in Ontario and as far as I know, the entire province has them installed, so this could get very interesting to say the least. Or until the meters are all updated, well I was always against them to begin with. Our hydro rates have done nothing but increase since they've been installed. Right on track for 22c/KWH by 2016 baby! [financialpost.com] Gotta love it.

Re:

[PPH]

One of the major overhead costs of your residential service is (was) reading your meter by having someone drive around in a car, jump out at every few houses and write down the readings.

I guess utilities could offer an option to people who think they'll get hacked or the meters will make their junk fall off: Pay for a meter reader and they'll leave the old analog one on your house.

Re:Is anyone surprised?

[localman57]

What's a bantex ass mart meter? I don't want to click it and find out, because I'm fearful it's probably NSFW....

Smart GRID not METER

[Anonymous Coward]

stop spamming the thread with crying about your smart meters, this is much much bigger than you

Re:

[Neil Boekend]

Correct me if I am wrong, but doesn't a smart grid require smart meters?

it is small wonder it took them so much time...

[stanlyb]

The main problem is that only the hackers that have not tried to hack their system, did not hack their systems. And the more terrifying truth is that there is not even one vendor with secure solution out-there. I am just amazed of how they even put the word "secure" in there product!!!!

This is a Good Example

[chill]

This is a good example of why the gov't is worried about cyber security for critical infrastructure. Just like there are minimum standards for building and fire safety there needs to be minimum standards for IT infrastructure security.

Re:

[kiwimate]

Err, not saying that I agree or disagree, but wasn't there a story on these hallowed pages [slashdot.org] yesterday saying exactly the opposite?

Re:

[chill]

Sort of. They were saying this isn't enough, claiming an "offensive" component is also needed.

Keep in mind, it was penned by an ex-military guy who just joined a cyber security consulting firm. He's saying we need to change the laws to allow something like a digital version of Blackwater.

It is like he read his first cyberpunk novels and thinks Shadowrun was a good idea for real life.

Inigo Montoya

[guttentag]

...investigating a sophisticated hacker attack spanning its operations in the United States, Canada and Spain... and they're not running what they think they're running.

Sounds like they need a modern-day Inigo Montoya to do their security: <SPANISH ACCENT>"You keep using that software, I do not think you're running what you think you are running."</SPANISH ACCENT> And if the worst happens, he can exact revenge: "Hello. My name is Inigo Montoya. You killed my power grid during a level 85 raid. Prepare to die."

Keep Banging That War Drum!

[IonOtter]

That's right, keep banging on that war drum. While the leaders are making all the big noise and keeping everyone distracted, the governments and their military are already engaged in full-on, no-holds-barred combat.

We took out 50% of Iran's nuclear capacity with nothing more than a USB stick loaded with Lady Gaga albums and porn.

But at least Iran was smart enough to put an AIR GAP between their critical systems and the rest of the world. We had to rely on a human to use the Sneakernet to infect those cent

Re:

[stanlyb]

Don't ask, dont tell....

Re:

[snspdaarf]

That's the firewall that keeps the people just passed over for promotion for the third time from running amok in your servers.