Breached Water Plant Employees Used the Same TeamViewer Password and No Firewall (arstechnica.com) 80
An anonymous reader quotes a report from Ars Technica: The Florida water treatment facility whose computer system experienced a potentially hazardous computer breach last week used an unsupported version of Windows with no firewall and shared the same TeamViewer password among its employees, government officials have reported. The computer intrusion happened last Friday in Oldsmar, a Florida city of about 15,000 that's roughly 15 miles northwest of Tampa. After gaining remote access to a computer that controlled equipment inside the Oldsmar water treatment plant, the unknown intruder increased the amount of sodium hydroxide -- a caustic chemical better known as lye -- by a factor of 100. The tampering could have caused severe sickness or death had it not been for safeguards the city has in place.
According to an advisory from the state of Massachusetts, employees with the Oldsmar facility used a computer running Windows 7 to remotely access plant controls known as a SCADA -- short for "supervisory control and data acquisition" -- system. What's more, the computer had no firewall installed and used a password that was shared among employees for remotely logging into city systems with the TeamViewer application. [...] The revelations illustrate the lack of security rigor found inside many critical infrastructure environments. In January, Microsoft ended support for Windows 7, a move that ended security updates for the operating system. Windows 7 also provides fewer security protections than Windows 10. The lack of a firewall and a password that was the same for each employee are also signs that the department's security regimen wasn't as tight as it could have been.
According to an advisory from the state of Massachusetts, employees with the Oldsmar facility used a computer running Windows 7 to remotely access plant controls known as a SCADA -- short for "supervisory control and data acquisition" -- system. What's more, the computer had no firewall installed and used a password that was shared among employees for remotely logging into city systems with the TeamViewer application. [...] The revelations illustrate the lack of security rigor found inside many critical infrastructure environments. In January, Microsoft ended support for Windows 7, a move that ended security updates for the operating system. Windows 7 also provides fewer security protections than Windows 10. The lack of a firewall and a password that was the same for each employee are also signs that the department's security regimen wasn't as tight as it could have been.
How does this keep happening? (Score:5, Insightful)
Re:How does this keep happening? (Score:5, Informative)
Re:How does this keep happening?
In my experience, generally the only people who care about security are security professionals. Sure there are always exceptions, but I work with highly technical developers all day, not the city employees in this case, and they do whatever it takes to get the task at hand completed with no regard for security, let alone the best way to accomplish the task. It is not surprising to me at all that this keeps happening.
Re: How does this keep happening? (Score:4, Insightful)
Re: (Score:2)
Re: (Score:2)
And when something goes wrong they'll hold a team meeting about how "we all need to be doing better". I know "The Office" was mostly a parody but I swear the parts where Michael Scott would call an office wide meeting on something like racial insensitivity when he's the one that committed the offense was spot on.
Re: (Score:2)
Even then, security fatigue sets in. You need to be able to use things at the end of the day, and good security is very hard.
Re: (Score:1)
Re: (Score:2)
They presumably shared a password because they did not have a license to TeamViewer, and used the free version.
As for password managers, they create their own set of risks. They have their place, but are not especially secure.
Re: How does this keep happening? (Score:1)
Re: How does this keep happening? (Score:2)
That your manager let's you do it is the problem.
If everyone just visibly goes drink some coffee every time there's a hold up because of IT there is some impetus to reduce the overhead to a minimum ... but still a documented and designed minimum, not some ad hoc bullshit.
Re:How does this keep happening? (Score:5, Insightful)
It's worse than that. They don't just not care about security, they see it as a barrier to getting their job done. It's literally an obstacle that they have to overcome.
Re: (Score:2)
Of course, there's also developers that don't care and just do the bare minimum to get things working
Re:How does this keep happening? (Score:5, Insightful)
Of course, there's also developers that don't care and just do the bare minimum to get things working
And it's also a management issue about project load.
Lets say I'm given 4 projects and a 40 hour week to get them done.
To write them to a quick and dirty "it'll work for now but needs to be cleaned up later" level it'll take about 8 hours each. To do it properly it'll take 20 hours each. Notice the issue? To do it properly there's not enough time (and next week they'll be throwing a new set of stuff at you).
When the metric is "get as much done as possible" security suffers.
hardware / software stuck on old windows & 500 (Score:2)
hardware / software stuck on old windows & 500K+ to up grade to software / hardware that will run on better OS's?
Re: (Score:2)
The only way security gets done right is if every dev is a trained security professional, and that is expensive. And then those developers drive up all your other costs, like insisting on new versions of Windows (and commensurate hardware upgrades). And that is expensive. The short answer to how this keeps happening? Voters are cheap, politicians know that, and so budgets don't account for security costs. Because that is expensive.
Re: (Score:3)
The guy at the top, needs to deal with a lot of general information, the details on what is going on doesn't concern them.
Each person under him, and further down, are more and more focused on details on what is going on, but rarely concerned about the details from other, unless it affects their details.
Security really gets in the way of that model, because it requires a lot of detail on what is going on, across all the details that everyone else is working on. And quite often Security requests make the jo
It is the Russian hackers. (Score:4, Funny)
Them damned Commies keep re-entering the brains of those poor sods via 5G wireless piggybacking on the Coronavirus and making them do it. It is like a Plague of Putins, if you will.
Re:How does this keep happening? (Score:5, Insightful)
Re: (Score:1)
How does this keep happening?
yea I don't know how the editors keep re-posting stories either
https://it.slashdot.org/story/... [slashdot.org]
Re: (Score:2)
Couple years ago, I had a slashdot user explain to me that this does not happen on systems like this. Experience tells me otherwise when I see industrial techs use teamviewer on a regular basis to access system across the world and most of them use the same passwords for everything. And don't get me going on the open ports directly to PLC's.
Here we are today, same ol' crap going on. Nobody cares about security.
Re:How does this keep happening? (Score:5, Insightful)
Attrition happened, because everything seemed to be working and there were no disasters, so the city abandoned maintenance on critical infrastructure to pay for more visible projects.
In a town of less than 15,000 people in Florida? Nobody there is competent to run the system. There probably isn't enough work to hire someone who is competent, especially at the rate they'd have to pay the person. They likely had a contractor install the system, and perhaps (perhaps) at the time it was even reasonably secure. But password management was a pain, and the firewall kept popping up annoying message boxes. It was inconvenient to have to access the system on-site, so why not just plug it in to the network? And the city wanted a new hockey rink.
The most recent budget the city seems to have on-line is FY2014/2015. They spend 2% of the city budget on IT. They have an IT staff of 3--one of those is IT support, one is a GIS person, and one is the person who actually runs the IT systems. In 2014/2015, one of the city's goals was to replace their Windows XP computers with Windows 7, which came out in 2009. Windows 10 came out in July 2015. They were planning to replace their EOL XP systems with Windows 7, an OS that at that point was on the verge of being obsoleted by 2 versions. And apparently, they never upgraded again.
They don't prioritize IT, they certainly don't prioritize IT security, they don't pay for adequate or competent staff, they aren't aware of their risk exposure, the people in charge prefer hockey rinks and business corridor improvements over basic infrastructure requirements, and as long as nothing goes wrong, they all think they're doing a heckuva job.
Re: (Score:2)
Because employees aren't charged with criminal negligence anymore, since they all play the "CYA" game.
Re: How does this keep happening? (Score:1)
Re: How does this keep happening? (Score:1)
Re: How does this keep happening? (Score:1)
Re: How does this keep happening? (Score:1)
Re: (Score:1)
The usual current operating procedure with SCADA (Score:2)
Re: (Score:1)
They don't even need to update the SCADA system. They just need to front-end it with a bastion host that has an updated firewall and possibly 2FA.
We know - the air gap - or an air gap attempt (Score:1)
Re: (Score:2)
Anyone setting up industrial systems don't really care about that. They'll just install this:
https://www.ixon.cloud/knowledge-hub/secure-industrial-remote-access-to-any-plc-or-hmi
Just to punch through the firewall. When there's a market in the industrial sector to bypass IT security, you know things are pretty messed up.
Do you blame windows? (Score:2)
I think the biggest problem with Windows is the people who use it.
So many windows users are sitting on out dated versions of the system, with nearly every security mechanism switched off (to make everything easy). The number of times I've seen people change the permissions on a file share, or file system DACLS to grant full access to Everyone.
Re: Do you blame windows? (Score:2)
The laptop I use for PLC programming is outdated.
I know it will take me about 2 weeks to install everything again on a new win10 laptop. not counting any potential round trip times to manufacturers because their software license enforcing software is likely to throw a fit. So in total about a month probably. So I keep putting it off, because I have "real work" that needs to be done and can't wait a month.
Tl;Dr: industrial software is utter garbage.
yep (Score:1)
Stop putting stuff online (Score:4, Insightful)
I don't care if it runs on a Commodore 64 if it does it's job it doesn't need an "upgrade" but never put these things online no matter how modern it is; it all gets OLD. Good luck playing the never ending upgrade game in the real world when eventually you you can no longer run the special software required and are stuck in the past... with an addiction to online integration BS.
Emulator whatever... if it doesn't really have to be online, DO NOT PUT IT ONLINE!
This is why everything like this has to be open source; if the company drops support they must be forced to open the source. Not that it protects you from the online addiction but it sure would help if systems would disable EVERYTHING that isn't necessary.
Windows? WTF? I wouldn't run a toaster on windows and certainly not ever online!
USERS have zero grasp of security. Stop letting them choose passwords! ASSIGN THEM! They'll write down whatever stupid shit they come up with anyhow.
Stop letting users put dangerous systems online or making an idiotic phone controller.
Re: Stop putting stuff online (Score:3)
Re: Stop putting stuff online (Score:2)
TeamViewer is used to login and reboot the machine when remote desktop refuses to work.
Re: (Score:2)
Rock vs. hard place (Score:4, Interesting)
I do understand that the jail industry needs more revenue but people are doing this because they need to do their work. They are not responsible for the budget and they will be considered troublemakers if they can’t do their work because security get’s in their way but somebody else can because they work around security measures.
The solution to password sharing is some form of two factor authentication with a hardware token, preferably with the same personal device they use for physical entry.
Security is a process and getting your work done is a process, too. These problems (password sharing) often are a result of implicit (unrecorded) and/or changing processes for the “getting your work done” part contradicting the security part.
If everyone at every time has their individual access with all the rights they need to do the work expected, people will not share passwords.
If people are (even implicitly) expected to work remotely but don’t have the access (yet), people will share passwords. So you have to make that impossible in a way management can see and understand. Using the personal card they use to get on site (and to track their time and - if possible - to pay in the canteen) is a good way.
Re: (Score:2)
Oh the we were only following orders excuse. Where have we heard that before ?
> They are not responsible for the budget and they will be considered troublemakers if they can’t do their work because security get’s in their way but somebody else can because they work around security measures.
WHy didnt you think perhaps their bosses should be criminally responsible
Re: Rock vs. hard place (Score:3)
Can you have different usernames or passwords and still log into the same account? Because logging in as a different windows user will in best case deny the user access to the controls, and in worst case the scada software freaks out and does something "undefined".
Re: Rock vs. hard place (Score:2)
Why industry find it easier to maintain LOTO discipline but rarely IT security?
I think the problem is the lack of good standards and conviction in IT (not just management) that security needs to be build around physical artefacts (smartcard, dedicated laptops, airgaps, physical paperwork to justify exceptions).
Re: (Score:2)
"There needs to be jail time for sharing passwords."
Like Netflix passwords? It has been for a decade.
https://www.businessinsider.co... [businessinsider.com]
Re: (Score:2)
Re: (Score:2)
For a small water treatment district there just aren’t enough people for 24x7 on-site supervision. It gets set up so you have maybe 16x7 or 16x6 or 16x5+8x2 and a pager. (Oh, and then someone quits and you are short-staffed so you end up with less than that.) Some facilities like this only actually have on-site staff 8x5, and do batches those five days a week and draw from them on off days.
The problem is the remote access system was done on the cheap, apparently not even paying for a Team Viewer lic
look at HISTORY (Score:2)
Most people remember a time before internet; even after internet it took many years for upgraded systems with support to exist and be purchased.
How did the world function without internet? The main reason for many of the computer controlled systems was to save money by automating jobs away so fewer human monitors were required. They had computers so they didn't need human monitoring 24x7 and there are stll some people alive who remember pre-computer control...
Many of these systems have no serious need for
Re: (Score:2)
We refuse to pay for offline monitoring, and we refuse to pay for securing the online monitoring.
We get what we pay for.
Wait? buy automation to cut labor then... (Score:2)
So we buy human monitoring to monitor our automated monitoring which exists solely to replace human manual monitoring?
They really do not need this; they have forgotten the recent history when the great new computer system made everything better but the golden goose was NOT enough after a while so then they shoved an ethernet cable up its ass and nearly cooked their goose!
People did this before the computer age and before the internet age.
What does that mean? (Score:2)
Would that be Win7, which Microsoft _does_ support as far as they can make money on it? They gladly support their Edge browser on Win7, There's a reason W10 "upgrades" were "free". You are the profit.
Re: What does that mean? (Score:5, Informative)
Re: (Score:2)
TeamViewer was the fault point. More specifically a widely available login was the culprit. They didn't use some Windows 7 exploit to compromise the system.
FYI Windows 10 has security holes too, some from Windows 7. So stop the carping on using old operating system.
That's hardly a unique case (Score:5, Informative)
I worked in the cinema industry, and boy do I have a nice list of passwords that will get you in 3/4 of the word's cinemas' servers and live streaming boxes. The DCI [wikipedia.org] standards are such a fucking joke it's not even funny.
I have no beef against the cinema industry so I have no reason to use them. But I can tell you the whole infrastructure is staggeringly insecure. Others have though, and have played minor havoc in certain cinemas. Specifically, I remember an incident where a cinema reported someone logging into the (open, public, password known to everybody and his dog) wifi intended for audio devices for the hard of hearing, and changed the wifi's AP's SSID to something offensive. Obviously done by someone in the know.
Sadly, that particular industry has other things to worry about, since COVID-19 essentially destroyed it, and it wasn't in such a great shape before the pandemic to begin with.
mce elevator install doc's say set router login to (Score:2)
mcesupport / mcesupport
what's the problem again? (Score:2)
Re: (Score:2)
Re: (Score:3)
What happens is the water quality sensors further down the line trip and the whole plant shuts down while an engineer investigates.
Re: (Score:2)
What happens is the water quality sensors further down the line trip and the whole plant shuts down while an engineer investigates.
But the water quality sensors had been turned off years ago because they were inconvenient. And they were some cheap model installed 15 years ago by the nephew of the major.
And we didn’t sent an engineer to investigate but the operator has sensors as a hobby so they did have a look at it and turned it off and on again.
Why don’t these stories happen? Why is IT different?
Technological Debt (Score:5, Insightful)
Also known as "IT rot". IT is expensive. Some people think they can do it on the cheap and just set up something, connect it to the Internet and let it run. These days that must be regarded as gross negligence.
Re: (Score:2)
Also known as "IT rot". IT is expensive. Some people think they can do it on the cheap and just set up something, connect it to the Internet and let it run. These days that must be regarded as gross negligence.
We can expect to see more and more as various levels of government struggle with insufficient revenue, and rapidly growing infrastructure deficits,
Few politicians want to make the hard decisions. Given the choice of spending scarce financial resources to make the IT infrastructure safe, or fixing potholes, IT repairs get deferred 99 times out of a 100. Politicians know few if any taxpayers care about IT upgrades, while the torches and pitchforks come out if potholes don't get filled.
Re: Technological Debt (Score:1)
Re: (Score:2)
But why? Any Linux/xBSD system comes with sshd and x-forwarding. And long-term updates. The real problem these people have is using tech that is cheap in the short run, but very expensive in the long run.
Nonsense, (Score:2)
Same as Florida COVID dashboard? (Score:2)
Does Florida use the same password on all of it's systems that are connected to the internet?
It's always TeamViewer. (Score:2)
Somehow, it always seems to be TeamViewer that's responsible for leaks like this. Of course, TV security is pretty bad, and it always seems to be getting hacked, but shared passwords written on stickynotes under the keyboard are a problem as well.
what a dumb article (Score:2)
- windows 10 would not have prevented this
- a firewall would not have prevented this
Re: (Score:2)
- windows 10 would not have prevented this
- a firewall would not have prevented this
Except that it was (likely) a known windows 7 security flaw that allowed files to be dropped through team viewer so that the team viewer interface itself was invisible to the physical user of the machine.
Also, this was a waste water treatment plant, not a drinking water treatment plant.
You get what you pay for (Score:2)
You get what you pay for - if you don’t pay for safe water supply you will not get it. The city does have a yearly budget for NaOH, why don’t they have a yearly budget for IT updates and security? The city does not allow working around safety equipment measuring PH, why do they allow working around IT security measures?
And I bet that “mouse cursor moved slowly and then ‘1’ key was pressed multiple times in a form field” is either a case of “stack of papers on desk c
Just use dedicated laptops ... (Score:2)
Firewalls? The automation needs to be behind a VPN. Passwords? The key for the VPN needs to be on a smartcard, in a laptop made unusable for anything else but logging into the automation. Or at the very least a laptop with a VM dedicated to the automation, still with the smartcard.
Maybe for emergencies have some one time use logins for remote use on an unsecured computer ... with logging and the absolute requirement to justify the use in writing afterwards, it needs to be extremely annoying to deal with the
Sucha Advanced Hardare! (Score:2)
Having worked in water quality for a few years, I am surprised they had a system as sophisticated as Windows 7. A lot of the smaller wastewater plants were still running Windows XP and even DOS to connect to their SCADA and PLC systems. Security is considered a big hassle as it takes a lot of training. Upgrades costs money that most municipalities simply do not have. When there is no budget, things don't get done, or done with a patchwork of parts. It is sad to see. I am not advocating throwing money at the
Ars is lolsing it... (Score:2)
Windows 10 would not have prevented this.
A firewall would not have prevented this.
The culprit was TEAMVIEWER. The login for TeamViewer was the culprit. Too bad reporting is now a matter of sensationalism.
Re: (Score:2)
Training would have prevented this whole thing from exploding. Keeping critical infrastructure off the internet would have prevented this. What the hell is is Teamviewer doing on a system that is linked to their SCADA? Air Gap anyone?
Re: (Score:2)
Re: (Score:1)
Auditors need to join the 21st century (Score:4, Interesting)
--
.nosig
Bets on having actually paid for Teamviewer ? (Score:2)