Botched npm Update Crashes Linux Systems, Forces Users to Reinstall (bleepingcomputer.com) 256
Catalin Cimpanu, reporting for BleepingComputer: A bug in npm (Node Package Manager), the most widely used JavaScript package manager, will change ownership of crucial Linux system folders, such as /etc, /usr, /boot. Changing ownership of these files either crashes the system, various local apps, or prevents the system from booting, according to reports from users who installed npm v5.7.0. -- the buggy npm update. Users who installed this update -- mostly developers and software engineers -- will likely have to reinstall their system from scratch or restore from a previous system image.
LOL (Score:5, Funny)
A shitscript package manager that does a chmod of /etc and /boot? This thing had to have been written by that Poettering asshole.
Re: (Score:3, Funny)
Re: (Score:2)
I'm guessing 'ownership' is too racist, capitalist, and phalocentric to be a valid concept in filesystem design form much longer.
Phalocentric?
What's my Dick got to do with this???
Re: (Score:2)
No ideas needed. It was tried in the past in the forms of FAT, even though FAT itself is owned by dear leader Microsoft rather than a public asset.
Re:LOL (Score:5, Funny)
Do you guys really need to inject your rightwing politics into literally EVERY story????
It's a memorial to the language which attempted to shatter the glass ceiling [github.com] but was stopped by the patriarchy.
Re: (Score:2)
Re: LOL (Score:3)
Your laptop is a pet. Your cloud servers ought to be cattle.
Re: (Score:2)
Re:LOL (Score:4, Interesting)
Re: (Score:2)
How come a script can adjust permissions on critical system directories?
It's always seemed a bit odd that Linux runs stuff like this as the superuser who can basically do anything. No granularity, no account just for installing JavaScript (!) packages... It's like Windows XP again.
Re: LOL (Score:2, Insightful)
I use npm daily as a non root user. People are just too lazy to take the extra 2 minutes to get it up correctly and instead just throw it to sudo. Run shit as root when there is no reason to and you're gonna have a bad time.
Re: LOL (Score:3)
This isn't unusual. It's stupid but not unusual. I've had 'Professional' software developers tell me this is how it's supposed to be. You'll find a lot enterprise software in a lot of industrial settings functioning this way. Giving root access to people that don't understand anything about the system to deal with faults or errors, etc.
This is now standard industry practice. Stupidity is now standard industry practice.
Re: (Score:2)
At least change the name!
Rescue mode (Score:5, Informative)
If it is a file permission issue... boot from install disk into rescue mode... chmod and reboot. I don't get it.
Re:Rescue mode (Score:5, Informative)
More precisely, I don't know exactly what should be readable by all vs readable by certain groups vs readable by root only in
I say again: It's not acceptable to make your users go through that work. And I also say again: automatically and implicitly trusting package maintainers to do the right thing is awful security policy and awful from a reliability standpoint. All updates should be tested before they are deployed. For home users this isn't practical and we have to rely on the distros to do this for us. Trust breaks down severely when fuckups like this go through and it lends credence to people who don't update their software automatically on the grounds mentioned above. This is bad when actual security fixes need to be deployed out, and it's all the more crucial for ALL software maintainers in OSS to make sure their shit works. Trust is the currency of OSS, and unlike dollars, you can't get some more by going to the bank, you have to earn it.
Re: (Score:2)
"automatically and implicitly trusting package maintainers to do the right thing is awful security policy and awful from a reliability standpoint."
And this is why this, 2018, is the year of Linux as THE desktop.
*whoosh*
Re: Rescue mode (Score:2)
Reddit - come for the inane trolling - stay for the censorship!
Re: (Score:2)
Re: (Score:2)
It's a breathtakingly poor development process that led to this collossal fuck-up, Do you really think they haven't tested at all though, or they just don't work in an enviroment that mimics their users, or don't test the same package that goes to users? How does a development change get to users with npm?
I've worked with QA teams that test different build artefacts than the ones that go to customers because they don't want to deal with installers and things like that in their automated environments. And
Re: (Score:2)
All they have to do is use a scratch system or recover from backup.
Re: (Score:2)
Right, because expecting that is reasonable.
Re: (Score:2)
so one script for each os base line? and not added (Score:2)
so one script for each os base line? and not any added software that has it's own per app rights.
Re:Rescue mode (Score:5, Interesting)
The people most likely to be using npm, and an apparently untested bleeding-edge version of it that gets pushed out automagically (there's a separate bug that pushed out 5.7.0 prematurely), deserve this rancid dog food. This is incontrovertible proof that the JS community lacks competence and leadership.
Re: (Score:2)
The people most likely to be using npm, and an apparently untested bleeding-edge version of it that gets pushed out automagically (there's a separate bug that pushed out 5.7.0 prematurely), deserve this rancid dog food.
So Javascript developers then.
Re: (Score:2)
No, /etc,
there are varied permissions in my etc folder
not everything is owned by root:root
do you know off the top of your head the permissions of all files and folders in
good on you.
Re: (Score:2)
Re: (Score:2)
https://github.com/MrMEEE/bumb... [github.com]
Re: (Score:2)
If it is a file permission issue... boot from install disk into rescue mode... chmod and reboot. I don't get it.
Chmod to what? Do you remember what the user and group permissions on every single file on your system were?
Seriously, it's easier to reinstall.
Re: (Score:2)
Re: (Score:2)
In general most developers don't really know any system. How do you think devops came to be.
Just shut up and run this bash script to start the system, you lowly op. And quit trying to show that you know nothing about software architecture...
windows has better group permissions and muilt use (Score:2)
windows has better group permissions and multi user ones.
Re: (Score:2)
setfacl -m u:joe_dragon:rwx filename
setfacl -m u:joe_nobody:rx filename
but how much software does it by ACL's? (Score:2)
but how much software does it by ACL's? some from repo installs may just do it the old non ACL way.
Re: (Score:2)
Other than the system tools, why should it be up to the software to deal with it at all?
I use ACLs in a few situations and take care of the apps that don't even know ACLs exist (all of them) using default ACLs on the relevant directories.
Re: (Score:2)
In general, you can achieve similar results using conventional unix permissions, but it takes a bit more work.
"
In practice it really only comes up in a small subset of use cases.
Permissions, abstractions (Score:2)
Granted, but that's not necessarily a good argument. Standard Unix permissions provide a fairly limited set of abstractions. You have groups, users, and anyone; read, write, and execute; and a few other flags. It's concise, capable of expressing anything you like, and very efficient. One can say the same thing about assembly code. In both cases, having better abstractions is necessary, and the most important factor is the amount of time required by the various developers. And on that note, I must say that I
Re: Rescue mode (Score:3)
Haha, hahahahaha, haaaahahahahaha.
In Windows you can indeed change boot file permissions and the system won't care a bit because everything ignores or is capable of ignoring it. That is wrong.
Also, recovery is easy for Linux, you can chroot and apt reinstall everything or manually recover permissions. Windows has no package manager much less repair installed applications.
Re: (Score:2)
Yes, but I've had problems before where different sub-directories had different permission requirements. Sometimes reinstalling is easier than tracking down every change, especially if you've got decent backups.
Re: Rescue mode (Score:5, Insightful)
Re: (Score:2)
Re: (Score:3)
"Windows recovers from some things well, others... not so much. Same can be said with Linux and OS X"
Don't know about Windows, I haven't run it in 10 years. But, I run KDE Neon on top of Btrfs and there isn't a better FS for Linux than Btrfs. Making snapshots of @ and @home take only seconds and doesn't use any HD space. Rolling back to your last snapshot to recover from anything takes less than 2-3 minutes.
Re: Rescue mode (Score:2)
BTRFS: Default works perfectly (Score:2)
How stable is btrfs?
I use it as my everyday FS on laptop (default for openSUSE), on smartphones (default for Sailfish OS), and on a few servers I take care of.
The default options work perfectly well nowadays. (CoW has been able to get me out of very tricky situation like hardware crashes)
The thing that are considered still unstable are features that aren't enabled by default (namely: RAID5/6, there has been progress lately, but it's still not considered stable. Unlike ZFS's RAID-Z/Z2)
The "paint yourself in a corner CoW madness
Feature (Score:2)
For everyday use, BTRFS is actually pretty stable, and is used by default by openSUSE.
(and used to be used by default by Jolla's Sailfish).
Some specific *features* (mainly : RAID5/6) aren't considered stable yet.
(But that's the case of even XFS: they working to add CoW and snapshotting too)
But if you stick to default settings, it works perfectly.
Re: (Score:3)
yum reinstall * and reedit any changes by hand (Score:2)
yum reinstall * and reedit any changes by hand?
That will be $75/HR or I can just reload from backup.
Re: (Score:2)
Sounds simple. But what about 3rd party apps installed or apps developed in-house? If there are a fair number of those you're hosed.
Re: (Score:2)
Re: (Score:2)
Every developers system I've seen was hosed. That's why never test on dev boxes.
Re: (Score:2)
You only charge $75/hour??? Have some self-respect for the profession, man!
Re: (Score:2)
And of course 3rd party and custom utilities installed there will not be recovered.
I remain of the opinion... (Score:5, Insightful)
Re: I remain of the opinion... (Score:4, Interesting)
Good luck with that. Having a packafing system for a language allows consistency across platforms. Otherwise you're at thw mercy of the platform team, or you have to maintain separate packages for platforms woth different release and maintenance cadences.
Re: (Score:2)
Otherwise you're at thw mercy of the platform team
You do realise this is why Linux "distributions" exist in the first place right? Seriously the dependency for everything in the system should be maintained by one group and one group only. Otherwise you end up with some package manager making a change to a system incompatible with another package manager and the entire mess gets royally screwed.
The only time I've ever been forced to nuke the root partition and start from scratch in Linux was due to complete loss of plot by a package manager.
Re: (Score:2)
I'd like to see all non-OS package managers just run as a user (not root). They can throw all the crap they want wherever they like in their own directories, but for all the root-owned stuff, use an OS package like everyone else.
Whilst I'm being a bit angry about this, I think it would actually lead to a far better (and safer) solution than we have today. The likes of NPM can run as the 'npm' user and can do nightly updates if they want, and I, the sysadmin will know that it's 'safe' in so much as it'll onl
Re: (Score:2)
What if you need to install python to run the python install scripts to install python?
Re:I remain of the opinion... (Score:5, Informative)
I'd recommend watching this talk:
https://www.youtube.com/watch?... [youtube.com]
or if you prefer, the excellent-as-usual LWN summary:
https://lwn.net/Articles/71231... [lwn.net]
I don't like the language-specific package manager situation either, but the way these languages split things up does not lend itself well to the distro packaging model either unfortunately.
Re: (Score:3)
Eh, not really, except *maybe* the rust situation, but even that has been accommodated (by having the major version of a package embedded in the name of the package).
First, the general answer that 'locks on to versions' is a recipe for security and bug nightmares. In that universe, no longer is it feasible for a foundational library to fix the problems they caused all on their own, becuase the apps statically linked. An openssl tweak means the world has to be rebuilt becuase everyone baked it in instead o
Re:I remain of the opinion... (Score:5, Insightful)
This.
Or nothing other than the system package manager should run as root. Create a top level sub directory and a product specific user/group. And then let it run in it's own file space as its own user. There is very little on a *NIX system that HAS to be owned by root. As long as it's readable and executable by all, that's good enough.
Re: (Score:2)
Indeed. Install as Root is the killer of both *nix and Windows. What is the point of having a permission system if every program we run needs to have root on install.
IOS taught us the right way. That we should be able to install third party software and not give it complete control of our systems. Well, almost...
Re: (Score:2)
Re: (Score:2)
Some npm packages are as little as 7 lines of code... they have more metadata than content. Do you really want all that (350,000 packages as of January 2017) cluttering up your OS-level package manager?
Re: (Score:2)
That's a problem in and of itself. There's no sane reason for such a small function to be a library all it's own, rather than coordinated in a larger, cohesive project.
Re: (Score:2)
JS kiddies take DRY to previously unfathomable heights of zealotry, and can't abide including code they don't need. Their eternal quest for perfect code mass optimization explains a lot of the problems with how JS has evolved.
Re: (Score:2)
Note that I agree with you, however people going after copr or ppa sorts of repositories have equal amounts of ability to screw over a system. The general impatience that leads to using the free0for-all package manager associated with a language will also lead to a free-for-all of yum/apt repositories.
Of course, the quality of rollback solutions is higher with the distro manage, hence my agreeing with you, but the problem is still complex beyond just using the crappier package management.
Re: (Score:2)
This might work if you only support one platform. But now you need to do something for macOS (ports, homebrew, appstore?) and for Windows (msi, appstore, something else?) All very different, so I'm pretty sure it's less effort and cost to have a common solution specific to your language. But perhaps there is a place for a language neutral cross-platform package manager, but that would take a lot of effort to overcome the momentum behind things like Perl's CPAN or Python's PiP.
Re: (Score:2)
Well, yes and no. While some managers, like Cargo, are strictly for build-time dependencies, others like npm, pip or RubyGems can also manage globally installed libraries and apps. This bug actually manifested itself in such global installation.
Re: (Score:2)
When I ship software, the "language packager" is my first step to make a distro package. However, a large contingent of my users use the language package management to get the applications to use directly, despite having zero developer interest.
Ugh (Score:5, Insightful)
1. There is no reason to run a language-specific packager as root, whether npm, pip, composer, maven, etc. Either the package manager makes packages available to the user in $HOME, or there exists some kind of virtual environment tool. Use them.
2. Why is NPM chowning anything?
3. Read the thread, the attitudes there are unfortunate to say the least. A new version of NPM is provided when using NPM to upgrade itself without any arguments, and it grabs a "pre-release" version without warning? The version number is 5.7.0, not 5.7.0-beta or 5.7.0-rc1 or whatever. The NPM people violated semver. So there was no obvious way to know this is not an official release.
chown? (Score:2)
Maybe those "developers" and "engineers" need to learn how to fix their systems with the super-advanced CHOWN command, instead of reinstalling??
Re: (Score:2)
This reminds me of the Jenkins (centos) RPM. It used to do all the package extract and then did a 'chown -R /var/lib/jenkins' - innocuous enough, given everything in there should be owned by Jenkins anyway. The trouble was, some of us had gigabytes of files and jobs in there (in my case, it got so big we had to put it on an NFS share), so the chown would literally take hours/days. It took 2 years to get the issue fixed: https://issues.jenkins-ci.org/... [jenkins-ci.org]
OS packaging is actually pretty hard unless you know ev
Re: (Score:3)
Easy, because there's a large body of content saying "it's ok to do all that stuff, because continuous delivery!".
Some people will say "but that's not what it means" and some of those will have very reasonable implementations and call it that. While they are doing right by themselves and their team, they only make the problem worse because others see 'success' and decide that downloading and random code in production is ok, because this other guy did it and it seemed to work out.
sudo bites again (Score:2)
What is not mentioned in the summary is that the bug only shows up when using sudo.
Sudo is a nightmare, both technically and psychologically (strangely, it's seems easier to run 'sudo npm' or 'sudo fuck_me' than running the same commands when logged in as root).
It makes me laugh any time when I try to build some shitty program (inside a vm, of course), and more often than not, it tries to run 'sudo' from the install rule and trash over my system by writing and overwriting files inside /usr and /etc, and ign
Re: (Score:2)
My first command when doing admin stuff is usually "sudo -i" to get me a root user shell. Repeatedly typing "sudo" is a waste of 5 keystrokes.
Re: (Score:2)
# sudo su -
Re: (Score:3)
It's really ubunutu that popularized sudo as 'the way'.
To be fair, the order of the day prior to that was that people logged in as root to avoid the hassle, sudo is better than that.
To those suggesting 'su is better' or 'ssh root@localhost' is better, any popularized approach would ultimately land in the same land of abuses.
Of course unless you setup sudo :NOPASSWD, that's just stupid and not the way any of those things come by default.
mtree(8) (Score:4, Interesting)
NAME
mtree -- map a directory hierarchy
SYNOPSIS
mtree [-LPUcdeinqruxw] [-f spec] [-f spec] [-K keywords] [-k keywords]
[-p path] [-s seed] [-X exclude-list]
DESCRIPTION
The mtree utility compares the file hierarchy rooted in the current
directory against a specification read from the standard input. Mes-
sages are written to the standard output for any files whose character-
istics do not match the specifications, or which are missing from
either the file hierarchy or the specification.
Why is root involved? (Score:2)
Always requiring the latest version of everything (Score:4, Informative)
This is why I always reject anything that has requirements that I install the latest version of everything and use a language specific package manager to manage dependencies. Javascript packages seem the worst for the "bleeding edge" requirement, but Java, PHP, Python, Ruby and even Perl have long had issues with requiring the language specific package manager to be used.
If my distro maintainers have not packaged it and tested to the level that the rest of the OS gets tested, then it has no place on my server.
Re: (Score:2)
This is why I always reject anything that has requirements that I install the latest version of everything
Please send me through your IP address. I look forward to having another internet connected machine at my disposal.
Re: (Score:2)
Re: (Score:2)
Yes I can see that. Its a very good machine to have at my disposal.
Root + installing right off the net. Brilliant. (Score:2)
This explains a lot. A couple weeks ago I was complaining about having to build a Javascript app written using Babel and NodeJS. The build was designed to automatically check for and download the latest version of all packages with NPM before building. After the inevitable swath of updates, things were so broken beyond repair that even re-installing all the packages didn't fix the build system. The only way I was able to get anything to build again was by resetting my VM image and reinstalling from scra
Far worse than the summary makes out (Score:2)
The first tweet in TFA really takes the cake:
"“if I run sudo npm --help my filesystem [changes] ownership of directories such as /etc, /usr, /boot”"
You don't even need to install or make a change to the system. Simply sudoing npm is enough to trigger the bug. Ironically those people who are ballsy enough to run everything as root are unaffected since this seems to be some iteration with sudo only.
Epic fail. Wait are we still doing Epic fail or are we just saying "Sad!" ?
Re: (Score:2)
yarn (Score:2)
Drop npm. Use yarn. It has a lock file, it's quick, doesn't DoS the DNS until it decides to timeout, it doesn't take half an hour to install the packages in Gitlab CI, and it feels professional... Yarn is just lovely.
Re: (Score:2)
"Well, it seemed like a good idea at the time. But looking back, I can see it was a bad idea giving guns to monkeys."
Re: (Score:2)
You don't have to be root to run it?
(that was a joke btw)
Re: (Score:2)
The only reason to run npm as root is the same as the only reason to run pip as root.
Instead, you should use nvm or something similar with node stuff, and virtualenv with python stuff, then you don't need root.
Re: (Score:2)
Chroot into your damaged install, mount a devtmpfs and a /proc, maybe /sys,
then, "for p in $(rpm -qa); do rpm --setperms $p; done"
Works on rpm-based distros at least.
If it's changing all the modes to 000 or something, do a "chmod -R 755 /etc" or something on whatever files are causing trouble, then run the above to restore them to the correct state.
Yeah, I was going to post something similar. Unfortunately, of course, you're still hosed on userland files and anything non-packaged. And the kind of people running bleeding edge npm are probably those who barely understand permissions to begin with.
On any RHEL system that's the first place I'd start, then try to pick through the debris of what was left.
Re: (Score:2)
If you are using something like npm, odds are the rpm databse is only a vaague hint of what a fraction of the system used to look like. That admin is already down the path of ignoring the OS packager, so expectation that it will repair completely is low.
Re: (Score:2)
Back in the late '90s, I was doing tech support for an ISP, and part of my job was running database queries (through shell scripts) while logged into a server via telnet. (The server was behind the firewall and you couldn't reach it from outside, so this was safe.) Some of the scripts used sudo, but I
Re: (Score:2)
So long as you mange the permissions on the scripts and commands the "sudo" user can run, to avoid changes, it's a great tool.
Best practice is for users to logon with their own account and 'sudo su -' to root, if they are root authorized. Otherwise, they get a list of sudo commands they can run, and they don't get root.
Re: (Score:2)
And how do you prevent whoever installed the OS in the first place from using it, especially if the installation program won't let you install without setting it? And, it's my home computer, nobody else uses it so why shouldn't I know it? (Yes, I understand that you're talking about work computers, especially servers, but there are cases where having somebody know the root password isn't a problem.)
Re: (Score:2)
so when I run "sudo apt-get update" i'm doing it wrong?
If I don't run "apt-get update" as root it doesn't work. Does that mean Debian has been doing it wrong for two decades?
Re: (Score:2)
People log on, then sudo to their admin account which is a functional root, or they sudo directly to root in some cases.
Sudo provides logging, and gives you the ability to delegate commands that can be run as root, even if a user doesn't get full root.
I don't get the hate, just typical refusal to change, the old ways are better?
Re: (Score:2)
If a system is "in production" but using npm to manage it, that backup database is not particularly relevant to the state of the system. I have seen projects that say to install an rpm, then immediately start modifying parts of it becuase the developer wanted to customize. Use of the package manager means discipline, and using npm to deploy stuff as admin represents *not* having that discipline.
Re: (Score:2)
Imagine if that had happened with Apple.
Slashdot would have completely filled-up their Servers by now with the Hand-Wringing and Apple-Hating Diatribe.
Think about it.
Do you mean the acutely difficult: sudo diskutil repairPermissions /
Re: (Score:2)
You blame his phone for using standard unicode characters that are unsupported by Slashdot?