Ubuntu Servers Hacked 330
An anonymous reader noted that "Ubuntu had to shutdown 5 of 8 production servers that are sponsored by Canonical, when they started attacking other systems. Canonical blames the community, saying they were community hosted, and were poorly maintained. However, kernel upgrades couldn't be done because of poor backwards compatibility with the very hardware that Canonical had sponsored! While people point fingers at each other it is pretty clear that both sides are equally to blame, the community administrators for practicing bad security practices, such as using unencrypted FTP transfers with accounts, not properly maintaining the system. However Canonical should have been well aware of what they are hosting. The question remains, if any of the files distributed to users have been compromised. A major blow for Canonical though who are attempting to enter the business market with Ubuntu Server."
New distro name (Score:5, Funny)
Hacked... (Score:5, Funny)
Re: (Score:2)
True, it has an entirely different meaning when applied to a FOSS organization rather than a commercial closed source company.
Re:Hacked... (Score:5, Insightful)
People in the industry are aware that "hack" used to mean "cleverly manipulate a device into doing something its designers did not intend." People also know that "wherefor" used to mean "why." In both cases, the original definitions no longer apply.
Language changes. You'll get over it. There are more important battles to fight.
Re: (Score:3, Insightful)
Re:Idiot (Score:4, Funny)
And to think, the only reason I post here is so I can be taken seriously by the people who really count.
Another dream shattered!
Re: (Score:2, Insightful)
Yes, it means exactly what he thinks it means. This whole thing with calling hackers "security researchers" is just silly beyond belief. Both of these little peccadilloes in terminology are reasons that no one who really counts takes the Slashsnot crowd very seriously.
I don't think you know what he thinks it really means. I think he want's to use hacking as a generic term, for doing stuff as in "I hacked together a working PC form all the junk in my basement" or "I hacked that new feature into my existing code.", and so the poster and many people who like using the word hacker for themselves but don't want others to immediately associate themselves with criminal hackers, tried to coin a new term for those people, "crackers". And while that term never caught on people
Gentoo also recently disclosed security breach (Score:5, Informative)
http://bugs.gentoo.org/show_bug.cgi?id=187971 [gentoo.org]
Re: (Score:2)
Re: (Score:2)
and it's usually in that order too.
Don't worry (Score:4, Funny)
Re: (Score:2)
Re: (Score:2)
Windows does give you crude and non-secure ftp client by default.
Any Unix machine is more likely to have an scp client than ftp client.
I would like to read a report (Score:5, Interesting)
This could really help the community as a whole, and I know I would enjoy reading it..
Re: (Score:2)
Well. I mean, 5 of 8 machines were already totally owned by the time they worked it out. I don't think documenting the discovery process is going to do anyone any favors. Unless we're going to be composing a Linux Administration HOWTO: Best of Bloopers.
Re:I would like to read a report (Score:5, Insightful)
Isn't that part of the Linux/Microsoft Double Standard? Now, if Microsoft this type of issue and had been less than totally open about the cause and methods, you know as well as I do that there would be a high-pitched wailing from the Slashdot World.
Re: (Score:3, Insightful)
Isn't that part of the Linux/Microsoft Double Standard? Now, if Microsoft this type of issue and had been less than totally open about the cause and methods, you know as well as I do that there would be a high-pitched wailing from the Slashdot World.
I'm not so sure this is any kind of double standard. The last time Microsoft was compromised there wasn't a "high-pitched wailing from the Slashdot World" demanding details. Nobody really expected to hear any details. And we didn't get any. I'm sure there were some who would have been interested in them... and others who didn't care. And this is the situation we're in now.
Some people care about these details and some don't. The parent apparently thinks there's nothing to learn. I disagree. There mi
Re: (Score:3, Interesting)
Could you imagine the data load if everybody wanted the information about how every windows server that ever got hacked (I assume M$ takes greater care of it's servers than general users, just as Canonical does).
Re: (Score:3, Insightful)
You mean the high-pitched wailing from the Slashdot World actually stops at some point?
Re:I would like to read a report (Score:5, Interesting)
I could fill about a 100 pages on my own from stupid things I've done and stupid things I've seen coworkers/customers do.
The funniest one is still one where one of my coworkers nuked /lib on a fairly important machine unintentionally because he just loves his spacebar:
rm -f /home/user/project /lib/*
Upon which of course by he proceeded to ask everyone "Hey, suppose I deleted something like /lib, is there a way to get it back?", followed by 10 people laughing, followed by a minute of silence as soon as we realized what machine he just did that on. He never got a root password for an important server after that incident. In hindsight, that was a funny incident, and a valuable lesson to us all (we all became paranoid of rereading what we just typed).
Yes, we had backups... Yes, tape drives are still slow
Re: (Score:3, Informative)
Your server was poorly administered.
Re:I would like to read a report (Score:5, Insightful)
"He never got a root password for an important server after that incident. In hindsight, that was a funny incident, and a valuable lesson to us all (we all became paranoid of rereading what we just typed)."
I hope the decision to deny him root access was based on more than that one unintentional incident. It could have happened to any of you. After all, why else would it be a "valuable lesson" to you ? Isn't the person who made that mistake the least likely to make it again ? And you did also say you "could fill about a 100 pages on my own from stupid things I've done".
Comment removed (Score:4, Informative)
Sounds like (Score:2)
Re:I would like to read a report (Score:5, Interesting)
It's important to note that the servers may not have been actually rooted. There is a large number of ssh dictionary breakin attempts on every machine I administrate on several completely different ip blocks. The worst hit is usually my personal server that tended to get hit with several thousand attempts per hour(enough that legitimate logins were a problem) before I installed countermeasures. Even now the countermeasures are locking out 5 to 8 hosts per day.
They have managed to get user accounts on a few occasions and most of the time they never even attempt to gain root. They just start scanning for new hosts.
I'm now running a python script called DenyHosts [howtoforge.com] to find and lockout dictionary attacks. "apt-get install denyhosts" for debian users. Even on much more liberal settings than the default it's lowered my cpu load considerably and locks out attacks in the first minute rather than the hour it would otherwise take me to notice.
Re: (Score:2)
Others can go say "bah security by obscurity" for all they like, I think they're mostly stupid/ignorant anyway
Actually what I do is run the ssh server on 127.x.y.z:someport and internal.ip:someport.
Then I have the firewall redirect all accesses to external.ip:extport to 127.x.y.z:someport.
That way even if the firewall rules aren't present (or messed up), it's likely that people ou
uh ho (Score:4, Funny)
Re: (Score:2)
The real test (Score:5, Interesting)
It sounds like that part at least is still underway, with a meeting (FTA) in "#ubuntu-locoteams on Tuesday, August 14, 2007 at 2:00PM UTC". Seeing as that's yesterday, we should probably reserve judgement a day or two to see how they respond.
sorry... (Score:2, Insightful)
who the hell places such exposed servers like these on the net without applying security patches and following simple rules? yeah, the freaking old hardware, compab problems, i sure understand that. but then make a fuss 'bout it. threat to stop maintaining the hardware if the networks cards aren't changed. if that REALLY is the only problem with the hardware which prevented updates, then i just don't understand how the hell this could happen. NICs, even though this would be no consumer
Re:sorry... (Score:5, Insightful)
Ultimately, I'd say that if this does wind up being an admin problem, then Ubuntu Server will not suffer. The bottom line is that a poorly administered server is a hacker target regardless of the OS.
Did they file bug reports? (Score:2)
The NIC's worked fine with version A.
The NIC's did not work with version B. Where's the bug report?
Breezy - this is where they stopped.
+ 6 months - Dapper - LTS, where is the bug report?
+ 12 months - Edgy - a bug report?
+ 18 months - Feisty - a bug report?
If you just CANNOT apply a patch then you HAVE TO make sure that EVERYTHING else is locked down AND INCREASE YOUR MONITORING OF THAT SYS
Re: (Score:2)
sftp (Score:4, Insightful)
Re:sftp (Score:5, Interesting)
Re:sftp (Score:5, Insightful)
Not like Debian (Score:5, Informative)
Bruce
Re: (Score:2)
I would assume that the Ubuntu source is safely stored offline somewhere and can be recovered but one of the lessons that has to be learned is the value of a standardised production environment that's been designed in
Re:Not like Debian (Score:4, Funny)
Re: (Score:2)
laziness and excuses (Score:2)
Re: (Score:2)
"tighter than a dolphins ass" (Score:4, Funny)
I suppose they'll have no choice but to flee to deeper waters.
Re: (Score:2)
Re:laziness and excuses (Score:5, Funny)
how ironic (Score:4, Insightful)
Re: (Score:3)
Re: (Score:2, Insightful)
Re: (Score:3, Informative)
If you had bothered to read the originating mail ( https://lists.ubuntu.com/archives/loco-contacts/20 07-August/001510.html [ubuntu.com] ), you would have seen that these servers were hacked through unpatched 3rd party web-applications running on these servers - namely:
Your argument is whiny and offtopic.
Re: (Score:2)
Constructively (Score:2)
No software is perfect,no package is absolutely secure.
Its good that these servers were compromised and detected too[i hope withing time].
This means either admins are not doing their job properly or the culprit packages are buggy.
Either way it is an eye opener to the community and especially Canonical.
This calls for better auditing and more effort to be put into security on Ubuntu server systems as well as packages which make their way into Ubuntu.
This may possibly mean mor
Re: (Score:2)
Re: (Score:2)
Most Windows problems tend to be about what the system will do by default, not what sort of ways you can screw yourself up if you really try hard and insist on ignoring decades of other people's mistakes.
Panic, They Might Have Gotten the Source Code! (Score:5, Funny)
It's like NT all over again [slashdot.org]. God only knows what bad things they can do with that.
This is why packages should be signed (Score:2)
Ubuntu seems to have something in place already, but from my look at it, doesn't seem nearly as insistent on security as it should be.
New NIC, Anyone? (Score:2)
no upgrades past breezy due to problems with the network cards and later kernels
So wait, this old hardware has no PCI slots? No USB ports? Nothing that could allow one to simply NOT USE THE UNSUPPORTED NIC CARD???
;)
What the HELL is going on here? This isn't just an 'oops', this is really, really friggen lazy! Last I checked, 3Com and Intel still have about a billion NICs out there in the great wide world. Hell, I could mail them a few myself...
No?
Re: (Score:2, Insightful)
Re: (Score:2)
Admin: You see, boss, I wasn't there. I can't exactly reach through the pipes!
Boss: I see. So should any hardware fail, it can never be replaced? No one has any kind of physical access to the hardware at all? I suppose the servers are encased in concrete??
Admin: Well no. Not exactly...
Sure, that'll fly. I'll use it on my boss. "I couldn't replace the drive from home, and didn't feel like driving in, sorry."
Sheesh
Re: (Score:2)
Call the datacenter. Scream at the staff. Scream at the staff some more if the NIC isn't installed after the first round of screaming.
It's not as if the datacenter isn't dying to help you for a fee.
That's not even getting to the mind numbingly obvious option of schlepping over to the datacenter.
Re:Older cpmpatible NIC, Anyone? (Score:2)
I wonder if they could use some of my NE2000 NICs. They should be compatible. I'll even toss in some 50 ohm terminations.
Further proof.. (Score:5, Funny)
And for bonus "hate" points, even MS servers can be secure if they are admined probably. Don't worry though, I have my flame suit on.
Some clarification (Score:5, Informative)
It happens (Score:5, Informative)
My site - http://screencasts.ubuntu.com [ubuntu.com] was one of them that was affected, so I was of course concerned that there might be some data loss. I only use SCP to copy files up to the site, and logon with my ssh key, so don't think that all Ubuntu community members are using FTP, weak passwords and really old software, it only takes _one_ though to naff it up for everyone else.
The Canonical system admins (on top of the work they already do) migrated the services from those servers to their own DC very quickly. My site went down on Tuesday and was back by Friday. For free hosting and oodles of bandwidth, I'm happy with that downtime - for a community site.
Re: (Score:2)
Re: (Score:2)
Soviet? (Score:5, Funny)
In Soviet Russia, server attack you?
Turns out the whole reason for the attack was... (Score:5, Interesting)
I used to be an ardent Ubuntu supporter but since Dapper and the wider adoption there has been too much emphasis on making things more Windows-like and less on best practices throughout the Ubuntu community (note I said the community, not the developers). Stuff like Automatix and the general feeling that any script that or line of code that is posted on the Ubuntu forums is guaranteed safe has led to lax standards. I've brought this up a couple times and any valid discussion quickly descends into a flame-fest and the mods (rightly so) lock it down.
The Ubuntu community has bent over backwards so far to prove they can include everyone they lost site of many of the things that make Linux a better choice for many people; time to get back to fundamentals and best practices, the sooner the better. Stop worrying about besting Windows at every silly thing (ahem, desktop transparency), stop trying to include aunt Tilly (who is never going to "switch" anyway) and remember that some things take more effort but are often worth it.
Re:Turns out the whole reason for the attack was.. (Score:4, Interesting)
Re: (Score:3, Insightful)
Re: (Score:3, Interesting)
And up until last week the most frequent answer on the Ubuntu forums for many questions was "use Automatix".
Possibly because web forum software is horrible on all fronts. It caters to a narrow, dangerous audience of experienced people who should know better. People who's been using the internet for long enough to know what a "web forum" is, but aren't familiar with mailing lists and IRC. So the forums were never planned for, but it eventually it was felt that the forums should be intergrated rather than continue to grow and divide the community.
Automatix in particular is a fantastic story of why I avoid forums.
Breaks happens all the time (Score:5, Interesting)
In mean time, there is a tradeoff between having one, LTS release which has rather old kernel with old drivers and new one, which has 18 month support but has everything up to date, including also unstable stuff of course. But in fact it doesn't even mather, because admin is who in charge.
So Linux is more secure than Windows? You bet. Then why such break-ins happens? Because of lazy or hobbist admins who have no time or maybe not enough knowledge to lock down server to protect it from attacks. To lock down such Windows server/workstation is much harder because of "black box" mentality such software has. But it is also possible.
So in resume - those are admins who are gulty persons here. Ubuntu Dapper and Feisty are secure enough releases to keep them locked down without causing trouble for services. And ohh, be careful to which persons you give access to and have good password management system.
To put this into perspective... (Score:3, Insightful)
Re: (Score:2)
Do you have evidence for this? Particularly for the "all the time" part.
Re:Following the M$ example. Re:BWAHAHAHA... (Score:5, Funny)
Well, if they _did_ get broken into all the time, then that would be pretty embarrassing. The last thing they would want to do is publicize the fact, so it only makes sense that they would cover it up and say nothing about it.
Since nobody has _ever_ said anything about frequent break-ins, it's clear that they must be happening.
Why am I the only person who can see how obvious this is?
Your Conspiracy-Fu is strong, young Grasshopper! (Score:3, Funny)
"The complete lack of evidence is the surest sign that the conspiracy is working."
- Jack Handey
Re: (Score:3, Funny)
I am surprised no one reports how oftem Linux source code is taken from company servers, they must get hacked constantly compared to MS.
Re:Following the M$ example. Re:BWAHAHAHA... (Score:4, Insightful)
Re:Following the M$ example. Re:BWAHAHAHA... (Score:5, Funny)
Well, I heard that Ubuntu [ubuntu.com] isn't very good at that either...
Re: (Score:2, Funny)
It boggles the mind.
Re: (Score:2, Insightful)
They got arrogant, cocky and lazy. They let their security slip on things a Windows uers wouldn't use or care about (ex. FTP vs SFTP, from a user perspective, the difference is minimal).
Does your reality distortion field go so far as to say that Windows
Windoze access should be read only / password free (Score:2, Interesting)
How insecure is it to leave a system accessible to Windows users on any front?
I won't give an gnu/linux account to any windows user because a minimum of 25% of them are part of a keylogging botnet [slashdot.org]. They are liable to access my machines from windoze and things go downhill from there, even if they use a better client. A system is only as strong as it's weakest link.
Ubuntu itself is dangerous because it includes non free software like Adobe Flash, but this should not be of concern to business users. The
Prevent Windoze at the packet filter (Score:3, Insightful)
You can back up your policy in the packet filter.
In iptables, look up osf and --genre.
For pf, look up osfp.
I am what I am and it is what it is. (Score:2, Funny)
I've seen this hundreds of times, but never bothered with it.
You made a good argument, but when you use terms like "Windoze" you lose credibility.
People who can't see though my wording probably won't believe the argument anyway. Brainwashing is strangely dehumanizing like that. The victims lose their sense of humor as well as reason. The term "windoze" implies both of those losses and that people who continue to use it are asleep at the wheel.
Re: (Score:3, Insightful)
Here we are, talking about a serious security breach at a prominent Linux distributor, and all you can muster is a hissy fit because not enough people are blaming Microsoft for it.
It's not clever. It's certainly not constructive. Worst of all, it reflects poorly on the community you claim to serve.
You're the rhetorical equivalent of a brick-throwing protester at a WTO meeting, foolishly believing that vandalism and insul
Re: (Score:3, Interesting)
How are those peaceful protests working out for you anyway? Weed is still illegal, the war in iraq went on, and the disparity between the rich and poor is stronger than ever. If one person throws a brick, hes a vandal, if a hundred thousand do it, its a revolution. Thats actually my main problem with protests, their peaceful nature. Its almost like the people just want a shell of a protest to look "cool" while in reality risking nothing of substanc
Re: (Score:2)
Jokes aside, my systems are working, so it probably another issue.
Re: (Score:2)
Not unless you clicked through a "these packages aren't signed" warning. The package signing system is specifically designed to handle compromised repositories.
Re: (Score:2)
How right you are! (Score:5, Insightful)
On the other hand, we all know that segregation & apartheid were both ended by paid professionals. If you want something big done right, only paid professionals can do it.
Re: (Score:2)
Re: (Score:3, Informative)
Re: (Score:2)
I put together a new machine Core 2 Duo on a new Asus board. I put Fiesty Fawn on it. It works great. I did notice a networking problem with the built-in NIC when trying to transfer large files to my fileserver. (DVD ISO) It would start and then hang with less then 1K transfered. Web, small transfers and such worked fine. I finaly had to make a SMB share on the machine and use my
Re: (Score:2)
Or do as I do at times (and as Oddscurity was suggesting I think) and use the old network card driver with a newer kernel
Re: (Score:2)
Now, whether or not this _should_ be the case is another matter, but it is a reason why this isn't necessarily a great idea.
Re: (Score:2)
They're not. The repository servers are controlled and maintained by Canonical. These were community-run servers for hosting Local Community Teams [ubuntu.com]. You can take the tin foil hat off now.
Mod -1 please (Score:3, Funny)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)