Trojanized SSH Daemon In the Wild, Sending Passwords To Iceland 171
An anonymous reader writes "It is no secret that SSH binaries can be backdoored. It is nonetheless interesting to see analysis of real cases where a trojanized version of the daemon are found in the wild. In this case, the binary not only lets the attacker log onto the server if he has a hardcoded password, the attacker is also granted access if he/she has the right SSH key. The backdoor also logs all username and passwords to exfiltrate them to a server hosted in Iceland."
SSH Got Bjorked? (Score:5, Funny)
Re:SSH Got Bjorked? (Score:5, Funny)
Re: (Score:3)
This is completely normal human behaviour.
Re: (Score:2)
Re:I use Gentoo (Score:2)
I only install from sources you insensitive clod!
Re: (Score:2)
Or you could source two binaries from two diff repo servers in two diff countries, and only accept the install if they both are identical.
Re: (Score:2)
Re: (Score:3)
Re:I use Gentoo (Score:5, Informative)
Unless you're auditing said sources, you're no better off than installing binaries.
You know that Gentoo checks the SHA256 SHA512 and WHIRLPOOL digests of the downloaded sources before compiling right? You also know that the digests stored in Gentoo's repository are signed using a PGP key of the package maintainer, right?
So can you please explain me again why I need to audit such sources? Sure sources can be compromised at upstream's servers and Gentoo maintainers can also make mistakes but this is not what TFA is about.
Re: (Score:2)
Re: (Score:3)
Everyone replying here seems to be obsessed in comparing binary vs source and totally miss the point. The point is that
Gentoo users do not have the habit of installing binary packages at all, for that reason this trojanized sshd BINARY does not concern us.
Wooooooosh!
You are missing the point. Install doesn't matter (Score:2)
How does the bad guy get the trojan on your system, if not from the repo, yo
Re: (Score:2)
How does the bad guy get the trojan on your system, if not from the repo, you ask? He gets access when someone else who is infected logs into your machine - your sysadmin, your hosting company, a vendor, etc.
Dude, this does not make any sense at all!
If person X logs into my machine (which is not infected) and that X is infected with the trojanized daemon. Then the attacker will not be able to find out the X's account password in my machine because the connection was made to my uninfected daemon and not to X's infected daemon. Get it?
Re: (Score:2)
Re: (Score:2, Funny)
Warner Bros. Records.
Re: (Score:3)
Re: (Score:2)
Re: (Score:3)
If it weren't for the mention of Iceland (Score:3)
I'd think this was a story about Cisco gear from not that long ago.
Re:If it weren't for the mention of Iceland (Score:5, Interesting)
I was asked to clean some exploited servers in the wild that had compromised SSH binaries in them. This was about 10 years or so ago, so this is really a not current news story.
Thankfully most script kiddies are exactly that, and their script left the source on the machine for me to review. Why? I guess to compile on the machine to make sure it used the local libraries. They generally didn't have passwords hard coded though, they used SSH keys. The passwords they stole were usually stored locally in a file, and sent to somewhere in eastern Europe or Asia. I have no idea if that was by the script kiddie design or the person who rolled it up. It's a pretty slick trick though. You get the script kiddies to gather intel from compromised machines, and feed it back to you. Now you have access to every machine the script kiddies compromised.
The funny part about most of the cleanups was, the version of SSH they put on was newer than the remotely exploitable version they got in with.
Re:If it weren't for the mention of Iceland (Score:4, Insightful)
I was asked to clean some exploited servers in the wild that had compromised SSH binaries in them. This was about 10 years or so ago, so this is really a not current news story.
Someone got murdered about 10 years ago, so there's no point reporting about current murders.
Or what is your point exactly?
Re: (Score:2)
Breaking and decorating (Score:3, Interesting)
Re: (Score:2)
Re: (Score:2)
Aw, I don't know why you were modded troll. That's funny. :)
And only ultra secure because it would work for a few days, and then hang, refusing all access. :)
Re:If it weren't for the mention of Iceland (Score:4, Funny)
Maybe it was an Open Source client, and they had to give you the source code to comply? :-)
Re: (Score:2)
I've seen that too. Mostly with fairly open hosting machines. Like, letting anyone sign up for free or low cost hosting. I saw a lot of them that were built for one platform (usually Redhat), where the libraries didn't quite line up with the running system.
At one place, I was collecting them. The script kiddie would put their SSH server on a high port. It worked great, except they couldn't get root with it, and they couldn't get to the daemon through the firewalls. It was rather entertaining.
I hat
Tip (Score:5, Informative)
I cleaned up an a backdoored Debian system after discovering md5 sum mismatches on all the ssh binaries from the original packages some time ago.
debsums is a nice utility to check this for you, granted that the attackers didn't modify the signing keys and installed their own package.
Re: (Score:2, Insightful)
So how is a compromised ssh binary getting on Debian?
Re: (Score:2)
Simple: It snuck in the back door.
Re:Tip (Score:4, Informative)
Poor passwords, other infected (non-repo) packages, exploits in services (like apache, etc) installed with too many permissions, etc.
Re: (Score:3)
Re: (Score:2)
Mostly weak passwords/lack of anti-brute-force, Apache vulnerabilities is probably the second most common. Nothing new.
That's why SSH - most compromised by the trojan (Score:3)
Re: (Score:2)
Re: (Score:2)
Yeah, I hate most Linux exploit articles. They always seem to be about what happens after the machine was rooted. Binaries changed out, configurations changed, permissions set, yadi yadi yada... They never seem to be about how they got rooted in the first place.
Unlike the MS world where typically some software maker (and often MS itself) has screwed up, Linux tends to be rooted because some administrator or user got careless. That is not very interesting and the details do not matter.
Re:Tip (Score:5, Insightful)
You can never clean up a system. MD5s help, but you know what one of the first things I'd do when rooting a system is? After making sure my rootkit didn't show up in directory listings, I'd patch md5, shasum, perl, and ruby to return the exact MD5 I wanted for every file I defined a magic string for.
You gonna catch me on some systems? Sure. You gonna catch me on an extremely common distro like Debian without installing out-of-tree software? Probably not.
Re:Tip (Score:5, Insightful)
Re: (Score:3)
Rule #1 of investigating a compromised system is you don't use the tools on the compromised system.
Security is a process. Running a scanner on a compromised system has no guarantee of finding out that the system is compromised. But 98% of the time it will work just fine - these systems are usually compromised by script kiddie tools that don't spend a great deal of effort to find the rootkit scanners.
I suppose it would be better if somebody came up with a well-developed package for cross-scanning systems o
Re: (Score:2)
I suppose it would be better if somebody came up with a well-developed package for cross-scanning systems over shared storage (does it already exist?) but that's also only going to reduce your 98% to 98.5%. I suppose all you need to really do is to validate the integrity of the host system's scanner (or rpm or dpkg, etc.) but then again the remote system access method could also be compromised (today it's TLA stuff to compromise e.g. nfsd such that it will only return the wrong (but valid) md5sum and sha1sum and [randomhashsum] for only the scanners you might check for) but it's not likely.
Taking down every system on a frequent schedule for an offline scan would be a good idea but then again the BIOS could be compromised and you start to run into the collision of business needs and absolute security.
I use NFS root internet servers connected to a intranet NFS server. On the last I use rsync to do backup and detection of any files change. The only permanent memory on the internet servers machines are the BIOS, ok, but I have yet to see an exploit that hack the BIOS in a way to compromise a TFTP + NFS boot without any detectable trace from the NFS server point of view. If your concern really go into this level of paranoia, then you can simply hack the write protection signal of your BIOS memory chip (see
Re: (Score:2)
Bootable CD or DVD with tools on it. Worth its weight in gold.
use a PPC mac. (Score:2)
No hacker can find the binaries to a PPC mac os 10.5
Duh, you can boot OSX from a DVD, if you are really that paranoid.
Re: (Score:2)
Re: (Score:3)
Re: (Score:2)
Re: (Score:2)
Duh, run md5sum from external machine (Score:3, Insightful)
Man, you would run md5sum on the actual compromised box??
Why not do it from a ISO booted linux, and nfs share the whole box , so you can sum it from the outside.
What you really need is all binaryies libs running of a partition that is read only. Kind of like android.
Re: (Score:2)
The nfs could be hacked to give the original binaries to your outside server.
Absolutely impossible because the hacked file will have his checksum changed from the NFS server point of view. As long as the NFS server is not compromised, this is very safe. I personally use NFS root internet servers since more than 12 years now and I can say that's a very efficient way to detect and recover from external attacks. In addition, it very reliable, comfortable to manage, and easy to backup.
Re: (Score:2)
If you hacked the box, you can certainly also modify the NFS server running on that box.
The NFS server is NOT the same box as the internet server that use NFS root as a client. Here is a simple schema:
RAID array NFS server box Firewall Internet server boxes (NFS root client) Firewall Internet
To archive a permanent modification of one of the internet server box, a file must be modified on the NFS root mounted filesystem. This modification is very easy to detect from the NFS server point of view. I personally use a simple script that parse the rsync log while it backup the internet servers
Re: (Score:2)
Arrg! Missed to see that the formatting removed chars on the graph. Below is a correct one:
RAID array ==== NFS server box ==== Firewall ==== Internet server boxes (NFS root client) ==== Firewall ==== Internet
Re: (Score:2)
Yes, "out-of-tree" is the key.
I have experimented a very successful way to solve the problem: Internet connected servers using NFS root from an intranet NFS server. All the machines are using a standard distribution. The intranet NFS server handle the backup of all the internet servers using rsync. It's really easy and efficient as it's just moving data internally of the RAID array. It's easy to detect unexpected files change, to compare internet machines binaries with the intranet machine binaries, and to
Re:Tip (Score:5, Funny)
This is, of course, assuming that you yourself are not running on another compromised virtual machine.
(There was one hack I was involved in where an investigator tried to get clever and started calculating MD5 checksums with a universal Turing machine operated using pencil and paper. Fortunately, I'd already trojaned base logic itself and managed to subvert alphanumeric characters to return the 'correct' values. Hacking the logical representations of arabic numerals? Now that's pretty advanced stuff. But then, there's always the worry that my own consciousness is running on something other than what I think is my own brain...)
Re: (Score:2)
I can always have a real hardware PPC or MIPS based linux or even old Solaris machines used as 'screeners, scanners' to verify the VMs.
Oh and I could use 2 identical machines that do a complete scan and compare results.
Re:Tip (Score:5, Funny)
If there is one thing I truly dislike, it's getting backdoored.
Effective (Score:5, Interesting)
I myself have set up modified versions of the sshd dameon/ssh client on various rooted machines in the past (mine was simply allowing root login without logging anything with a hardcoded password + writing logins/hosts/passwords, after encryption, deep inside a fake shared library on the system - this took less than an hour to implement in the openssh source).
In all cases, despite the extreme simplicity of this mechanism, it was very, very easy to then gain access to a lot of other boxes on the network, and eventually to all the network itself. Just wait for the clueless sysadmin to log in (dumping his pass), then wait for him to use ssh to log in to another box in the network (dumping his pass as well). Then install the modified version of the daemon/client on to this new box. Rince, repeat.
It is in a way terrifying to see how most sysadmins are clueless. So, IMHO, a few measures that should be compulsory on any network:
- Do not allow write access to any essential binaries (like sshd, ls, and so on). If you have to, make sure you have a stealthy daemon checking the hashes of all critical binaries at regular intervals to make sure they have not been tampered with. ...there are also other ways to implement an effective kernelspace rootkit, like VFS hijacking, so make sure you disable modules loading in the kernel before compiling (and check the hash of the kernel image itself regularly as well).
- Never, ever, log in to a server in your network from another server in the same network. Use port redirection, then log in from your initial box.
- Always use a dedicated server for system logs (syslog handles that very easily - sending logs over the network). That way, the logs of the initial system compromise will not be deletable. Do not ever log in remotely to this dedicated machine.
- After the initial system install, make a dump of the syscalls table of your kernel. Check it regularly to make sure it has not been tampered with (kernelspace rootkits usually hijack this table).
-
Those are the main sane measures that come to mind (this is all based on real life experience intruding into large companies/laboratories/university systems, without ever having been detected). Posting anonymously for obvious reasons.
Comment removed (Score:4, Informative)
Re: (Score:2)
Bad advice because ... (Score:3)
Re:Bad advice because ... (Score:4, Informative)
Security is usually one of three things: Something you know (password), something you have (key, token) or something you are (biometry). Alone, none of them is really a big security obstacle, at the very least you should combine two of them. If you're paranoid, require all three.
Using two of the same class is patently useless, as practice has shown. If one password can be sniffed, so can two and more. And people tend to store their keys and dongles and other work related gadgets in similar places (if you get their keys, you usually also get their ID dongles), so if one is gone, usually they are all gone. And I guess I needn't explain why fingerprinting and retina scans aren't really a sensible combo either (aside from the cost).
And preferably request them through two separate and independent channels. It's amazing how rarely this rather essential minimum of security is enforced.
Re: (Score:2)
Re: (Score:2)
What I wrote above should be obvious but I seem to have attracted five people that have decided I'm a pedant so want to get pedantic about me using "password" to make the my post easier to read.
Re: (Score:2)
Re: (Score:2)
There is no way to enforce this, except to scan people's home directories for unsecured keys. It's also awkward if they are using "ssh-agent" or other passprase unlocking tools on a shared host, such as a server where I have administrative access or when someone has set up software to use passphrase-free keys as part of a regularly scheduled task, such as a backup system for databases. I've used these approaches to borrow other user's SSH keys when needed, not always with their advance knowledge. (I always
Re: (Score:2)
And it doesn't help when we get so many people writing "don't use passwords" - so they take that advice to mean not securing their keys :(
Re: (Score:2)
Re: (Score:2)
Single factor authentication with a stealable key can be even worse than single factor authentication with a password in some cases.
Re: (Score:2)
Re: (Score:2)
No - the above poster wrote "don't use passwords". Taken at face value that sounds like recommending not to encrypt your private key with a password - because that would mean "using a password" - thus bad advice.
No - that is called reading between the lines, adding something that is not there and making an assumption that somebody that doesn't know better will not
Re: (Score:3)
No - the above poster wrote "don't use passwords". Taken at face value that sounds like recommending not to encrypt your private key with a password - because that would mean "using a password" - thus bad advice.
No, it doesn't. Everyone with half a brain regarding SSH should know what 'not using passwords' with SSH means - and it doesn't mean *not* using a pass 'phrase' for your SSH key. He never said that. I'm afraid your splitting hairs here to have something to say.
No - that is called reading between the lines, adding something that is not there and making an assumption that somebody that doesn't know better will not make - thus it doesn't change that the post above is bad advice.
No. You have totally misunderstood the post, and please don't tie yourself in knots like that. It was very clear what he wrote and you were the one who added in 'I think a password is a very good idea if combined with a key'. It's also not called a 'p
Still bad advice because ... (Score:2)
I'm not splitting hairs because ssh login with a key and no pass phrase is a frequent shortcut.
You don't know what he's talking about either - you are just adding things you think should be there but are not.. I'm making a comment on something that should be there if it's going to be good
Re: (Score:2)
Re: (Score:2)
Is this for real or just buzzword pizza? (Score:2)
This makes no sense. Once somebody has root and can write to those binaries you are already fucked and they can change access to anything they like, and then write over it to their hearts content.
Putting them on read only media sounds like a cool idea but is difficult to implement for anything other than the short term and can be circumvented if somebody roots your box anyway. There's the script kiddie toy of changing file attributes (and thus announcing
Re: (Score:2)
This makes no sense. Once somebody has root and can write to those binaries you are already fucked and they can change access to anything they like, and then write over it to their hearts content.
Quite right. Binaries are already write protected from users but if you're root you can do anything.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Simply use NFS root for the internet server. It's reliable, easy to backup, and fun to manage.
Re: (Score:2)
Once somebody has root (or full physical access and a screwdriver) they own the box, and about all you can do is make sure they can't easily get into other ones.
Re: (Score:2)
Except you can't. The securelevel [wikipedia.org] feature in FreeBSD (maybe others) disables the ability to write to files with the immutable bit set. The only way to write to these files is to reboot the box into single user mode, before the kernel securelevel is raised. Single user mode = no network running.
Setting securelevel is a one way thing, once it has been raised, you can not lower the securelevel.
Re: (Score:2)
Re: (Score:2)
old school: send remote syslog to a printer. real paper printer.
can't rm -rf paper; at least not remotely.
Define "wild" (Score:2)
The article says "...discovered on compromised servers."
If they are not distro(i.e. ubuntu, redhat,centos, etc.) servers, or openssh servers, this is a non-story. I can create a random domain, compile a version of openssh to sends passwords to me and host it on my server as an official ssh binary.
If I install stock wordpress with access to the file, would that then count a compromised?
Gather passwords with ssh? Hah! (Score:2)
I never use passwords with ssh. If ssh asks me for a password, I know something is wrong. I made this policy decision because of the endless attempts to log into my machine from all over the world. It was either something like a yubikey, or only using public key authentication. I went for the public key authentication.
Re: (Score:3, Funny)
So you don't password-protect your private key?
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
I use denyhosts - fail2ban is good too, the basic idea is the same.
I use SSHGuard here. I'd rather block all IP traffic from them entirely.
Re: (Score:2)
Denyhosts can be set to block all, not just ssh, if wanted. I decided not to because some might have ISP NAT and that would block many people from my webserver... Maybe unlikely though, and leaving them an option to find an exploit in apache or wordpress might not be that good idea...
Sorry it took a while to get back to you.
My point is I would rather have all traffic from an offending IP address be DROPped at the kernel by an iptables rule, than have the machine continue to receive packets only to have connections rejected by TCP Wrappers and the hosts.deny file.
Of course with TCP Wrappers you do have more fine-grained control. You can, as in your example, decide to block one service and not another. In my own use-case there is no scenario where I would want to deny SSH access t
Which Distro? (Score:2)
I may have missed it but was this a binary package for a specific distro (on their own servers) or a binary on an untrusted server not attached to any specific distro?
Re:Iceland? How hard could it be? (Score:5, Insightful)
In all likelihood the server is just another compromised machine.
Re:Iceland? How hard could it be? (Score:4, Interesting)
Yup. There are actually several reasonably priced hosting services in Iceland. I was shopping for international homes for some of my data, until I thought about it and none of my data is worth spending money to hide. :) Iceland was pretty high on the list though.
Re: (Score:3)
Re: (Score:3)
Maybe it wasn't a "who" but a "what" [www.iiim.is].
Re: (Score:2)
I don't think so. I suspect an extraterrestrial ship parked in a geothermal vent somewhere. They're out of sight, they get free energy, and they're close enough to a server to hack into it with their advanced computers. They've given up on those abductions and anal probes, now they're just hacking into the network to discover whatever might be on our minds.
Or, maybe it's occurred to them that we don't keep our brains next to our rectum?
Re: (Score:2)
That's funny, but for those that don't know: XOR is a well-known, poor man's encryption/obfuscation trick.
Re: (Score:2)
They have thousand of eyes looking at the source and working on ways to improve things. What are the Window's folks doing?
Re: (Score:2)
Knowing someone who is infected is the condition (Score:2)
Re: (Score:2)
It would be nice if ssh could enforce this and refuse to connect if you try to break the policy.