OpenSSH Package Trojaned 574
cperciva writes "The original story is here.
And more details are available from the guy's weblog here." Here's a mirror of that email message. Another reader writes, "Not really a trojan because all it does is make a connection to 203.62.158.32:6667." Still another writes "The tarball of the portable OpenSSH on ftp.openbsd.org is trojaned. The backdoor is only used during build - generated binaries are fine." There isn't much authoritative information available, but this appears legitimate - please be careful if you're updating any of your machines with code from ftp.openbsd.org, and we'll update this story with more links as information is available. Update: 08/01 19:13 GMT by M : OpenSSH now has an advisory.
MD5 sums (Score:5, Informative)
Idle curiosity (Score:3, Informative)
It wasn't orgianally like that. (Score:5, Informative)
So if the file was modified it happen later.
Re:How many people do check the MD5 checksum? (Score:5, Informative)
The same is true of other systems like the Red Hat Network.
Re:Since its only a build issue... (Score:3, Informative)
@ $(CC) bf-test.c -o bf-test;
The backdoor code will still be there, but it won't be built. Or, alternatively, just wait for it to be fixed. Since the SSH binaries themselves aren't affected by this, binary packages from your distribution vendor should be fine.
Trojan executes code read via /bin/sh (Score:3, Informative)
But it reads from the connection and executes the read code via /bin/sh. You call this not a trojan?
Re:203.62.158.32 (Score:2, Informative)
The C code is not that smart. It tries once per hour to connect to port 6667 on the machine 203.62.158.32 which is web.snsonline.net and waits for commands from the person or persons who 0wn3d the machine. Does it get an M, it sleeps for another hour. Does it get an A, it will abort. Does it get M, it will spawn a shell
I guess this answers your question
Re:hmmm.... (Score:4, Informative)
Further reading [freebsd.org]
Re:It wasn't orgianally like that. (Score:5, Informative)
The datestamp on the modified file was Jul 31, so it does look like it's been changed recently.
OpenSSH (Score:5, Informative)
M respawned the process.
A killed the trojan.
D launched
With the D command, as given _to_ the trojan, the daemon on 203:62.158.32:6667 was given control of the trojaned users system shell. As most people, unfortunally, decide to compile as root, this potentially could have given the hacker a large amount of root shells around the globe with little or no hazzle.
Funny, this is. Expect more trojans that look like this, but in a better disguise.
-- Hans.
Healthy versions still available..? (Score:2, Informative)
juan:~> md5sum openssh-3.4p1.tar.gz
459c1d0262e939d6432f193c7a4
ftp://ftp.fr.openbsd.org/pub/OpenBSD/OpenSSH/po
My analysis (Score:4, Informative)
So the backdoor is in the Makefile, not the OpenSSH software itself.
One thing to mention is that IMHO this is not a fault of OpenBSD. As anyone can read in their FAQ [openbsd.org] www.openbsd.org (and ftp.openbsd.org) is run on Solaris.
Slashdotted (copy of the weblog) (Score:5, Informative)
01 August 2002 - 19:10:23 - OpenSSH 3.4p1 package trojaned
And all I was thinking was "Oh! I should upgrade ssh on these two machines before there are problems...". The beauty of FreeBSD is that it goes like this:
[~] edwin@k7>cd
[/usr/ports
[/usr/ports/security/openssh-porta
Easy euh? It went well, except for the second step:
===> Extracting for openssh-portable-3.4p1_7
>> Checksum mismatch for openssh-3.4p1.tar.gz.
Make sure the Makefile and distinfo file (/usr/ports/security/openssh-portable/distinfo)
check, type "make NO_CHECKSUM=yes [other args]".
*** Error code 1
Euh... I didn't remember seeing a change in the FreeBSD ports regarding this. And I didn't see an announcement for it from the people from OpenSSH... Oh well, it happens. I downloaded the new openssh-tarball:
-r--r--r-- 1 12187 mirror 840574 Jul 31 16:47 openssh-3.4p1.tar.gz
-r--r--r-- 1 12187 mirror 232 Jun 26 08:20 openssh-3.4p1.tar.gz.sig
That's weird, they've rerolled the tarball without updating the signature file. I asked a couple of people on irc (#sage-au) if they have had troubles with compiling openssh the last days. Yups, ^Sarge^@bofh.snsonline.net also had it, he had a checksum mismatch.
Curious as I was, I extracted the old and new tarball and this were the differences:
[~/test] edwin@k7>diff -r -u openssh-3.4p1-old openssh-3.4p1
diff -r -u openssh-3.4p1-old/openbsd-compat/Makefile.in openssh-3.4p1/openbsd-compat/Makefile.in
--- openssh-3.4p1-old/openbsd-compat/Makefile.in Wed Feb 20 07:27:57 2002
+++ openssh-3.4p1/openbsd-compat/Makefile.in Thu Feb 1 08:52:03 2001
@@ -26,6 +26,7 @@
$(CC) $(CFLAGS) $(CPPFLAGS) -c $bf-test.out; sh
$(COMPAT):
$(OPENBSD):
Only in openssh-3.4p1/openbsd-compat: bf-test.c
At this moment I asked a couple of people on irc (#sage-au) if they have had troubles with compiling openssh the last days. Yups, ^Sarge^@bofh.snsonline.net also had it, also a checksum mismatch. Time to go deeper into it...
bf-test.c is a weird file. It talks about HP-UX PL.2 systems, it talks about _CRAY notes, it talkes about none-T3E machines, it talks about _ILP64__ and it does an epcdic2ascii() call. I'm not very skilled in computers (well, I am
bf-test itself is pretty harmless, it only prints things to the screen (remember the change in the makefile? execute, redirect the output and execute the output). The shell script it prints creates a C program and tries to compile it. If it doesn't succeed at first, it tries to link other libraries (everybody who has ever ported a Solaris knows that you have to explicitely link to libresolv et al). So it's cross-platform
The C code is not that smart. It tries once per hour to connect to port 6667 on the machine 203.62.158.32 which is web.snsonline.net and waits for commands from the person or persons who 0wn3d the machine. Does it get an M, it sleeps for another hour. Does it get an A, it will abort. Does it get an M, it will spawn a shell. Some people will build it "normal" privileges and install it as root: they will get a shell with "normal" privileges. Other people will build it with "root" privileges and the shell will have "root" privileges.
While analyzing the code on #sage-au and mentioning the hostname, ^Sarge^ looked strangely at me (well, it's IRC so you never know but that's what I would do): "That is my machine.". The good news is that I didn't have to worry about finding out who manages the machine!
The next step is to inform somebody who manages the openssh-packages: The OpenBSD team. Up to right now, I have had no experience with the OpenBSD team (if you check my website you'll see that I'm more a FreeBSD guy
*** MavEtJu has joined #openbsd
Euh... anybody from the openssh-team here?
I have some news for you...
What's up?
I have contact! Marius asked me the standard questions (how did you find out, how can I see it, when did you find out) and after some investigation he said "I think I'd better call (and now I have forgotten the name)". Coolies! I think I found a right person to talk to! It looks like things are going to roll now, I can take my hands of it.
The last things I did were writing some emails to a couple of mailinglists and guide ^Sarge^ to #openbsd. For the rest I wasn't of very much use anymore, so I just kept monitoring #openbsd. And the logfile of my website, which went ballistic.
Aftermatch
* The portable version wasn't the only which was trojaned, the normal version was also.
* It seems it took only six hours before somebody was alert enough to see that there was something wrong, all thanks to the checking of the MD5-checksum [insert a sweet 'aaaaaahhh' here]
* OpenSSH itself wasn't trojaned, the tarball was. There is nothing wrong with OpenSSH itself (this time
* The building of a port (under FreeBSD at least) is done as root with all its privileges. This is a wrong approach. For a time I tried, as an experiment, to build ports as user "port". This worked fine except for the "make clean" part, in which I couldn't remove the files created during the "make install" phase and the files which were made during the building of the RUN_DEPENDS ports.
Re:Idle curiosity (Score:4, Informative)
Re:How many people do check the MD5 checksum? (Score:2, Informative)
I don't know what OS you are talking about. Debian apt automatically [debian.org] checks MD5sums, Red Hat network uses cryptographic certificates to verify package integrity, even Windows has a package verification system.
Re:Checksum...? (Score:3, Informative)
Re:Gentoo... (Score:3, Informative)
Re:MD5 sums (Score:3, Informative)
Re:How many people do check the MD5 checksum? (Score:1, Informative)
Re:How many people do check the MD5 checksum? (Score:5, Informative)
Re:New catch phrase (Score:2, Informative)
Next it will be "one remote hole and one 'harmless trojan' in the default install, in really very close to 6 years!"
No it wont, because the trojan was only in the source to the portable version. OpenBSD ships with a binary which is from the unpatched source.
Re:203.62.158.32 (Score:1, Informative)
Re:Since its only a build issue... (Score:3, Informative)
This was just one type of trojan. Some others could go dormant for several hours before contacting the world outside. Simply "building the binaries with plug off the wall" is not a solution. It's a knee-jerk reaction and still at fault. The correct way is to check the package against a MD5sum or (preferably) GPG signature - and if possible, these should be at a different machine on a different network from the tarballs.
On the other hand, just looking at the trojan source quickly, it looks very much like a slightly evolved version of those found in irssi, BitchX, dsniff and fragroute configure scripts. This has already been noted by some other individual here as well. See his post for links.
Re:203.62.158.32 (Score:1, Informative)
Building and rebooting is not enough. Next time, back up the entire machine so you can examine it for traces of those who compromised it, and then perform a clean reinstall from verified media.
Re:203.62.158.32 (Score:3, Informative)
But in this circumstance, I don't believe it is the case. The trojan connects to port 6667, which is usually ircd. Outgoing connections to irc servers are not exactly uncommon in those boxes that run any kind of flavour of *nix. Hence, it's not a connection that really attracts attention by itself. It looks like a connection to a stand-alone ircd in netstat reading. Also, because irc is so common service to use, the firewall setups are likely to allow this through.
The other end of that connection, however, was more than likely running something totally unrelated to irc. After all, the connection itself is somewhat like a backward rsh. (I believe it actually bears the name "bindshell"...) This was a very basic case of trojan: install a backdoor that calls home and allows to execute commands remotely.
Re:Not the fault of OpenBSD? (Score:4, Informative)
Re:203.62.158.32 (Score:2, Informative)
Cheers,
Brian
Re:Idle curiosity (Score:2, Informative)
I checked out this FAQ:/ securing-debian-howto/ch7.en.html
http://www.linuxsecurity.com/docs/harden-doc/html
In current Debian stable (Woody) there is a package called debsigs, containing the gpg signatures of the package maintainers, and another called debsig-verify, which when installed will cause all package installations to be conditional on checking against the keys in debsigs.
I ran an install on debsigs and debsig-verify in aptitude, but having installed debsig-verify _first_ apt refused to install debsigs on account of the fact that it could not verify the signature on the package. Silly, but kinda reassuring. Easily fixed by removing debsig-verify and explicitly reinstalling in debsigs _first_.
I hope they'll make debsigs a required main package in future, so that installation of debsig-verify will be completely painless.
blocking network traffic (Score:3, Informative)
In particular, if the machine in question is a server (usually the reason you have SSH on a box), you should make every possible effort to remove outgoing traffic. There's usually no reason for a server to create outgoing connections to the internet, and if it needs to connect to any specific local resources (e.g. a database machine), limit the iptables/routers appropriately.
Re:203.62.158.32 (Score:3, Informative)
Take a look at www.linuxfromscratch.org to see how this works for an entire system.
Re:Trojan (Score:2, Informative)
Bad? Yes. It could be worse, though.
Rebuilding from source, and paranoia (Score:3, Informative)
http://www.acm.org/classics/sep95/
Step 2: Decide for yourself if you're ready for the tinfoil-helmet brigade.
Step 3: Type 'make world' if you dare.
Re:203.62.158.32 (Score:3, Informative)
Recompiling the compiler doesn't do anything to rid you of trojan code/back doors. If you need to ask why, take a look at "Reflections on Trusting Trust" by Ken Thompson [acm.org], and the description of a back door [tuxedo.org] in the jargon file.
The short story is, that the compiler decides what to output. So you have to trust your compiler.
The reason for the recompiling of gcc is something else entirely: It is to allow the source code of gcc to rely on gcc, and to allow an optimized compiler to be created.
For additional information consult a good book on compiler writing.
Re:Idle curiosity (Score:3, Informative)
This is not quite up to the standard I had come to expect from Debian.
My apologies go to anyone who was misled by my earlier statements.
Re:GnuPG a good idea (Score:4, Informative)
Real cryptographic signing, like that mentioned in the grandparent post, involves hashing the tarball using a strong hash, and then encrypting that hash using the private key of the signer. Then, any person can retrieve the public key of the signer and verify that the "signature" is legitimate by attempting to decrypt the hash (this is probably not strictly correct with the way things really work, but the concept is, AFAIK, correct). The point is that the only person who can "sign" the tarball is limited to only legitimate people (or organizations). So, if the signature for the tarball is valid, you can guarantee it was signed by someone you trust, and so you can trust the code.
Re:Bad news and Good news (Score:5, Informative)
And it looks like they're not "eating their own dog food," and eating Sun dog food instead
did you ever think there might a reason [openbsd.org] for that?
then you can't trust a web server to give you a web page with an unaltered MD5 sum. Surely this is common sense?
I am not sure, but this just might be the reason why systems like BSD ports and Gentoo portage store the MD5 sums in the ports trees, and don't in fact get them from websites.
The real solution is digital signatures (i.e. an MD5 sum encrypted with a private key).
WOW! what an original and fresh solution! you sir, are some sort of genious for coming up with this.
congratulations, you've managed to regurgitate several of the things that have been said, literally, hundreds of times today already. I think the Society for Prevention of Cruelty to Dead Horses might have a bone to pick with you.