Peter N. M. Hansteen writes "In real life, zombies feed off both weak minds and the weak passwords they choose. When the distributed brute-force attempts stopped abruptly after a couple of months of futile pounding on ssh servers, most of us thought they had seen sense and given up. Now, it seems that they have not; they are back. 'This can only mean that there were enough successful attempts at guessing people's weak passwords in the last round that our unknown perpetrators found it worthwhile to start another round. For all I know they may have been at it all along, probing other parts of the Internet ...' The article has some analysis and links to fresh log data."
None of my systems allow passwords via ssh - and I run log-guardian.pl to "3 strikes - you're out" the idiots who do the brute-forces by putting them into iptables
Anyone with passwords turned on is not secure IMHO
IMHO if the passwords are strong enough there is nothing to worry about, unless you get pissed off with flooded log files and the waste of bandwidth.
None of my systems allow passwords via ssh
Exactly, using public key authentication and disabling PAM/Password authentication solves the problem.
and I run log-guardian.pl to "3 strikes - you're out" the idiots who do the brute-forces by putting them into iptables
sshguard is nice too, they will be firewalled in no time. (watch out for DoSs though)
However, it is not just ssh. Http and servers also suffer a lot from automated breakin attempts. Anyway.. I'm glad services running on port 22 are not in the same security leve
"IMHO if the passwords are strong enough there is nothing to worry about"
The problem is that most users are not capable of choosing a strong password. Even when you try to enforce policies about minimum password strength, users will manage to choose weak passwords; aside from the world's most common password (password1), there are plenty of people who use their own username as a password -- and requiring non-alphanumeric symbols won't stop them: jane123 will just becomes j@ne123. Minimum password leng
The problem is that noise in the logs makes it hard to find real problems. That and the risk of a bug in openssh or openssl, though the current environment is a pretty good test for the software.
I disagree. Should a new bug arise in openssh, I sure feel a lot better knowing that while I do enforce key-only authentication, I also restrict access to specific IP addresses. It's pretty hard to crack a service that you can't reach on the network due to packet filtering.
I don't allow password-based logins either (SSH keys only), allow SSH only from specific IP addresses, and I use fail2ban [fail2ban.org] across all services that involve any kind of authentication (mail, ftp, http auth, etc). I've got it set to "two strikes and you're out"; every day I still get hundreds (some days thousands) of IPs banned in the logs. It's pretty sad.
From a quick look at fail2ban it looks like one of it's features is that the blocking only lasts until the next log rotation. Considering that attacks are temporary and automatic firewall rule generation could become a script kiddies playpen this is actually a good thing. If they work out you have this and spoof a few adresses as a denial of service attack the system will recover over time without needing someone to go through all the firewall rules. I'm still a bit nervous about allowing malicious third p
I only allow public key connections, and am only listening on port 2022 (I have no issues telling the world that). My auth.log is completely empty of password attempts. Am I missing something simple-stupid or is the bot net only going after port 22?
I would suspect it's going after port 22 only. If your smart enough to move the port from 22, your probably smart enough to use key pairs and then what is the point of trying to brute force you?
Focusing on default 22 is a good strategy because you'll find those who have completely defaulted settings, weak passwords, etc.
Likewise - I use fail2ban with iptables to drop any packets from someone who fails auth about 5 times in a few minutes. I've toyed with the idea of adding them to a global blacklist for all servers in all locations, but in reality this solution works just fine.
Nice in the short term but giving people an easy way to add rules to your firewall may create hassles later once miscreants know that is what you are doing. Some people have scripts that implement temporary blocking so it doesn't hurt much on the day that some script kiddie decides to have fun with them by forging attacks from different addresses.
passwords are still the most portable lightweigth authentication method out there, that's why. I had a system that got caught by this when a user with shell access set a weakish password. the user was sandboxed with only a limited whitelist of applications they could execute, so no harm done, but i did log all of the bot's attempts and the IRC channel it connected to along with passwords to other bots. it was very interesting to take the attack a part, at it's core it runs a simple dictionary attack and onc
The odds of them getting into a system like this must be quite low, but I guess they're after the low-hanging fruit. Running your services on a high port rather than the default reduces this, as does disabling password login and using 2-factor authentication. Quite easy to do, and very, very secure.
We did remapped SSH to a higher unused port and then took it a step further blocking access to that port on the hardware firewall from every IP address except for the office. If I need to connect to the server, I first have to connect to an OpenBSD box in the office.
We have 3 - 4 PCI scans a day (seems like every payment gateway we support for our clients scans the server daily. None of them even see SSH.
A fair point. I've set up on our production servers two lists for ipset [netfilter.org], one each for China and Korea. Bullshit accesses to SSH and HTTP dropped way way off once I did that.
With 719 unique CIDR blocks for China and 430 for Korea, we get a lot less garbage traffic to our servers. Worth the hour it took to set up, too.
Use SSH keys in addition to passwords. Disable ssh root logins. Use the AllowUsers command in sshd_config to restrict what accounts can log in with ssh. Edit/etc/hosts.deny and add IP ranges [iana.org] for where you are unlikely to login from. Use iptables rules to block people [itwire.com] who are hammering your ssh server from the same address. Use tools like Fail2ban [fail2ban.org] and DenyHosts [sourceforge.net] to block other abusers and share abuser information with other victims.
It's security theater. There are good reason for allowing (private key only) root login, allong with autorized_keys command= directives. Furthermore password+ssh keys is rather pointless.
mostly good advice. you might consider using ssh keys instead of passwords, depending on your environment. the only thing i'd outright disagree with is pre-denying IP ranges based on a guess of where you're likely to log in from. i've had to leave the country on business unexpectedly on very short notice; it'd suck to have been locked out when i landed.
Roll out SPA / Port knocking, their IP shouldn't be touching your sensitive ports without a rule, table, or chain specifically allowing access. FORGET THE PASSWORD!
Use a script like denyhosts, and I'm sure there are a ton of others out there that are just as good if not better. Unless your password is weak enough to be guessed in five attempts and the attacker isn't already in the denyhosts list, you shouldn't have to worry about too much. And, most importantly, just peruse your auth logs every now and then, it's not really that big of a chore.
For those already familiar with Peter Hansteen's website [home.nuug.no], I'll offer a Thumbs Up recommendation for his Book of PF [amazon.com].
There's already been several stories on Slashdot either submitted by or about him, and I don't recall any mention of his book. I'd say his efforts if not his humility deserve some kind of reward, and the reduced sale price of $19.77 is a bargain.
I've used a script to block servers that failed a certain number of attempts along with AllowUsers. That worked well for a couple of years, but was annoying in that you could see the attempts being made and knowing that if you made a config error you could be vulnerable. It seems to me that even after I got several hundred systems in my block list it wasn't making a difference since the pool of zombies was so large.
Now I just use key only access and AuthUsers and feel a lot more secure. I'm thinking I may add a white list of IP addresses as well. That would really lock things down pretty well.
This has been going on for years. Really. I've been seeing this crap in my logs since we started running an Internet-facing SSH host nearly ten years ago. It's always the same password based login attempts with the same dictionary/script used in the attacks. This is probably just some training exercise for Chinese hackers at some state-run school to see who can break into the running-dog Yankee Imperialist's computers the fastest.
Just checked my auth logs and I'm seeing hundreds of various IPs, some of which are connecting up to 20 times. Definitely a new twist. I'll have to do some poking around to see what kind of machines are doing the probing. ( Is it compromised windows boxes?)
Be proactive on port 22 as well. At the advice of another comment I saw on/. a year or so ago I'm running a honeypot, with three static ports (one of them 22) and 4 roving ports. Establishing a TCP connection to any of them causes your IP to be instantly added to an iptables blacklist. It's worked pretty well; I've got about 2-3 unique addresses trying per day, and about 294 have been blocked since mid-December 2008. It takes care of both port scanners and bots deliberately connecting in order to try and g
While it's true there are a variety of techniques that can increase security, I've found simply moving to a new high-numbered port eliminates all random login attempts. Yes, security through obscurity is all it's cracked up to be, but for now, I've eliminated the problem with a pretty quick fix.
There sure are a lot of people who didn't bother to read the article. The point of these attacks are that it's a coordinated botnet attack. Meaning if you block any single IP, or even a large subnet, you've cost the attacker nothing. Fail2ban, denyhosts, all of these won't even slow these attacks down.
Once again, we have a built in linux goody which helps us out; -A INPUT -p tcp -m tcp --dport 22 --tcp-flags SYN,RST,ACK SYN -m recent --set --name sshattack --rsource -A INPUT -p tcp -m tcp --dport 22 --tcp-flags SYN,RST,ACK SYN -m recent --rcheck --seconds 300 --hitcount 3 --name sshattack --rsource -j LOG --log-prefix "SSH Drop: " -A INPUT -p tcp -m tcp --dport 22 --tcp-flags SYN,RST,ACK SYN -m recent --rcheck --seconds 300 --hitcount 3 --name sshattack --rsource -j REJECT --reject-with tcp-reset -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
The above allows three connections in a 5 minute period to port 22. After that it rejects any further connection attempts until the 5 minute timer is up.
Which means this won't work, since what I see in the logs (at least in the last go-round with this stuff) is one attempt per remote IP, yet a coordinated distributed dictionary attack. What we need is a ruleset that properly spots this attack pattern, then locks down port 22 entirely except for defined, fixed-IP administrative exceptions, just if there is any pattern of failed attempts over a longer period.
In a slightly-different recent case, I saw someone get ahold of an ftp login. The uses of that login were one-off per remote IP. The IPs were in Ukraine, Russia, China, Texas... with hours between successive attacks. From what was attempted on the logins, it was obviously a generic attack, not well-directed at the particular server. Quite likely an infected client system divulged the login to the botnet that infected it, and the botnet then focused the distributed attack. As with ftp, so with ssh. They don't want you to even notice it in the logs. Each IP will be used for one transaction attempt with you, spaced hours apart.
We need tools to spot these slow-moving distributed attacks. The single bad login attempt needs to be correlated with others from completely different IPs, over a span of days, not just minutes.
I was having similar brute force attacks. I've made some alterations to protect my server from brute force SSH attempts.
1) Moved SSH to another random port 2) Bound the SSHD to an IP address that is not used for Web/Mail/FTP, etc.. So the IP should generally see less traffic 3) Disable Password Authentication, Users who are given SSH access must use a password protected key file 4) Disabled Root SSH Login 5) Setup the system that 3 failed logins add the entire IP Subnet(X.X.X.0-X.X.X.255) for 15 minutes, 5 failed attempts 1 week, anything else is a never ending ban. (iptables and hosts.deny, just in case)
Um. You realize, of course, that remote desktop is a lot less secure than ssh, right?
It doesn't matter if people are trying to pick the lock on your door. What matters is whether they can pick the lock. Use RSA-based authentication, and no amount of brute force is going to improve the odds of their breaking in to the point where it's worth bothering.
Remote desktop, on the other hand, is completely brute-forceable. If you're not seeing brute force attacks, it's because nobody's bothering, not because
Um. You realize, of course, that remote desktop is a lot less secure than ssh, right?
Remote Desktop uses TLS and X.509 certificates, so it's not exactly trivial to crack. It's easily as secure as SSH using password-based authentication. It's definitely *more* secure if your users never bother to actually check unknown server keys.
Now, compared to SSH using only key-based authentication, Remote Desktop isn't as secure. Unless you use smart cards for authentication with Remote Desktop, which are probably more
why are passwords even allowed? (Score:5, Informative)
Anyone with passwords turned on is not secure IMHO
Re: (Score:3, Informative)
IMHO if the passwords are strong enough there is nothing to worry about, unless you get pissed off with flooded log files and the waste of bandwidth.
None of my systems allow passwords via ssh
Exactly, using public key authentication and disabling PAM/Password authentication solves the problem.
and I run log-guardian.pl to "3 strikes - you're out" the idiots who do the brute-forces by putting them into iptables
sshguard is nice too, they will be firewalled in no time. (watch out for DoSs though)
However, it is not just ssh. Http and servers also suffer a lot from automated breakin attempts.
Anyway.. I'm glad services running on port 22 are not in the same security leve
Passwords are a terrible idea (Score:3, Informative)
The problem is that most users are not capable of choosing a strong password. Even when you try to enforce policies about minimum password strength, users will manage to choose weak passwords; aside from the world's most common password (password1), there are plenty of people who use their own username as a password -- and requiring non-alphanumeric symbols won't stop them: jane123 will just becomes j@ne123. Minimum password leng
Re: (Score:3, Interesting)
If you only allow public key authentication then there's really no need to bother blocking them: you'll never block the entire botnet.
Just let them try - they'll never guess the right private key.
Re: (Score:2)
If there's a bug on in openssh (Score:2)
... then there's no need to bruteforce it, and therefore blocking a botnet doing that is futile.
Re: (Score:2)
Re: (Score:3, Insightful)
grep -v for the win!
Re: (Score:3, Insightful)
Re: (Score:2, Insightful)
Re: (Score:3)
Re: (Score:3, Funny)
Re: (Score:3, Informative)
Re: (Score:3, Interesting)
I'm still a bit nervous about allowing malicious third p
Re: (Score:2)
I only allow public key connections, and am only listening on port 2022 (I have no issues telling the world that).
My auth.log is completely empty of password attempts. Am I missing something simple-stupid or is the bot net only going after port 22?
Re: (Score:2, Insightful)
Re: (Score:2)
Likewise - I use fail2ban with iptables to drop any packets from someone who fails auth about 5 times in a few minutes. I've toyed with the idea of adding them to a global blacklist for all servers in all locations, but in reality this solution works just fine.
Re: (Score:3, Insightful)
Re: (Score:3, Informative)
Poor Odds (Score:5, Informative)
How-to (Score:5, Informative)
Parent
Re: (Score:2)
We did remapped SSH to a higher unused port and then took it a step further blocking access to that port on the hardware firewall from every IP address except for the office. If I need to connect to the server, I first have to connect to an OpenBSD box in the office.
We have 3 - 4 PCI scans a day (seems like every payment gateway we support for our clients scans the server daily. None of them even see SSH.
Not seeing it yet (Score:4, Funny)
Re: (Score:2)
A fair point. I've set up on our production servers two lists for ipset [netfilter.org], one each for China and Korea. Bullshit accesses to SSH and HTTP dropped way way off once I did that.
With 719 unique CIDR blocks for China and 430 for Korea, we get a lot less garbage traffic to our servers. Worth the hour it took to set up, too.
Oh... (Score:5, Funny)
[puts shotgun down]
Re:Oh... (Score:4, Informative)
[puts shotgun down]
Don't be so hasty. There are real people herding the zombies.
Parent
Re: (Score:2)
Protect yourself (Score:5, Informative)
Use SSH keys in addition to passwords. Disable ssh root logins. Use the AllowUsers command in sshd_config to restrict what accounts can log in with ssh. Edit /etc/hosts.deny and add IP ranges [iana.org] for where you are unlikely to login from. Use iptables rules to block people [itwire.com] who are hammering your ssh server from the same address. Use tools like Fail2ban [fail2ban.org] and DenyHosts [sourceforge.net] to block other abusers and share abuser information with other victims.
disabling root login is idiotic (Score:2)
It's security theater.
There are good reason for allowing (private key only) root login, allong with autorized_keys command= directives.
Furthermore password+ssh keys is rather pointless.
Re: (Score:3, Insightful)
Re: (Score:3, Informative)
SPA / PORT KNOCKING - Bye Bye Brute (Score:4, Insightful)
Roll out SPA / Port knocking, their IP shouldn't be touching your sensitive ports without a rule, table, or chain specifically allowing access. FORGET THE PASSWORD!
Another solution (Score:3, Insightful)
Not So Shameless Plug (Score:5, Informative)
For those already familiar with Peter Hansteen's website [home.nuug.no], I'll offer a Thumbs Up recommendation for his Book of PF [amazon.com].
There's already been several stories on Slashdot either submitted by or about him, and I don't recall any mention of his book. I'd say his efforts if not his humility deserve some kind of reward, and the reduced sale price of $19.77 is a bargain.
Denial (Score:3)
I've used a script to block servers that failed a certain number of attempts along with AllowUsers. That worked well for a couple of years, but was annoying in that you could see the attempts being made and knowing that if you made a config error you could be vulnerable. It seems to me that even after I got several hundred systems in my block list it wasn't making a difference since the pool of zombies was so large.
Now I just use key only access and AuthUsers and feel a lot more secure. I'm thinking I may add a white list of IP addresses as well. That would really lock things down pretty well.
In all seriousness (Score:5, Informative)
I'm safe... (Score:5, Funny)
Re: (Score:3, Funny)
pinkie? C'mon, gimme a hint.
Re:I'm safe... (Score:5, Funny)
Parent
Re: (Score:3, Funny)
reference [bash.org]
Stop collaborate and listen. (Score:2)
Just checked my auth logs and I'm seeing hundreds of various IPs, some of which are connecting up to 20 times. Definitely a new twist. I'll have to do some poking around to see what kind of machines are doing the probing. ( Is it compromised windows boxes?)
Don't just run it on a higher port. (Score:2, Interesting)
Be proactive on port 22 as well. At the advice of another comment I saw on /. a year or so ago I'm running a honeypot, with three static ports (one of them 22) and 4 roving ports. Establishing a TCP connection to any of them causes your IP to be instantly added to an iptables blacklist. It's worked pretty well; I've got about 2-3 unique addresses trying per day, and about 294 have been blocked since mid-December 2008. It takes care of both port scanners and bots deliberately connecting in order to try and g
quickest fix (Score:2, Interesting)
Goodness. (Score:5, Informative)
There sure are a lot of people who didn't bother to read the article.
The point of these attacks are that it's a coordinated botnet attack. Meaning if you block any single IP, or even a large subnet, you've cost the attacker nothing. Fail2ban, denyhosts, all of these won't even slow these attacks down.
iptables goodness (Score:5, Informative)
Once again, we have a built in linux goody which helps us out;
-A INPUT -p tcp -m tcp --dport 22 --tcp-flags SYN,RST,ACK SYN -m recent --set --name sshattack --rsource
-A INPUT -p tcp -m tcp --dport 22 --tcp-flags SYN,RST,ACK SYN -m recent --rcheck --seconds 300 --hitcount 3 --name sshattack --rsource -j LOG --log-prefix "SSH Drop: "
-A INPUT -p tcp -m tcp --dport 22 --tcp-flags SYN,RST,ACK SYN -m recent --rcheck --seconds 300 --hitcount 3 --name sshattack --rsource -j REJECT --reject-with tcp-reset
-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
The above allows three connections in a 5 minute period to port 22. After that it rejects any further connection attempts until the 5 minute timer is up.
Re:iptables goodness (Score:4, Informative)
Which means this won't work, since what I see in the logs (at least in the last go-round with this stuff) is one attempt per remote IP, yet a coordinated distributed dictionary attack. What we need is a ruleset that properly spots this attack pattern, then locks down port 22 entirely except for defined, fixed-IP administrative exceptions, just if there is any pattern of failed attempts over a longer period.
In a slightly-different recent case, I saw someone get ahold of an ftp login. The uses of that login were one-off per remote IP. The IPs were in Ukraine, Russia, China, Texas ... with hours between successive attacks. From what was attempted on the logins, it was obviously a generic attack, not well-directed at the particular server. Quite likely an infected client system divulged the login to the botnet that infected it, and the botnet then focused the distributed attack. As with ftp, so with ssh. They don't want you to even notice it in the logs. Each IP will be used for one transaction attempt with you, spaced hours apart.
We need tools to spot these slow-moving distributed attacks. The single bad login attempt needs to be correlated with others from completely different IPs, over a span of days, not just minutes.
Parent
Re: (Score:3, Interesting)
then locks down port 22 entirely except for defined, fixed-IP administrative exceptions
This is my default configuration, with a port knock pattern for emergency access.
My Solution (Score:4, Informative)
I was having similar brute force attacks.
I've made some alterations to protect my server from brute force SSH attempts.
1) Moved SSH to another random port
2) Bound the SSHD to an IP address that is not used for Web/Mail/FTP, etc.. So the IP should generally see less traffic
3) Disable Password Authentication, Users who are given SSH access must use a password protected key file
4) Disabled Root SSH Login
5) Setup the system that 3 failed logins add the entire IP Subnet(X.X.X.0-X.X.X.255) for 15 minutes, 5 failed attempts 1 week, anything else is a never ending ban. (iptables and hosts.deny, just in case)
Re: (Score:3, Insightful)
Um. You realize, of course, that remote desktop is a lot less secure than ssh, right?
It doesn't matter if people are trying to pick the lock on your door. What matters is whether they can pick the lock. Use RSA-based authentication, and no amount of brute force is going to improve the odds of their breaking in to the point where it's worth bothering.
Remote desktop, on the other hand, is completely brute-forceable. If you're not seeing brute force attacks, it's because nobody's bothering, not because
Re: (Score:3, Informative)
Remote Desktop uses TLS and X.509 certificates, so it's not exactly trivial to crack. It's easily as secure as SSH using password-based authentication. It's definitely *more* secure if your users never bother to actually check unknown server keys.
Now, compared to SSH using only key-based authentication, Remote Desktop isn't as secure. Unless you use smart cards for authentication with Remote Desktop, which are probably more