The Slow Bruteforce Botnet(s) May Be Learning 327
badger.foo writes "We've seen stories about the slow bruteforcers — we've discussed it here — and based on the data, my colleague Egil Möller was the first to suggest that since we know the attempts are coordinated, it is not too far-fetched to assume that the controlling system measures the rates of success for each of the chosen targets and allocates resources accordingly. (The probes of my systems have slowed in the last month.) If Egil's assumption is right, we are seeing the bad guys adapting. And they're avoiding OpenBSD machines." For fans of raw data, here are all the log entries (3MB) that badger.foo has collected since noticing the slow bruteforce attacks.
Solution: Public Key Auth (Score:5, Interesting)
The obvious solution is to use public/private key authentication and disallow password logins.
This is much safer anyways, since your private key and your passphrase stays on your local machine always, so even if the server is compromised and the SSHd is bugged, no one will have immediate access to your login token.
AI (Score:5, Interesting)
It's my (admittedly probably crazy) idea that we WILL begin to see "emergent intelligence properties" out of some sophisticated system at some point in time, whether it be Google, an AGI lab, or a botnet. I shudder at the prospect of our first AI of power will have grown from one of these botnets.
NOTE: I'm not saying this will happen tomorrow, but extrapolating the current state of botnets relative to the current state of other systems leads me to believe, on a relative basis, systems may be complex relative to one another as they are today. If that is the case, well... that would be bad.
Speculative but interesting (Score:3, Interesting)
The conclusions are a bit too speculative, nonetheless the research is interesting. I am not sure if a few hundred hosts are enough to conclude that the "bad guys" are coordinating and sharing attack output. And as far as avoiding OpenBSD, come on..."OpenBSD is a bitch." Why is this a surprise?? :)
OpenBSD hosts make stupid targets... (Score:5, Interesting)
There is still reason to avoid them, though. Because OpenBSD is something of a niche system, you can make plausible inferences about the systems running it. Specifically, they most likely have admins who are interested in security and are watching activity fairly closely, and are more likely than average to do something about it. If you are doing something illegal, why attract such attention?
Botnet solution (Score:5, Interesting)
Bots were knocking on my door to the point I was worry about performance degradation. I know there are many ways to defeat these but here was my solution.
In hosts.deny /var/www/html/allow.txt
-----------------
sshd:ALL EXCEPT
-----------------
Create a simple cgi-script (password protected and accessed via secret random url) that writes your browser IP address to the allow.txt file and all those nasty botnets and go to hell.
I want to see a death bounty for these people (Score:5, Interesting)
These people are a tremendous illness upon the world. If it were legal, I would contribute to a bounty on the lives of the people responsible for this stuff. These people make me beyond sick. I have said it many times and sometimes I actually mean it -- if I knew of someone involved in this sort of business close by, I would appear on the news shortly thereafter. And I am pretty sure I am not alone in this sentiment.
How do the botnets know it's OpenBSD? (Score:4, Interesting)
How would the botnet know they are attacking an OpenBSD box (vs Linux or something else)?
Is there some sort of server signature involved (that I'm not aware of)
My (Linux) ssh server at home just responds with a password prompt. I don't see any easy way to determine the underlying system from that.
BTW. On my server at home I use Hashlimits to limit each IP to 1 attempt per minute (maximum). This has taken the attacks down from hundreds / thousands per day ( The most attacks I ever got was ~7,000 from one IP) to about 3 to 6. This is typically, 1 attempt each, they then get blocked, and then they go away.
Re:Solution: Public Key Auth (Score:5, Interesting)
Since changing my SSH ports to something really high (above 50000), I have had exactly *zero* failed password attempts in the last 14 months.
I know the plural of 'anecdote' is not 'data', but this is the case across *all* my servers.
Economics (Score:5, Interesting)
Don't forget about the economies surrounding botnets. There are two sides, those that profit from the botnets (the operators), and those that profit fighting the botnets (the fighters). Additionally, there are those that profit from providing botnet remedial "solutions" whilst not being in either of the primary (operator or fighter) categories. If botnets ceased to exist, there would be a *lot* more lost on the fighter and solution side than on the operator side. So... like SPAM, this raises the question of just who actually benefits the most from botnet existing.
Fail2ban? WTF? (Score:2, Interesting)
Posting as AC because the people running botnets can be nasty...I had most of their hosts banned two weeks ago and it got more interesting.
To the people who say: "Use fail2ban" --it won't work unless you jail the host on the first failed login forever. They'll be back once every six hours on my system.
After I had a week worth of logs, I added them to hosts.deny--and now things are getting interesting. I'm working on compiling the pattern now--but it looks like there's "micro wordlists" being thrown at it until they get picked up in fail2ban...two or three a day from new hosts.
Re:I want to see a death bounty for these people (Score:5, Interesting)
Nobody keeps you from putting a bounty on the head of a spammer and botnetter. You can't ask for them being killed, but you can without a problem issue a bounty on them, payable to whoever tracks down a botnetter and drags him to court.
Re:Solution: Public Key Auth (Score:5, Interesting)
I didn't change my ssh port to something that high, but I changed it to something above 1024, and the botnet attacks have stopped, so you can add my anecdote to yours...
Re:Solution: Public Key Auth (Score:5, Interesting)
Unfortunately, this is often too hard for your users.
What's really scary is that I'm starting to see really good passwords coming through (I modified the OpenSSH source to log the password sent for one of my jails.) I'm seeing passwords that have no particular rhyme or reason (in other words, they're either random or are generated through an obfuscated scheme.) I have to assume that they're passwords which were harvested in some way. It really makes me wonder where they're getting them.
Re:Economics (Score:5, Interesting)
As someone being in the latter group (to avoid confusion, the ones fighting them), yes, we make some money fighting that crap. Looking at the money being made on the other side, some are already wondering why we stay here.
We stay on this side because we (well, most of us) hate botnets. Most people I met at various conventions and meets are somewhere between zealous, fanatic or outright crazy, but generally see the money as some sort of pleasant side effect.
Believe me one thing: We know we cannot fight it, we know it's almost impossible to track them down and we know how it works. If we were in it for the money, we'd switch sides before you're done reinstalling your system. There's about ten times the money to be gained on the dark side.
Conservatively estimating, that is.
If spam and botnets ceased to exist overnight, we'd gladly return to more interesting and maybe also more profitable professions. Most of us are network experts. Some know more about the way Windows works on the "inside" than most people at MS. And if everything fails, we could actually maybe even create a copy protection system that is hard enough to break that nobody would willingly do it (after all, we spend a good deal of our time with disassembly). Do you really think that any of the (good) spam and botnet fighters would have a hard time finding a "honest" job that maybe even paid better than this?
I could enjoy having a life again, instead of this sorta permanent on-call duty. Again, no christmas for me, because yes, this is one of the hottest times of the year (many people at home, many new computers needing infections, so many new opportunities for botherders...). I would also prefer to create something, like some new software to make people happy or more productive, instead of poking at malware and trying to find a sensible way to detect it. It's not really good for your ego if your product is seen as the necessary evil that steals valuable computer time instead of something that people actually want to have.
Thanks for hearing out the rant. Now we're back to your scheduled program.
Re:Solution: Public Key Auth (Score:5, Interesting)
That means that you haven't been attacked by a portscanning bot yet.
I don't know that any exist yet, so you would be safe until they do. Really, wouldn't any port other than 22 that isn't used for anything else bots attack work?
Re:OpenBSD hosts make stupid targets... (Score:3, Interesting)
So, Mr Formal Education In Operating Systems, will OpenBSD refuse a valid username and password combination because the person logging in has a hidden evil deep in their hearts, unlike Windows which has blind faith in all valid passwords?
You're very confused. It's true that, if configured to accept username and password authentication, any system will treat a valid username and password as sufficient. That's why most professional administrators use public key authentication with good private key protection policies. But given an equal configuration of username and password, OpenBSD will be just as trusting as Windows.
Re:I want to see a death bounty for these people (Score:4, Interesting)
Their attacks will make the internet stronger by helping it evolve defenses it would not otherwise have.
Some steady pressure spurs evolution. So long as it does not kill the host we should smile and welcome the challenge.
"Correct" Remote Access Protocols (Score:3, Interesting)
I've looked at the TFA and the hard data and it seems like admins are the ones making the IT mistakes. With so many attempts for root and none of the other users personally identifiable, I can personally just set up a Bot to run tracert routines on failed attempts and report them for trying to access Root or Admin.
When it comes to multi-user sites however public key auth is standard, but your user ID and password have to match. What I don't understand is why everyone immediately resorts to AI development.
Clearly musing, he is. AI means "Self-adapting code". Self-adaption is too slow in real time and is only controlled by small control variables in games. Botnets have a heard. IT's the ADMIN's fault for being hearded, but they can have a techie d/c the power cord to save the rest of the world. Theres no real threat to secure folks because physical disconnection is trivial over a router (I just disable my IP assignment and I'm disconnected until I get another techie to do it physically) but more of a threat to people who can't control it. People controlled by the law, such as big-time Admins.
Sure, sure, the server won't crash when you're watching it, sure. But how boring will that be?
Here's the real issue: Remote Access
There has to be a way for the slow bots to get into root or admin or a remote access. I usually disable root or admin from working outside the internal loopback - 127.0.0.1 - standard Class A IP Address. I could technically configure a Bot to run Tracert (traceROOT) routines on all of those people (yes, windows user here) and have them reported to the federal government. It can't mess up my personal account, nor can it mess up DNS servers with sheer volume. It's small-scale.
so, the solution is proper remote access protocols. I remember NEVER activating remote access but at the same time using public-key authorized third party demo services to make minor changes remotely, including shutting the system down. I used logmein.com, free demo version, pathetically, but it's actually more secure as long as I have no idea how why I should do it myself. Once I used the shutdown signal it could not boot itself up unless someone would physically press the button. I have to call a physical person in the house to do that myself, so unless demons from hell can use an on/offswitch and my BIOS password without my permission, it ain't starting on it's own nor does it listen for a restart signal until I sign into windows for the first time (Windows XP here). My system has never been breached before, but it constantly deadlocks to save itself from burning the CPU out. It has a thermosensor and cutoff only in the power supply unit, however. Stupid laptops weren't designed for gaming even though thats how its advertised. How do I pull an all nighter at this rate? I'll just remove the sensor in my power supply and WHAM there goes my processor for not having heat sensors. Stupid dell power supply. Rocket fish will at least deadlock my system without damaging my hard disk.
Re:Solution: Public Key Auth (Score:5, Interesting)
Re:Solution: Public Key Auth (Score:3, Interesting)
look at all the attempts for "Root".
Well, that's why nobody got in. Every OS with a root account is also CaSe-SeNsItIvE!
This is not a game changing tactic. (Score:5, Interesting)
I do computer and network security for a university.
This distributed SSH password guessing is not a new tactic. We have seen and tracked this tactic off and on for over a year.
If this tactic was a game changer, we would have seen it ramp up before now. It would occur all the time. But it doesn't. It only seems to occur during holidays.
At it's heart, this tactic is not any more effective than non-distributed password guessing. Either way, the attacker has to enumerate the same number of guesses before finding a hit. If a machine is vulnerable, it will be successfully attacked by either approach to password guessing. If it is not vulnerable, neither approach will work.
Modern hacking is a economic activity. It must balance risk and reward. This attack doesn't offer any more reward than conventional password guessing. It's main feature is to try to change the risk side of the equation.
Conventional SSH password guessing is noisy. One machine will portscan for TCP/22. Then it rapidly guesses passwords against everything that responds. That one machine is usually lost to the attacker. Automated defense systems block it. Also, defenders report it to the owning ISP. The only way this works for the attacker is if he can harvest more that he loses.
The distributed guessing attack is also noisy, but in a different way. Currently, we see the attacker start by sacrificing 1 computer to do a TCP/22 portscan. At this point, he has already risked as much as a conventional password guessing attack. Then he feeds the results to a bunch of bots. Each bot then takes turns guessing passwords. Each bot guesses 1 password at a time. However, each bot guesses against multiple SSH servers at the same time.
This attack is inherently more risky that conventional password guessing. The attacker exposes many of his computers. If we can detect and respond, this attack is not as cost effective as conventional password guessing.
It is easy for my university to detect and respond to these attacks. We detect it in three different ways.
1) Each attacker has a distinctive network behavior pattern. We can automate detection by looking at aggregate Cisco netflow data.
2) It is trivial to pick off this attack using a SSH honeypot.
3) We use a network visualization tool to watch aggregate SSH activity. This password guessing is obvious on our visualization tool.
Once we have detected the attackers, we respond to them in the normal way. We block them. We inform our peer institutions and the authorities. We inform the owning ISP.
The main difference in this situation is that detection and response is easy if you have access to aggregate traffic or multiple SSH servers. It is difficult if you only manage 1 SSH server.
I don't expect this form of attack to last much longer. I am sure that everybody else is adapting. Once the defenders adapt, this tactic is too expensive to be used.
Miles
Re:I want to see a death bounty for these people (Score:1, Interesting)
I completely disagree. In my opinion if a problem is so bad you wish the opposition dead then you're going about solving it wrong. You should be confident in letting the bad guys guess as many times as they want. Passwords likely to fail to that attack? Fine. Then passwords are the problem. I'm not saying I have a better solution just now, I'm just saying that it's more likely that there's a better scheme than passwords than it is that death threats will help a security weakpoint.
Re:Solution: Public Key Auth (Score:4, Interesting)
You can still use the standard port, just install a simple defense system in iptables.
iptables -A INPUT -p tcp --dport ssh -m state --state NEW -m recent --update --seconds 99 -j DROP
iptables -A INPUT -p tcp --dport ssh -m state --state NEW -m recent --set
Now any particular IP address can only open a tcp connection to your ssh server once every 99 seconds, or longer if they keep trying during the blackout period^^
Maybe put some whitelist rules before that. Change it to 900 (fifteen minutes) if you don't log into your server that often from other addresses.
Re:Solution: Public Key Auth (Score:3, Interesting)
Comment removed (Score:5, Interesting)
Re:Solution: Public Key Auth (Score:3, Interesting)
I've had the opposite experience. I run a website for a small choir and we have a contact form on there. This is something I wrote myself, not some popular package, and it's very tightly tied down so that the worst which can happen is that an attacker can send more junk to me.
Over the last year I've had at least two repeated and persistent attacks against this script. They were random bits of text with a random URL (not working or registered) at the end. After playing cat and mouse changing field names and blocking certain phrases which kept reoccurring I only managed to stop it in the end when I completely blocked the ability to include URLs in any message (which I didn't really want to have to do). We are a very small site and none of the attacks ever worked - but someone spent a considerable amount of time trying to break our site.
The moral is that noone is safe and it's just the luck of the draw if someone decides to focus their attention on you.
Re:Solution: Public Key Auth (Score:3, Interesting)
I have noticed brute force attempts for years now. I have a simple script that adds hosts with a number of failed attempts to /etc/hosts.deny automatically. If a host logs in successfully, all its past "mistakes" are forgiven.
This has helped cut down on the invalid login attempts by >90%. This is by no means a perfect defense, since each botnet slave has three "shots" at guessing my passwords, but it still helps mitigate the problem.
To use my script, you need to add this to your sshd_config:
MaxAuthTries 3
And this to your root cronjob:
@reboot tail -F /var/log/auth.log | ~/bin/AutoDenyAttacker.pl &
(Replace the path for AutoDenyAttacker to fit your needs). You can download the script here: http://apocalypse.rulez.org/~upi/AutoDenyAttacker.txt [rulez.org] (This is a perl script -- rename it to .pl after you download.)
This script works well for debian etch and lenny, and I expect it would work on other systems too, perhaps with a bit of tweaking.
Regards,
UPi.
Re:AI (Score:4, Interesting)
My understanding of botnets is that all their activity is centrally coordinated: the bots sit in an IRC channel waiting for orders and do what they're ordered to do.
For comment spam it's more sophisticated than that: I monitor all attempts at adding comment spam to several sites I run. One site is interesting because it requires several distinct requests in order to post a message (and you have to visit each of those pages in turn in order to be successful at posting). The bots can perform these steps -- I watched as the controller in the Ukraine first worked it out manually -- but they do it from random IP addresses in turn. However, the cookie that I send in the first request is faithfully sent back by the other IP addresses.
These are not human attacks using something like Tor - far too quick for that.
So the bots communicate that cookie back to their "master" between each request, and that happens in sub-second times.
Rich.
Apply bayesian spam filtering? (Score:3, Interesting)
Since the attacker doesn't know the set of valid userids on the target system, it's hard to see how this could be countered. Spam authors know how normal email looks, but still can't defeat bayesian spam filters.
Re:Solution: Public Key Auth (Score:3, Interesting)
ok, get this
This criminal gang ("The Botnets") are rampaging across the city breaking into cars and stealing the stereos. But, get this, they are all lazy ass fatsos and at the multi-storey carpark they only ever break into cars on the ground level because its too much trouble to walk up to the upper levels. I mean, not many people park up there and these guys are not very good at picking locks anyway. If I had a car, I would park it up a few levels because I like the exercise and I don't like wiping greasy fingerprints off my door all the time. Seems like those other two guys do that too.
You're welcome.