KentuckyFC writes "Recommendation systems attempt to guess what books, movies, or news people are likely to be interested in. Companies such as Amazon, Google, and Netflix have developed algorithms to mine vast databases looking for correlations that they then use to recommend new items. Now a team of computer scientists has used some of the same filtering techniques to predict the origin of malicious Web attacks so that they can be blacklisted in advance. The team mined a database of hundreds of millions of security logs looking for correlations between victims. The correlations were then used to produce a predictive blacklist of potential attackers. The team says its algorithm is up to 70 per cent more successful at predicting the origin of attacks than current state-of-the-art predictive blacklisting."
Why do both have to be mutually exclusive? Why can't the problem be approached from both sides by different groups whose skillsets are appropriate for what they're doing?
Perhaps I should included my title in my post. No doubt this useful given the current situation. But we wouldn't be in this position so much if we had well designed systems in place from day 1.
I do think this is interesting how we can use massive data sets to predict and map trends so much quicker. But I'd rather not have to worry about them in the first place.
I think the underlying issue has come from the fact that people have been more focused on making computers do what they want them to do, and not focused on making them do it securely. It's great to sit on the sidelines and talk about how it should have been done better/smarter/more securely in the first place. That perspective does not take into account the reality that computers are relatively new and new functionality comes out almost every day. To consider another aspect of security, we've been living
Unix was designed from day 1 with the notion that it is a multi-user system that needs serious integrated security. Windows was designed for a home PC with a single user. It wasn't designed with the notion that it would be on the internet, or need much in the way of security.
It isn't to say that we couldn't have forseen security concerns to design it correctly in the first place. Most *nix systems were always designed this way. Windows opted not to follow that model.
Bruce Schneier [schneierfacts.com] says that give a choice between security and dancing pigs on your computer, people will take the dancing pigs every time.
When Windows came out, it was perfectly secure - there's only one user in the universe, and she's allowed to do whatever she wants. ("Format C: "? Sure!).
Unfortunately, while Unix was designed from the beginning for security, it didn't always _stay_ designed for security, and some of the things that were done for security had serious tradeoffs. Networking was usually th
In truth, all you need to do is read the Art of War, and you'll know that implementing proper Windows permissions couldn't possibly the the answer to security. You'll also realize that collaborative filtering couldn't possibly the the answer to security either.
The only answer is to be one step ahead of the attackers, and to think up what they're going to throw at you next.
(That's not to say that proper Windows permissions don't help, and that collaborative filtering doesn't help, but security is war, and th
but shouldn't we be more concerned about true security?
What is "true security" against the main threat of the modern era: social engineering? How does your operating system protect you from from responding to that e-mail you've just received from your long lost uncle in Nigeria? How do you protect a user that will click on the user account control pop-up as many times as is required to install that cool "weather forecasting" program that sits in his task tray?
Or were you referring to "true security" in the context of firearms, expendable redshirts and moats filled with laser wielding sharks?;)
Don't underestimate sharks with friggen laser beams!
I agree that Social Engineering is likely the number one threat in many cases.
UAC is security theater in that people are trained to simply click allow, absolving Microsoft of responsibility.
What I mean by true security is sandboxing and accountability. Look at Chrome's design, in that a browser window (process) has limited access to data on your HDD.
Users in an enterprise environment frankly shouldn't have access to install software at all. And the more I
Users in an enterprise environment frankly shouldn't have access to install software at all.
Which leads us to the true security question/issue. The only truly secure system is one users don't have access to. In any other environment, where people are trying to get work done. a completely locked down environment can impede the business. The end goal, whether the security types like it or not, isn't a secure environment. It's to make money or reach some other objective. Security is relevant in that it sup
And if you just take the PCs away from the silly users and lock them away in safes they'll be 100% secure! Seriously sandboxing is a bandaid on a bullet wound, and is as much bullshit as "as long as they can't get root its okay". Well, no its not. If I have control over your network connection why would I give a shit if it is sandboxed or not? As long as I can get the user to visit my site and load up my malware I can spew spam, I can DDOS, etc. Just like if you get a hold of the local user account you can
As long as I can get the user to visit my site and load up my malware I can spew spam, I can DDOS, etc.
Not true. If neither the user nor app have admin/root access, and you're using a secure browser (say, Chrome) then your malicious web site can't do squat. The biggest hole here right now would be that plugins aren't fully sandboxed, and Acrobat has a serious vulnerability every other week. But that is partially why I keep recommending to businesses to use Foxit as opposed to Acrobat.
sers in an enterprise environment frankly shouldn't have access to install software at all.
Unfortunately it's rarely that simple. I've worked in two "enterprise" environments in my IT career. One (my current job) makes this fairly easy to implement -- most of our operations run around web based database apps and Office. Very easy to lock users into restricted accounts.
The other enterprise I worked for was an insurance agency. The insurance industry has so much legacy software that restricting users to non-admin accounts is not possible unless you are willing to sacrifice needed functionality
Both of the two enterprise environments I've worked in have used proprietary legacy apps that "need" admin rights.
Most of the time, all the app really needs is write access to a certain folder. However, in the rare instance that the process truly does need administrator access, I make the app/process into a Windows service that starts automatically at login with System level access. The user doesn't have admin access, and other apps don't. That one app is elevated.
What is "true security" against the main threat of the modern era: social engineering?
Social engineering will always be a problem but there is a simple fix. Restrict the user on damage they can do on their own given the worse case circumstances and you will also end up with the same prevention of malware in the process.
Speaking of which... Why does a web page ever need to communicate with the OS to make file changes to the OS? Why?! Why I ask?!
This is a flawed premise and will solve 99% of the problems we f
Some of us would be happy if one company would give us a central repository that we could manage for our own networks. Software whitelisting isn't exactly a new concept.
" "Well I want a pony!" you reply "But you'll just have to deal with a limited OS because we can't have nice things because they keep installing viruses on their machine!" "
I used to work with an ex-Navy guy - our lab became much neater once he joined us, and more secure as well. But different organizations have much different concepts of what it means to "secure a computer" -
The Army sends out computer technicians to look at log files.
The Navy ties the computer down with ropes and netting to keep it from bouncing around in rough seas, and does whatever it takes to keep the computer room water tight.
The Marines send a squad of guys with automatic weapons to make sure no
I truly agree, bad OS design, some lack of security based on too much money it would cost, and not enough people really taking security seriously, there will always be that one person with a password equal to a dogs name or their birthday!!!
You can't be held responsible for blacklisting sites right now, what makes this any different from any other Blacklist?
If you want to get traffic to/from the site then Whitelist it.
Now they say their approach is 70% better than existing Predictive Blacklist technology, well how good is that, 70% better than horrible false positives and annoyed customers is not enough. Throwing darts at the DNS listings is also not optimal, so how good is this new technique.
BTW Amazon and Netflix recommend crap to me I don't
Well, according to TFA it's not quite ready for prime time.
There are some potential problems to iron out. For example. the team isn't quite sure how to handle the constantly changing pattern of malicious attacks and malicious attackers may soon find that it's not too hard to fool recommendation systems if you try hard enough.
Yeah, and I'm pissed that I can't get Clippy working on Ubuntu!! The little dog in the search box too!! What, when you start to go geeky you can't have pets anymore? It's just WRONG, I tell you!! I'm going to send some hate mail to Canonical, and find out what the deal is. This just pisses me off!!
sarcasm off
Seriously - all those user agents and stuff should have been a tipoff. A corporation that offers cartoonish characters as part of a "serious" operating system can't be trusted with secur
... wouldn't blocking people's access in advance considered an attack in and of itself? So the service should simply block itself off and be done with it.
That, or go Skynet. The ideal way to stop all web attacks would be to bring down the internet itself. I so hope these guys did their homework [xkcd.com].
Obviously they have developed hacking technology to break open all these systems to get at their logs to determain if they have been hacked. Well they will be blacklisting themselves later this afternoon.
This sounds great, but only if it requires human intervention to implement the block. I used to work in a NOC, and we would have loved to throw up a warning on the big screens that an attack is 80% likely from the following netblocks in the next N hours. That way we would have a strategy developed for defending before it even started and would be able to minimize downtime.
On the other hand, if you make this automatic you're going to piss off a lot of people very quickly because it's going to be wrong more often than you want.
Exactly. Because even if it's true, and it's 70% more accurate... I've yet to see a predictive system that's even remotely accurate. It may predict say... 50% of the sources of an ongoing attack (assuming a collaborative effort to determine when attacks are happening, and that you're not the first one hit), but that's far from enough to prevent a DDoS attack. And if you "accidentally" block... Say Canada (which I've seen before), then that's a LOT of costumers you just pissed off, but hey... Doesn't matter,
What about the people who are blacklisted unfairly? If the false positives are 1%, a huge number of servers will be blocked. This is the same problem with lie detectors and drug testing -- innocents get snared in the net. You need a way to confirm the positive, and not just blacklist based solely on this algorithm.
If site is shitty, +.1 If site has a "clever" name, such as bit.ly, +.1 If site's name has become widely used as a verb or other part of speech, +.1 + unique users in the last 24 hours / 100,000,000
Calculate the monetary factor
If site sells something, +.05 If site makes revenue through ads, +.05 If site is partnered or associated with a megacorp like a bank, ms/google, etc., +.1 + dollars lost per minute of downtime (based on the last 24 hours) / 1,000,000
Thanks, I knew that there was a perfect tag for this story! Marking it as such allows two benefits which I can easily define:
Just because a query originates from within an IP address block will not make it an attack. It's like assuming that someone from a bad neighborhood will steal from you.
No doubt useful (Score:4, Insightful)
But this is still treating the symptom as opposed to the core problem, which is poor security in OS and app design.
Microsoft is starting to come around on this to an extent (not running as administrator), but shouldn't we be more concerned about true security?
Re:No doubt useful (Score:4, Insightful)
Parent
Re: (Score:2)
Perhaps I should included my title in my post. No doubt this useful given the current situation. But we wouldn't be in this position so much if we had well designed systems in place from day 1.
I do think this is interesting how we can use massive data sets to predict and map trends so much quicker. But I'd rather not have to worry about them in the first place.
Re: (Score:3, Insightful)
Re: (Score:2)
Unix was designed from day 1 with the notion that it is a multi-user system that needs serious integrated security. Windows was designed for a home PC with a single user. It wasn't designed with the notion that it would be on the internet, or need much in the way of security.
It isn't to say that we couldn't have forseen security concerns to design it correctly in the first place. Most *nix systems were always designed this way. Windows opted not to follow that model.
Choosing Security vs. Dancing Pigs vs. Unix (Score:2)
Bruce Schneier [schneierfacts.com] says that give a choice between security and dancing pigs on your computer, people will take the dancing pigs every time.
When Windows came out, it was perfectly secure - there's only one user in the universe, and she's allowed to do whatever she wants. ("Format C: "? Sure!).
Unfortunately, while Unix was designed from the beginning for security, it didn't always _stay_ designed for security, and some of the things that were done for security had serious tradeoffs. Networking was usually th
Re: (Score:2)
In truth, all you need to do is read the Art of War, and you'll know that implementing proper Windows permissions couldn't possibly the the answer to security. You'll also realize that collaborative filtering couldn't possibly the the answer to security either.
The only answer is to be one step ahead of the attackers, and to think up what they're going to throw at you next.
(That's not to say that proper Windows permissions don't help, and that collaborative filtering doesn't help, but security is war, and th
Re:No doubt useful (Score:5, Insightful)
but shouldn't we be more concerned about true security?
What is "true security" against the main threat of the modern era: social engineering? How does your operating system protect you from from responding to that e-mail you've just received from your long lost uncle in Nigeria? How do you protect a user that will click on the user account control pop-up as many times as is required to install that cool "weather forecasting" program that sits in his task tray?
Or were you referring to "true security" in the context of firearms, expendable redshirts and moats filled with laser wielding sharks? ;)
Parent
Re: (Score:2)
Don't underestimate sharks with friggen laser beams!
I agree that Social Engineering is likely the number one threat in many cases.
UAC is security theater in that people are trained to simply click allow, absolving Microsoft of responsibility.
What I mean by true security is sandboxing and accountability. Look at Chrome's design, in that a browser window (process) has limited access to data on your HDD.
Users in an enterprise environment frankly shouldn't have access to install software at all. And the more I
Re: (Score:2)
Users in an enterprise environment frankly shouldn't have access to install software at all.
Which leads us to the true security question/issue. The only truly secure system is one users don't have access to. In any other environment, where people are trying to get work done. a completely locked down environment can impede the business. The end goal, whether the security types like it or not, isn't a secure environment. It's to make money or reach some other objective. Security is relevant in that it sup
Re: (Score:3, Interesting)
And if you just take the PCs away from the silly users and lock them away in safes they'll be 100% secure! Seriously sandboxing is a bandaid on a bullet wound, and is as much bullshit as "as long as they can't get root its okay". Well, no its not. If I have control over your network connection why would I give a shit if it is sandboxed or not? As long as I can get the user to visit my site and load up my malware I can spew spam, I can DDOS, etc. Just like if you get a hold of the local user account you can
Re: (Score:2)
As long as I can get the user to visit my site and load up my malware I can spew spam, I can DDOS, etc.
Not true. If neither the user nor app have admin/root access, and you're using a secure browser (say, Chrome) then your malicious web site can't do squat. The biggest hole here right now would be that plugins aren't fully sandboxed, and Acrobat has a serious vulnerability every other week. But that is partially why I keep recommending to businesses to use Foxit as opposed to Acrobat.
Re: (Score:2)
sers in an enterprise environment frankly shouldn't have access to install software at all.
Unfortunately it's rarely that simple. I've worked in two "enterprise" environments in my IT career. One (my current job) makes this fairly easy to implement -- most of our operations run around web based database apps and Office. Very easy to lock users into restricted accounts.
The other enterprise I worked for was an insurance agency. The insurance industry has so much legacy software that restricting users to non-admin accounts is not possible unless you are willing to sacrifice needed functionality
Re: (Score:2)
Both of the two enterprise environments I've worked in have used proprietary legacy apps that "need" admin rights.
Most of the time, all the app really needs is write access to a certain folder. However, in the rare instance that the process truly does need administrator access, I make the app/process into a Windows service that starts automatically at login with System level access. The user doesn't have admin access, and other apps don't. That one app is elevated.
Re: (Score:3, Interesting)
What is "true security" against the main threat of the modern era: social engineering?
Social engineering will always be a problem but there is a simple fix. Restrict the user on damage they can do on their own given the worse case circumstances and you will also end up with the same prevention of malware in the process.
Speaking of which... Why does a web page ever need to communicate with the OS to make file changes to the OS? Why?! Why I ask?!
This is a flawed premise and will solve 99% of the problems we f
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
I think there is a company willing to do that. They're in Redmond, Washington.
Re: (Score:2)
Re: (Score:2)
" "Well I want a pony!" you reply "But you'll just have to deal with a limited OS because we can't have nice things because they keep installing viruses on their machine!" "
Thank you, you've just made Jonathan Zittrain's point exactly [futureoftheinternet.org].
Except he thinks this is a bad development and can still be changed.
What the military means by "Secure a computer" (Score:2)
I used to work with an ex-Navy guy - our lab became much neater once he joined us, and more secure as well. But different organizations have much different concepts of what it means to "secure a computer" -
Re:No doubt useful (Score:4, Insightful)
"True security" is a fantasy. No such thing exists, nor will it ever.
We should be concerned with balancing risk reduction with its cost. We should not be concerned with your silly fantasy.
Parent
Re: (Score:2)
Designing the OS to be secure as opposed to chasing people attacking vulnerabilities left by design in the OS is silly?
Re: (Score:2)
I truly agree, bad OS design, some lack of security based on too much money it would cost, and not enough people really taking security seriously, there will always be that one person with a password equal to a dogs name or their birthday!!!
Minority Report (Score:2, Interesting)
What about false positives? Can they be held responsible for blacklisting an innocent site?
Re: (Score:2)
You can't be held responsible for blacklisting sites right now, what makes this any different from any other Blacklist?
If you want to get traffic to/from the site then Whitelist it.
Now they say their approach is 70% better than existing Predictive Blacklist technology, well how good is that, 70% better than horrible false positives and annoyed customers is not enough. Throwing darts at the DNS listings is also not optimal, so how good is this new technique.
BTW Amazon and Netflix recommend crap to me I don't
Finally a use for this technology (Score:3, Funny)
There's finally a use for this collaborative filtering technology.
Re: (Score:2)
Well, according to TFA it's not quite ready for prime time.
Re: (Score:2)
If it doesn't, they won't have much of a website left after today.
Did I read this right... (Score:5, Interesting)
recommendation systems may soon be providing you not only with books and movie tips but a happier surfing experience too
I am a little weary of making my surfing experience happier by allowing the system to do my thinking for me. Just think, "clippy" for the browser.
Re: (Score:2)
sarcasm on
Yeah, and I'm pissed that I can't get Clippy working on Ubuntu!! The little dog in the search box too!! What, when you start to go geeky you can't have pets anymore? It's just WRONG, I tell you!! I'm going to send some hate mail to Canonical, and find out what the deal is. This just pisses me off!!
sarcasm off
Seriously - all those user agents and stuff should have been a tipoff. A corporation that offers cartoonish characters as part of a "serious" operating system can't be trusted with secur
Re: (Score:2)
Umm... (Score:2, Funny)
Re: (Score:2)
That, or go Skynet. The ideal way to stop all web attacks would be to bring down the internet itself. I so hope these guys did their homework [xkcd.com].
the new 404 (Score:3, Funny)
"People..." (Score:5, Funny)
"People who attacked this site ALSO attacked..."
Re: (Score:3, Funny)
"Was this review helpful? Yes or no"
Re: (Score:3, Informative)
Amazon should patent "1-click attacking"
Ptech [google.com] already has it patented!
Re: (Score:2)
We used to get targetted many, many moons ago by people searching Google for "phpBB version x.y.z". If you want to predict web attacks, Google says:
Results 1 - 20 of about 80,600,000 for "phpbb version x.y.z"
The Article is obviously a fake (Score:4, Insightful)
Or greatly exaggerated...
"The team mined a database of hundreds of millions of security logs"
Nobody actually keeps security logs, certainly not hundreds of millions of somebodies.
The kind of people that DO keep security logs probably wouldn't hand them over either.
I call shenanigans
Re: (Score:2)
Obviously they have developed hacking technology to break open all these systems to get at their logs to determain if they have been hacked. Well they will be blacklisting themselves later this afternoon.
Re: (Score:3, Insightful)
Fixed:
Now it makes more sense, and is quite believable, no?
False positives (Score:2)
False positives, here we come...
Meatware needed (Score:5, Insightful)
This sounds great, but only if it requires human intervention to implement the block. I used to work in a NOC, and we would have loved to throw up a warning on the big screens that an attack is 80% likely from the following netblocks in the next N hours. That way we would have a strategy developed for defending before it even started and would be able to minimize downtime.
On the other hand, if you make this automatic you're going to piss off a lot of people very quickly because it's going to be wrong more often than you want.
Re: (Score:2, Insightful)
Exactly. Because even if it's true, and it's 70% more accurate... I've yet to see a predictive system that's even remotely accurate. It may predict say... 50% of the sources of an ongoing attack (assuming a collaborative effort to determine when attacks are happening, and that you're not the first one hit), but that's far from enough to prevent a DDoS attack. And if you "accidentally" block... Say Canada (which I've seen before), then that's a LOT of costumers you just pissed off, but hey... Doesn't matter,
False positives (Score:2)
Predictorator (Score:2)
Calculate the annoyance factor
If site is shitty, + .1
If site has a "clever" name, such as bit.ly, +.1
If site's name has become widely used as a verb or other part of speech, +.1
+ unique users in the last 24 hours / 100,000,000
Calculate the monetary factor
If site sells something, +.05
If site makes revenue through ads, +.05
If site is partnered or associated with a megacorp like a bank, ms/google, etc., +.1
+ dollars lost per minute of downtime (based on the last 24 hours) / 1,000,000
Calculate the brought it up
Sounds vageule familiar... (Score:2)
They *guess* that you may be guilty before it happens and blacklist you.
Re: (Score:2)
a "coorelationisnotcausation" tag
Thanks, I knew that there was a perfect tag for this story! Marking it as such allows two benefits which I can easily define:
Re: (Score:2)
You may joke about it, but I wrote a "slashdotted" snort rule for a web development and hosting company.