NETI@home Data Analyzed

An anonymous reader writes "The NETI@home Internet traffic statistics project (featured in Wired and Slashdot previously) has a quick analysis on the malicious traffic they observed. It's a rough world out there." Perhaps not suprising, but still disheartening, the researchers find among other things that a large portion of typical end-user traffic consists of malicious connection attempts.

But did they find intelligent life?

That's what we need to know.

Re:But did they find intelligent life?

Well, they found people with a bunch of Windows Services on and all their ports open. Does that answer your question?

Re:But did they find intelligent life?

Coming soon: NETI@home discovers sentient penguins and daemons... "Penguins were seen to be working alongside daemons, cultivating apples and mischievously breaking windows..."

Considering..

Considering these malicious programs aren't following any kind of 'standard' to reduce bandwidth utilization when checking over entire subnets of IPs that have been checked by 100000x other copies of the virus, it doesn't suprise me one bit.

It would be like setting up a massive feedback loop on a mail server. When user X gets message X, he passes message X to user Y, who upon receiving message X sends it back to user X.

Standards for viruses?

You can't impose a standard upon viruses. What will you do if a virus doesn't follow the standard? Find the author and punish them unless they fix it and release a new version that fully supports the standard?

The only way viruses will ever get standards is if the authors agree that they will get a considerable benefit by working together. I can't see that happening.

DSL/modem/router

Its insane the ammount of bandwidth this is sucking up (i remember a time when virus's and worms were relativly well programed, still as bad but less collaterol dammage).
I would like to see more ISP isntead of suplying basic DSL modems with those overpriced sign up deals but instead a proper firewall/router/Dsl modem.
This would save us all alot of pain in the long run .

Recent Worms DO organize to manage utilization

Most of the interesting recent viruses *do* have some level of organization to reduce duplication of effort, and the postulated "Warhol Worms" designed to take over the entire Internet in 15 minutes would need to do so, because otherwise they're not as effective. Some of them pre-scan the net to find a list of vulnerable machines to infect first, and then haul around parts of the list. Others partition the address space quasi-deterministically (e.g. Phase 1 scans all of the valid /8 address spaces until it's infected some machine in each one, Phase 2 scans all of the 256 /16 address spaces within its /8 until it's affected one in each, Phase 3 scans all of the 256 /24 addresses within its /16, Phase 4 scans all the 256 addresses within its /24.

Code Red II [caida.org] implemented a randomized variant on this: "1/8th of the time, CodeRedII probes a completely random IP address. 1/2 of the time, CodeRedII probes a machine in the same /8 (so if the infected machine had the IP address 10.9.8.7, the IP address probed would start with 10.), while 3/8ths of the time, it probes a machine on the same /16 (so the IP address probed would start with 10.9.)" It means the worms don't have to keep track of phases, but it gets similar effects, and while there is more chance of overlap, it's not too high until the worm's infected most of the net, and the added random searches help make up for machines that didn't successfully infect their netblocks due to firewalls or failures or simple slowness.

At least one worm that took this sort of approach had a bad random number generator, so it kept hitting the same territory too hard and missing other wide-open spaces, which protected a few parts of the net from infection.

RBL of infected/malicious sites?

Does anything like this exist already? It would be nice if I could filter, say, ssh traffic coming from "known" naughty sites, and report sites that portscan me, though probably I should look at using smartcards or something more secure at this point. I can't just restrict the ssh port at the firewall, since people could be coming in from pretty much anywhere because of travel to remote sites. Aside from complaining to upstream providers (which so far has yielded zero responses) when I see people banging away at ssh, I don't see much else I can do.

Re:RBL of infected/malicious sites?

Why can't you restrict access to ssh from the firewall? One solution could be port knocking. You only let your firewall open up ssh after a series of connections on pre-defined parts are made. So say you choose "233 457 69 876 2094 576" to be your "password". You would make a client that would connect to those ports in that order and only after that initiate an ssh connection on port 22.

Time to drag out this old chestnut

"Those willing to give up a little security by using a little obscurity deserve neither security nor root privileges".

-Benjamin Franklin

Don't use SSH password authentication

You really should be using RSA or DSA keys instead of passwords. Hardly a day goes by that my systems don't get at least one script-kiddie SSH password guessing scan. Since I'm requiring keys for authentication, they're wasting their effort; if someone manages to crack a public key, we have far worse problems than password guessing.

Re:RBL of infected/malicious sites?

There are some. This site [bluetack.co.uk] has several different blocklists, such as ad-hosts, anti-p2p bodies, spyware companies, hackers, trackers, trojans etc. The link above lists what's available. Sure, the lists aren't 100% acurate, but they are a lot better than nothing.

Very highly recommended. With the case of p2p, it's good to keep your head down. It's the tall ones that get their heads chopped off...

They also have software to convert the lists to various formats for use in different firewalls. iptables fans should check out "linblock". Beware though, a large list can take an hour to parse on your typical recycled firewall box, but the tool merges the ranges to keep the tables as short as possible.

More malware, slower computer and net connection..

Hmmmm, must be time for a new computer and a better ISP

Not necessarily a Bad Thing...

ISPs could use this data to great benefit, if they'd put out some effort.

Assuming that the statistics show which IP address ranges are the worst offenders for malicious traffic, the ISP(s) responsible could simply shut down the outbound connection(s) of the "problem" users until they de-virus their systems and KEEP THEM THAT WAY.

Perhaps that will help to finally clue people in that having Internet connectivity is a privilege, not a right, just like driving. If you're going to enjoy an Internet connection you need to show some responsibility for making sure your own system isn't going to be a problem to others.

I -still- think there should have been Internet user licenses, just like we have driver's licenses...

Keep the peace(es).

Re:Not necessarily a Bad Thing...

The ISP KNOWS the physical addresses of the cable/dsl modem a home user has. It's not like the ISP has no idea which ip addresses are home user or account is using at any given time. How do you think they can reliably (for the most part) identify people for the likes of the RIAA when they ask. Likewise, with modern hardware and software its a pretty trivial task for an ISP to turn your internet access down to a crawl or off with the click of a button. They can do this, they just don't want to.

Maybe it would be a good idea to throttle the users down to a bare minimum and redirect all http traffic to a gateway page to tell them they have a problem with their computer they need to correct. It seems to work for wireless access points in hotels/airports/coffeeshops. Why can't big ISPs do the same thing?

Re:Not necessarily a Bad Thing...

I'm pretty sure internet connectivity is neither a privilege nor a right. It's just a service, plain and simple. You pay ISP, they provide internet connectivity. You don't pay, you don't get internet. No rights or privileges involved.

In other news...

Yeti@home [phobe.com] has yet to yield conclusive results.

Root of the problem

Ignoring all complaints about Windows, the root of the problem goes back to having access to the network in the first place. If ISPs would spent a few bucks on implementing passive traffic analyzers to search for the viral/trojan patterns and null route offenders, we'd clean things up pretty quick. Why do we have all these piracy probes going on to sue people and no infected probes going on to cut people's access?

Now, stepping back to the Windows complaints...wouldn't the ISP turning off your access motivate you to get a BASIC education in computing and maintain your PC?

To make an analogy, in most states you need to have your car inspected (and some require emissions inspection, too). PUBLIC roadways means you share it with other people...an unsafe car affects more than just you. When you're connected to the net, your PC affects everyone else. I'm not suggesting the ISPs make an inspection system or a law passes to force ISPs to monitor traffic, but the same logic applies....someone should be doing checkups and flagging the offenders.

Cheap access means unsafe computing

Sadly, while some customers might get motivated to learn something, others would just be motivated to switch ISPs. Which costs the ISPs money, which means that they won't do it.

At least such is their thought process as often presented. I suspect it's bad cost-benefit analysis; if your dumber customers leave, it's probably a net win for you. Smarter customers mean less bandwidth (at least, they don't act as spam zombies maxing out the bandwidth) and fewer tech support hours explaining how to fix the cup holder.

The big players (AOL, Comcast) are the best targets for this logic, but they live for those left-side-of-the-bell-curve customers. They're the "default" ISPs that people get because they're so readily available, so they get all the customers who don't know better. (Hell, I don't know better; I use Verizon for my DSL but I don't let them do anything but provide me bits.)

So AOL and Comcast are in a bit of a bind; they don't want these customers, but they don't want to lose them, either. I think that they're probably going to have to use gentle persuasion to say, "Hey, it looks like you've a spam zombie. Please call your cousin's best friend to clean the crap off your computer again and give you a stern talking-to. And please stop downloading Bonzi Buddy."

Re:Root of the problem

Amen to that. Car analogies have just plain run out of gas. People get too much mileage on them. They start more flamewars than a Pinto.

Re:Root of the problem

If ISPs would spent a few bucks on implementing passive traffic analyzers to search for the viral/trojan patterns and null route offenders, we'd clean things up pretty quick.

Bollocks.

The aren't running a network in their parents basement you know. Their networks are massive, with nodes LITERALY spanning thousands of miles. The volume of traffic they deal with is HUGE. They use cutting-edge routers just to keep up with the demand.

How on earth do you do traffic analysis on that level? You might be able to catch some of the more obvious spammers, but how do you differentiate (on the IP level) between: a) a residential user b) a commercial user who maildrops willing customers c) a zombie d) a community group or e) blah. Blocking someone based on traffic is not possible, unless you want to lose your valid customers.

What they should do is be more responsive to complaints. If a customer of theirs is a zombie spambot or acting as a stepping stone for some script kiddie, they should have their connection suspended until it is remedied. But they can only do this based on a complaint.

Besides, what's the profit in spending any resource on the problem in the first place? Until that is affected, they won't care about it.

In a few minutes...

...they will realize that there isn't anything more malicious than the traffic from Slashdot.

malicious?

I've only scimmed the paper, but from the looks of it, a lot of not all that harmful trafic could be labeled "malicious", for example nmap port scans. I use them all the time, not to find valunerable services, but for more general sysadmin stuff.

Flow observation conclusions... news u can use

It's good to know the IP addresses of machines active searching dark IP space. If you can see those statistics in real time, you have useful information.

ISPs are already starting to work together on this type of information. If an ISP sees malicious worm spreading behavior, it can upload the offending IP into a global db that all ISPs can use to block at their borders.

Again, the authors conclusions are that nothing beats having a nice dark block to trigger alerts.

Next Step?

Modify the Neti@Home client to do dynamic blacklisting?

The biggest problem in Intrusion Detection Systems (buzzword for firewalls with more intelligence than a typical rule-based firewall) is that metrics gathering is occuring at a specific site, making it difficult to discern malice intent from dropped packets or bad coding.

Any time the central server sees a certain threshold of malicious attempts from a single IP, it adds it to a short term blacklist... Make the term length just slightly longer than the reporting period so if it persists it'll remain on the list but if it stops, the IP is cleared in short order.

Spyware?

To collect data, Internet users must volunteer to run the software package on their end hosts. Once the package is installed, the NETI@home client will collect net- work statistics from the end host and periodically send a report back to the NETI@home server. Volunteer by downloading the NETI@home toolbar with new "we are watching you" emoticons

proposal

I would like to submit this proposal for your review. I am seeking funding for a new research project. Please grant me the funds needed so that I can deploy rain sensing equipment to every residence in the Seattle area.

This project will record 3 years of data and prove once and for all whether or not it actually rains in seattle.

sincerely,
Kelly H.
Head research scientist
Darington Univeristy of Heretics

The Most Illegible Graphs. Ever.

Shouldn't there be a butt-ugly histrograph warning?

Randomly Generated Topics?

This paper looks almost exactly like
one of the randomly generated research papers
i got from that MIT research groups website..
(Questions...)

I passed the randomly generated paper around campus to a bunch of C.S. kids and they all bought it without thinking.. Quite amusing...

neti samples

From what I've seen the real challenge would be to find significant samples. I don't imagine crackers would go for the neti software.

If you build it, they will portscan

Been to Borders and seen the honeypot books on the shelves amongst the rest of the become-a-security-guru-in-$29.95-easy-steps books?

Does it prove or disprove simple A==B logic to note that these incidences of spyware and insecurity are growing at the same time as adoption of Linux variants? Just musing on the "l33t win script kiddie finds Linux religion" phenomenon I've been seeing lately.

Anyhow, this does suggest further that security is where it is at for the future skillset of interest at interview time.

Maybe analyze their own network trafic?

It can't be good to have a 8731x1276 GIF as a logo [gatech.edu] on their first page, especially when being slashdotted.

April 27th-30th?

Look at that "Daily usage for April 2004" graph...

Apparently this site will be linked to by Slashdot in two days, but it hasn't been yet...

Big deal.

I've been doing neti [wikipedia.org] at home for several days trying to shake a sinus infection brought on by allergies. :-)

It remains to be seen if I'll find positive results.

-Peter

Neti?

I've been using neti [healingdaily.com] for years to improve my nasal bandwidth. I had no idea they made it into a distributed.project...

PDF, ack

A PDF warning would be nice next time around, folks.

Collecting packets

so who "owns" these packets?
Sounds like echelon in a new suit.

Anyone get this working on OS X?

Has anyone managed to compile this on OS X? I am trying but I get 4 errors on line 469.

Slashdot generates "malicious" traffic too

My firewall figures slashdot checks my ports for open proxies and bans it for 20 hours, I added slashdot to my trusted networks because of it.

If I didn't know it/ ignorant etc, I would see 100s of port scans from a huge , evil T class machine.

Oh btw CmdrTaco, don't hack my machine :)