Network Intrusion Detection Systems Fail to Impress 215
TheBongPipe writes "I'm reading a nice test here about 7 commercial IDSs. Who won the prize? Nobody..." They also looked at Snort, but found that all the products generated way too many false alarms.
What does everyone use (Score:1, Interesting)
Re:What does everyone use (Score:1)
But it has helped me catch nimda and other nasty tidbits running around the network, allowing me to repair the systems.
Re:What does everyone use (Score:5, Funny)
Yeah, me too. All that special lab equipment to refine it, and the look out always saying the cops are coming when half the time it's just a meter-maid....
Re:What does everyone use (Score:2)
Re:What does everyone use (Score:1)
IPCop. Woks great and really easy to install
Re:What does everyone use (Score:1)
Re:What does everyone use (Score:1)
I Know. I just ASSUMED that people would know it uses snort
Too bad no Dragon (Score:3, Informative)
Difficult to make (Score:1)
False Alarms (Score:2, Insightful)
Too many false alarms isn't necessarily a bad thing. In intrusion detection you'd rather take the false positives, than the alternative.
The rate of false fire alarms, and false burgular alarms is VERY high compared to the actual number of real emergencies.
Car Alarms (Score:5, Insightful)
You would hope that the increase in false positives decreases the number of false negatives but that isn't necessarily true either.
Re:Car Alarms (Score:2, Interesting)
I have been to a certain obscure country visiting my relatives where a common joke is this: A car alarm sounding means another dork can't figure out how to enter their car.
The only flaw with this analogy is that car alarms are supposed to alarm people (i.e. draw their attention) who have no personal interest in the safety of your car to pay attention, and alarm the crook that they have been detected so they will hopefully run away without stealing the car.
I am no security professional, but I would expect that such intrusion detection software does not give the cracker any warning that they have been detected by a real person or security system, so they have no reason to leave the system alone. It also does not give the IP of the potential threat to everyone else on the network so that the threat can be DDOS'd.
Re:Car Alarms (Score:3, Funny)
Yup, yup, I *know* what you mean!
I've got RAID array in my office that's part of the main production file server and there's this alarm that's been going off for, like, 16 MONTHS on the thing. Don't worry, it's not important - it's only a fan in the back of the drive tray that gets stuck sometimes, then it works itself loose and everything goes back to norm@#&$%@#$89d sifsd00JE{PGJE....
The problem with false alarms (Score:5, Insightful)
Spoken like someone who does not carry the IDS support pager at nights and on week-ends!
The problem with too many false IDS alarms is that the staff tend to treat it like the boy who cried wolf. After awhile, you disregard the pages or treat them with less consideration because the last n pages have all been false alarms.
I think that IDS is important, but if there are too many false IDS alerts, it becomes difficult to put up with. Because they are strictly reactive systems, it is improbable that there will ever be a perfect IDS that never raises false alarms, but clearly there is a lot of work to do. I am surprised that Snort did so poorly, since it really is a nice system, but it takes a long time to build up a good set of heuristics...
The rate of false fire alarms, and false burgular alarms is VERY high compared to the actual number of real emergencies.
That's right. And in my area, if the police department are called out to the same location for three false burglar alarms in one year, they will not respond to any subsequent alarms automatically. And the fire department charges a fine of $300.00 per incident if they receive more than three false fire alarm calls to the same location in one year. Why? Because, as you said, the number of false alarms is much higher than the number of actual emergencies. The false alarms cost time and money and if all the resources are busy dealing with false alarms, there is nobody left to help when a genuine emergency occurs.
Re:The problem with false alarms (Score:2, Insightful)
Re:The problem with false alarms (Score:3, Insightful)
I completely agree with this statement, but can't understand why all the "false alarm" complaints. When you build an IDS, you need to start with all the alarms turned on then identify which non-intrusion signatures cause false alarms, then TAKE THEM OUT. This is the first rule of IDS's. Any site which explains how to install snort will tell you that you need to customize the config for your situation. Another thing that gives snort the edge(besides speed and cost) is that its open source, so the real TCP/IP hackers out there will always be tinkering to make it better and more efficient at catching true intrusions.
Everything's OK! (Score:3, Funny)
Homer: Now, here's my "Everything's O.K."alarm!
[Homer flips a switch on the device, and it begins to emit a high pitched, incredibly loud beep. The rest of the Simpsons cover their ears as Homer speaks up]
Homer: This will sound every three seconds, unless something isn't okay!
Marge: Turn it off, Homer!
Homer: It can't be turned off! [alarm fizzles out] But it, uh, does break easily.
-- "The Wizard of Evergreen Terrace"
This sounds about as useful.
Snort and GUI (Score:4, Insightful)
Come on, reading the article I saw the guy said a Snort disadvantage is not having a GUI. What kind of technical user this guy is?
Snort GUI... (Score:4, Interesting)
Re:Snort and GUI (Score:2)
Obviously we didn't get the whole thing up and going in a day, and we still spend time updating/tweaking signatures; but it wasn't rocket science.
Re:Snort and GUI (Score:2)
False Positives... (Score:4, Funny)
Re:False Positives... (Score:1)
That's basically the reason it 'failed to impress.'
Re:False Positives... (Score:1)
Re:False Positives... (Score:2)
If I had an intrusion detection package that made a similar kinda log, it's easy to imagine that it'd constantly fill with new log events. I wouldn't know what to look for until the damage had been done.
Get what I mean?
Re:False Positives... (Score:5, Insightful)
ALERT: Intrusion attempt detected! User Liora mistyped his password!
the rate of false-positives is high enough that
ALERT: Account Liora has recieved login attempts from three different IPs in the past 12 hours!
you stop paying attention; unlike with AIDS testing (which has a very high false positive rate) the user is simply likely to ignore the system even if real threats occur
ALERT: 1,262 attempts to login as user "root" in past 7 seconds!
So it becomes desirable to lower the false positive rate to a manageable level, WHATEVER the rate of false negatives is, because otherwise you won't actually catch anything.
The purpose of the AIDS test is to assure you that you don't have AIDS. False negatives are unacceptable, false positives can be dealt with.
The purpose of the IDS is not to prevent intrusions - that would be nice, but it's not going to happen. The purpose of the IDS is to identify the (coloquially) hackers, so that you can retaliate against them / deter them, before they get you too many times, or get too many different people once. To do this, you need a deterence-level set of positives which is small enough (and true positive rich enough) for you to actually act on them.
Oh, and because I get off on it when people with agree with me: this is no substitute for real, human-level, security measures. Someone who expects any system of this kind to protect them from lousy sysadmin decisions deserves the rusty metal sodomy they will no doubt recieve.
Re:False Positives... (Score:3, Insightful)
This is so true, I wish the PHBs would get this! Detection and prevention are two different things, it is like comparing a pregnancy test to a condom.
You should have at least used the later, but to be sure, use the former as well.
Re:False Positives... (Score:2, Informative)
You SHOULD be testing your systems constantly.
You SHOULD be installing new patches.
You SHOULD be subscribed to CERT style mailings.
You SHOULD NOT think you are safe because you're small. Security though obscurity is the biggest false sense if security I've seen. Former employees, especially the guy you replaced are a pretty large threat.
For beginners out there, here are some places to start... (Some of these are OLD links, but still contain some useful information and yes, they're Linux oriented.):
Beginners Guide to Armoring Linux [enteract.com]
Linux Security Guide [nic.com]
Nessus [nessus.org]
Traditional HOWTO [tldp.org]
Pattern matching based on behaviour (Score:1)
Is not exactly an accurate science; either one has to deal with false alarms, and probably more than a pleasant number of them for the sake of paranoia, or simply be more sure of their systems.
Informal poll who among us uses an IDS, and why? I've always figured them for a way to be lazy, but that perspective is likely not shared. I'd love to see some other opinions!
Always compare with a placebo (Score:5, Funny)
Re:Always compare with a placebo (Score:5, Funny)
Re:Always compare with a placebo (Score:1)
Need more detail (Score:4, Insightful)
Gee, whose fault is that?
Re:Need more detail (Score:2, Insightful)
Remember--there are significantly more computers than talented sysadmins out there, which means that tools such as these must be made to not only work for experienced and knowledgable admins as well as "n00b" admins.
I think it's at least partly Snort's fault. Make it easier to use, so that configuration errors are less common.
A False Alarm is still an Alarm (Score:5, Insightful)
They also go on to mention all ask too much of their users in terms of time and expertise to be described as security must-haves. IDSs are not screen-savers. Those who are setting up an IDS better have a good understanding of how they work and how to configure these applications. Point-and-click doesn't really apply to something this involved.
Re:A False Alarm is still an Alarm (Score:2, Informative)
I've run NAI's IDS (the one that came bundled with PGP 7.1) for a year on Win2k, and it hasn't crashed yet. It does come up with false positives, especially if you configure it to be "sensitive", but once they occur, you can determine whether or not you want to continue to listen for them. It consistently tagged something the Mac's on our LAN were doing as a "fraggle" attack, so I turned off "fraggle" detection.
Not a perfect solution, but soooo much better than nothing.
Re:A False Alarm is still an Alarm (Score:3, Informative)
I can vouch for this also. About six months ago I setup a snort logging to mysql box at work to monitor our class-C and have had zero problems. It does take a while to tweak and prune things to eliminate all the portscan misdetects (any/any is certainly not advised) and such, but overall it's performed sweet. The price is right certainly, comparing it to those listed in the article. As far as console analysis, you simply cannot do without ACID as the parent said. Head to CERT and download a copy. My only complaint is that the db lookups perform a little sluggishly on the p2-233.
Re:A False Alarm is still an Alarm (Score:2)
Bad test == Bad results (Score:2)
From reports of the test the "wu-ftpd" exploit used wasn't an actual exploit, but was only a replay of one of the signatures of a decade old exploit. Since not all of the systems use signature detection, and since the "exploit" didn't actually exploit anything (*gasp*) some of the IDS systems didn't pick it up.
Lancope Stealthwatch M100 (Score:1)
According to the chart, it also didn't detect code red worm, SYN flood, or wu-ftp exploit.
Was this box even operating correctly?
Re:Lancope Stealthwatch M100 (Score:1, Informative)
It functions by analysis of traffic flow.. system x communicating to system y on ports a,b,c. if the pattern changes beyond a threshold, an alarm.
$20,000 for crap (Score:3, Interesting)
IDS are complex systems. Anyone pretending they have a packaged solution should rot in jail.
Re: (Score:2)
False Alarms get annoying for real admins (Score:3, Interesting)
Imagine the fun the first time we try to deploy an antivirus package to his desktop just to be blocked for -- are you sitting down? -- an attempted NetBIOS intrusion.
After the second time we tried to deploy (and failed) BlackIce locked down the system so that it couldn't be accessed across the network by any other workstation, despite our having adminsitrative rights. That was cute.
Just throwing up a little real world example of how annoying these false alarms can be.
Re:False Alarms get annoying for real admins (Score:1, Informative)
had firewalls in place and had never had an intrusion on our network.
HAHAHAHAHA. seriously. HAHAHA. This is called the hard-candy security princple.. Hard and crunchy on the outside, soft and chewy inside. Perimiter firewalls are insiufficient for medium-large businesses, and generally small ones as well. Got VPN connections? Allow ANY traffic in to a DMZ or internal network?
As for BlackICE.. 'NetBIOS intrusion' isnt even an attack type. What happened was probably that the user blocked the netbios PORT via the firewalling and it showed as a 'NetBIOS port probe' with the reason=firewalled. Also, BlackICE does not disable ALL network activity. By IP address or port, yes. Not by network or everything. That would be stupid. See http://www.networkice.com/advice for BlackICE attack details.
Re:False Alarms get annoying for real admins (Score:1, Interesting)
Keep your damned NOC hands out of my Fricking PC's. I'll deploy my OWN updates than you. (I am always at least 2 weeks ahead of corperate on every update, and I have NEVER had a virus origionate or propagate from my offices.. the last 2 Virii propagated from the NOC computers...)
The local admin is who is in charge... you remote dweebs can keep your damned fingers out of my stuff.
Re:False Alarms get annoying for real admins (Score:1)
What a wonderful idea! (Score:5, Interesting)
I hate to tell you this but, at this day and age when everything is being outsourced, some users feel they need to protect their machines against the "IT support".
Re:False Alarms get annoying for real admins (Score:1)
Sounds like you need to lock your users down a little more; there's your primary security problem.
Network adminstrators are a real pain in the ass. (Score:2)
The really shitty part is that just when I get the anti-virus off my system, IT (in their infinite wisdom) pushes an "update" onto my system.. sending me back into blue-screen land.
I finally installed BlackIce on my system and set it up to deliberately block the fuckers. Yeah, IT gets pissed and thinks I'm stupid or something, but at least my computer doesn't consistently bluescreen anymore.
Re:Network adminstrators are a real pain in the as (Score:2)
work done. Let's see, risk getting fired because I'm better at protecting my machine than the IT department & they are whining about me giving them the metaphorical finger, or get fired because I'm not getting any work done & all my boss hears is how the "IT department is full of incompetents".
I believe I'll take the risk over the sure thing. I even get to keep some of my self-respect.
Re:Network adminstrators are a real pain in the as (Score:2)
No, the problem _IS_ fixing the machine. I need the machine working to get my job done. The IT department is NOT fixing the machine, therefore to get my job done, I have to fix it myself. (The IT department wasn't exactly incompetent, just understaffed and overloaded. They were perfectly happy that somebody was able to fix their own machine.)
Idiot blanket policies not allowing anybody to alter anything on any machine just prevent _me_ (supposedly an expensive company resource) from getting my job done.
BTW, if you think that kind of policy is useful at stopping _malicious_ user activity, you're completely in dream land. Users have _PHYSICAL ACCESS_ to their machines. There's nothing that the IT dept can do to stop them from installing or using anything they want on their machines. A competent malicious user will do anything they want on that machine.
All the IT dept can do is try and limit the fallout from _accidental_ user mistakes, set up a good secure network architecture & provide some competent monitoring to try and discover if anything out of the ordinary is occurring.
One thing is for sure, if the IT department thinks that establishing complete control over everyone's machine is more important than actually doing the work that keeps the company alive, then everyone in the IT department needs to be fired. They've got to find the balanced solutions that provide decent security while minimizing the inconvenience to the people actually doing the work.
I'm not with that company anymore (left amicably), but when I was, it was doing just fine, thank you for the concern. Most of the reason is because most of the managers were more interested in helping us get our job done rather than thinking they had to maintain absolute control over all our actions.
The _best_ way to reduce the probability of malicious insider activity (or to increase the probability of discovering such malicious activity) is to make sure that everyone knows what everyone else is doing (transparency). (Not everyone in the company, obviously, but a wide enough circle of peers to provide decent self-monitoring.) This also has the added benefit of improved communication between team members & less unproductive screwing around (to avoid looking bad in front of your peers).
Someone completely missed the point. (Score:2)
Nathan
Info for commercial client (Score:3, Informative)
Nowadays you really have to be selective about what ruleset you use, logging too much isn't a good thing. This is part of the reason you need a qualified Intrusion analyst who have the expertise to determine which ruleset is useful and which isn't.
The worst thing that can happen (which does happen quite often) is after paying for the expensive distributed sensor IDS system, the logs are never processed or read by anyone.
As stated by the article, an IDS is suppose to log anomalies, that is any abnormal behaviour. But anomalies is only useful if you have a technical guy capable of analysing the traffic. In fact, I would rather have a faulty IDS system that misses packets than to have a good IDS system and all logs go down the drain at the end of the day.
Re:Info for commercial client (Score:2)
That's the beauty of Snort. If you use Snort as your IDS you have saved $20,000 that you can then spend on the services of someone that knows how to set Snort up. 20K buys a fair amount of consulting, and from the article it sounds like you are going to need expensive help no matter what IDS you use. It also appears that Snort is at least as robust and useful as the competition, so you might as well go for the least expensive option.
Unrealistic expectations (Score:2, Informative)
But Opus One's servers run OpenVMS, not Windows. Even though it is trivially easy to figure out what operating system a Web server uses, not one of the IDSs did so. Instead, they collectively generated literally millions of alarms about attacks that never happened.
That's an unrealistic expectation to place on an IDS, from the start. You get an IDS to log attack attempts first, not the attacks themselves - if the attacks are known (have signatures) your machines should be protected against them in the first place.
Prevention (Score:2, Interesting)
CycSecure [cyc.com] from Cycorp [cyc.com] the makers of OpenCyc [opencyc.org], the AI reasoning system, helps prevent attacks by using an AI engine to simulate attacks on your network to identify problems.
It's worth looking into.
still looking for a good free IDS (Score:2)
So, anything out there besides Snort? I just installed 1.8.7 on my Linux machine and was (un)pleasantly surprised to find that my (un)favorite Snort feature was brought back: random mysterious death of the snort daemon, with no logging or other diagnostic. But only in daemon mode, mind you, making the problem fun to debug.
Luckily it doesn't do it on FreeBSD which is where I really need it running, but it is really frustrating and doesn't instill a lot of confidence. Grumble grumble, bitch, moan, etc.
Re:still looking for a good free IDS (Score:1)
Different Experience (Score:2)
Re:still looking for a good free IDS (Score:2)
More recently I've run short 1.8x on a dual P3 w/ ECC Ram & hardware RAID for a year now, first on kernel 2.2.19, and more recently 2.4.17.
Not a single failure in 12+ months of either snort or the kernel. I suspect that the combination of a marginal backplane and IDE disk, coupled with the older kerenel probably caused the first machine's problems.
This review was poorly done (Score:5, Informative)
Actually, most of his posts tend to have interesting (and qualified) views on IDS> sure he is biased (a vendor) but his commentary is usually thought out and not vendor-ish.
> From: Andrew Plato [mailto:aplato@anitian.com]
> In-Reply-To:
> >http://www.nwfusion.com/techinsider/2002/0624sec
> Next time they should do RealSecure on one of my Win2k
> appliances.
No.
While it is true that the reviewer found a bug with the Nokia platform that
doesn't exist on Windows or Solaris, there wasn't anything especially wrong
with the platform.
The issue is that the reviewer was hostile towards IDSs. A customer wants
his product to work, so when they don't, they will keep calling tech support
until it does. Reviewers want the products not to work, so they will
construct the nature of the test in order to make sure this happens. The
reviewer, in this case, never called ISS; the first we heard about him was
at the end of this review, not at the first crash of the Nokia box.
RealSecure has a unique feature called "audit" events. These are supposed to
trigger on normal traffic, such as every HTTP GET request. These are useful
either to create audit trails, or as "anomaly detection": turn on all
audits, then turn off those that trigger normally on your network.
This reviewer turned on audit events, which flooded the console. The setup
that Nokia provided them (256-megs of RAM and a database limited to
2-gigabytes) is perfectly reasonable for the network they had, but not if
all audits were turned on. (The Nokia bug we fixed was related to the fact
that it didn't have enough memory to handle the event load). The reviewer
complained about an overload of false-positives and the box crashing, but
this was because the reviewer drove the product to the point where this
happened.
In truth, it isn't always obvious which of our events are "Audits" and which
ones are "Attacks"; this is an issue fixed in 7.0 of our product. I doubt
this would have made a difference in the review: 7.0 has a lot more audits,
allowing reviewers to overload the product even more if they desire.
Imagine a review of automobiles, where a reviewer grabs a Ford Explorer and
starts complaining that it still crashes, even with the Firestone tires
fixed. One might ask if the there is a problem with the Ford, but one might
also ask if the reviewer intentionally drove the car until it crashed. Next
time you are driving down the freeway, violently jerk the steering wheel all
the way to the right. If you survive, you'll understand what I mean.
I'm not saying the review is wrong. As the reviewer said, he learned a lot
about IDS during the process of reviewing these products. If you, too, don't
know much about IDS but are planning to install one, you will likely get the
same experience: being overwhelmed with alerts that are "false-positives",
and a general sense that the product isn't working. The first few months of
running the IDS are likely to be particularly frustrating. I suggest (a)
working with a consultant to tune the system, (b) working with the vendor's
support in order to get suggestions from them, (c) learning more about the
system. You are going to do (c) anyway: after a few months, you are going to
have learned a heck of a lot more about hacking and defense then you ever
dreamed possible. Read the review: take it with a grain of salt knowing the
reviewer wanted all the products to fail, but realize that this likely to be
your experience the first few months after installing the product, you are
likely to be overwhelmed with events and unlikely to be impressed during the
first few months of ownership.
Robert Graham
Chief Architect
Internet Security Systems
Re:This review was poorly done (Score:2, Interesting)
Who are these people? (Score:3, Interesting)
There are complaints about false-positives. I've played with Snort and there are ways to decrease the alarms put up. For example, a certain number of bum packets in a certain length of time. Not each and every packet.
Looking at the info at the bottom of the article, the authors should know what they are doing. But given the misrepresentations and inaccuracies releative to Snort, why should I believe their testing of non-Free software was any better?
Maybe it was eWeek or some similar publication about six or nine months ago did a similar check. The article was much longer and more in depth. They were also more appreciative of the programs out there. Now, some will say "just to appease their advertisers". Well... Maybe. But if that is the case, why did Snort get their nod as the best?
NIDS will not catch everything (Score:3, Insightful)
Signature based detection is only good if the attack utilize abnormal or unique traffic to exploit the vulnerability. It will not pick out attacks that uses normal common traffic (for obvious reasons).
IDS evasion techniques are also heavily worked on, plus all application level evasive techniques (eg. sidestep). We can just never be totally dependent on the NIDS for telling us intrusion has occured. It works for most attacks but will fail for some.
Re:NIDS will not catch everything (Score:3, Interesting)
The catch with such a system is that you have to be very careful about measuring what your "norm" is. If you capture a profile on a very noisy network, then a lot of potentially dangerous traffic could go unreported.
As with most things in security and system administration, your solution will only be as good as the person or persons who design, implement and support a system. If you don't have a trained analyst evaluating and tweaking your IDS solution, you're in trouble. There's currently no such thing as a true IDS appliance.
-buffy
Snort with ACID and MySQL (Score:5, Informative)
I've been running Snort for some time now, and love it! I'm using MySQL logging with ACID and ADODB under Apache for a front end. You just can't get any easier than fill-in-the-blanks SQL querys and intuitive packet layouts. Obviously, they want a strictly out-of-the-box product, and aren't willing to invest any time to make a solid IDS.
As to the false positives, I can concur that in the beginning it was daunting seeing the flood of alerts, but in time, you figure out what is normal and what is not. A little restructure, or a few rule overrides, or rewritten rules, and it's seamless. All it takes is time. This is akin to bitching that your fresh *nix install doesn't have everything just the way you want it, with all your custom apps and modules. You can easily reduce the number of snort alerts by passing the command option as:
snort -D -o -i eth2 -c
This (the -o) changes the rules order to Pass:Alert:Log killing home network normal activity before alert processing. It helps immensely!
Re:Snort with ACID and MySQL (Score:2)
I've read that sentence about six times and i still can't parse the end.
Managed Security Services (Score:2)
IDS only as good as the person reading it (Score:3, Interesting)
I have used Snort and Qualys (the high priced commercial outsourced IDS) and both give false positives quite frequently. However, proving they are false positives is part of the skill of a good human sysadmin. This is why IDSes will never replace a good sysadmin. He or she should be able to see the report and say without any shadow of doubt in his speech that any particular exploit shown by the IDS is a false postive or not.
This still means that each IDS has its good points; but why anyone would pay a lot for a system that cannot, by definition, be any better than an up to date Snort and human reading of the report, and knowing your network inside out. Those who buy into big commercial IDSes clearly are investing in software when they should be investing in people, training those people, and understanding those people. Too many middle managers think their sysadmin speaks a language they will never learn, and therefore need these things to understand. But a good sysadmin should try hard to find ways to communicate with them, and can if need be annotate a nice little Snort report and be done with it.
great testing (Score:3, Insightful)
Snort was off the air at the time of the attack because of misconfiguration on our part.
I don't have a lot of confidence in their results.
Re:great testing (Score:1)
too many false alarms ... (Score:1)
What? IDS vendors err on the side of caution!? (Score:2, Interesting)
No GUI for Snort? Acid! (Score:5, Informative)
Just say no. (Score:1, Informative)
1. Turn off unused services, and disable suid software. If this is not possible, then don't use the application.
2. Take steps to minimize compromise of at-risk services. Use chroot, aplication specific uid/gid.
3. Leran how to apply packet filters to ingress/egress points.
4. Apply patches for applications and O/S
5. Stop going to expensive, lame 'security' seminars.
fini!
IDS will someday approach NMS systems in uselessness
IDS Systems (Score:1)
As long as they all suck... (Score:2)
They were not properly configured and tuned! (Score:3, Informative)
Not how IDSes should be deployed... (Score:3, Informative)
Clueless reporters (Score:4, Insightful)
To wit, we've used all of the products they complain endlessly about, and all I can say is RTFM. All of the problems they encounter are either configuration problems or worse, PEBKAC.
If you want to really learn about IDS, and you don't have the budget to buy a commercial IDS, download a copy of snort [snort.org] and learn for yourself. This report strikes as the type of complaing you get from an IT customer that wants to buy a product, turn it on, never configure it and expect it to magically work.
Wow! What a revelation! You mean you have to know what you're doing and it actually takes time to configure these powerful tools?! In a word, DUH. IDS'es must be tuned. IT products must be configured properly. These things take time, sometimes a lot of time. The core of their complaints revolve around their inability to do either of these things well. Given that lots of people manage to do this effectively everyday and have been for years and years, we're left to conclude that these reporters were not up to the task. And here it is:
These folks actually expected NIDS to be plug-and-play, and thats what they seem upset about. NIDS are powerful sniffers, they need to be tuned, they need to be configured and yes, this IS an ongoing process - but they are not plug-and-play devices.Futhermore, all of IT is an ongoing process. A big, circular, ongoing process that requires competent personnel to manage, maintain, tune, test, patch, configure , deploy and yes, spend TIME on. Anyone that expects to be able to deploy close to a dozen different IDS products as plug and play devices into a production network in 90 days with questionable expertise is fooling themselves.
And then they say as much. Again, this report is total waste of time. Its overly sensationalized and stems from a lack of expertise on the products in question. Skip it, download snort or buy one of the commerical products, take a class, read a book and learn for yourself. You won't learn much from this report that common sense wouldn't have told you already.Re:Clueless reporters -- windoze users (Score:2)
So how come alarm companies exist? According to this logic, everyone should send a co-op to Home Depot and have them install the company alarm system. Maybe it'll take a day.
Where have I seen this attitude before? Oh yeah, the guys who turned on their new computer, plugged it in, and called it their web server. The ones that scan my boxes' port 80. The ones that are owned by Code Red, Nimda, and Klez. The "network administrators" with an MSCX certificate on the wall. The Windoze users.
Re:Clueless reporters (Score:2)
IDS is not a simple technology and anyone who expects to filter and analyze WAN traffic with the click of a mouse should scurry away with their MCSE between their legs. IDS takes tuning. Snort was originally written with the intent that its users would write their own rules to adapt to their own environments. (Apologies to Marty if I am not 100% accurate here.) Instead, so many excellent rules have been written and distributed that the work has been done already for most of us and the project has grown stable and accurate enough to go commercial - and compete impressively.
IDS is a science and an art, not a prepackaged app that you can stick a label on: "good", "fair", "sucks!". YMMV according to the time and research you invest in making the product work to its full potential.
I can't imagine using an IDS in this fashion (Score:2, Insightful)
Clear Winner: Snort (Score:2)
Snort and False Positives (Score:2, Informative)
IP address vs domain name (Score:3, Insightful)
This statement alone gives me reason to doubt their abilities. Combine this large amounts of traffic they have with a reverse-DNS lookup for each and you would have crippled your DNS servers.
This is configurable in Snort, and mentioned in detail in the Snort docs.
A good solution is to either dedicate a DNS server to the IDS box, or use a script/utility to do reverse-lookups on the items you are interested in. Not live, but when a human is looking at things.
They are right about one regard -- IDS configuration, monitoring and maintenance isn't for non-professionals. You *need* to know what you are doing.
Snort needs to be tuned... (Score:2, Interesting)
You need to take quite a while (based on your network) and OPTIMIZE your rules for a product like SNORT so that you are getting alerted to the types of things you want to know about while minimizing false positives.
It is pretty obvious that the tester didn't do that, and as a result he had nothing but bad things to say.
Let's let an experienced Snort user configure his conf files and then run the test again. I think you may find that the results are different.
Topology (Score:2, Insightful)
What?! (Score:2, Interesting)
Gee, I wonder if they should learn how to configure Snort before they test it.
STAT IDS Framework, one that actually works. (Score:2, Interesting)
You can find more info about STAT at
http://www.cs.ucsb.edu/~rsg/STAT/
STAT already has 2 extensions, NetSTAT and USTAT that watch for common network and unix-level exploits. Other projects include making java-level IDS's and mobile agent IDS's. It's a great project and it blows everything else out of the water. If you're dissatisfied with IDS's as they are, check out STAT.
Darned "false alarm" criterion. (Score:2, Funny)
Curses, foiled again! If it weren't for that pesky "not too many false alarms" requirement, I'd be able to create terrific security software. I'm picturing a system that generates a "WARNING: NETWORK SECURITY BREACH" message every five minutes, rain or shine. Keeps the sysadmins on their toes, and foils all network intruders who aren't fast enough to be in and out in five minutes.
Enterasys Dragon (Score:2)
Fine tuning... is the key to a succesfull IDS (Score:3, Informative)
If you know your web servers are running on OpenVMS, why do the even bother to log IIS directed attacks ? An IDS needs fine tuning to only log what is needed, or more likely what is NOT needed. And every so often, you need to check the logs and fine-tune the rules, and update the rules for new signatures.
To most security specialist, this is common sense...
What we need are intruder-locating systems (Score:2)
We'll probably see this: Your local intrusion detection system will connect to some big service, probably run by Verisign, which can query account info for major ISPs, SS7 phone call data, logs of recent packets, and related info, and then relays it to the National Infrastructure Protection Center, which will pick a few script kiddies every day and send out big guys in suits to bring them in. About once a month, a big, televised raid. A few months of that, and the attack rate will go way down.
A few thousand honeypots tied to such a system will put a big dent in the distributed attacks, too.
A Great front end for Snort - PureSecure (Score:2, Informative)
Things I like about it:
installation script is truly a marvel (installs snort, mysql, apache, perl modules)
Login screen/authentication
Big Brother like monitoring
File integrity checking
IDS using Snort sensors
free to use for non-commercial use
no I don't work for them I just like the software.
Where's the frigging methodology? (Score:3, Informative)
I don't trust any "real world" shootout that doesn't show how the IDS were plugged into the network, how they determined an attack, and other such key points. You can't just say "we plugged it in and nothing worked." IDS are much more complicated than that. How and where were they plugged into the network fabric? Were they using switch port mirroring or passive ethernet taps at the uplinks? How do they know these attacks happened without initiating them themselves? That last one is the biggest single problem with "real world" testing. Unless you're launching the attacks yourself you do not know, and unless I missed it, they were relying on attacks to just happen out of the blue.
Now, they do raise some important issues with the backend storage of events and the need for clarity with the false positives and false negatives, but many of these can be dealt with by implementation of a real-time security console that does some form of event correlation from multiple security devices that says "The IDS sees this as a problem, the firewall sees it as a problem, and the target sees it as a problem. It's probably a problem. RED ALERT!" It's a much more intelligent way of dealing with events than just forwarding each one to a pager.
We've always said security is a process which must be maintained and firewalls/IDS systems are not a panacea to network security. As someone who's been responsible for a large scale IDS roll-out at Enron Broadband Services, where we were ISS' single largest customer for RealSecure before everything went to hell, I feel confident saying that Network IDS is a very useful tool, provided you keep it out of the hands of people who have absolutely no clue what they're doing with it, like the three gentlemen who are responsible for this article.
JosephAlerts != Alarms (Score:2, Informative)
For example as an IDS admin I want to see alerts for failed telnet attempts internally. If it's 5 within one minute directed at one host, it's not a problem, probably just human error. If it's 1000 per minute, then I want to see an alarm, and get paged. The "reviewers" of the IDS products would have understood this, and taken it into consideration had they ever deployed an IDS in a production environment.
Alerts are not a bad thing even if they are false positives. False positive alarms are a bad thing, especially when they wake you up at 3AM. Again it comes back to tuning.
I can however verify what they experienced with ISS. At my last company we had ISS come out and install, configure and tune Real Secure. I can tell you from first hand experience, that the ISS products suck. I ended up installing Snort in order to keep the ISS products honest. My experience was that ISS had a nasty habit of dropping packets. Snort had no problem keeping up with our frational DS3 (30 Mb).
More recently I also had the priveledge of seeing Real Secure crash repeatedly on a Nokia 530 (installed and configured by ISS engineers) during an IDS pilot. We kicked ISS to the curb and are now in the process of installing Snort with support for ACID, and MySQL.
IMO the reviewers were idiots. They obviously didn't spend much time with any of the products. Which is unfortunate, because some of the good products got lumped in with some of the bad ones, which were all failed together because some reviewers obviously didn't RTFMs.
FYI there is also a really pretty GUI for Snort:
www.demarc.org
Well.. (Score:2)
An IDS is nothing more than something to alert you to any abnormal conditions. It's a tool to help filter out the noise and show you what you want to know.
IDS is not magic (Score:2)
Neither of them is flawless OR a complete solution.
NONE of them are going to be perfect out of the box. It takes skill and experience to know what's important and what's not. None of these IDS systems are going to catch the guy doing a slow map of your IP space. ALL of them will false positive on some things, and miss others completely.
It takes a human at the other end to look and see and decide what's a threat and what's not.
We won't go into the lack of information on how they configured each one of these IDS systems. I use snort at home on my LAN, but have worked with NFR and Cisco's Netranger - and each has it's advantages and disadvantages. If you're SERIOUS, you're combining something from a commercial heavy duty IDS, with Snort, with dumps from all your syslogs, some kind of host based IDS, and putting together the individual pieces to see what's happening. Then you might be able to detect a skilled intruder.
It's NOT for the faint of heart, the clueless, or, it seems, the media pundits.