Follow Slashdot stories on Twitter

 


Forgot your password?
Close
typodupeerror
×
Security IT

Testing So-Called 'Unified Threat Managers' 98

Posted by Soulskill from the ain't-no-silver-bullet dept.
snydeq writes "The InfoWorld Test Center has released vulnerability testing results for four so-called 'unified threat managers' — single units that combine firewall, VPN, intrusion detection and prevention, anti-malware, anti-spam, and Web content filtering in lieu of a relay rack stuffed top to bottom with appliances. The lab threw nearly 600 exploits of known vulnerabilities in a wide range of popular OSes, applications, and protocols, and despite being designed to thwart such threats, the UTMs as a class allowed hundreds to pass through. Why did the UTMs miss so many exploits? A lack of horsepower to perform the necessary deep packet inspection under load is suspected, as the lab pushed the limits of each unit's throughput with legitimate traffic. 'The upshot is, although the vendors have packed these devices with additional gateway security functions, clearly many UTMs are still strictly firewalls at heart.'"
This discussion has been archived. No new comments can be posted.

Testing So-Called 'Unified Threat Managers' More

Testing So-Called 'Unified Threat Managers'

Comments Filter:

  • general purpose != good (Score:3, Insightful)

    by KingFeanor ( 950059 ) on Wednesday May 27, 2009 @11:35AM (#28111257) Homepage
    Is it possible that single purpose security applications and appliances do a better job? In combining make various technologies in one device, how watered down was each individual component?

    • Re:general purpose != good (Score:4, Insightful)

      by houstonbofh ( 602064 ) on Wednesday May 27, 2009 @11:59AM (#28111655)
      Part of the solution is the tool, and part is how you use it. The Pix firewall can be very secure, but it is very hard to configure correctly. So many people just opened it up, making it very insecure... If a unified tool can be more easily configured securely than many best of bread applications, it will produce a better result every time. It will also have better cross communication than other applications as it is designed that way, not tacked on to support many other things.

      Now could you personally out do that? Probably. Could your typical business person? Not likely...

      • Re: (Score:2)

        by rthille ( 8526 )

        PIX can also be very stupid. We had one which would drop a packet while rsync'ing email from one system to another because the packet looked like an exploit to the PIX.

        • Re: (Score:2)

          by Hyppy ( 74366 )
          False positives and false negatives are bound to occur. It's like a see-saw, and finding the right balance is tricky. Swinging back and forth between the two isn't nearly as effective as shrinking the fulcrum, but either way there will always be a few.

      • If a unified tool can be more easily configured securely than many best of bread applications

        Sounds like a half-baked idea ;-)

    • Re: (Score:3, Insightful)

      by fuzzyfuzzyfungus ( 1223518 )
      Depends. In practice, I strongly suspect that quality suffers pretty markedly in many cases. If "UTM" is the new hotness buzzword, every last maker of firewalls, IDSes, packet shapers, and god knows what else will be rushing to ship a "UTM". This will probably mean taking their existing product and hacking together, or aquiring, enough other functions to make it qualify. That will typically mean that the result is good at something, with a bunch of other near broken crap hanging around and eating CPU time.
      • I agree. When UTM isn't a buzzword and vendors have had a couple of release cycles to get their act together, get quality pieces put together and design a reasonable user interface, then these things may well be better than a mish mash of various components. There may be great benefits in time from gathering these things together, but we aren't there yet! And hopefully the vendors won't reduce the 1000 different options you had in the mish mash that perfectly served your business's needs with 10 pre conf

        • Re: (Score:3, Informative)

          by mvdwege ( 243851 )

          I have news for you: UTM is old news. Vendors have been selling this stuff for years already. And yes, the complaint remains: a mish-mash of badly integrated components that eat up a significant part of your performance.

          I admin these things for a living, and they're a pain. Their management interfaces suck, the false negative rate sucks, and turning on the various protection methods eats up to 80% of your bandwidth.

          Mart

      • This will probably mean taking their existing product and hacking together, or aquiring, enough other functions to make it qualify.

        Watchguard's been doing that since 1996! I really do like the Firebox Core after using it for a year or two, but man, you can tell that they've taken work from multiple unrelated development projects and strung them all together with a "manager" that simply launches bulky, inconsistently designed apps which then in turn launch more inconsistent smaller apps. Great feature set and very fast, though, once you get past the decent learning curve and annoying support contracts.

    • Re:general purpose != good (Score:4, Interesting)

      by agristin ( 750854 ) on Wednesday May 27, 2009 @06:12PM (#28116875) Journal

      UTM is a crock. It loads multiple single purpose apps on to a general purpose computing device and then tries to do it quickly.

      The best thing in this field I've seen recently is Palo Alto Networks firewall (www.paloaltonetworks.com).

      Knows the applications, even web apps. It can tell the difference between Gmail and gchat. Bittorent and wow torrent patching. Can do user based rules when integrated with AD. And can proxy SSL to look in the SSL stream if necessary. Malware blocking, url filtering via subscription. Because ports or protocols != applications and IP address != user anymore.

  • Strange (Score:5, Insightful)

    by Reason58 ( 775044 ) on Wednesday May 27, 2009 @11:36AM (#28111273)

    Focusing on doing one thing well yields better results than trying to do everything. Who'd have thought?

    • that's why I bought 10 pieces of hardware before connecting my computer in my home office to the internet. I think the issue ends up being scalability vs. robustness

    • Your UTM gave me a UTI.

      Thanks for sharing the virus!

      Actually I think the idea is not to bundle all apps into one but allow data communication between them to be better. I think it could be communication pathway would be more of permiable barriers which get smaller down the line. Firewall to AV to Spy/Greyware to deep scan heuristics on the hard drives.

      • So I can let my rootkit directly interface with all products trough standardized interfaces? Sweet! I take ten!

    • Re:Strange (Score:4, Insightful)

      by Rene S. Hollan ( 1943 ) on Wednesday May 27, 2009 @12:30PM (#28112159)

      Disclaimer: I am employed by one of the companies represented in the trial but do not speak for them.

      Unfortunately, security is a process and affects all interacting systems. Placing them under one umbrella in a UTM device allows security issues to be dealt with in one place. This is better than having "something else" misconfigured somewhere undo all the efforts one has made in a particular place.

      Yes, by layering SPAM filtering, virus scanning, and application protocol validation, one can achieve the same effect, and each appliance can excel in it's area, but this comes at the complexity of having to configure many things independently (not "atomic security changes" spanning multiple issies), adds to complexity (the bane of security), and may give rise to an "end run" if these units run in parallel, instead of sequentially (which yields latency issues).

      The bottom line is that the market likes the convenience of unified threat management, and the price to be paid is generally not quality but performance.

      • The bottom line is that the market likes the convenience of unified threat management, and the price to be paid is generally not quality but performance.

        I dunno.. TFS said it did a pretty crap job keeping things out... I'd call that a cost in quality.

        • Re: (Score:3, Insightful)

          by Rene S. Hollan ( 1943 )

          The bottom line is that the market likes the convenience of unified threat management, and the price to be paid is generally not quality but performance.

          I dunno.. TFS said it did a pretty crap job keeping things out... I'd call that a cost in quality.

          That's a different problem, since signatures can be updated over time. But, now that you mention it, space constraints in a UTM do limit the size of signature databases it can hold.

          The answer is, of course, to get a bigger UTM, and address performance with clustered UTMs.

          Sadly, one does not have to be perfect, one just has to be better, for some definition of better, than the competition.

        • Did it say, that groups of separate items before that did it better? You know: Everything is defined by its relation.

      • Well I hope you work for Sonicwall. All the rest did terribly.

      • "Unfortunately, security is a process and affects all interacting systems. Placing them under one umbrella in a UTM device allows security issues to be dealt with in one place."

        Unfortunately, putting all those interacting systems under one umbrella in a UTM device allows security to be tresspassed by jumping out just one choke point. Security by depth==0.

  • Comment removed (Score:3, Insightful)

    by account_deleted ( 4530225 ) on Wednesday May 27, 2009 @11:40AM (#28111337)
    Comment removed based on user account deletion

    • Re: (Score:3, Informative)

      by John Hasler ( 414242 )

      > It would have been nice to see how the ASA5500 series appliances stood up to the test.

      If you send them one I'm sure they'll test it. It appears that Cisco wouldn't.

      • Re:No Cisco product? (Score:4, Interesting)

        by houstonbofh ( 602064 ) on Wednesday May 27, 2009 @12:04PM (#28111731)

        > It would have been nice to see how the ASA5500 series appliances stood up to the test.

        If you send them one I'm sure they'll test it. It appears that Cisco wouldn't.

        They also didn't include Untangle, http://www.untangle.com/ [untangle.com] which is available free, and is a direct competitor to the things tested. So it might be other reasons...

        • Re: (Score:2)

          by evol262 ( 721773 )
          Or IPcop, pfSense, m0n0wall, Shorewall, etc. Why? Because they're not appliances.

          • Re: (Score:3, Insightful)

            by houstonbofh ( 602064 )

            Or IPcop, pfSense, m0n0wall, Shorewall, etc. Why? Because they're not appliances.

            monowall is not a UTM, it is a firewall. I am a dev on it, and I should know.
            pfSense is also not a UTM, but it has a lot of plugins that can get close. But since it is a lot of plugins, it is not really "Unifiied."
            Untangle, and both of the above are available as supported appliances, or installable on standard x86 hardware, or appliance like hardware.
            I have not used IPcop or shorewall, so I can't speak on them.

            • pfSense is also not a UTM, but it has a lot of plugins that can get close. But since it is a lot of plugins, it is not really "Unifiied."

              I tried shopping around with various free firewalls/UTMs when I wanted to try something different then pfSense for SOHO installs. I ended up finding out that things like Endian, Untangle, etc. all are lacking the freedom that pfSense does. I'll stick with pfSense, thanks.

              • I'll stick with pfSense, thanks.

                I'll second that emotion.

                For a while, I whished for a layer 7 filter for pfsense but in the end, using squid + squiguard eliminated almost all unauthorized net access (p2p, im, sending of zombie generated spam).

                I still believe the best policy is to have a talk with the users about proper net usage and the consequences of not following guidelines, but there will always be someone who thinks he can get away with it and get everyone (mostly me) in trouble.

              • I keep all three (Untangle, m0n0wall, and pfSense) in my toolbox. In some cases I have used 2 of the above.

        • They also didn't include Untangle, http://www.untangle.com/ [untangle.com] which is available free, and is a direct competitor to the things tested.

          Free with an asterisk on it. It seems if your needs go beyond the very basic, you have to pay for the professional version.

          According to their website, creating different policies for specific groups of users or time-based is not available in the free version. Nor is wan failover.

          I'm not against paying for the product, it seems quite capable and the $250 a year subscription i

        • I'd vote for http://www.clarkconnect.com/ [clarkconnect.com]. They've got a nice set of features.

      • Re: (Score:2)

        by Amouth ( 879122 )

        eh.. it's one of the why bothers?

        when your looking at a purchase price that high.. people arn't exactly looking for 2 page review articals .. for a 200$ graphics card sure.. but dropping 10-30k .. you don't care about the 1-2 page articals..

        they through 600 things at it.. out of how many? how did they pick them? also the "as configured" and the options they had set
        what are they? where is teh indepth..

        if i'm going to send someone a 10-30k peice of equipment to review and put in a head to head.. you bett

        • Re:No Cisco product? (Score:4, Insightful)

          by Hyppy ( 74366 ) on Wednesday May 27, 2009 @12:26PM (#28112083)
          Could you point us to something with more in-depth information, by all means. All we can find is marketing propaganda from Cisco and Checkpoint. Unbiased, timely reviews with real-world information like this are far and few between.

          • Unbiased reviews do exist, but they are generally paid for by someone. One example would be a document that was a good 200 pages long, pitting various log aggregation and correlation devices/software against each other (Netforensics and Arsight to name 2). It was extremely thorough, and useful, but was done by a consulting/contracting company (with no vested interest in any of the products or organizations) for a large Federal/DoD entity. The damn thing was wrapped in so many NDA's that no one outside t

          • Re: (Score:2)

            by Amouth ( 879122 )

            can i point you to an in-depth review of UTM's - no i'm sorry but i havn't run into nore have i looked for one.

            Other stuff yes.. but i don't focus on UTM's

          • Re:No Cisco product? (Score:5, Insightful)

            by vlm ( 69642 ) on Wednesday May 27, 2009 @01:04PM (#28112629)

            Could you point us to something with more in-depth information, by all means.

            Your interpretation was backwards. He's looking for less because it's expensive.

            When purchasing a $200 graphics card in a corporate environment, the technical staff will read 200 page technical documents, search google for hours, write reports, run simulations, justify the upfront cost vs long term labor savings, basically spend at least a grand or two of labor costs to pick the best $200 card.

            However, when purchasing a $30K "buzzword of the month" the decision will be made at a high level by a manager whom is proud of being non-technical based on:
            1) What they saw on CSI and/or 24 last night, or maybe Obama's latest speech.
            2) Whom has the scariest marketing material (buy this expensive magic widget or you be p0wned)
            3) How much he enjoyed the sporting event the sales exec took him to, or how much he enjoyed the sales exec in general.
            4) The cheapest, or the first one he saw in a magazine, or perhaps a brand that will offend one of his enemies (you know, like he hates the guy who happens to love Cisco products, so if the enemy of my enemy is my friend, then ...)

            • You were modded funny, but you should have been modded insightful. This happens all the time, and constantly, depending on the organization.

              It also seems the bigger the company is, the more vulnerable they are to this kind of thing.

              Though, I do prefer to think it happens because of smooth sales pitches and multi-thousand dollar "business trips" to Tahiti that do the trick. Mostly because I'd like to be there someday, though I probably never will. Heh.

        • I can vouch that at least Sonicwall will let you evaluate their firewall for free before you choose to purchse. Barracuda Networks also does this and it's an incredibly great policy as you get to play with the device to find out if it's too clunky for your purposes.

          I do find it interesting that Cisco wasn't added to the mix but as another poster probably said, this was based on units available for review and Cisco is usually pretty tight lipped about a lot of their products. In one year of looking for WAN

          • Re: (Score:3, Informative)

            by John Hasler ( 414242 )

            > Of course, these days most people don't perform due diligence...

            They never did, and it made IBM billions.

          • Re: (Score:1)

            by Atticka ( 175794 )

            You can download the Astaro software free on their website (limited IP's and concurrent connections) and will send a demo/eval unit on request as well.

    • They don't do well I'm sure. That being said, I'm surprised they don't have samples from Cisco, Checkpoint, or Juniper - the three market leaders. Even if the companies don't provide freebies, they should have bought a few and tested them.

    • Re: (Score:2)

      by Hyppy ( 74366 )
      Exactly. Cisco's ASA and Checkpoint's UTM products are on the short list for us. Having a solid review of the difference between those two would be far better than picking 4 quasi-no-name brands.

      • "Nobody ever got fired for buying Cisco", right?

      • SonicWall is no where close to "No-name", but otherwise yeah. I'm a bit shocked that Cisco didn't provide a box for them to test. Info World isn't exactly a huge security publication, but it's a half-decent generalist magazine. People looking for UTMs tend to be generalist network/systems types in small to mid-sized companies, not security specialists in large ones.

  • I used to love Sonicwall (Score:4, Interesting)

    by C_Kode ( 102755 ) on Wednesday May 27, 2009 @11:51AM (#28111513) Journal

    I used to be a big SonicWall fan, until I joined a company that required IM messaging and used Vonage. Sonicwall causes a bunch of issues with AIM's protocol. IM will go into a blackhole, a user cannot connect, etc. We were using them at the small remote offices, but we replaced them with Juniper SSGs. The Vonage and AIM issues vanished once we switched over.

    • Isn't Juniper quite a bit more expensive than Sonicwall? Also the AIM thing doesn't surprise me as that is probably something many companies want to (for some pretty good reasons) block.

      • Re: (Score:3, Informative)

        by C_Kode ( 102755 )

        An Sonicwall TZ190 starts around $500 and an SSG5 can be had for about $500. They are comparable products. This is the base router without the annual subscription for filtering and virus type scanning extras that they both support, but are unnesessary for use.

      • Reaoons being? Well, for AIM... I can understand that. But for XMPP/Jabber? That's as stupid as blocking e-mail or phone. Why not seal the doors, shut all windows, and put it all in a bunker under the sea. :P

    • That's funny, Sonicwall is heralded as one of the best firewalls for VOIP support these days. How long ago was this? As an admin that deployed Asterisk company-wide using Sonicwalls as head-ends with VPN tunnels to remote locations and zero issues handling any voip traffic.

      I think you'll find things have changed dramatically and that Sonicwall is much cheaper than the same level UTM from Juniper.

      The thing that surprised me was the disparity between Sonicwall versus the other provides as it was an entire s

    • I liked Sonicwall (I run 3060s on my networks), but they really pissed me off last fall when all security services stopped because Sonicwall's license activation servers went down for a day (I just renewed the licenses about a month earlier) - apparently if they don't see the activation servers they immediately stop. I'm looking at alternatives now....
    • You're right, Sonicwalls can cause issues with SIP (e.g., Vonage) but so will any firewall you drop between IP telephony devices and the internet. I can't think of a single device that will work out of the box with that, including Netscreens. Not that I'm a big fan of Sonicwall either, but usually you have to fiddle with their stupid ALGs and NAT features to get SIP to work properly -- and once you do, it works pretty well. At least as far as SIP-based VoIP is concerned.

      I realise that's of little use t

  • Re: (Score:2, Interesting)

    by account_deleted ( 4530225 )
    Comment removed based on user account deletion

    • The biggest problem with most open source offerings, is lack of support. Businesses want support, and while yes some of those offerings may have support (I have not looked), the quality is most likely no where near close to what pure commercial entities can offer.

  • Testing Criteria? (Score:1, Interesting)

    by Anonymous Coward
    Tests like this usually favor the company that supplied the criteria and/or funding for the test. The results are exactly what you can expect. I'm sure that the other 3 companies tested could have supplied a criteria that favors them as well.
  • No, its not shocking that a tool for the job beats a "jack of al trades" - its shocking that the jack still missed third or more exploits. Hundreds out of 600? The odds are less that the janitor will break the server. ... heres the kicker, its happened before.
  • Having used Sonicwall products in the past, I can believe the results. They weren't the models tested but they were fairly effective for their price and performed well for a fairly small environment (around 100 or so employees). Sourcefire has some nice stuff as well. I'm sure other posters much more experienced with hobbitmon can chime in on the configuration and deployment of that but from what I've seen it was a nice component of home-built threat managers that also had snort and open-source firewalls on
  • Comment removed based on user account deletion

    • Talk about a dangerous power hungry scenario. At least use two servers for a highly availably solution man. Otherwise the setup is like something I did before I got funding for a proper Sonicwall. Maintaining a beast with that many different products is a pain. I'll give you an example. VPN Client wouldn't connect through the whole rig. Opening a port on the firewall is just the beginning, you need to whitelist the IPS, create a proxy routing rule and then hopefully all will work.

      Of course a rig like that

  • It seems to me - and the headline implies this - that a "Unified Threat Manager" is a firewall that has had Marketing's claws in it.

    As Bill always said [youtube.com], "If you work in marketing, kill yourself."

  • Flawed by Design. (Score:5, Insightful)

    by canipeal ( 1063334 ) on Wednesday May 27, 2009 @12:44PM (#28112347)
    The notion of having a single point of failure "security" device contradicts one of the primary foundations of security principle: Defense In Depth. Multiple layers of security is essential in safe guarding your systems, placing them all one one unit is nothing short of moronic.

    • Re: (Score:3, Insightful)

      by ZouPrime ( 460611 )

      Defense in depth refers to the principle of having multiple, overlapping security controls. For example, I've seen some companies use dual-firewall configurations where they will use two different brands of firewall. Or they will use a main network firewall as well as host-based software ones. So if one control fail the other is there to protect the asset.

      This has nothing to do with UTM, who are about hosting *complementary* controls on the same device. In this case, there is a real benefit in term of manag

      • Re: (Score:2, Interesting)

        by jonnyt886 ( 1252670 )
        Multiple security controls, yes, but these must be independent.

        If I have a firewall and an IDS on the same machine, and someone exploits a hole in the TCP stack or the IDS to get local root/admin priviledges, they then have control of not only the firewall but also the IDS. If I have two separate machines, a firewall and an IDS, if one gets compromised it does not affect the other.

        Thinking about it, the way to get around it in the case of a UTM is to use VMs for each task, but that will have a hit o

        • Re: (Score:2, Interesting)

          by jonnyt886 ( 1252670 )
          Er, I should add that I totally agree with your point about the ease of management - this is definitely a benefit, particularly in smaller businesses (the cost of a UTM is also lower than that of separate IDS/firewall/anti-virus/etc appliances).

          My point was just that from the technical perspective is isn't optimal. Realisticly, it is a good compromise for those who can't afford/don't need anything better.

        • Re: (Score:2, Insightful)

          by ZouPrime ( 460611 )

          True... but that's not "defense in depth", that's "not having a single point of failure".

          I agree that one big box to do everything has its issues. It's certainly not acceptable for corporations. But I think the cost/benefit is worthwile for a lot of small business who most of the time don't have shit (although this is getting less and less true).

          It's a bit like those Linksys routers: sure they sucks, but they are so cheap and so commonly available, and so better than being only jacked right in a modem, they

  • Neglected to test (Score:3, Insightful)

    by Runaway1956 ( 1322357 ) on Wednesday May 27, 2009 @12:48PM (#28112415) Homepage Journal

    They should have used a control for this test. Put each of these unified conglomerations up against one good Sysadmin with a clue.

    No one tool will ever be "THE Solution". No matter how many doodads are attached to a Swiss knife, some sack of warm tissue has to fire a few synapsis to put the knife to use. If the sack of warm tissue is lacking in the synapse department, he fails.

  • Is when the products that are being used to protect your network, themselves have vulnerabilities.

    I will use a large, very large company as an example. They make AV, they make IDS's (although crappy), they make firewalls, and they sell smaller stuff to the general public... It starts with a giant S....

    Long story short, a few years back they had a vulnerability in the way their stack did deep packet inspection, this particular piece of code was shared across their entire product line. Well, their all in on

  • Unified Threat Management is a dead end concept. We've been there and done that and we left it in the past.

    With disaster recovery concepts, decentralized administration on the rise again, and cloud computing we once again come full circle to the whole reason we left mainframes for client server architecture.

    "Who Watches the Watchman" is a line that comes to mind. The IDS should be keeping tabs on the Firewall, not part of the firewall. TRON should be an independent keeping tabs on the MCP not part of the MC

    • 3 separate realms.

      Policy to define what's allowed (you haz a policy, whether it is written down or even thought about).

      Enforcement of that policy. FW, IPS, application fw. The higher in the stack the fw goes, the closer it should be in the net topology to the target it defends.

      Audit the enforcement of that policy. IDS, stats, flow.

      And rather than tie everything together, how about focus on the 3-4 sources that really kick ass? FW logs are not useful. Focus on what your targets are doing, not what the mi

  • UTM = Universally Targeted Machine

    So much from learning from the phrase "all your eggs in one basket..."

  • Where was TippingPoint? (Score:3, Interesting)

    by c_g_hills ( 110430 ) <chaz@@@chaz6...com> on Wednesday May 27, 2009 @06:44PM (#28117171) Homepage Journal
    I do not think much of a UTM test that does not include any products from TippingPoint, the current market leader.

Slashdot Top Deals

Aren't you glad you're not getting all the government you pay for now?

Close