SSL Optimization Over WAN Needs Scrutiny

coondoggie writes with word of the expansion of WAN optimization appliances to handle SSL traffic and the security concerns this brings up. From the article: "With more and more WAN optimization vendors extending their capabilities to include encrypted traffic, corporate IT executives have a decision to make: Should they trust the security these devices provide? Rather than passing through SSL sessions between clients and servers located in remote data centers, some WAN optimization gear can terminate the SSL sessions, shrink the traffic, and re-encrypt it for the next leg of the trip. These chains of encrypted sessions introduce potential vulnerabilities that different vendors address in different ways. SSL traffic represents a growing percentage of total traffic on WAN links, according to Forrester Research. So SSL support in WAN optimization appliances will become more important to businesses that want to keep traffic secure while minimizing the size of their WAN links."

Never trust an analyst

[Watson Ladd]

The cited analyst is confusing proxying conducted by someone the site designates vs. yourself. One doesn't let anyone do anything they can't already. The other means you have all your confidential information sitting on a device that is not engineered for efficency.

What about Akamai?

[Anonymous Coward]

What about Akamai's SSL encryption? http://www.akamai.com [akamai.com]?

Re:

[pipacs]

From TFA:

The fact that SSL sessions are being proxied shouldnt scare anyone away, says Joe Skorupa, an analyst with Gartner. Akamai, for instance, proxies SSL transactions - some of them involving e-commerce - for its customers, he says. Theres precedent for doing it and doing it where significant amounts of money are at risk

Let's just hope they are all good guys.

Re:

[Anonymous Coward]

If you read the fine print of SBC/ATT's "private WAN" contract, even for frame relay service, they have the right to put your traffic on the public internet w/o any encryption. Who knows how much they do.

Re:

[jafiwam]

I am tired of cheap ass providers that have the shittiest proxy server service on the damn planet letting my users think the server is "under construction" for two days because of their pile of shit proxy decided to poke while I was rebooting and is too niggardly to go and check for a new version quicker than 48 hours later.

Make your shit work right, and nobody will notice it, dumbfuck.

How bout them apples Mr. Too Chickenshit To Use A Real Name?

Re:

[AI0867]

what makes you believe those proxy servers actually listen to those headers? I've had proxies completely cache an interactive website, after seeing exactly the same data three times I noticed the timestamp was that of half an hour ago.

Right, what these devices actually do.

[Anonymous Coward]

You do realise it's simply not possible for ISPs to screw with SSL traffic? That's the whole point of this.

These products aren't used for Internet connections. They're used for either dedicated WAN links or for VPN links. Say you have an office out in timbucktoo and a head office. You put one of these on/instead of the the gateway router at the head office, and another at the same gateway at the branch (just inside your network) so all your traffic is going through them. (And these devices are designed to help get the best out of that private frame relay link you've got as well as VPNs.)

What do they do? They automatically cache repeated TCP traffic patterns against all traffic sent to and from the head office. (Basically a matched pair - when one end sends a particular packet the pair of devices has seen before, just the reference to that pattern is sent. Yes, they have to keep in synch and all that.)

See what that gets you. Transparent caching of all interal web browsing. FTP downloads. Transparent caching of file share traffic. IMAP traffic. (Imagine if you send the same 100 kb word attachment to every end user. Now imagine it only gets sent once, without any software seeing anything but normal, ordinary, IMAP.) It just recognises patterns in the TCP traffic, not caring what those patters actually represent. (Lower latency too - because that 100 kb word doc comes out of cache rather than across the line.)

Now SSL disables all this. Each SSL session is unique - the data patterns are low entrophy nearly random noise, from the point of view of these devices. The patterns are never repeated, and thus you don't get to cache anything.

You DO have the option, of course, doing a man-in-the-middle-attack, where you decrypt the traffic on the device, subject that to caching, and re-encrypt it once it leaves the other matched device (presumably the WAN-accelerator-to-WAN-accelrator link is also encrypted.).

BUT ISPs cannot do this. To pull off that M-i-t-m, they have two ways: [1] to screw around with your SSL system's root certificates. Corporates can do this, just deploy their own "trusted" CA. Piece of cake. ISPs can't. [2] Have the private keys for the destination server loaded into the device. Again, Corporates can (sometimes). Edge SSL applicance owned by the same organsiation as the end server can. ISPs, again, can't.

These devices are aimed at corporates, where the network and end-points are all owned by the company. Of course, if the wan-accelerator-to-wan-accelerator link isn't encrypted then you have an issue (can you trust the device designers? That's the POINT of this article!) If that fake CA private key got out, you've also got real issues too!

Re:

[TheLink]

Actually in theory your ISP could supply you with a fake root cert. Unless you do stuff like downloading your browser using a different ISP or a different channel, or compare notes with someone who has.

Re:

[petermgreen]

they could do that and they could also use the nasty flaw in IE that allows owners of versign certs to MITM ssl connections made by it provided you own a verisign issued certificate.

but people who share thier connections with broadband routers who plug in a PC with firefox already installed will not have any reason to run the ISPs software and hence will notice this and there isn't really anything they can do about it.

and it would only require one sufficiantly tech savy user using a browser that didn't have

Microsoft ISA Server

[Anonymous Coward]

Microsoft's ISA (Internet Security and Acceleration) Server has been providing functionality to accomplish just this (reverse proxy publishing of web servers, acting as the SSL endpoint, passing on unencrypted or re-ssl'd traffic to the webserver post-inspection of the http traffic) for quite some time.

Not only does it work very well for microsoft and non-microsoft webservers (and perform application-layer inspection of a number of other protocols), but neither ISA 2004 nor ISA 2006 has had a single exploit

Re:

[Anonymous Coward]

Nu vulnerabilities in ISA 2004 ?
One quick google and http://nvd.nist.gov/nvd.cfm?cvename=CVE-2006-7027 [nist.gov]

Re:

[jafiwam]

Yeah, except if you try to authenticate through it, it rebroadcasts the session and screws up a WebDav login.

Only solution, put in a complete bypass. Bypass so you can login. Great security that.

Apache has done this for a while...

[SuperBanana]

It'll also publish multiple sites (say, your /sharepoint, your /cacti, your /owa, and your /intranet sites) using path rules on the same IP address - one point of entry (or multiple points, with an array of ISA boxes), with a collection of disparate apps in your WAN / DMZ behind it

In other words, something Apache has done for more than half a decade?

Re:

[beuges]

Apache is a web server, ISA is a gateway server. The proper comparison would be apache to IIS, which has also done this for half a decade, if not longer.

Whiskey Tango Foxtrot?

[ewhac]

...some WAN optimization gear can terminate the SSL sessions, shrink the traffic, and re-encrypt it for the next leg of the trip.

Um... Isn't that the very definition of a man-in-the-middle attack?

Er, no. Thank you very much for your kind offer, but I would prefer my encrypted data were not "optimized" in this way.

Schwab

Re:Whiskey Tango Foxtrot?

[Professor_UNIX]

Er, no. Thank you very much for your kind offer, but I would prefer my encrypted data were not "optimized" in this way.

Thanks to Enron, many people don't have a choice in the matter anymore. Allowing encrypted connections out of your LAN without scanning the data included could open you up to law suits or criminal prosecution now. Expect that SSL termination will be a growing trend in the years to come as more and more companies come on board.

Re:

[IchBinEinPenguin]

Thanks to Enron ... Expect that SSL termination will be a growing trend in the years to come as more and more companies come on board.

Which, like most 'security' measures, is a great way to keep honest people honest.
But then, I guess, it's not about stopping the bad behaviour, it's about being able to deflect lawsuits by being able to prove that you put measure in place, however ineffective, to try to stop it.

Re:

[Anonymous Coward]

Um... Isn't that the very definition of a man-in-the-middle attack?

Er, no. Thank you very much for your kind offer, but I would prefer my encrypted data were not "optimized" in this way.

If you manage your network infrastructure devices and servers properly, this is no more risky than simply adding more webservers to your DMZ.

In fact, it probably makes your infrastructure as a whole more resilient, since your webservers are no longer directly receiving traffic from clients, and data can be sanity ch

Re:Whiskey Tango Foxtrot?

[Melkman]

If you are a network admin SSL traffic is a bitch. Normal web traffic can be scanned for malicious content with ICAP. With SSL traffic you're out of luck. Also a lot of remote access programs tunnel their traffic in an SSL connection over port 443 (cool service LogMeIn, NOT). That means if you allow encrypted web traffic you also allow people on the network to give control over their PC's to external people. I've dealed with both. Some moron downloaded an infected program of an https site, which was luckily caught by the local virus scanner. And some nitwit called the helpdesk of an ASP which took over teir desktop while he was having sensitive data on his screen. After these events we enabled SSL termination on the proxies and feed all SSL traffic to the ICAP scanner and disallow any traffic that doesn't strictly follow HTTP. If someone has traffic that must bypass these rules the can request an exception, usually this concerns shitty java applications.
that was about a year ago. We were already using Bluecoat proxies, formerly known as CacheFlow.

Re:

[Sloppy]

Some moron downloaded an infected program of an https site, which was luckily caught by the local virus scanner.

Which just goes to show that the approach of attempting to detect malware (as opposed to preventing it from being run, or running it harmlessly in a sandbox, etc) is a fundamentally wrong approach to malware. I'm glad that you saw the local scanner's success as "lucky" because luck is exactly what it is.

Re:

[Anonymous Coward]

It is a MITM. But I hate burst your bubble, but the majority of large ecommerce presences terminate SSL at hardware level appliances(SSL offloaders) and usually don't reencrypt it to the server.

Re:

[starfishsystems]

Um... Isn't that the very definition of a man-in-the-middle attack?

Good point.

It's not an attack but it is definitely man-in-the-middle. A server proxy could make good architectural sense, for example when the SSL proxy is a load balancer in front of a server farm. As the client, you can validate that the load balancer is what its certificate claims it to be, say www.foo.com.

What is privately architected beyond that, you don't know, and really shouldn't care. Just think of the whole thing as one big

Re:

[UnderCoverPenguin]

Note that the proxy can only succeed at representing itself as having the expected identity because it has been configured with the private key corresponding to that certificate and its identity. In other words, no random person can set up this man-in-the-middle.

Mostly true. Be aware that in a corporate environment, the IT department can install its own SSL CA certificates into the web browsers (and other applications). Once this is done, corporate proxy can masquerade as any entity the IT department chooses - up to and including auto-generation of SSL certificates, signed, of course, with the IT department's CA certificate.

This, of course, is no surprize.

Also, I assume that applications, including web browsers, could be configured to use a central CA certific

Re:

[starfishsystems]

Once this is done, corporate proxy can masquerade as any entity the IT department chooses

We have to be careful when talking about certificate identity. It's not the case that a proxy can ever "masquerade" as something else. By being configured with a given certificate and -- critically -- its corresponding private key, that proxy in identity terms becomes that other entity. As you say, this should be no surprise, but this is not familiar territory for many people, and it bears emphasis.

Often people m

Re:Don't trust SSL!

[modeless]

The protocol is not insecure; In order to enable the proxy to decrypt the SSL session, one of the endpoints must cooperate. It could work the same way with any of the protocols you mention.

Re:

[gweihir]

True, but in VPN and SSH no endpoint will usually cooperate. With SSL this seems to be assumed as something that should be allowed, and possibly without telling the user. Bad, bad design IMO and a fundamental flaw.

Re:

[hal9000(jr)]

Sure they if they are configured to do so. It;s not that hard and has been productied for years.

Nothing new, let's move along.

Re:

[modeless]

There are really no foolproof solutions; SSL's users can't judge the authenticity of certificates so asking them to verify anything isn't an option. Usable yet bulletproof security is still an unreachable holy grail. But I'd like to point out that you can still trust your SSL connection to Amazon as long as you really do trust all the people with certs in your trusted root list.

And I'd also like to point out that unless you manually verify the SSH keys using a seperate secure channel during your first con

Re:

[Beryllium Sphere(tm)]

Yes, but that cooperation may not be the result of informed consent. It's painfully easy for an untrusted application to stick a new root CA into IE's list, after which it can sign its own certificates and use them to intercept SSL. MarketScore, reportedly, did this.

Re:

[modeless]

You have a good point. Also, organizations that pre-load their own root certs on their computers get the power to intercept SSL connections made from those computers. I never really thought about it before, but that means if I'm doing online banking at work, the IT department could read it all. Theoretically.

Re:

[jsm300]

Yes, but this would be illegal (which might not stop them). Companies may have the right to monitor all traffic on their networks, read email, etc. They also certainly have the right to block all traffic to your bank. But for them to read your online banking transactions it would require them to forge your banks SSL certificate (i.e. they would need to create a certificate that claims it belongs to your bank, but they would sign it with the companies pre-loaded root cert). IANAL, but I'm fairly certain that

Re:

[Tony Hoyle]

The whole point of these devices is that *don't* need to forge the bank's SSL certificate - they're breaking the end to end nature of SSL and inserting a proxy inbetween that allows the admins to get at the banking data in plaintext.

You have no way of verifying it because the ability to verify the SSL certificate is taken away from you (every site returns the certificate of the proxy).

Yes reading such data would be actionable - as would reading most emails without explicit written consent. Hasn't stopped t

Re:

[it073281]

If you not trust SSL, how you keep security trade through SSL?

Re:Don't trust SSL!

[slamb]

The fact that this ''optimization'' is even possible, demonstrates that this protocol is insecure!

If they were able to do this without cooperation of the endpoints, it'd be a man-in-the-middle attack, and yes it would demonstrate that SSL is insecure.

The article's background information skips over an important point: the WAN accelerator device must be owned by the same group that runs the server. So this is for your data centers, not for your corporate headquarters.

I've never heard of this sort of device before, but I'll give my own background, filling in the gaps from their later information, my understanding of SSL, and a couple guesses.

The original (unencrypted) problem: Say you have a low-bandwidth link between a bunch of servers and clients. You want to make the best use of your link's limited bandwidth through compression (as well as QoS and such). In fact, you want to say "these next 426 bytes are the same as chunk 12345, which I've previously sent to some other client". The original setup looks like this:

Server <-slow-> Clients

The solution: obviously you can't say "just access chunk 12345" to client B if you sent chunk 12345 to client A. That's why they use a pair of these devices straddling the slow link to do this compression. They compress when sending and decompress when receiving. In the end, you still need to be able to send all the original bytes to each client. You might have many slow connections, but we'll say it's a "fast" link because in aggregate they're fast enough. So the setup looks like this:

Server <-raw/fast-> Accelerator.Server <-compressed/slow-> Accelerator.Client <-raw/fast-> Clients

Now, I'm not sure how common this setup actually is. Apparently you've cheaped out and gotten a T-1 or something across town, but you still have to pay for all the upstream bandwidth as the data leave your control or diverge. But let's just go with it.

Now add encryption. You could look for commonalities in the encrypted streams, but you're unlikely to find much. For the compression to be effective, it needs to work on unencrypted data. And you can't just forge a certificate on-the-fly to make proxying transparent, unless you've found a man-in-the-middle vulnerability in the SSL setup. Short of that, this is possible under one of two conditions: (1) the validators trust a CA certificate to which the device has a private key, so it can issue new certificates on the fly, or (2) the device has been preloaded with keys for every certificate it is expected to proxy.

The choice they've made seems to be this: you're assumed to control the servers but not the clients. So if you want to have an SSL connection to a given server, you need to load that server's key somewhere onto this pair of devices. (Preferably the Accelerator.Server for security.) That's what they're talking about in this paragraph:

In any SSL proxying architecture, the server must give up its certificates to at least one other device. In the cases of Blue Coat, Certeon and Riverbed, that device is a WAN optimizer that sits inside the data center where it is as physically secure as the server itself. ...

I'm not sure if they support SSL client authentication, but if they did, the choice is the same: load all the clients' keys onto the device or have your servers trust the device's CA.

huh?

[Doppler00]

"some WAN optimization gear can terminate the SSL sessions, shrink the traffic and re-encrypt it for the next leg of the trip."

Doesn't this completely defeat the point of SSL? An encrypted link from client to server? If you got to an SSL website through this WAN link, it's not going to be encrypted anymore unless you trust the keys from the WAN access point. So, unless each WAN has it's own trusted key through a signature authority this isn't going to work. Seems pretty stupid to me. Isn't SSL traffic alrea

Re:huh?

[flyingfsck]

It only works if your user name is Administrator and your password is 123... ;) This kind of proxy equipment needs the end points to co-operate. It cannot be done on just any ssh or stunnel connection. I don't really see the use for it. If your SSL connection is slow, then you have to adjust the cypher or something.

Re:

[rviana]

Imagine that you have 4000 branches all around the world. And imagine that the average latency across these links range from 10ms when its close, to over 400ms when its far. If you've ever tried to load a website with 30+ images/javascript/css with IE's measly 2 http simultaneous connections. ONLY then, will you realize how necessary this is. Because its either WAN optimization (more for latency reduction...not so much for bandwidth reduction), or local servers in 4000 branches.

Now throw in a little CIF

Re:

[jmauro]

Don't use SSL. Encryption and Decryption take time, there's nothing that can be done about it.

Re:

[Beryllium Sphere(tm)]

Nothing? Buy a crypto accelerator from nCipher or any other company that sells them. AES in smart implementations is in the neighborhood of 16 cycles per byte anyway.

Nothing can be trusted...

[antirelic]

Its a good thing that these devices can offer SSL, but it should only be talked about as "additional" security measures. There seems to be a trend in the IT world where one "new" security tool comes along and renders some "other" (other, not older) security tool/method discarded. These lessons have been learned, and shouldnt be repeated: Defense in Depth.

This is gross irresponsibility

[Animats]

Any administrator who puts in a security hole like that to get a minor reduction in bandwidth is grossly irresponsible. No device in the middle of the network should have the end to end decryption keys. Ever.

If you have too much traffic to your secure servers, take a look at what they're sending. Maybe the canned images can be moved to a non-secure server, where they can be cached locally. You're probably not actually sending huge volumes of secure data to a browser.

Or maybe you just need to find out who's downloading videos and stop them.

Re:

[hab136]

SSL is used for more than just browsers. Many business data feeds are XML over HTTP over SSL (I've seen this in growing use at financial institutions). Many oddball applications took their standard protocol and wrapped it in SSL because their clients demanded it.

Re:

[UnderCoverPenguin]

If you have too much traffic to your secure servers, take a look at what they're sending. Maybe the canned images can be moved to a non-secure server

In theory, this sounds good, however, many browsers put up a warning when a web page contains mixed "secure" and "unsecure" content. (I just checked this with FireFox 2.0.0.2 with a page on my internal server; I got such a warning.)

I hope such warnings will continue to be the default. As such, the work around is either reduce the number and size of images in your web page, and/or use SSL/TSL acceleration.

Re:

[VENONA]

Not sure what you meant by, "the middle of the network." A decent network design will likely have all of this happening on the DMZ. One positive thing about doing things this way (in addition to optimizing WAN traffic) would be that endpointing before the host allows you to place a network intrusion sensor on the wire.

If you have to comply with policies that require NIDS & retention of their logs, you don't have any other good option that won't load the host. That load will be disk I/O and it's associat

Legality

[digitalgimpus]

As I recall it's criminal to intercept financial communications without permission from the institution.

That means, if an employee at a company that uses a BlueCoat device were to say login to a bank... the sysadmin would technically be violating the law.... unless the bank gave explicit permission (most likely would need to be in writing).

Even if you were to ban online banking in the workplace, that wouldn't change the impact on the sysadmin's libility if a user broke the rules. Go ahead, fire the employee... they just let the bank know about the security breach, and who the "hacker" was.

Seems to me like the laws may need some adjusting.

And that's why I don't login to a bank from work. I know sysadmins in many companies steal SSN's, credit card #'s, etc. by using Timbuktu, VNC, etc. as well as sniffing packets. It's just what it is. Why should I put my identity at risk? I don't see a good reason. I wait until I'm home for that. It's one thing to read an email, it's another to jack by SSN.

Why is this news?

[todd3091]

I don't understand why this is being portrayed as something "new" to IT. I work for a medium sized company, and we have been doing this for nearly 5 years. There are a slew of vendors in this space including F5 Networks [f5.com], Citrix, Sun, and Microsoft. The technology works be installing the SSL certificate onto the appliance rather than the server. If designed correctly, this is no less secure than the conventional model, and can save significant processing load on web servers.

Not sure about over the WAN..

[_hAZE_]

I've actually been around SSL Accelerators for a few years now, working in the web hosting industry. The idea is pretty much the same; your customers connect to your website using https, except they're actually talking to an SSL Accelerator/load balancer device, not the actual web server. The SSL Accelerator holds the SSL certificate, decrypts the traffic, and sends it to the web server in plain text. The web server replies back in plain text, the SSL Accelerator encrypts it, and send it back to the customer's web browser. Being a device made specifically to handle this type of traffic, the SSL Accelerator can encrypt/decrypt a lot faster than the web server, leaving the web server to do it's task of serving up content without all the overhead involved with SSL.

Normally, the SSL Accelerators are in very close proximity to the web servers (both network and physical distance-wise). The thought of doing something similar with enterprise WAN traffic over long distances, however, is a bit more frightening. I guess like many other things out there, it all depends on who your vendors are and how you implement the solution. Just be careful.. the Internet can be a nasty place.

always adjust your password

[it073281]

If your SSL connection is slow, you better always adjust your password.

to keep your connection security.

SSL

[june_c21]

Areas of concern that SSL addresses: 1. Unauthorised access to cached data at each access point Problems that SSL poses: 1. Slower throughput due to encryption process