Marriott IT Exec Shares Network Horror Story

alphadogg writes "Neil Schubert is only partly kidding when he calls Marriott International's move toward a converged network a horror story. 'I'm here to tell you a terrifying tale of network design, support and administration,' he said at an IT conference in Boston, referring to a major bandwidth crunch caused by guests wielding Slingboxes and other network devices that overran the hotel chain's outdated network. 'One of the things we've learned about our guest networks is we have one of the most foreign, hostile environments known to man in the network administration world ... I can take 100,000 customers a night on that infrastructure and we actually have less incidents of harm than we do on our corporate back-office infrastructure.'"

Where's the horror?

[SilverJets]

Marriott has a network and customers use it. Marriott realizes the network is overwhelmed by the customer use and is now upgrading it.

So where's the horror? Or is this just Marriott's way of advertising their new network?

Re:Where's the horror?

[luvirini]

It is a "horror story" because the network manager was not prepared for the customers to actually USE the service...

Re:

[superphreak]

I didn't see much horror or much story.

Sub-headline: "Hotel chain wants each property to have one network for guests, staff" Maybe that's it?

Re:

[Bastard of Subhumani]

You must be new here.

Re:

[superphreak]

no, not really...

Re:

[dwater]

Of course that's not an accurate description. It was that he was not prepared for them to use it to the extent they did - he wasn't prepared for the degree of success that occurred.

On that note, I wonder what turns a healthy network into a broken one? Is there something different between a broken network and one that's just very slow?

It reminds me of the problems we had in Asia (I'm in Beijing) earlier this year due to the earthquake in Taiwan. Network congestion was so bad that we figured using a 56K modem

Re:

[Lehk228]

if you did a 56k dial in to a modem bank at your destination you would have, if it was 56k to a local ISP you would have been just as fucked

Re:

[dwater]

Well der, of course I meant a modem on the other side of the blockage, else there wouldn't any point at all.
I was thinking of something along the lines of an AOL or earthlink account, or just have someone hook up a modem at our US office...

Re:

[rbarreira]

With a 56k modem, you have guaranteed bandwidth between you and the ISP, not between you and the internet.

Re:

[dwater]

Yeah, so?

If the ISP is on the other side of the blockage, it effectively gets me onto the other side of the blockage, no?

Re:

[magores]

...Totally off topic post here....

Dwater

I'm also in Beijing. Let's grab a beer sometime.

We can discuss proxys, etc. Write it off as a business expense. ... Off topic post ends now...

Re:

[dwater]

Well, my mum told me I shouldn't arrange to meet people off the net'...but, er, maybe...

Re:

[Peter La Casse]

Is there something different between a broken network and one that's just very slow?

One way for a network to be broken by design is if the total capacity decreases as the utilization increases; in other words, past a certain point, adding more load decreases the total throughput of the network.

Re:

[dwater]

> Is there something different between a broken network and one that's just very slow?

One thing that has since occurred to me is wrt the various time outs that are set as default. For example, how long will a computer wait for a response from a DNS server? I guess those timeout values might set a minimum service level, or mean that some services should be given priority else the network would just not work.

Anything other than DNS?

Re:

[SharpFang]

Yes. Because the network was not overengineered and supported the expected traffic, not maximum possible traffic. And then the ad came. I work at one of bigger portals, and when the Business Unit is to comission a TV/frontpage/other media advertisement of certain service, they are required to give a 2-week notice to the IT, and the IT is free to veto the ad in case the infrastructure is not ready for the extra traffic. If an ad gets aired without IT's knowledge, and as result a service goes down, the BU is

Come on!

[Anonymous Coward]

For running this slashvertisement, Zonk's getting one years free accommodation from the Marriott chain, so what's the problem?

Can't an "Editor" graft in peace?

Re:

[edittard]

Can't an "Editor" graft in peace?

In some parts of England, "graft" means hard work rather than the modern American meaning relating to corruption. This could easily lead to confusion and ambiguity, and yet in this case it doesn't. Puzzling, most puzzling.

Re:The Horror

[eln]

Sounds like a lack of capacity planning. If the marketroids were planning a huge marketing push, the IT manager should have been given the resources to increase network capacity prior to that push. If the IT manager was given proper warning and funding prior to the marketing campaign, it's his fault for not effectively utilizing it. Either way, the problem could have been avoided with proper management.

Re:

[Kyojin]

Whoever modded this "insightful" needs to get a clue. The article doesn't even have 3 pages, it has 2. Also the only mention of "Linux" is in the ads around the story, not in either of the two pages of the story.

hurm?

[spazmonkey]

So what was the point of that article again? I must have missed it. Perhaps the PR flak who subbed it could explain it to me. I want that two minutes of my life back now /.

Re:hurm?

[CrazyJim1]

And here I was hoping the comments would be exciting to make up for my boredom reading that article.

Re:hurm?

[WarlockD]

Agreed. Whats the freaking point? The article is a fluff piece as it doesn't even describe what, if any, the problems they had to overcome. I rather have it explain WHY the hotel customer network was safer than there internal network than just it be said:P

Marriott a sponsor of Slashdot?

Re:

[DonChron]

This is one of the worst networking articles I've ever read. There is nothing remotely interesting or informative in those two pages. There's not enough detail to offer any insight into networks, project management, system design or anything else. Why?

Why? Why? Why? Why? Why? Why? Why? Why? Why? Why? Why? Why? Why? Why? Why?

yep!

[PaulBu]

Trying to parse it on lexical level is OK, moving to syntax is a bit challenging (have to make some assumptions!), but then on semantic level (taking above-mentioned assumptions into account) -- IT JUST DOES NOT PARSE!!! :) There is a feling of some text semi-randomly generated but not that smart an AI... (in other words, reminds me of spam!)

On which network he could accomodate 100,000 customers, the one before the great unification, or the one after? Which network gives him his headaches?

Now, I'm curious t

Re:

[Angostura]

the thing I took away from this, is that even some well educated people don't know the difference between "less" and "fewer". The rest of the article didn't really match up to that exciting insight.

Re:

[Midnight Thunder]

One of the points was that their current network implementation sucked. They have multiple networks, each handling a different type of medium, be it telephone, computer data and TV. This makes for huge complexity when administering the networks. Their current data network was not designed in an optimal manner and quickly got over run. Their plan is to provide one IP network for all data types (voice, fax, data and TV), instead of dedicated networks, and then have a box in each room which would provide an in

Re:

[NateTech]

So an old company with lots of "legacy" shit has to re-design their bad network.

I'm still trying to figure out why that's even slightly interesting on a Geek News website.

If I'd have known, I could have had that guy write some stupid articles about everyplace older than 5 years old that I've ever worked for.

weilding slingboxes?

[ChiChiCuervo]

I just get this mental image I'm not going to be able to shake....

"Some call it a slingbox, I call it appleTV. nnnngggggghhhh"

One lesson from the article...

[BrowserCapsGuy]

There needs to be better coordination between marketing and IT. IT had no idea marketing was running commercials showing customers using all this high-bandwidth stuff so there's no way IT could be prepared for it. Imagine 160 customers just trying to view websites on one DSL line! I admire this guy for his honesty if nothing else. He'll probably catch hell for it from his superiors!

Re:One lesson from the article...

[timmarhy]

that's what marketing do dumby! they make outrageous claims then handball it to technical, and when technical can't make the impossible happen marketing make it look like the techies failed. marketing will then tweak their bullshit slightly to cover their own ass's and make it look like they saved the day.

Re:

[adamruck]

You just described the entire IT industry :)

Re:

[chthon]

Agreed. +Infinity Insightful. (Hmmm, is it firefox which does not support &8734; ?)

Re:

[Kjella]

that's what marketing do dumby! they make outrageous claims then handball it to technical, and when technical can't make the impossible happen marketing make it look like the techies failed. marketing will then tweak their bullshit slightly to cover their own ass's and make it look like they saved the day

Marketing aren't half bad, they haven't got a clue what they're talking about and answer "yes" to most questions when they have no idea or not. Most have a fairly relaxed attitude to that and know that thei

When evaluating marketing

[sjames]

Whenever evaluating marketing, the first and foremost thing to keep in mind is that it is their JOB to successfully pass frosted dog turds off as wedding cake. No matter who they work for, the number one thing they will market is THEMSELVES. Once they successfully market themselves to their employer, they may (or may not) actually market the product. After all, they don't HAVE to make the product look good every time, just convince the employer that they did!

Re:

[Orleron]

Does this crap actually happen in real companies like Marriott?? I thought that only my fellow marketing and IT people in the biomedical industry were that stupid.

Re:

[ls -la]

Never underestimate the stupidity of a CEO who is trying to save money by cutting things he has no clue about.

Re:

[Anonymous Coward]

i've met many CEO's in my time. without fail, they have all been weasles with no clue how their business runs. even the ones that founded it.

i currently work for a multi billion $ venture and the CEO is required to sign off on outside contractors HOTEL STAYS. i shit you not. he will usually approve anything you slap infront of him, but sometimes he must get this urge to interfer and ask a bunch of fucking stupid questions and attempt to derail it.

Re:

[Wolfrider]

Parent +4 Insightful ;-)

Biggest lesson learned......

[Nick Driver]

...is that the worst threats to your network do not necessarily come from outside.... they almost always come from your very own moronic employees.

Re:

[sjaaklaan.com]

Are they charging for Internet access? In Europe, in most you have to pay a large sum of money just to get enough bandwidth to check your mail.

Re:

[MCZapf]

In general, I've found that expensive resort or business hotels charge for Internet access, but budget hotels do not - even when the parent company is the same (in this case, Marriot).

This is more typical than horrifying to me...

[kungfoolery]

The business units of most organizations typically make promises to their customers without comprehending or even considering the IT implications. Account Executive to customer: "Sure! We can provide you and your thousands of users seamless B2B connections from your network to ours wirelessly from any global location!" Account Executive to IT department: "Ok, you guys figure out how to do that."

Re:This is more typical than horrifying to me...

[jd]

Yep, seen that at plenty of places. At one place, the IT department clearly spelled out what was possible and in what timeframe. Marketing oversold anyway. However, this isn't just IT, and that's the scary part of it. Airlines routinely overbook aircraft. Package holiday companies sell hotel rooms for hotels that haven't yet been built. You too can place advance orders for books that haven't been written or buy computers that have not yet been built. (One company I know has made substantial money off a computer for which even the prototype does not yet work.)

The problem with marketing is that it is not about selling what you have, it's about selling what the person wants to buy. If there's a discrepancy between the two, well, that's not your department. Complaints is three doors down, across the hall from Abuse.

I have to note with aircraft overbooking

[Sycraft-fu]

That the reason they do it is because statistically a percentage of people don't make a given flight. Sometimes it ends up in conflicts, especially since they tend to err on the side of being full rather than no conflicts, but there is good reason to do it. If 100% of people who wanted on a flight showed up, they'd never overbook. However about 10-15% of people cancel their reservations or otherwise fail to show. Thus it makes sense to overbook the aircraft.

Re:overbooking/non-refundable

[evought]

That the reason they do it is because statistically a percentage of people don't make a given flight. Sometimes it ends up in conflicts, especially since they tend to err on the side of being full rather than no conflicts, but there is good reason to do it. If 100% of people who wanted on a flight showed up, they'd never overbook. However about 10-15% of people cancel their reservations or otherwise fail to show. Thus it makes sense to overbook the aircraft.

That might almost be an excuse except that they sell non-refundable "you die, you fly" tickets, supposedly for the exact same reason. Those empty seats are already paid for. They are trying to make additional money off of them at the cost of double booking. Like most businesses, they get you both ways and make you deal with the mistakes and inconvenience.

Re:

[MCZapf]

It's not like refundable tickets aren't available anymore. You can still get them if you want. Business travelers often do. There's some on every flight and it's probably they that are most the unpredictable. The number of non-refundable tickets sold is a factor the airlines use to decide how much to overbook.

There was a NYTimes article on this recently: http://travel.nytimes.com/2007/05/30/business/30bu mp.html [nytimes.com]. It does, in fact, cost more to the airline to have empty seats than to pay off passengers that

Re:

[ryanov]

My most recent trip home from New Orleans on Continental was overbooked and they were offering initially $300 and then $400 to get off and take a flight 3 hours later.

I personally didn't take it because I was feeling ill and had a doctor's appointment that evening that I'd have missed on the later flight... but the bump would have paid for two round trip tickets at the price I paid.

(wouldn't you know, my train broke down on the way to the doctor)

Re:

[Monkeybaister]

The problem with marketing is that it is not about selling what you have, it's about selling what the person wants to buy. If there's a discrepancy between the two, well, that's not your department. Complaints is three doors down, across the hall from Abuse.

Don't forget the legal department.

There's this legal thing, promissory estoppel [wikipedia.org] that has come over to the US from English common law, and hasn't been taken out. Where I work I think we got another company's legal department to reel in a sales represen

Re:

[OnTheEdge]

I've seen this as well - but on a much smaller scale...just look at the average resume. We all do it, it's economics. Speaking of which, it still amazes me how so many posters here slam some company for trying to find cheap labor (and yes, I know the cheapest is not necessarily the best value), or do something else to save money, while at the same time mimicking that same behavior themselves. I know the most "loyalty" I show to any store comes from laziness, not true loyalty. My shopping loyalty typically g

Re:

[Dadoo]

Ok, you guys figure out how to do that.

To be honest, I've never had a problem with that. For me (and most of the people I've worked with), it only becomes a problem when they add, "Oh, and we need it tomorrow", or "There's no extra money on the budget to spend on this", or both.

Re:

[NerveGas]

Even better is the "We just sent out 200,000 flyers with (X) promised, we need you to make it happen."

Anybody have a "buzzword bingo" card for this?

[MonkeyBoyo]

I may have understood some of the article but it seems to have been mainly an exercise in trotting out what somebody thinks are the most trendy buzzwords.

Re:

[Zantetsuken]

Whats there to understand? Basically he just said that Marriot is bumping up their network backbone to better hand VOIP and Slingbox [wikipedia.org] type devices (basically broadcasts and manipulates your tv via the net, and you can watch it via the Slingbox app on your laptop or supported cell-phone), along with adding docking stations and cables to hook up electronics and laptops and whatnot to your Marriot room's A/V system and network connection...

Really there weren't any buzzwords at all in it - Slingbox isn't a bu

Unrealistic convergence plan

[Bruce Perens]

Placing all access points in a single telecom closet for what are generally rather spacious properties requires that 2.4 GHz signals be carried through coaxial cable that is very lossy at that frequency - it might be fair to expect up to 90% of the signal to be lost in the wire. There is an FCC limit on the transmitted power, and even if you manage to boost that at the antenna you will be boosting noise as well. And this attenuation and noise will of course hurt receiving too. This is in general going to result in lower wireless quality than desired, much lower than possible.

Instead, get zero-management access points that do not do NAT, routing, etc, and treat them just like antennas once you set the SSID. Do the protocol processing in the telecom closet with a higher grade of hardware than consumer equipment. Cache DNS and web transfers there. Work with Slingbox to engineer channel aggregation with multicasting that bypasses the home units while transmitting the same programming, because so many of those folks are watching the same sports game. I can think of some interesting approaches to the possible legal issues with Slingbox aggregating channels, no doubt they can as well. Can an in-house video alternative be made as attractive as Slingbox? That's another solution.

Bruce

Re:

[Beatlebum]

>> Work with Slingbox to engineer channel aggregation with multicasting that bypasses the home units while transmitting the same programming

Oh sure, Slingbox is going to switch from being P2P to a multicast provider of copyrighted content in order to let Marriott go cheap on bandwidth.

>> Can an in-house video alternative be made as attractive as Slingbox? That's another solution.

This is non-trivial and does not solve the bandwidth problem completely.

Along the same lines I have a solution:

Cache t

Re:

[Bruce Perens]

Oh sure, Slingbox is going to switch from being P2P to a multicast provider of copyrighted content in order to let Marriott go cheap on bandwidth.

I am well aware of the legal issues and did not mean to pose them as being simple. Hotel chains have lucrative relationships with sports enterprises.

Cache the Internet on a local server and connected each room via a quad fiber ATM connection.

This proposal not solve problem, which is limtied by the pipe to the building rather than the pipe within the building. Ca

Re:Unrealistic convergence plan

[Leto-II]

Slingbox is not P2P at all. You stream content from your computer back home to wherever you want to watch it.

Re:

[dangitman]

Slingbox is not P2P at all. You stream content from your computer back home to wherever you want to watch it.

How is that not P2P? Your computer at home is one peer, and the computer in the hotel room is the other peer.

Re:

[Franso6]

In that definition everything that's unicast is Peer to Peer.
The term is nowadays used for a form of content distribution that's based on using end-user-owned, non-specialised machines working collectively.
What you're referring to is a client-server model.. Usually considered as more or less the opposite of a P2P model.
I agree that the naming peer to peer is unfortunate, though.

Re:

[dangitman]

In that definition everything that's unicast is Peer to Peer.

Correct, so what's wrong with that definition?

The term is nowadays used for a form of content distribution that's based on using end-user-owned, non-specialised machines working collectively.

If they want it to mean that, then why not use a term that actually specifies that, rather than something vague?

What you're referring to is a client-server model.. Usually considered as more or less the opposite of a P2P model.

But then isn't P2P also a client-server model, just with lots of different servers and clients?

Re:

[cyclocommuter]

More accurately, you don't need a computer at home to stream remotely from a Slingbox to your hotel room anywhere in the world... it is a server unto its own that hooks up to your cable/satellite set top box, Tivo, or all of those.

Re:

[billsoxs]

Cache the Internet on a local server and connected each room via a quad fiber ATM connection.

So if I understand correctly, you are suggesting that Marriott needs to build a Google installation for each one of its hotels. Do you suppose Mattiott would be able to stay in business if it does this? When are they going to hold the auction on used bits left over from the former Marriott Corporation? I'd like to get a rack full of computers and raid drives... - or - Maybe I am misreading what you are suggestin

Re:

[Anonymous Coward]

You clearly misread. You'd need *two* racks full of computers and raid drives to cache "the internet".

Re:

[afidel]

Cisco has a solution that puts the PHY on the antenna out in the premises and just send back the partially processed signal to the MAC built into either an AP or linecard.

Re:

[TooMuchToDo]

Can an in-house video alternative be made as attractive as Slingbox? That's another solution.

Yes, it can. It just needs to be cheaper to build/purchase it then bandwidth.

Re:

[scottv67]

Placing all access points in a single telecom closet for what are generally rather spacious properties requires that 2.4 GHz signals be carried through coaxial cable that is very lossy at that frequency - it might be fair to expect up to 90% of the signal to be lost in the wire. There is an FCC limit on the transmitted power, and even if you manage to boost that at the antenna you will be boosting noise as well. And this attenuation and noise will of course hurt receiving too. This is in general going to re

Re:

[Bruce Perens]

I wasn't talking about leaky coax, but TV coax - RG-6, -57, -59, etc. Some of the "antenna distribution systems" not only use that, they might even use the same wire that is used to distribute television around the building. Obviously a solution that does not require wire pulling is attractive to these properties, that's why so many of the hotel-room wired Ethernet devices are really a sort of short-haul DSL piggybacked on the room phone line. I bet they feed the WAPs with that pseudo-DSL, too.

When you sai

Re:

[Wolfrider]

--Bruce, sounds like you should send a friendly Note to Teh Brasshats before they screw up their implementation again. ;-) // YA, SRSLY :)

Astroturf?

[UESMark]

This seems like a thinly veiled ad for Marriot internet access.

Re:

[owlstead]

"Plans for hotels include a distributed antenna system with a converged voice and data network, having all access points located in the telecom closet, up to four antennas used for coverage, and up to four access points used for bandwidth."

Dunno, if it is an astroturf, then they probably don't want the author to write articles like this. Sounds like a clueless person on the technical side of things.

Can anyone please explain what this means? I presume these antenna's are fixed to the access points, so of cou

Re:

[Peter La Casse]

In the same sentence, the article refers to both an "access point" and a "distributed antenna system," indicating that when they say "access point" they're probably not talking about the $2.99 device from CompUSA.

It seems like when they say "access point" they mean "connection to another network outside the hotel". Currently, data might come in a variety of places -- maybe they have a directional antenna on the roof for each local HD tower, plus a satellite dish, plus an internet connection, plus a cable

Re:

[scottv67]

Actually, that section of the article sounds a LOT like the solution from Mobile Access:

Plans for hotels include a distributed antenna system with a converged voice and data network, having all access points located in the telecom closet, up to four antennas used for coverage, and up to four access points used for bandwidth.

The wireless access points live in the wiring closet and then Mobile Access' antenna infrastructure is used to get the signal out closer to the wireless clients. The wireless acc

Re:

[scottv67]

Can anyone please explain what this means? I presume these antenna's are fixed to the access points, so of course there are four of both. And then there are four access points all in the same closet? What's the use of *that*? Or are there four locations, each with its own antenna, with four access points each? Is that even possible?

Instead of the "classic" model where the antenna and the wireless access point are separated by a few feet of coax and the pair are installed in the ceiling near the clients,

Networked Door Locks ?

[The Media Mechanic]

They want to put door locks on the hotel network ??? This has got to be the smartest idea I've ever heard. Because we all know that plugging Ethernet cord into something instantly gives you more protection and security!

Job security

[adarklite]

At first I thought Marriott was going to fire this guy. Then I realized that he just gave himself job security. What network admin will want to deal with this particular nightmare? Not I.

Microsoft update + Public Network = Instant DOS

[James_G]

While I was at the 2005 MysqlUC in Santa Clara, Microsoft put out one of their massive patch releases. Despite a large number of people running OS X or some Linux variant, there were still enough geeks in the hotel running Windows that they essentially DoS'ed the hotel's internet connection for about 2 days. I went down to reception at the time to find out if it was just the wireless, but the front desk people were similarly frustrated (they shared the same connection). Packet loss was at about 99%.

That was probably just a T1 or something, but still, pretty funny. I wonder if Microsoft realises the damage potential..

Re:

[imac.usr]

What kind of fucking idiot updates their laptop during a conference? You wait to do that shit until you get back home in case it screws your machine.

Re:Microsoft update + Public Network = Instant DOS

[Nezer]

What kind of fucking idiot updates their laptop during a conference? You wait to do that shit until you get back home in case it screws your machine.

Let's see... At a conference your computer is connected to hostile networks nearly all the time. Depending on the conference, there are potentially a LOT of people that know about 0-day exploits and might want to try something dumb.

I dunno. I can see your argument but there may be very good reasons to patch your system ASAP. I used to work in an environment where NOTHING got patched because they were afraid of fucking-up production services. I argued until I was blue in the face that we needed to do something and have a plan for deploying patches. I even went so far as to make proposals explaining the benefits, the risks, and the costs. No one would listen to me because I was a UNIX admin on a Windows team. Eventually I was let go and no one else took-up my cause (perhaps the cause was a large reason I was let go). No one on the team, except me, felt that there was any risk because the networks were "isolated" behind three layers of firewalls. About three months after I left some nasty work managed to find it's way into this "isolated" network and wreak much more havoc than we ever could have patching the damned servers.

I know that this isn't exactly the same thing as updating your laptop while on the road, but sometimes the updates are just worth the risk.

Perhaps the hotels should consider a caching proxy for just these sorts of events. Let the first user wait for the the download to come down the pipe and everyone else can leach from the proxy.

Re:

[Red Flayer]

No one would listen to me because I was a UNIX admin on a Windows team. Eventually I was let go and no one else took-up my cause (perhaps the cause was a large reason I was let go).

You seem a bit bitter.

About three months after I left some nasty work managed to find it's way into this "isolated" network and wreak much more havoc than we ever could have patching the damned servers.

Hmm. Let me get this straight, it "managed" to find its way onto the network about three months after you were let go? Sounds s

Re:

[Nurgled]

At least by default, Windows Update will download updates in the background and then offer to install them once the download has completed.

Re:

[simong]

You do realise that in any conference, there is at least one fucking idiot? It's a rule of life.

Lucky

[Esc7]

"The good news is bandwidth will never be more expensive and it will never be slower than it is today."

Someone doesn't have cable!

Ironically...

[EReidJ]

With some irony, I am reading this story from a Marriott hotel room at the Marriott Boca Raton. I've had mostly no problem with their services, but here are a few things I would call interesting: * My laptop can often see multiple nodes, some very fast, some blazingly slow. If you stay in a Marriott, try out the different nodes you can see. * Some Marriott properties give free Internet access, some cost $10/day. I wish Marriott would be consistent across all their properties. * If you stay in a Marriott that does charge for access, as for a low floor. This is because often there's a single wireless connection in the business center that is free, but other access points cost money. So if you can get a room near the business center, you'll be able to hook up to that one for free. * Finally, I've never had a problem with BitTorrent uploading at any Marriott property. I don't know if they leave all their ports open or what, but I traditionally leave my uploading port for BitTorrent open on 34567, and I've never had a problem with a torrent at a Marriott.

One quote stands out...

[idontgno]

I can take 100,000 customers a night on that infrastructure and we actually have less incidents of harm than we do on our corporate back-office infrastructure.'"

That says less about the robustness of the hospitality net and more about the poor planning and administration of the enterprise intranet.