Forgot your password?
typodupeerror
IT Technology

As Hurricane Season Looms, It's Disaster-Preparedness Time 117

Posted by Soulskill
from the make-sure-the-anti-zombie-alarms-have-batteries dept.
Nerval's Lobster writes "In 2012, hurricane Sandy smacked the East Coast and did significant damage to New Jersey, New York City, and other areas. Flooding knocked many datacenters in Manhattan offline, temporarily taking down a whole lot of Websites in the process. Now that fall (and the tail end of hurricane season) is upon us again, any number of datacenters and IT companies are probably looking over their disaster-preparedness checklists in case another storm comes barreling through. Ryan Murphey, who heads up design and capacity planning for PEER 1 (which kept its Manhattan datacenter running during the storm by creating a makeshift bucket brigade to carry fuel to the building's 17th floor), offers a couple basic tips for possibly mitigating damage from the next infrastructure-crushing disaster, including setting up emergency response teams and arranging contracts for maintenance and fuel in advance."
This discussion has been archived. No new comments can be posted.

As Hurricane Season Looms, It's Disaster-Preparedness Time

Comments Filter:
  • by edibobb (113989) on Friday September 27, 2013 @05:13PM (#44975257) Homepage
    Looming? Most North American hurricanes this year have already happened! Is this some kind of spam?
    • by Anonymous Coward on Friday September 27, 2013 @05:29PM (#44975421)

      Since NY was affected by a hurricane in the fall, hurricane season now officially starts in the fall. Everyone else be damned.

      • Makes sense. [samefacts.com]

      • by PopeRatzo (965947)

        Since NY was affected by a hurricane in the fall, hurricane season now officially starts in the fall. Everyone else be damned.

        Since the Northeast is by far the most densely populated section of the US, they kind of have a point.

        I think only something like 1/3 of the population lives West of the Mississippi, even though it's over 2/3 of the land area.

        The financial center of the US is in New York (and one of the financial centers of the world) and to a great extent, the cultural center is there too.

        I know the

        • by Maxo-Texas (864189) on Friday September 27, 2013 @07:56PM (#44976401)

          I think you are ignoring some critical points.

          Like the population affected by gulf and southern atlantic hurricanes.
          You know... texas with 25 million (up by 5 million since 2000 alone) vs new york with 19 million ( essentially no growth since 2000).

          Another 16 million in Florida (which has been hit by almost every major hurricane at some point). 10's of millions more in loiusiana, alabama, mississippi, the carolina's, and arkansas.

          You have an argument culturally (tho california has come on a bit with it's 33 million people). Nothing is replacing Broadway and off Broadway.

          New York is visible financially (tho it's slowly being routed around due to cost issues). The largest component of job loss recently has been financial jobs (20,000 in 2009 alone).

          The hurricane season officially began on June 1 and will end on November 30.
          It was predicted to be a rough season.

          So far, it's a dud.

          In the 20th century, of the 64 major hurricanes to hit the US, 51 hit in september and august.

          • The NYC metro area alone has 25+ million people, the most densely populated region in the country. Add in the rest of the Northeast Corridor and it doesn't take much for even a small sized storm to cause a lot of damage. While Texas and Florida have the numbers, they are relatively spread out and a hurricane rarely effects the entire population at once.

            All of the major tropical storms that have hit NJ since I have been alive have struck between the end of September or in October, its when the ocean is the w

          • When "official" assholes predicted a far worse than average hurricane season this year, I snapped a picture of Good Morning America's graphic.

            May 24, 2013k
            GMA 2013 Hurricane Season
            NOAA
            13-20 Named Storms
            (Normal 12)
            7-11 Hurricanes
            (Normal 6)
            3-6 Major Hurricanes
            (Normal 3)

            These clowns have no real scientific knowledge about even basic things like regression to the mean, or a simple grasp of chaos theory and statistic. They are acting (probably deliberately, the alternative that they are stupid is also unsettlin

            • by tzog (83893)

              Your post is wrong and offensive, good job.

              Regression to the mean is all nice and good, it may be applicable for the next couple decades lumped together, but it's not of any use to attempt to predict the outcome of a single season.

              It is not just NOAA that predicted an above average season, everyone that I'm aware of that issues a seasonal forecast did. Those guys are scientist, not politician. Their funding doesn't depend on AGW, and if it did, they are aware that there is no consensus on the impact of glob

              • Regression to the mean is all nice and good, it may be applicable for the next couple decades lumped together, but it's not of any use to attempt to predict the outcome of a single season.

                That was, of course, my point. They are overestimating other effects, and treating it in such a way as to garner headlines, like the proverbial "the end is nigh!" types, or psychics making failfull yearly predictions that people rarely check up on the next year

                I could be wrong about any given year's prediction vs. theirs,

                • Remember I am not predicting hurricane cycles. I am making a meta-prediction, that their predictions, for reasons listed, are not reliable.

          • by PopeRatzo (965947)

            New York is visible financially (tho it's slowly being routed around due to cost issues).

            I think the concentration of the financial industry in New York is actually increasing, due, paradoxically, to technology.

            In the 20th century, of the 64 major hurricanes to hit the US, 51 hit in september and august.

            I understand that. My comment was addressing the point that in an important way, there's a good reason that hurricanes that hit New York have a greater impact on America than those that hit lower on the coa

            • The damage to Florida alone from just Hurricane Andrew was catastrophic and slightly larger than Hurricane Sandy. And that was with an atypically small wind field.

              The damage to Louisiana alone was *triple* that to New York ($75 billion vs $25 billion) and just about killed the city which is a major jazz hub and a major cultural center in the country.

              I know you think New York is more important... and it used to be.... but there are many cultural, financial, educational, and large population centers around t

        • by msauve (701917)
          "The Northeast, and specifically NY, is mainstream America."

          +1 Funny.
    • by ackthpt (218170)

      Summer is the Pre-Season - Early Autumn is the real season, depending upon how moody Dame Nature feels.

    • Well, so far, we've had two category ones and 7 tropical storms. In early August, NOAA predicted 13-19 named storms, including 6-9 hurricanes, 3-5 of them major.[1]

      so, expect a big pileup in October and November. Maybe even a storm to ring in the new year

      • Well, so far, we've had two category ones and 7 tropical storms. In early August, NOAA predicted 13-19 named storms, including 6-9 hurricanes, 3-5 of them major.[1]

        so, expect a big pileup in October and November. Maybe even a storm to ring in the new year

        Or not.

        The traditional formulas for statistical prediction of hurricanes have been patently knocked for a loop. So we can say that the statistics indicate that we've some catching up to do, but then again, where I live, we're double-plus overdue over multiple years.

        So I won't "expect" a pileup or even a big storm. Though I will be prepared. Because frankly we don't have a clue anymore.

    • Looming? Most North American hurricanes this year have already happened! Is this some kind of spam?

      So, you can predict the future? Hurricane season runs from June to November. September is the most active month for hurricanes in North America. Still, I would not call it "looming". That's the kind of thing one would say in April or May. That's when we start thinking about them on the Gulf Coast.

  • How many emergenices has lower Manhattan had in the past 12 years? Stupid is, is stupid does. MOVE!

    • First, calculate the chances of a disaster vs. where you can actually find customers and IT talent. There are not too many highly trained systems administrators in rural New Mexico. Also, can you get enough customers to pay the bills for your multi-million dollar co-location facility? Seems to me the location is worth the risk. If you are well prepared, you can mitigate the damages. It is more of a risk to have a crappy business location.
      • by mythosaz (572040) on Friday September 27, 2013 @06:28PM (#44975899)

        While rural New Mexico might be extreme, there's a reason that places like Phoenix AZ are filled with new data centers and skilled IT staff. Need a disaster recovery site? Put it here, or in Nevada. Flood? Hurricane? Earthquake? Tornados? You must be kidding.

        Sure, it's not the tech density of San Jose, but it's kitten-safe from a disaster standpoint.

        • by khallow (566160)
          They still have flooding and water shortage issues. But flooding is something you can get mostly right by proper site selection (you still have to worry that there are dependencies outside your control such as the power company putting your substation in a flood-prone area).

          Water shortages aren't usually considered a data center problem, but if you're doing something like using water-based cooling towers to remove heat, that can impact your business.

          But having said that, these are issues that Californ
        • Flood? Hurricane? Earthquake? Tornados? You must be kidding.

          Yes, earthquakes. No matter where you go, it's earthquake country, at least on a long enough time scale. Remember, the Northridge Earthquake of 1994 [wikipedia.org] was on a previously unknown fault, so the fact that there's no record of earthquakes in the area doesn't mean as much as you'd like it to.
          • by sjames (1099)

            When is the last time you heard of an earthquake in Az? Now compare to Ca.

            Yes, in theory they could have one, but it is extremely unlikely. If they did, it would be strange enough that geologists would still be talking about it centuries from now.

        • Flood? [...] Tornados?

          Yes. [azcentral.com] and Yes. [azfamily.com]

      • by sjames (1099)

        Just how many SAs do you need onsite? These days, even commodity servers have a BMC built in so they can be managed from anywhere. You do need a few local to handle lost connectivity to the management network and such, but you can probably get half from anywhere else in the U.S.

        I find it really funny how many datacenters pay a huge premium on space in order to be in an 'it' location when the whole premise of the net is communication over a distance. Really what's needed is somewhere with stable infrastructu

  • WTF? (Score:5, Insightful)

    by scheme (19778) on Friday September 27, 2013 @05:16PM (#44975271)

    Hurricane season has been going on for a few months now. Why the hell would a data center or organization review their hurricane/storm related disaster checklists now instead of, oh, you know, before hurricane season? Any organization complacent and negligent enough to wait till the end of the hurricane season to review/correct their checklists probably isn't going to actually care about the checklist anyway.

  • Oh cool (Score:5, Funny)

    by the eric conspiracy (20178) on Friday September 27, 2013 @05:18PM (#44975299)

    I was looking for a reason to lay in a supply of scotch. I already have the generator and transfer switch.

    Arberg here we come!

  • >> which kept its Manhattan datacenter running during the storm by creating a makeshift bucket brigade to carry fuel to the building's 17th floor

    No fire code violations there, right? I'd love to be an attorney near this one. "So, you burned down the building trying to keep a couple of servers running, when you could have just co-located your equipment in a smarter place (like anyone who knows what they're doing would have done)."

    • by Shoten (260439) on Friday September 27, 2013 @05:54PM (#44975657)

      >> which kept its Manhattan datacenter running during the storm by creating a makeshift bucket brigade to carry fuel to the building's 17th floor

      No fire code violations there, right? I'd love to be an attorney near this one. "So, you burned down the building trying to keep a couple of servers running, when you could have just co-located your equipment in a smarter place (like anyone who knows what they're doing would have done)."

      When you're doing things like HFT, colocation in a different geographical area is a non-option. They're eking out every microsecond they can, even going so far as to use microwave for communications when possible instead of fiber simply for the reduced latency. Putting the servers way the hell out somewhere away from Wall Street is not helpful.

      Surely you don't think that these companies have large datacenter operations in Manhattan just for the cheap real estate?

      • by BitZtream (692029)

        Putting Wall Street on Wall Street is the first example of STUPID in that equation, the rest follows quickly from there.

        Every company I've ever worked with that has datacenter in manhattan JUST so they can say 'We have a datacenter in XXX' as a bragging right. (Note, never worked with financial traders)

        Yes, SOME companies want the shortest length of transit to the markets, MOST of those data centers are marketing fluff for ignorant companies who think showing how wasteful you can be with money is impressiv

      • by RockDoctor (15477)

        When you're doing things like HFT, colocation in a different geographical area is a non-option. [Reason] Putting the servers way the hell out somewhere away from Wall Street is not helpful.

        And it's also a false dichotomy. The dichotomy is not between

        • - putting the server next to the trader's desktop, and
        • - putting the server in disaster-proof New Madrid

        It's between

        • - putting the server and generator on the 17th floor, necessitating a bucket-brigade or a 17-floor capable fuel pump, and
        • - putting the server o
  • by peter303 (12292) on Friday September 27, 2013 @05:29PM (#44975423)
    Never modern have there been so few hurricanes by Oct 1 as there were in 2013. Meteorologist blame a "cooler" ocean. We still have another couple months to the season. nd bad ones have occurred late like Sandy.
    • by i.r.id10t (595143)

      Thing is, currents changed with the big earthquakes that have happened in the past few years. If it was enough of a change to affect migration patterns (it has), it was also probably enough of a change to alter the predictable long term weather models. Anyone know if the modeling software has been updated with the new data?

  • by bobcat7677 (561727) on Friday September 27, 2013 @05:30PM (#44975433) Homepage
    We have our servers in a data centers in inland Oregon/Washington. There has never been a hurricane or typhoon within a thousand miles, seismic events are rare, the area is used to large amounts of rain so flooding has minimal effect, the weather is temperate so there is rarely extremes in heat or cold and Tsunamis would have to get past the coast range mountains to be an issue. Basically, nothing ever happens there. I would recommend anyone with important data at least have a DR location in a low risk geographical area.
    • by edibobb (113989)
      Maybe, but according to a web page the Internet (the finding the link is left as an exercise to the reader), Mount Rainier, Adams, and Hood are past due and will erupt any day now.
      • Sorry, forgot to mention volcanoes. Mount St. Helens could erupt again too. But the servers are located outside the lava and mud flow paths for all of these (there are maps widely available that show these things). Even a major eruption would be unlikely to pose a physical threat to the servers and damage to electrical or internet infrastructure would be temporary and easily routed around at worst. Probably the most likely thing might be ash from an eruption potentially clogging cooling systems...but th
    • It seems everyone has their own notion of 'low risk geographical area'. Mine
      is Utah.

    • hate to tell you, but the AWS servers In Oregon are right near a very, very large river, (The columbia I think is second to only the mississippi) Right near several volcanoes, In an area that commonly gets lots of freezing rain and high winds. Its also a few miles from a major chemical weapons depot. (I guess the servers wouldn't care, but nobody would go onsite to replace failed equipment) and not very far from Hanford, where there is tons of nuclear waste.

  • by Anonymous Coward

    I am enjoying your website. I would like to buy your wormhole technology. Please contact me.

  • by VortexCortex (1117377) <VortexCortex.project-retrograde@com> on Friday September 27, 2013 @05:42PM (#44975553)

    Look, we don't have to re-invent the wheel. A hurricane preparedness kit is EXACTLY the same as Zombie Survival Kit minus the shotguns.

    • by slew (2918)

      Look, we don't have to re-invent the wheel. A hurricane preparedness kit is EXACTLY the same as Zombie Survival Kit minus the shotguns.

      Depending on what state you live in, a hurricane preparedness kit would necessarily include a shotgun (of course that means an extra shotgun in those states).

    • by Anonymous Coward

      Actually, if you want to protect your business from potential looting after the storm, you still need the shotguns.

      • Nonsense. If your business is washed away by a storm surge you won't need to worry about looting.

    • by i.r.id10t (595143)

      My hurricane check list includes checking the ammo supply level... I live across the road from just over 1000 acres of forest, has the potential to be a grocery store....

      If I lived more urban, the check list would still include an ammo check, for "social issues" that may arise.

    • Look, we don't have to re-invent the wheel. A hurricane preparedness kit is EXACTLY the same as Zombie Survival Kit minus the shotguns.

      You clearly haven't waited in line for fuel in New Jersey before.....

  • I'm not sure I want to take the advice from a guy who's emergency plans ended up with him using a bucket brigade to keep his DC open...

    No, sorry. Being clever in the midst of a disaster does not make you good at emergency IT planning.

  • by sexconker (1179573) on Friday September 27, 2013 @07:07PM (#44976159)

    Sandy was a tropical storm. Not a hurricane. No, there wasn't anything "super" about it.

    • by sl149q (1537343)

      Why the mod down. Sandy was a tropical storm (as measured by wind speed) when it hit New York.

      The super is due to the amount of damages which is due to the amount of expensive infrastructure that was built in areas that were vulnerable to that type of storm.

      • Actually, Sandy was technically not tropical, and therefore not a hurricane or a tropical storm when it made landfall. It was "Post-tropical cyclone Sandy". The "Superstorm" designation was more catchy, but also attributed to the fact that Sandy had merged with another low pressure system (which is what caused it to make the very unusual sharp turn West).
    • Actually, Sandy covered an enormous amount of territory for a tropical storm/hurricane (something close to twice the area of the next largest major storm to make landfall in the U.S.), which is what made it a "super-storm". The fact that it hit New York only compounded that.
  • 1. Require all new housing within 10 miles of a coast to be built either on stilts or with a ground floor only used for garage, mud room, and guest room.

    2. Remove all insurance subsidies for housing withing 10 miles of a coast. All of them. No exemptions.

    3. Raise power systems 10 feet up, to allow for storm surge flooding.

    4. Redraft flood drainage and screening to anticipate storm surges 10 feet in elevation in all locations. Yes, this means almost all of Florida.

    5. Mandate emergency power systems for apart

    • Require all new housing within 10 miles of a coast to be built either on stilts or with a ground floor only used for garage, mud room, and guest room.

      That makes sense. I live 1 mile from the coast. Of course I also live 200 feet above sea level. What sort of storm surge should I worry about?

      Remove all insurance subsidies for housing withing 10 miles of a coast. All of them. No exemptions.

      But keep them for places subject to earthquakes, volcanic eruptions, tornadoes, forest fires, river flooding and flash floods?

      Raise power systems 10 feet up, to allow for storm surge flooding.

      You do realize that most of the power losses from Sandy were due to the winds breaking above ground power lines, right?

      Redraft flood drainage and screening to anticipate storm surges 10 feet in elevation in all locations.

      That sounds useful - Sandy had a 13 foot storm surge.

      • That sounds useful - Sandy had a 13 foot storm surge.

        When I first came here, this was all swamp. Everyone said I was daft to build a data center on a swamp, but I built in all the same, just to show them. It sank into the swamp. So I built a second one. That sank into the swamp. So I built a third. That burned down, fell over, then sank into the swamp. But the fourth one stayed up. And that's what you're going to get, Lad, the strongest data center in all New York.

      • Depends on the geographical environment you live in - in some places surges can go up to 100 feet due to geographical restrictions, in others it peters out fairly quickly.

        I did not recommend insurance subsidies for earthquakes, tordanoes, forest fires, flooding and flash floods within 10 miles of the ocean shore.

        Power losses would not have been critical if buildings had their own solar panels and passive solar - part of the problem is the load factor to bring power back, actually.

    • by sl149q (1537343)

      10 miles is silly. First in many areas its not enough. Around New Orleans and probably in Florida lots of places farther inland still in danger.

      On the other hand I live less than a kilometer from the Pacific Ocean. But at 100m above sea level would be pretty safe. More worried about things higher up coming down than anything coming up.

      • Actually, in the Ft. Lauderdale/Palm Beach area the Atlantic is rather deep and not going to produce a big storm surge. You need shallow water for the water to pile up. They have shallow water in most places in the Gulf but not all of Florida is susceptible to major storm surge issues.

        http://www.wunderground.com/hurricane/surge/miami_mom5h.png [wunderground.com]

      • 100 meters up less than a kilometer, you're usually at risk from slides. Erosion from waves and storm surges can trigger those.

        Depends on the nature of the soils you're on, and landscaping.

    • Your basic idea is sound, but to be possible it needs to be less drastic. The first step is that any new buildings do not get subsidies for flood insurance (and it does not matter how far it is from the coast). Part of that is that if you re-purpose a building, the flood insurance subsidy is the lesser of what would be available for the original purpose or the new purpose. That goes into effect immediately. After a short length of time(2-5 years), any residential building that is not the primary residence o
      • No. It needs to be more drastic.

        First, remove all subsidies. Second, change all zoning codes FOR BUILDING NEW building or revising current buildings.

        Never wait. Always do.

        • Good luck getting that passed. I am pretty sure that it would be possible to get my idea passed. I am overwhelmingly confident that it would be impossible to get yours passed. In addition, it is almost always a mistake to pass laws which make drastic changes to existing law because of unintended consequences.
    • by Anonymous Coward

      Electrical power is not something needed for in an emergency. Electricity does not support life, you can't eat or drink it either. You can argue that it can keep you warm but a few 1800 watt heaters running on it is nothing great. Some sleeping bags will keep your warm too. All of these idiots who think that having a generator solves 90% of their emergency problems are sitting ducks. Take the money you would spend on a generator and fuel to keep it going (most burn about a gallon an hour), and buy som

    • by iggymanz (596061)

      whether climate change is here or not, there is no evidence whatsoever of anything unusual about frequency or strength of hurricanes. you might be interested to know the worst hurricane ever to hit the USA was over 200 years ago.

      calm down, stop believing hype

    • 1. Require all new housing within 10 miles of a coast to be built either on stilts or with a ground floor only used for garage, mud room, and guest room.

      Note that New Orleans is >10 miles from any coast.

      Note also that it is below sea level.

      Defining a solution to a problem in terms of "miles from the coast" shows enough ignorance of the subject that anything you suggest can safely be disregarded....

      • New Orleans is a river delta we no longer allow to be periodically flooded. As a result, it sinks.

        I can't solve all your problems.

  • For your emergency survival kit, take a good long look at all the options for camping gear. If you aren't prepared for a month of camping, alone, way out in the wilderness, with just your emergency kit, you aren't prepared for a fire/flood/hurricane/earthquake. You'd be amazed at the dozens of small things you will need (or seriously want) but would never think of until you try a few days of camping.

    * Hiking water filter or lots of purification tablets (1 gallon per person, per day)

    * Multi-gallon water co

    • You've got some good things on your list. You also have some stupid things. Like deodorant. Who needs that? If your camping, you don't need it. If there is a break down in social order, you don't need it. If you don't have access to shops for a month, you don't need it (as no one else will have it either...).

      Camp shower? I'm pretty sure where I come from they call those waterfalls. Or you could just go swimming in the lake. (Just avoid all those small microscopic bugz and shitz.)

      Further, how are you going t

      • by evilviper (135110)

        Like deodorant. Who needs that? If your camping, you don't need it. If there is a break down in social order, you don't need it.

        You don't strictly need deodorant, just as you don't strictly need toilet paper, aspirin and antacid, but they're tiny items that'll last a long time. And you'll be feeling a bit miserable if you omit these tiny conveniences.

        Camp shower? I'm pretty sure where I come from they call those waterfalls. Or you could just go swimming in the lake.

        In an emergency, you don't have the luxur

  • That makes sense. Hurricane season has been going on for a few months now. Why the hell would a data center or organization review their hurricane/storm related disaster checklists now instead of, oh, you know, before hurricane season? when Hurricane come then electric may be cut. that why you need a generator . i think we should visit website: http://mayphatdiencu.vn/ [mayphatdiencu.vn] and buy a new generator.
  • 2013 is the year hurricanes seem to be taking a vacation. /Florida!

He who is content with his lot probably has a lot.

Working...