Catch up on stories from the past week (and beyond) at the Slashdot story archive

 


Forgot your password?
Close
typodupeerror
×

Case of the Great Hot-Site Swap 119

Posted by Zonk from the collaboration-makes-things-easier-you-don't-say dept.
BobB writes "Two universities — Bowdoin in Maine and Loyola Marymount in Los Angeles — have entered a unique arrangement under which they are backing up each other's web sites, email and servers on different ends of the continent. They say this could be a disaster recovery model all sorts of organizations could follow. From the article: 'When Bowdoin switched over to Exchange e-mail, so the schools would have similar e-mail infrastructure, LMU staffers were their guides and advisers. "We implemented that pretty quickly," says Davis, the Bowdoin CIO. "When we launched Exchange, we had just eight calls to our help desk." And the shared experience of the infrastructure components then forms a kind of informal help desk, where managers and staff can reach out for advice, brainstorm and troubleshoot problems with their colleagues a continent away.'"
This discussion has been archived. No new comments can be posted.

Case of the Great Hot-Site Swap More

Case of the Great Hot-Site Swap

Comments Filter:
  • does it have to be an equal agreement?

  • The Great Exchange (Score:5, Funny)

    by IAmGarethAdams ( 990037 ) on Saturday August 04, 2007 @07:41PM (#20116761)

    When we launched Exchange, we had just eight calls to our help desk.
    So they had Exchange running their helpdesk phone system too?

    • Re: (Score:2, Insightful)

      by Kamokazi ( 1080091 )
      As much as it's fun to knock Microsoft products, there's really not anything that compares to Exchange.

      • Re: (Score:1, Interesting)

        by Anonymous Coward
        As much as it's fun to knock Microsoft products, there's really not anything that compares to Exchange.

        Notes does a lot of what Exchange does, and does a whole lot more.
        • I'll admit I have not used Notes from the admin side, nor used any version of it for about 4 years. The places I used it at were a few retail locations (I was never personally impressed with the program), so possibly it is preferred from supporting a lot of remote users, but this was before RPC over HTTP in Outlook as well. The people I work with in IT used Notes several years ago and often use it as a humorous comparison to some other piece of software, essentially refering to how much they disliked the
        • Notes does a lot of what Exchange does, and does a whole lot more.

          Having been an administrator of both systems, there's only two spots in which Notes exceeds Exchange/Outlook: in Notes, when I set an item on my calendar as "out of office", it asks if I want to set an out of office message for that time. Fantastic, but that's available in E/O 2007, so it's only an edge if you're still on older versions of the pair. Second, I can set the number of people a conference room will hold so that users can't ov
          • > Sorry, but Exchange is simply install and forget

            Wow, "simply install"? That's gotta be a freaking joke. Any email software that requires a bare minimum of THREE servers is so insanely not-simple. My single GroupWise server is a bit old, but it still runs GroupWise for quite a few clients (granted, not 14,000) and I don't get any calls for it except two people who managed to remove their "sent items" folder (and, as you said, spam-block checking).
            • It's called "Ghost", learn it, love it.

              • Re: (Score:2)

                by hesiod ( 111176 )
                Does ghost not require having an existing install from which to make an image?
                • Yes, so you spend all your time building one box and make it "perfect", then push the image to the other servers. You still had to build that one Notes box. A few hours to build the first, then 10 minutes to build the remainder. Your original point seemed to be the number of servers involved and the time it takes to set it up. My point was the additional servers do not take a significant investment in time. The 16 servers that make up our environment were all built in two days. After the initial build

                  • Re: (Score:2)

                    by hesiod ( 111176 )
                    The "multiple servers" can't be ghosted because they have individual, specialized roles, and thus cloning them would only be useful as a backup. That's why I was confused on the Ghost thing.
                    • No, but you can ghost a base OS image, then have it sysprep on first boot and install Exchange in its proper role. It was a bit more work than necessary for that many servers, but when a server goes down, I just have to plug my 2G flash key into any convenient machine, edit one of about 50 text files each describing a different server config and put the name and IP of the machine in. Then, plug it into the server and reboot. 20 minutes later, the server's up and running like it never went down.
            • Actually it requires 3 different 'server roles' that can be installed on separate servers or the same one. (there are two more optional roles too, one of these, the Edge Transport Role must be installed on a separate server, preferrably outside the domain for security purposes) It depends on the number of clients you have if you need multiple servers to handle the processing load.

              And as far as smiplicity, you can just hit "Typical Installation" and it will install the required 3 roles and only ask for basi
      • What about Scalix?

        e

  • Not nearly as exciting as I first read it... (Score:5, Funny)

    by Franklin Brauner ( 1034220 ) on Saturday August 04, 2007 @07:45PM (#20116799)
    Case of the Great Hot-Wife Swap

    Pity. It being a Saturday, I kind of wanted to read that article.

  • Higher Ed. (Score:4, Interesting)

    by saintlupus ( 227599 ) on Saturday August 04, 2007 @07:48PM (#20116815)
    From what I understand, this is pretty common in higher ed -- in fact, the college that I work for is currently setting up something similar with another college in the area. Not cross-continent redundancy, true, but enough to keep things going should there be a smaller disaster in the area. If all of Western New York is wiped out, I don't really care if people can get their email.

    This really came to the forefront with the beating the New Orleans area colleges took during Katrina; from what I recall, Loyola and Tulane were really unprepared and suffered for it.

    --saint

    • Re: (Score:3, Insightful)

      by ushering05401 ( 1086795 )
      I am not familiar with specifics of these agreements, so perhaps you can tell me... is security jointly administered (blanket policies/configs etc), does the host institution have oversight, or does the institute that originates the data have oversight of the remote servers.

      Not that it makes a huge difference... my sister had all of her data stolen (and consequently her credit was hijacked) through infiltration of a Bay Area college by ID thieves. No off-siting involved.

      Regards.
    • It doesn't surprise me either. It would surprise me if my university system (Wisconsin) wasn't doing this by copying to other campuses within the state. From what I've gathered from our IT department, we have IPv6 backbones running throughout the UW system (but oddly, not "used" for anywhere outside of the system). Unfortunately, they have not taken them further than the server rooms. The only way I could get IPv6 to work when I lived on campus was through tunnelling....

    • Re:Higher Ed. (Score:4, Interesting)

      by Linker3000 ( 626634 ) on Sunday August 05, 2007 @05:42AM (#20119625) Journal
      Good point.

      My company has 30 sites and so it was easy for us to install (Linux) servers at multiple locations and arrange overnight rsync backups of data, server-located 'My Documents' folders, email & Intranet redundancy etc. for business continuity. I am a school governor for my son's local primary school and their backup procedure comprised a disk-to-disk copy from their main student server to another Windows-based server on the network, with an occasional dump to a removable hard disk.

      When the school decided to improve their backup (after a disk failure and realisation that their backup process had not been working for a while, naturally!), they approached their incumbent IT supplier for a recommendation - which turned out to be a new main server with Windows 2003 Server, enough CALs for the children, a dual Xeon processors, SCSI-based RAID 5 and removable tape - very functional, very corporate and very expensive (approx £6,500) for a school that teaches 5-11 year olds!

      Having approached me for my comments, we are now looking at a two-way peering arrangement with the local secondary school comprising two Linux-based servers with SATA RAID 1 (the school is only using the server for low-volume file and print services so Samba and CUPS are just what's needed), and an overnight backup strategy through the education WAN. Total cost is approx £750 for the two servers.

      The only thing that may not make this fly will be County Hall red tape.

      • Re: (Score:1, Insightful)

        by Anonymous Coward
        Recover file xxxxxx.yyy that I accidently deleted three weeks ago please.

  • How about just collocation? (Score:1, Informative)

    by Anonymous Coward
    Eh, this is hardly unique. The university's IT department I work for has a similar arrangement with three other universities in place. We offer collocation space for each other and in one case even make each other's IP address space available on both sides. Is something really bad happens our colleagues can bring up our web site at their location and vice versa. In addition to that we also use SunGuard [sungard.com] so that the administration is able to keep on running even if our campus falls off the world.

    I guess we sh

  • VPN connection over a 30Mbps link. (Score:4, Insightful)

    by More_Cowbell ( 957742 ) on Saturday August 04, 2007 @07:57PM (#20116879) Journal
    I run backups for a large web hosting company; this is to say I do this 40 hours a week for thousands of servers around the globe.

    We have many clients that mirror backups between East and West coast. They may be connected at each end at that speed, but they are almost assuredly not achieving throughput at that rate.

    YMMV, but there are 3000 miles to deal with here. I've never been able to achieve speeds like that, and we have some seriously fat pipes in our data centers.

    • Re:VPN connection over a 30Mbps link. (Score:5, Informative)

      by quanticle ( 843097 ) on Saturday August 04, 2007 @08:40PM (#20117153) Homepage
      Remember, these are universities, so they get access to the Internet2 pipes.
      • Remember, these are universities, so they get access to the Internet2 pipes.

        It doesn't matter how fat the pipes are, the speed-of-light will still bite you in the ass when you are replicating data from one coast to the other.

        • rsync it, with compression. (Score:4, Insightful)

          by khasim ( 1285 ) <brandioch.conner@gmail.com> on Saturday August 04, 2007 @09:09PM (#20117309)
          That way you only take a real hit during the first copy.

          After that, you should be able to copy just the changes and the new files. It is amazing.

          • After that, you should be able to copy just the changes and the new files.
            We use software that does exactly that with block level backups to Netapp filers. Taking a wild guess here, I would say they have quite a bit of new data daily on the exchange servers of two large universities.

          • That way you only take a real hit during the first copy.

            After that, you should be able to copy just the changes and the new files. It is amazing.

            Even better, use rdiff-backup, which uses rsync to transfer minimal deltas, and preserves a complete history on the backup server. I do this with a number of systems that I back up and it works very, very well. When I first set it up I put extensive effort into building a script to automatically clean out backups so that I could keep dailies for a week, weeklies for two months, monthlies for a year, etc., but in practice I just keep dailies forever because all backups except the most recent are stored a

        • Re: (Score:3, Insightful)

          by moosesocks ( 264553 )
          The speed of light is more of a limiting factor for latency, and not throughput.

          That said, it's not even a big deal for latency -- light travels at 186,282 miles/second. New York to LA is approximately 2,800 miles.

          Most of the latency/bandwidth lag comes from routing or congestion along the tubes.
          • The speed of light is more of a limiting factor for latency, and not throughput.

            And throughput is affected by latency. Which was the original poster's point. A huge round-trip time will affect the number of Megabits/sec that you can get through a pipe regardless of how big the pipe is.

            That said, it's not even a big deal for latency -- light travels at 186,282 miles/second. New York to LA is approximately 2,800 miles.

            Light travels through fiber at a slower speed. You'll never see data moving thr
            • And throughput is affected by latency. Which was the original poster's point. A huge round-trip time will affect the number of Megabits/sec that you can get through a pipe regardless of how big the pipe is.


              Not necessarily. Given that these are pretty reliable links, you can set the transmission window relatively high without incurring very many penalties. That way, even if there is significant latency in the connection, you can maximize bandwidth.
              • you can set the transmission window relatively high without incurring very many penalties.

                I agree, some TCP tuning will help offset the challenges introduced by the large round-trip time.
          • I never used to think the speed of light was much of a limiting factor in my day to day life, but once I was living somewhere without access to wired broadband and so I looked into satellite internet. Then I found that geostationary satellites are about 22,200 miles above the surface of the earth, and that the best-case round trip ping would be about 500 ms. Which is awful for gaming or working via remote terminal.

            And suddenly the speed of light seemed strangely slow :)
          • The difference between a cross-continental trip, and shooting a signal up and down again to a geostationary satellite is a factor of 20.

            And even then, satellite communication is perfectly usable for applications where latency isn't a huge deal, as long as your transmission protocols are properly tweaked for a high-latency connection, as long as you're not sharing it between a whole lot of customers -- the big drawback is that it's rather expensive (for some rather obvious reasons) to have a large portion of

      • Re: (Score:2, Informative)

        by More_Cowbell ( 957742 )
        OK, must admit this was new to me. But from what I can tell through some quick research-

        http://en.wikipedia.org/wiki/Internet2 [wikipedia.org]

        There have been times when the media have reported on a network called "Internet2." This is misleading since Internet2 is in fact a consortium and not a computer network. "Internet2" is sometimes used, albeit a misnomer, for the Abilene Network.

        So I take a look at Abilenes website and find this map: http://abilene.internet2.edu/peernetworks/domestic .html [internet2.edu]

        From what I can see here

    • Re: (Score:2)

      by rmm4pi8 ( 680224 )
      I actually do see speeds like this. We pay $450/mo for a 3mb guaranteed/100mb burst pipe at our Vericenter cage in Boston, and something like that much for a backup facility in Dallas. We have about 300mb of new data an hour to transfer, which we send in hourly batches, and it usually completes at something like 37mb/s. Now this certainly isn't a guaranteed rate, and I wouldn't want to run an application that depended on that kind of throughput, but it shows you what's possible across Sprint's backbone f

    • Re: (Score:1)

      by mtmra70 ( 964928 )
      get with the times; you need phat pipes!!! :)

  • One of the main problem is... (Score:5, Insightful)

    by KokorHekkus ( 986906 ) on Saturday August 04, 2007 @08:00PM (#20116895)
    ...that you might have to accept the legal responsibilies of the site that is being backed up. It's not just a simple exchange of providing corresponding services... Take it down to a personal level... who would you trust to use your personal computer as a backup server (in a reciprocal manner)? No one that hasn't your full and complete trust is my guess. Encryption would provide some protection but this isn't about data backup but service fallback.

    So unless you have some kind of legal agreement covering your actual risks it's not for everyone. But for large scale organisations, with real legal clout, like universities it might makes sense. But not for individuals.

    • I don't know... companies that offer "online backup" still don't actually take responsibility for the integrity of your data. Also, you'd find out pretty quickly whether the person on the other end of your backup is doing a good job. Try accessing your data, running checksums, etc.

      After all, you don't really need to make sure that the person you have the deal with never once loses a piece of data, but only that the chances are remote of him losing a piece of data at the same time you lose that same piece

      • Re: (Score:3, Informative)

        by ScrewMaster ( 602015 )
        Everything you send offsite should be encrypted, and anyone accessing your computer should be jailed somehow.

        The problem is, a lot of the people trying to access your computer nowadays want to put you in jail somehow.
        • Correct me if I'm wrong here, but giving someone you know a jailed FTP/SFTP account shouldn't present a significant security risk for the server the account is on.
    • Comment removed based on user account deletion

  • Maine (Score:2, Interesting)

    by nuclearpenguins ( 907128 )
    It's always nice to see something from Maine featured in technology news. We're a tiny state population-wise, but there are many high-tech companies here. Hell, even in my own town there is a company that developed and makes the MK47 Advanced Lightweight Grenade Launcher as well as a bunch of other armaments for the military. Sure, this post is off topic and my karma blows already, but I always have to say "Go Maine!" whenever I read about my little state on any technology forum/site.
  • The summary mentions a university using Exchange successfully. Anathema, I say! Anathema! Have we no sense of decency? Must we sit idly by and let such disinformation blot out a beacon of truth here on the Internet? Let us take up our arms, brothers, and march onward. Er, roll onward, since all our computer chairs have wheels on them. We shall destroy the enemy while sitting upright in an ergonomic fashion! We shall upend the world from the depths of the basement!

    • Simple answer (Score:4, Funny)

      by schon ( 31600 ) on Saturday August 04, 2007 @08:59PM (#20117255)

      How'd this make the front page?!
      Simple - and you answer it with the first sentence of your post:

      The summary mentions a university using Exchange successfully.
      You're trying to say that this *isn't* front-page news? :)
      • Well done, you win.

        But, know this: you haven't seen the last of me! You shall rue the day you caught me in my own logic trap!
      • The change has been rather surprisingly successful. I had been very happy running our previous Sun servers and the control I had over everything via LDAP, but now all email changes are under the control of the Microsoft systems. However, the new email has been relatively easy to use, with fewer problems (though the web client does not due nearly enough justice for non-IE users, when compared with the old Java interface), still supports all the features I actually use, and if it helps in this Hot-Site effo
  • that my mail is being watched by 'the authorities' (you know, the ones which are keeping us safe from terrorism so successfully these days), and that if I should ever lose any I can just make a baseless accusation against myself to the police and have them restore it from their backups.

  • Too bad ... (Score:1, Redundant)

    by ScrewMaster ( 602015 )
    When Bowdoin switched over to Exchange e-mail ...

    Oh ... I'm sorry.
  • The alternate nameservers for many Universities are often at other schools. Not the same thing, but interesting to note:


    mtnBook:~ $ whois rochester.edu
    Name Servers:
          NS1.UTD.ROCHESTER.EDU 128.151.2.1
          NS2.UTD.ROCHESTER.EDU 128.151.7.6
          SIMON.CS.CORNELL.EDU
          DNS.CS.WISC.EDU

    mtnBook:~ $ whois cornell.edu
    Name Servers:
          BIGRED.CIT.CORNELL.EDU 128.253.180.2
          DNS.CIT.CORNELL.EDU 192.35.82.50
          CAYUGA.CS.ROCHESTER.EDU

    mtnBook:~ $ whois ucsb.edu
    Name Servers:
          NS1.UCSB.EDU 128.111.1.1
          NS2.UCSB.EDU 128.111.1.2
          KNOT.BROWN.EDU



    There's a bunch more NYU/UCBerkeley, WUSTL/ULA, etc.

    • Re: (Score:2)

      by pyite ( 140350 )
      Further evidence:

      aman:~ $ whois rutgers.edu

      ...

      Name Servers:
      DNS3.RUTGERS.EDU 198.151.130.254
      DNS4.RUTGERS.EDU 66.187.157.84
      RU-UFL.RUTGERS.EDU 128.227.128.162
      DNS1.RUTGERS.EDU 165.230.144.131
      DNS2.RUTGERS.EDU 128.6.21.9

      aman:~ $ whois ufl.edu

      ...

      Name Servers:
      LOWER-NAME.SERVER.UFL.EDU 128.227.128.254
      NAME.UFL.EDU 128.227.128.24
      RUTGERS-NAME.SERVER.UFL.EDU 128.6.224.82

    • Re: (Score:1)

      by xaxa ( 988988 )
      Cross continent:

      whois cam.ac.uk
      Servers:
      authdns0.csx.cam.ac.uk 131.111.8.37
      authdns1.csx.cam.ac.uk 131.111.12.37
      dns0.cl.cam.ac.uk 128.232.0.19
      dns1.cl.cam.ac.uk 128.232.0.18
      dns0.eng.cam.ac.uk 129.169.8.8
      ns2.ic.ac.uk
      bitsy.mit.edu

      Strange
  • The University of Cincinnati, The Ohio State University, and the University of Miami (Ohio) are already doing this.

  • Not for everyone... (Score:2, Insightful)

    by fuffer ( 600365 )
    "They say this could be a disaster recovery model all sorts of organizations could follow."
    For private businesses maybe, but I'm sure hosting backups on other organizations hardware is not acceptable under SOX.

  • My mail is backed up in at least 5 centres (Score:5, Funny)

    by flyingfsck ( 986395 ) on Saturday August 04, 2007 @09:13PM (#20117333)
    CIA, FBI, CSIS, KGB, MI5...

    It is the most secure backup system in the world.

  • Survival/Preparedness community (Score:3, Interesting)

    by zogger ( 617870 ) on Saturday August 04, 2007 @10:15PM (#20117657) Homepage Journal
    This is common for survivalist and preparedness minded folks. You and a trusted relative or friend exchange backup critical gear/necessities/copies of records, etc. In case of catastrophic loss of either abode, the other person has a decent "backup" for you to fall back on. Arrangements like this have been quite common for some decades now, usually they include mutually assured lodging, should full long term evacuation be required. IMO, it is quite a sound idea. Remember on the news, you see the same scene all the time, those scenes from..take your pick, fires, floods, hurricanes or whatever.. the newsies always zero in on those folks who are all freaked out and sad, and EVERY time they say "We lost EVERYTHING!"..well, there's no need for that if you take the time in advance to preposition enough of your gear so it doesn't fall into the "everything" category. The situation will still suck, but having a nice set of backup everything will sure help mitigate things and make the situation suck *less*. As to what to exchange/store, use your imagination, what would you like to have as a backup if for some reason your home just got wiped out? Spare sets of clothes for everyone, favorite toys for the kids, some electronic gear, tools, sporting goods, books, other media of importance to you, family photos, household records, personal mementos, etc. Salt to taste there. Even just a stuffed closet is good enough, that and the place to evacuate *to*.

    That, and what we call BOBs, or "bug out bags" are good ideas. A "bob" is a backpack or other container (backbacks are good in case you get stuck on foot), that has enough critical essentials to keep you alive for a week or so, enough even on foot to get you out of the disaster area most likely. It's called a bug-out bag from the old army term, and it is designed so if you have zero notice-hear on the radio local railroad has a tanker car full of chlorine leaking, nasty forest fire heading your way, and it's close, etc, that you can grab it and go, out the door within less than one minute. Very high speed emergency evacuation. The deal is, you hope you never need it, but if you do, it literally could save your life.

    Interesting subject, and although it is not directly related to the main parent IT topic, the concept is very similar.
    • A BOB is also essential in the case of a zombie invasion.

      BTW, back in 1993, we used to hold annual DR swaps with another federal agency who had similar systems to our own. We also had an MOU to mirror critical data and use each others facilities in case our primary site was hosed. (or is that swarming with the walking undead?)

      Xix.
  • So when the RIAA/MPAA come a knockin', they can subpoena the data from both universities in one shot.
  • They would store it in the middle (instead of on the edges, where it could fall off), somewhere like Lebanon, Kansas [roadsideamerica.com], equally convenient to both coasts.
  • So does this mean that if one University doesn't give up information to the RIAA, then the RIAA could just try to get it from the "backups" from another University ? Not that I would ever suggest that the RIAA would ever do such a thing.
  • last I heard, Maine and Los Angeles were on the same continent.
  • Exchange isn't so problematic because it is written poorly. The problem is that it is so frequently administered poorly.

    It's also problematic because exchange experts are few and far between. But then again how many sendmail or qmail experts are there?

    I wonder though - are they using exchange just for e-mail? Or are they using it for scheduling, shared folders, etc.? I can't see implementing shared schedules university wide and only receiving 8 help desk calls. You'd think more than 8 people would be c

    • Re: (Score:2)

      by jabuzz ( 182671 )
      And there was me thinking the problem was the shit that is the Exchange Jet storage engine is the biggest pile of dino turd going. We know from PostPath that Exchange really is a pile of junk. That gets five times the throughput on *IDENTICAL* hardware, talking the same binary protocol down the wire. I remember back when Exchange first came out everyone said it just chewed through CPU cycles and IO bandwidth compared to other solutions. Now we know it that it really is down to rubbish programming on behalf

  • And? (Score:1, Insightful)

    by yellowalienbaby ( 897469 )
    Im not sure what special about this? replace University with DataCentre and this happens all the time. With SAN's and dark fibre, you can get machine A at datacentre A writing data directly to tapes in an automated library in DataCentre B, and vice versa. and they generally backup a hell of a lot more than email and websites.

  • A friend and I have been talking about this. He's on the west coast too, but about 1000 miles away, so my backups would probably be safe.

    http://www.fluffy.co.uk/boxbackup/ [fluffy.co.uk]

    Any one using this?
    • I've experimented with it (an earlier version though) and it was pretty slick.

      I didn't ever finish setting it up and getting it running, but I can't remember off the top of my head what problem I must have run into. I think I got distracted by the prospect of setting up ext3cow, which is an automatic timeshifting/versioning filesystem (but it pretty early/beta, not for production) and generally ADDed out.

      I should go back and take another look at it. One of the things I really liked was that it was developed
  • ...Or Not.

    I wish we could do a similar setup where I work(a hospital system). But, HIPPA regs would prob nail both parties/partners to the rulebook cross :-(
  • Hmmm, a cross-continental backup system, set on college campuses.

    Sounds like something the defense department might be interested in.

    Wonder what they'll call it?

  • Great news for those who want to follow: We run such a service for our customers. Using Askemos for tamper proofed process replication [askemos.org] we run a network of nodes owned by several companies.

    We have a peering agreement, no matter what dies, be it a host or even a hosting company, our customers websites continue to run - undisturbed.

Slashdot Top Deals

"What man has done, man can aspire to do." -- Jerry Pournelle, about space flight

Close