Forgot your password?
typodupeerror
Communications Software IT

Email Offline At the Home of Sendmail 179

Posted by timothy
from the snowballs-happen dept.
BobJacobsen writes "The UC Berkeley email system has been either offline, or only providing limited access, for more than a week. How can the place where sendmail originated fall so far? The campus CIO gave an internal seminar (video, slides) where he discussed the incident, the response, and some of the history. Briefly, the growth of email clients was going to overwhelm the system eventually, but the crisis was advanced when a disk failure required a restart after some time offline. Not discussed is the long series of failures to identify and implement the replacement system (1, 2, 3, 4). Like the New York City Dept. of Education problem discussed yesterday, this is a failure of planning and management being discussed as a problem with (inflexible) technology. How can IT people solve things like this?"
This discussion has been archived. No new comments can be posted.

Email Offline At the Home of Sendmail

Comments Filter:
  • by bobstreo (1320787) on Sunday December 04, 2011 @05:37PM (#38259942)

    It's the backend. When you have too many connections on too few servers, with not enough storage
    you usually see this kinda issue.

    • by grcumb (781340)

      It's the backend. When you have too many connections on too few servers, with not enough storage you usually see this kinda issue.

      I see it as yet another failure for the client/single server model.

      It surprises me that people are still investing so much time and effort on centralisation of services when obviously the most practical technical[*] answer is the opposite. Simple, common protocols and decentralised infrastructure are the most robust model for overall survival of a communications system. DARPA proved that some time ago, but we seem intent on forgetting as much of that lesson as possible.

      ----------------
      [*] Okay, I don't wan

    • by vlm (69642) on Sunday December 04, 2011 @06:06PM (#38260164)

      It's the backend. When you have too many connections on too few servers, with not enough storage
      you usually see this kinda issue.

      Knowing the speed and flexibility of university upgrade policies, and knowing sendmail was born around 4.1BSD, and knowing the -BSDs were VAX only until 4.2 or 4.3 or so in the 80s, I'm guessing they're still using the original VAX it was developed on?

      • by CAIMLAS (41445)

        Many educational institutions lag behind because they're an ever-evolving door. Even when they've got dedicated and experienced IT staff, most of it's just in a managerial role for the student work studies (it saves money, of course).

  • by Colin Smith (2679) on Sunday December 04, 2011 @05:37PM (#38259948)

    It's an economic one. It needs an economic solution.

    e.g.
    Have people buy a $10 ticket to get an account on the email server.

    • by StikyPad (445176) on Sunday December 04, 2011 @05:56PM (#38260056) Homepage

      Pretty sure that's what tuition is.

      • by Anonymous Coward on Sunday December 04, 2011 @06:14PM (#38260256)

        no I'm pretty sure tuition is more than $10

      • Clearly email is an afterthought thrown in for free.

        If you want a service to work, you have to fund it. You can try to fight for budgets against the football team or you can simply charge and the money automatically goes where it's needed.

        Think of money as little packets of information. You buy something there is a need for it, you don't buy it, there is no need. Resource allocation without dozens of layers of management.

        Maybe nobody cares about email and they can just shut it down. Charge for it and find o

        • Maybe they should offer redirection for free and paid storage. I find it useful to have an email address within my college domain, but I redirect everything to my main account.

        • by Nimey (114278)

          At my university we split it into tuition and student fees. Fees go to all manner of things like clubs and activities and the rec center and other stuff.

      • by Jim Hall (2985)

        Pretty sure that's what tuition is.

        Tehnically, that's actually covered by a student fee, usually a "technology fee" in most universities. So yes, this cost should already be built into the cost of attending university. Whether that fee is enough to cover everything, including email, I'll leave to Berkeley.

    • I hate it when people try to act as if IT isn't subject to budget constraints and having to prioritize spending like any other department of a large organization. Sure the money comes out of the "client" departments, but it's an issue that IT does have to plan for and deal with.

      The summary asks "How can IT people solve things like this?"

      Forward the emails and responses to the demands for planned capacity growth to the public.

      Oh, you didn't keep the email from your manager refusing to pay for a neede

    • Indeed, that is exactly what happened at my alma mater. First they blamed the Squirrelmail front end, then they bought a black box solution from Mirapoint, and when the Mirapoint solution proved too expensive they just went with Google.
      • Re: (Score:2, Funny)

        by Anonymous Coward

        Wow, Squirrelmail. So at least they managed to migrate from pine at some point.

      • This is really quite common. It happened at my alma mater as well. Servers could not handle the POP requests, so they started blacklisting students that checked their mail more than four times an hour. A month later a RAID drive failed and email for 17k people (including a hospital) was completely offline for 3 days. It is sad that seemingly anyone can be a high paid "IT Professional" these days, but without a clue about HA.
        • One can have all the clue in the world, yet be powerless to prevent failures if not funded to purchase the appropriate equipment.
        • by AK Marc (707885)
          HA isn't an IT problem. It's a business issue. I've never seen a business who put a dollar amount on their downtime. I know they are out there, but every "real" place I've worked has never quantified their costs. How do you justify having a cold spare if it's a "waste" of money planning for an outage that is "free" (after all, if downtime were a problem, then someone would calculate the cost of downtime). IT did what they should, gave the users the best they could within their budget. That the budget
    • by lucm (889690)

      Outsourcing would work, because when there is another failure they will have another party to blame instead of pointing fingers to a decision made in Spring 2011 (even as a total stranger I could feel the bitterness under that bullet point in the slides).

  • When I started college in 1991 I was amazed by the telnet access I had to the email account given to me by the University. I hadn't had an email address prior to that. Now I have an email addresses through hotmail, gmail and yahoo that I use for different things and facebook also gives me an email address. So, I doubt students really need email addresses provided by the university anymore. As for the NYC Dept of Ed example, I think it just shows that trying to build IT competence into a government agenc
    • Re:Telnet (Score:5, Insightful)

      by slimjim8094 (941042) <slashdot3@justconnected . n et> on Sunday December 04, 2011 @05:58PM (#38260080)

      Students need school email addresses because that way all students have an email address.

      At my school, students are expected to check their university email at least once every 24 hours. Many people forward it to a personal account, and obviously most people check it more frequently than that, but if the university issues an account to everyone, then there can be no debate about how they didn't get the email. The school takes responsibility for the email system (and any failures), and then professors can be assured that if they send an email out to the class, it will be (or should have been) read, leaving the onus on the student to actually do it. It's similar to why we provide computer labs - that way, each student unequivocally has a way to do electronic assignments, even if nearly everyone has their own machine.

      • by Nimey (114278)

        They also need school email addresses because Hotmail et al are liable to mark a university as a spammer if enough luser students decide to click the "mark as spam" button for university-sent emails. Happened to my university a few times before It Was Decreed that all students are required to have a Google Apps for Domains email account resolving to our domain.

        • Given my university's propensity to send irrelevant e-mails to all students, students marking some e-mail from your university as spam was probably quite legitimate.
          • lusers reporting as spam because they don't want to bother unsubscribing is a problem. but can one unsubscribe from those emails without missing out on the important/relevant emails? that's what I wondered about as I ran into the same problem

            • by tibit (1762298)

              I've had this problem at a big ten school. They have a mailing list where the disclaimer at the bottom of every email is that if you unsubscribe (even though they won't let you anyway), you may miss some important stuff (I'm paraphrasing). I've archived all of those email from almost a decade, and one afternoon went through them all. All of six messages out of almost 3000 were relevant to me, and most of them would be irrelevant to 90%+ of students they were reaching. Yet there was no way of unsubscribing:

              • by Nimey (114278)

                Heh, yeah, that's legit. We send a bunch of crap that most of the *staff* don't care about, let alone the students, bulk emails about Spanish Club, bake sales, and so on. From two different email systems, yet.

      • Re:Telnet (Score:4, Interesting)

        by QuantumRiff (120817) on Sunday December 04, 2011 @11:31PM (#38262422)

        When I was admining at a small college, we DID NOT provide email for students, only for staff. We ran a listServe (sympa) and if the students gave us their personal email address, and checked a box, they would be added to a mail list for every class automatically..

        Any student that didn't have an email would be sent to the library, where they would be shown how to sign up for a hotmail, yahoo, or gmail account.

        We had students thank us, since they have gone to other schools, and though it was silly to have to check yet another account, when they already had 3 or 4.

        The ONLY reason colleges give out emails is because they have been doing it since before email was a common thing. There is no actual reason for it.. (but I have heard some neighboring colleges give very, very very good sounding arguments on why they needed to drop a few hundred grant on a SAN and exchange)

      • The school takes responsibility for the email system (and any failures), and then professors can be assured that if they send an email out to the class, it will be (or should have been) read, leaving the onus on the student to actually do it.

        That is peculiar coming from a scientific institution. Email offers no guarantee of reception. I grant that an extremely high amount of mails do arrive well. But if I were to avoid distracting discussions, I'd communicate with more than just email as a medium and I wouldn't rely on it being high available.

    • Re: (Score:2, Interesting)

      facebook also gives me an email address

      When did this start happening? Does it actually interoperate with other email services?

      • From news articles, it started a year ago, and it does interoperate with other email services (you get a [username]@facebook.com) but it doesn't let the sender choose the Subject line, add CCs, forward, etc.

        • What I really care about is being able to communicate with Facebook users without having to actually open a Facebook account. Thank you for the information, I will look into this in more detail.
    • No. (Score:5, Insightful)

      by damn_registrars (1103043) <damn.registrars@gmail.com> on Sunday December 04, 2011 @06:03PM (#38260122) Homepage Journal

      Now I have an email addresses through hotmail, gmail and yahoo that I use for different things and facebook also gives me an email address. So, I doubt students really need email addresses provided by the university anymore.

      You are quite wrong. Email addresses - especially .edu addresses - are still quite valuable. At lot of academic resources that take registration via email won't allow registration to go to a throwaway account (a la hotmail, gmail, yahoo, etc). Many organizations that are interested in real information on users insist that users use an actual unique account and not a freebie. And when you're in college and making very little money a lot of those things can be important.

      I think it just shows that trying to build IT competence into a government agency basically a waste of money because the institutional culture of government

      You're not very accurate on that, either. Government organizations need to be able to keep track of their email - especially internal communications - which they would not be able to do if they outsourced email and other telecom.

      In short, all of these kinds of organizations could just offer email through gmail/google business or any number of other providers that will scale up almost infinitely.

      With the various privacy breeches that have occurred, that would be a terrible idea. And on top of that, IT is a lot more than just email. Do you want the government to turn to comcast for networking support while their at it? What if the IRS web servers go down on tax day? Do you want them to have to lean on an outside company to get it back up?

      • you can get an email address through google with any domain name you want... so, the company I work for runs its email from google but we all still have email addresses that say mrbigshot@seriousbusiness.com so I don't think the point about the .edu ending of the address is really valid.
      • by matty619 (630957)

        In short, all of these kinds of organizations could just offer email through gmail/google business or any number of other providers that will scale up almost infinitely.

        With the various privacy breeches that have occurred, that would be a terrible idea. And on top of that, IT is a lot more than just email. Do you want the government to turn to comcast for networking support while their at it? What if the IRS web servers go down on tax day? Do you want them to have to lean on an outside company to get it back up?

        If you watched TFV...that is exactly what they're going to do. They're either going to go with Microsoft's 365, or Google's Gmail. They're just working out the contracts.

    • by mrmeval (662166)

      It's safer if the school has no access to your email so they should just drop that service except for those students who provably don't have the money for it.

      I'd never use their servers the same as I won't use my ISP's. I use ones I have some insulation from and if I need more privacy than that I use encryption and if it's life or liberty threatening a drop off to a hollow tree in some random park. ;)

    • Yes, but for most students, I'll bet that their official school email address is still their primary email address for all the important stuff.

    • Re:Telnet (Score:4, Informative)

      by Mr. Freeman (933986) on Sunday December 04, 2011 @06:38PM (#38260490)
      Having a .edu address gains you a lot of credibility when communicating with people outside the university. They are quite valuable. You can often get very quick responses to questions that most companies won't even respond to if they came from a name@gmail.com or name@yahoo.com.

      Also, email is used for a lot of very important stuff like sending reports, design files, etc. Having someone on campus that can fix problems is quite valuable. Your campus email will never be "accidentally" seized, locked out, etc. like people have experienced with google and yahoo. Because the campus maintains backups (or at least, they should), you data will never be suddenly gone with no chance for recovery like people have experienced with google and yahoo.
    • Actually, most schools require an Official school email address. This guarantees the uptime from the faculty's point of view; you can't claim you never got the assignment or that you turned it in on time and nothing was there. It's also important for them from a liability standpoint; my Registrar will not send me any bills unless it's to my .edu account, and professors are instructed to ignore any student emails from any other domain. They're also organized by real name, so the school has a working internal

      • by tibit (1762298)

        Um, because the "internal directory" can't be implement using LDAP?! What's wrong with LDAP anyway? Even OpenLDAP seems to scale decently and I'd hardly consider it a "bother".

    • by Jonner (189691)

      If the university can't competently provide its own IT infrastructure, why should they be expected to provide anything else competently? Perhaps it's just time to privatize the whole thing. I can't wait for GoogleU and Inteliversity.

  • Maybe it has something to with the fact that the state of california has cannibalized the funding for my beloved alma mater.
    • Re:Funding (Score:4, Insightful)

      by DesScorp (410532) <DesScorp@NOSpam.Gmail.com> on Sunday December 04, 2011 @06:10PM (#38260202) Homepage Journal

      Maybe it has something to with the fact that the state of california has cannibalized the funding for my beloved alma mater.

      They wouldn't have to if they didn't have too many colleges (they do), and try to send too many kids to college (they do), many of whom may have no business being in college (they don't). Tax revenue is not an infinite resource. But California seems to have a community college on every two dirt roads, and several 4 year (or higher) colleges in a similar area.

      • I agree that the overall system is probably too large, but we are talking specifically about the flagship university of the UC system. arguably the best public university in the world, and it is getting hurt just as bad as UC Riverside. that's absurd and embarrassing.
      • And lets not forget... they have the highest population count of anyplace in the USA also. Want to build a few more uni's in the Dakotas?
  • by mysidia (191772) * on Sunday December 04, 2011 @05:48PM (#38260022)

    Briefly, the growth of email clients was going to overwhelm the system eventually, but the crisis was advanced when a disk failure required a restart after some time offline.

    Capacity planning is supposed to account for reduced capacity due to component failures, system outages, and temporary demand spikes due to restart events.

    • In my experience this type of "planning failure" is caused when IT repeatedly tells management they need money to maintain and upgrade systems, and management consistently says no because they don't have the money for it. Not enough money or people to configure, install, support, and maintain any new systems because the budget won't allow any more. Yet somehow there always seems to be money for shiny new iPads and iPhones for the executives.

    • First and foremost, it has to account for budget, and the rationalization thereof. It's scary how often suits (and more and more "engineers") say things like "Come on really; how often does that kind of thing actually happen?" This is usually uttered after staring at a couple dozen slides of metrics that detail exactly how often it happens...
  • http://slashdot.org/comments.pl?sid=2556922&cid=38249652 [slashdot.org]

    IT should have unions so they are not the fail guy for management mess up's / lack of funds and or planing.

  • IT people need to move into management at a more useful rate. Instead most of the people who ultimately make the financial decisions for IT centers around the world have little grounding in IT and hence limited understanding of what is actually important beyond the bottom line.

    Of course, this requires IT people who are willing to put their foot down. We don't seem to have many of those...
    • by quetwo (1203948)

      Of course, the smart IT people are often not allowed into management -- they are too useful at their current level (keeping the systems running, turning the screws, etc), and would be a pain to be replaced. So they promote people who are easy to replace into management. For a smart IT person, usually the only way to get ahead is to move sideways, not up. Go somewhere else, or do something else... If you are good at what you do, there is little incentive for people to move you up.

    • by Ziest (143204)

      Unfortunately, those smart IT people who have spent years in the trenches and understand, in detail, how to build a robust and resistant infrastructure are often overruled by the a CFO who's only qualification is they have an MBA. In many tech companies the group that handles the infrastructure (DNS, email, backups, etc.) reports to the CFO not the CTO. Why is this? After 25 years in the computer field I still have not heard a rational explanation for this idiocy.

  • by arthurpaliden (939626) on Sunday December 04, 2011 @06:02PM (#38260120)
    IT goes to management and says "based on current usage/loadings etc the system will fail in 6 months to prevent it we need to do this....." Management says "Really, that's not what the sales man told me and its his equipment so he should know".
  • At the school where I teach, whenever there's a discussion of how much it costs us to run our own email, someone suggests outsourcing (e.g., to gmail), and then someone else says, "No, we can't do that because of privacy laws." Am I right in guessing that privacy laws don't in fact prevent outsourcing to google? I suspect the argument is basically a way for IT folks to have job security. There are certainly laws that say, e.g., that we can't give students' grades to third parties. But it's hard to believe t

    • by Tridus (79566)

      That depends on where you are. Privacy laws in Canada most certainly do restrict Government agencies (and probably educational ones) from using Gmail because it's American and the Patriot Act is a rather severe problem that can't be mitigated.

    • Re:outsourcing? (Score:4, Insightful)

      by betterunixthanunix (980855) on Sunday December 04, 2011 @06:13PM (#38260240)
      Since both my alma mater and my current institution have migrated to Google, and both are covered by FERPA and other privacy laws, I am inclined to say that that argument is bogus. However, I have a separate issue with outsourcing student email: third parties get to set the rules for student conduct without any action by the university itself.

      Typically universities have acceptable computer policies and at those institutions that run their own mail servers, such policies usually govern email. Students and faculty can demand changes to university policy if the policy does not properly align with the academic mission of the institution. Students and faculty have essentially no power over the terms of use that Google or Microsoft or any other third party email service imposes on them. It is easy to say, "Well, it is not like Google is going to demand something outrageous!" but there is really nothing preventing Google from doing so (if you do not think they have done so already). Google does not have the best interests of academia in mind when it sets its policies, nor is there any reason for Google to care about academic needs.
      • by swillden (191260)

        nor is there any reason for Google to care about academic needs.

        Sure there is: If they don't meet the need, they'll lose the customers to someone/something else who does.

      • by msobkow (48369)

        Can you provide an example of a policy that would get the students and educators up in arms?

        Can you provide an example where the terms of access to outsourced services are not set by the organization who set the terms in their contract rather than blindly accepting whatever the outsourcing company tabled as a "standard" contract?

        I thought not. More FUD. "Please panic, people, because they might possibly maybe perhaps breach their contract and do something we don't like."

        Pfffftttttt.

        • The first thing that comes to my mind is the requirement that nobody try to learn how the services are implemented (usually stated as "reverse engineering"). On principle, schools should have a problem with policies that forbid students from attempting to learn how their email software is implemented, although in practice schools generally have no problem with such agreements (perhaps they simply assume that students will never try to reverse engineer anything). Both Google apps for education and Microsof
          • by msobkow (48369)

            I'm just going with past experience on outsourcing details. In every case where I've seen outsourcing done, the terms of uptime guarantees, access guarantees, priveleges, security -- all were negotiable items with give and take by both the outsourcing provider and the service purchaser.

            If Google and Microsoft aren't allowing customers to set terms on critical things like system access and the validity of content, then companies are being very foolish to contract with them at all. Your provider should n

      • Students and faculty have essentially no power over the terms of use that Google or Microsoft or any other third party email service imposes on them.

        There's a more mundane problem. Unless you are an incredibly huge customer the large service providers are just not going to care if there is an outage. One example I ran into last year is a University of 45,000+ students that lost their student email hosting (hotmail) for a week due to a DNS typo for a machine in a hotmail MS Exchange server farm. To get a

    • by GIL_Dude (850471)
      Tons of schools use Google as their email provider. Here's a quote from a Time article from 2009:

      Google now manages e-mail for more than 2,000 colleges and universities, enabling students to transform accounts capped at 100 mb into Google-managed inboxes that allow for 70 times as much mail. Microsoft also provides free Web-based mail for thousands of schools, including colleges in 86 countries.

      Here's the article: http://www.time.com/time/business/article/0,8599,1915112,00.html [time.com]. Now, a specific school?
    • That's a very difficult question. You need to sit down with your company's legal department and figure it out.

      The answer depends on:
      1) Where you are and thus what laws are applicable to you.
      2) Who you are. Healthcare, university, private company? If you are a university, are you a public university? If so, there may be additional laws and regulations.
      3) What's being emailed. Patient records, classified documents?

      What's acceptable for people in similar situations may not be acceptable for you. I go to
  • Seriously, is Berkeley like the only college campus that hasn't outsourced their e-mail to Google yet?

    • by mactard (1223412)
      UCLA and UC Irvine do their email in-house as well.. I'm sure it's just a University of California thing. So many smart people here, but so many dumb decisions are made every day.
    • The University of Washington has outsourced its student email as of a couple years ago; but for faculty and staff the old "deskmail" servers are still available with no announced EOL (yet) - although many do use Gmail too, since it offers an order of magnitude more storage.

      Technically UW offers both Gmail and the cloud Windows mail. It used to be just Google Mail and Calendaring, but before he left UW President Emmert dictated the university needed to have Exchange available to everyone - probably coinciden

  • Only 70000 accounts? That's not a big system at all. I was running systems with over million email accounts ten years ago, and by today's standards even those would be considered small.

    • by thogard (43403)

      Two decades ago we were supporting 78,000 users on a machine that was almost as fast as 4 Nintendo 64.

  • by farnsworth (558449) on Sunday December 04, 2011 @06:14PM (#38260254)
    In the video, they don't even mention sendmail at all. Are they using it?

    Also, they mention that the cost of the system is something like $1.30 per account per month. I don't know much about IT budgeting, but that seems like a really low number for something as critical as messaging and calendaring. I have to imagine that they spend more money per user just cutting the grass around the campus.
    • They no longer use Sendmail; they use Exim.

    • by lucm (889690)

      Also, they mention that the cost of the system is something like $1.30 per account per month. I don't know much about IT budgeting, but that seems like a really low number for something as critical as messaging and calendaring. I have to imagine that they spend more money per user just cutting the grass around the campus.

      Totally agree. One of my client did a major cost-cutting initiative for its email platform, and there was just no way to make it reliable under 9$ a month (per account). And this is when there is no Crackberry (which brings the numbers way up).

    • by TClevenger (252206) on Sunday December 04, 2011 @08:22PM (#38261336)
      That's amazingly cheap. I don't know how you'd do it any cheaper outsourced. Microsoft is $8.80/user in qty. 20,000, and while Google starts at $4.17/user, I couldn't imagine that even 70,000 accounts could bring down the price that much.
    • by guruevi (827432)

      That's a lot of money actually. $1.30/user is $13,000/month for what a simple server cluster (Postfix etc.) should be able to handle (~10,000 accounts). That's over $150,000/year.

      I used to work for a hosting company and our server cluster (4 servers with a 10TB backend) handled well over 50,000 accounts about 5 years ago. Sure the load would be at 10 or so but that's not a big deal for a mail server.

    • the point was "how the mighty have fallen", not something about sendmail in particular

  • IT cannot solve this (Score:2, Interesting)

    by decora (1710862)

    it's like saying IT can do heart surgery or IT can provide pscyhological counseling to a trauma survivor. IT is IT, it is not management and it is not leadership. IT is IT.

    of course, shit rolls downhill, and leaders nowdays are incompetent buffoons who gain their positions largely through bribery, kickbacks, extortion, and other 'features' endemic to societies where the rule-of-law breaks down thanks to a greedy, corrupt elite.

    again, IT cannot fix that.

    • by iggymanz (596061)

      oh man, good IT people are leaders, whether in management or not. They try to identify resource problems before they become issue, and have solutions.

  • by linuxwrangler (582055) on Sunday December 04, 2011 @06:34PM (#38260430)

    I've only heard from people on one side of this but the story that I hear is that in the past, many departments had their own IT, mail servers, web, etc. When the campus built its centralized computing services facility, there was great pressure on departments to move to the central system. There was some griping about the costs for central services often exceeding the internal costs the departments formerly had but there was, I'm told, much need to justify the expense of and to pay for the new center. I've heard that some departments have been able to resurrect their internal systems to get through the outage.

    Perhaps someone with more inside knowledge than I have can fill in and/or correct information from both sides of the story.

    That slideshow is pure management-spin right from the opening "look how complicated and difficult this is..." I love how the "solution" to a system that is soon to outstrip its capacity is to stop expanding (and, it appears, properly maintaining) said system and hope it doesn't implode before you can toss the potato to an external party (who can then take the blame). Guess I was never learned at that school of capacity "planning".

  • by Above (100351) on Sunday December 04, 2011 @06:35PM (#38260446)

    The press pretty much reads like this to me:

    1) We didn't size the system large enough to handle the possible outages.

    2) The outage we didn't size for happened, basically taking everything down.

    3) My team is now working on a band-aid solution, which basically involves hobbling the application.

    4) Since we're incompetent, we're going to outsource this next year.

    I mean, if I was the CIO's boss I would have fired him on the spot. Maybe outsourcing is a better answer than putting in place a proper system and looking at that analysis could be interesting. I see no indication any of that was done here, basically the CIO gave the Barbie response, "Mail is hard, let's go shopping." If he doesn't understand how to do it in house, he won't understand how to arrive at a good outsourcing agreement.

    Which means this pretty much sums up everything that is wrong with large org IT today.

    • "Mail is hard, let's go shopping."

      IT isn't a university's core business. When IT was in its infancy there was a case for letting the CS faculty run IT with students volunteering. As IT has advanced in the mean time, the CS faculty can now concentrate on CS and reduce its hands-on stuff to leading edge technologies (e.g. research in super computing, semiconductors, etc...). Nowadays for the vast amount of tasks -even for most CS tasks- ample computing resources, technologies and software can be easily made available.

  • ... at the university. And they haven't been for decades. Microsoft and Google have thrown money and brains at these kinds of problems -- money and brains in quantities that university IT departments can only dream of. Both have economical, reliable, scalable, secure and user friendly solutions to this problem.

    Look up Microsoft live@edu and Google aps for education.

    • by iggymanz (596061)

      Microsoft exchange server is a reliable, secure and scalable solution? Bwhahahahaha, my employer uses that shit, it is none of the above. If it were, the internet's backbone would use it

  • Sigh...

    Look at the first bullet point of the timeline. Productivity suite approved, upgrade to Calmail cancelled. Then a week ago, they decided on an interim upgrade because not upgrading in the first place caused problems. So, rather than a planned upgrade, the IT folks were thrown into panic mode because their (probable) proposed timeline for safely doing an upgrade, including burning in and testing of new hardware, was cut to a fraction of what it should've been.

    You can argue about the budgets, or the IT

(1) Never draw what you can copy. (2) Never copy what you can trace. (3) Never trace what you can cut out and paste down.

Working...