IT Crises vs. Vacation: Sometimes It Isn't Pretty 352
CWmike writes "It's true that IT systems have become essential to business operations, but the successful functioning of the IT department shouldn't rest on any one person's shoulders. All told, vacations serve as mini tests to prove if a department can function when key players are away. That's the theory, anyway. In reality, IT departments sometimes flunk. The results can either be comical or turn out to be a serious wake-up call to organizations that need a better Plan B. To prime your mental pump before your own vacation, Computerworld compiled anecdotes about good vacations gone bad."
Another reason for vacations: crooked employees (Score:5, Interesting)
I'm personally aware thru my late father, who was an accountant, of two companies that had employees embezzling funds for years. One telltale sign was that they never took vacations, because their replacements would have discovered what they were up to. Businesses should insist that their personnel take time off, just to make malfeasance easier to detect.
Re:The real problem (Score:5, Interesting)
at least in the states is, companies have figured out they can get one person to do the work of two and pocket the other guy's salary.
This.
I'm working at a small hospital. Our entire IT department is just three people - one clinical liaison, me, and our manager.
The clinical liaison is awesome at what she does, but she can't build a server or fix a network issue or any of that. Her job is to train the nurses and explain the issues they have in terms we can understand and things like that. She isn't supposed to be technical support.
My manager is certainly skilled... But he's stuck in meetings most of the day, or working on grant proposals, or putting together purchase orders, or whatever. He's rarely available to fix technical issues.
Which means that all of the day-to-day support, and most of the longer-term projects, fall on my shoulders. I've been begging for another technical person for months, and it just isn't happening.
So I'm getting stuck working longer hours... And support is still suffering. It takes me longer to get to the little things, which gives them time to grow into bigger things. And the bigger things are getting fixed as quickly as possible, which means corners get cut. There's not enough time to properly plan/implement/train on new projects, which results in more things going wrong...
The end result is that I'm doing sloppy work, and causing more problems, which means more sloppy work... I can see what is happening, but I can't really do much about it. There's only so many hours I can work before becoming absolutely useless (As the 17-hour day I put in last week showed very clearly. I was completely useless the next day.)
I've got vacation time coming up next week... Which I've had to cut short, due to go-live for a new product... But I've still got a few days off. And, while I'm really looking forward to the break, I'm kind of dreading what I'll face when I come back to work.
Vacation? we don't allow that nonsense here! (Score:5, Interesting)
At a previous company I worked for I went away for 3 days (friday-sunday) at a company that only worked mon-fri, so it was essentially just a single day off. I made everyone well aware that I would be out of town and unreachable.
I returned on Monday to find my boss had flown in from out of town and was sitting cross legged on the server room floor with one of our servers in pieces all around him. He informed me that there had been a hardware failure over the weekend and that I should have been there to deal with it. After I finished fixing the original problem, and the problems he had created by trying to fix things, and once everything was up and running again, he asked me for my security pass and escorted me off the premises citing "budget cuts".
Probably much better that I don't work there anymore...
Re:There can be only one... (Score:3, Interesting)
The flip side to being "one deep" is you are more valuable. I would lean towards hoarding knowledge and being on-call. I don't WANT my employer to be comfortable functioning without me.
Be good at giving verbal instructions and at typing them on the fly for emailing.
Vacation? How about WEEKENDS? (Score:5, Interesting)
My current company, has no vacations. You simply tell them when you are not going to be there, and they decide if they want to fire you for the absence.
They also do not have weekends. On the Friday before each customary "3-day" weekend the owner declares an emergency that, somehow, MUST be finished by Tuesday.
No one wants to work there for very long. Turnover is very high. Projects don't get finished, precisely because of the turnover. Other projects do get "finished', but don't work, also because of the turnover
The owner doesn't seem to realize that he is sabotaging his own projects.
Every time (Score:5, Interesting)
Every time I go on holiday, something weird will happen with a system that's been running perfectly for years. It's guaranteed. And it won't just be because I've been kicking that system back into action all the time and "would get around to fixing it", I would get the really esoteric interconnected problems that suddenly crop up out of nowhere and you're never entirely sure you've solved until months afterwards.
However, my employer knows I'm on the end of a phone if it is indeed an emergency. They have called on me in Italy several times. The trick is to take holidays FAR AWAY from your place of work, and then they can't do anything but cope without you. Flying back to fix a company server? No thanks. Not unless you provide DOUBLE the time I'd taken off in lieu as compensation for ruining my long-planned holiday through poor planning / hiring. If a company can't cope without any single individual, then its hiring policies suck. What would you do if he went under a bus and was *never* coming back?
The worst that's ever happened to me is that I lined up my own brother to go into the company should the emergency they were having not be solved by my instructions. It was, however it would have be after-hours, because he works too, but they would at least have someone there who knew the right switch to press, could be talked through a RAID rebuild, etc. and not have to be led every step of the way and incur only a single day's downtime without making things worse.
Think the downtime wasn't important? It was a school with automated billing, parental contact, phone system, heating controls, registration, medical records, salaries, you name it, not to mention IT lessons and exams. Without registration, etc. the school is legally not allowed to open because they have no records of which children are where, no medical records, etc. Guess what? They coped for the day because they had contingency plans (i.e. cancel all IT lessons and do something else instead, catching up again next week, manual financial control, manual registration, etc.). There are very few companies that *can't* carry on if the IT dies. It might be inconvenient, it might mean harder and more work, but it's rarely impossible unless you're something like an ISP or a datacentre.
If there was nobody else suitable to come and fix the problem? Not hard - hire an IT guy to come in. You do have support contracts for your gear and software, yes? Or you could organise an emergency contractor to visit and fix your problem? It's not hard and the only problem there is finding the right guy (i.e. someone who *can* walk into the middle of a mess and at least get something working enough to last until the "real" IT guy gets back).
If you honestly, genuinely can't cope without an employee - you need him to train an assistant, or even two. It won't be perfect but it's better than nothing. Failing that, you need a large enough team that you can do something on the guy's instructions. Failing that, you need your support contracts which pretty much come as standard with business-level hardware/software. Failing that, you need a contractor at short-notice. If you can't do those four and get to a working system of some fashion within 24 hours, you were always going to be in deep shit whenever anything went wrong anyway. What would you do if the guy left and you had to find a replacement? What if he died? What if he suffered amnesia and forgot all the passwords? What if he was arrested? What if, what if, what if. Or you could just do the normal IT-thing and have backups - lots of them.
Nobody is that invaluable that they have to abandon holidays and drive away from their kids to come back after-hours. Sorry, it's just not true, and if it "is" then that's only the company's fault. It's purely a money saving solution rather than hiring someone else to fix the mess - get the guy who's away on holiday and pay him for a few extra hours - it's cheaper than calling on your support contracts or call-out fees for an emerge
Re:There can be only one... (Score:5, Interesting)
I off and on contracted for a place where this was hugely important.
What they'd do is come up with some bullshit reason they were giving a bunch of people 2 or 3 extra weeks of vacation but it must be taken within a short time frame (that quarter or the next two quarters), usually times to align with the summer already planned vacations, and sometimes not entirely bullshit.
Either way, if you were gone for about 3 weeks, and no one really needed you for that time off, your job was going to be axed shortly. Maternity leave? No problem, your job will definitely be here when you get back because we'll try not to fill it at all, and if we don't need it, you're gone as soon as we're legally allowed when you get back.
It's slimy, but it's business.
Fault tolerance is a serious problem. If you only have two people who know a system, both of whom work in the same area, and both get the same infectious disease for a week you have a problem. On the other hand, having 3 or 4 people with redundant skills is a waste of money. I can see the appeal of cloudsourcing to a 3rd party in that regard.
On a personal basis, if you don't have something you, and only you can do until the day you retire you're taking serious risk. That doesn't have to be technical of course, you can be the only one who knows how to deal with the crazy redhead secretary in another department who bothers you all the time, or you could be the only one who knows how stuff in storage is laid out or whatever. It's a tricky balance between 'manpower intensive to replace' and 'crippling the company if you get hit by a bus'.
Re:The real problem (Score:5, Interesting)
I used to think like you do. I believed that it is every man for themselves and that money in the bank was the only way to assure my freedom.
But now that I have a ton of money in the bank and am effectively retired and can do as I please, I am not so sure. I am the richest person I know. Everybody else I know still puts in the 9-to-5 (or often 8 to 7 and sometimes weekends) grind.
I don't think it is feasible to expect everyone, or even a simple majority, to achieve what I have achieved. Maybe its because, looking back, I can see that ~90% of my success is due to nothing more than being in the right place and the right time while only 10% is due to my own fortitude. Few people will ever be lucky enough to find themselves in similarly favourable circumstances.
So while I am still strongly in favour of some the things you wrote, like a competitive market for healthcare. I am not so sure that a go it alone approach is ever going to be successful given the array of forces organised against the modern peon.
Happened once (Score:4, Interesting)
They panicked at Boeing when they couldn't get me after hours. So I told them to hire backup. They transfered a guy in from IT who (supposedly) knew Perl (and other stuff). So, on the first day when I was showing him the ropes, I opened up one of the CGI files to show him some of our coding style and conventions. He stared at the file for a few minutes /usr/bin/perl staring him right in the face) and asked, "What language is this written in?"
(with #!
Management had a bad habit of calling everyone down to the shop floor at any hour if any little thing went wrong. I had remote login capability and could generally fix anything from home. One night, I get a call. Panic! All hands to the shop! So I ask, "What's the problem? Maybe I can fix it from here." I'm told I've got an attitude problem. So I get dressed and drive in. An hour and a half later, after reading the error message on the ATE equipment console, sure enough, a 15 minute fix from any terminal in the world.
Often, crisis are engineered to make people look important. Build a system that never dies and nobody notices you. Build one that causes trouble (or hire people who move their lips when they code to maintain it) and you get noticed. And promoted. We had a saying at Boeing: Heads roll uphill.
Vacation not as bad as quitting (Score:4, Interesting)
Supposedly, they are interviewing replacements, but so far I think they are patting themselves on the back for saving money (about half an IT person's salary, as a guess) and spreading around his work mostly to the overworked development group, including myself, who are now getting surprisingly little development done.