Deferred IT Maintenance Is a Ticking Time Bomb 186
snydeq writes "The underfunding of routine hardware replacement purchases and the degradation of aging enterprise apps pose systemic risk for many IT organizations, thanks to a ballooning 'deferred IT maintenance debt' in the decade since Y2K fears pushed enterprises to invest heavily in essential system upgrades, InfoWorld's Bill Snyder reports. And with sysadmins 'scrambling to keep systems up and running with budgets that barely cover the basics,' this 'IT debt' promises only to increase in the coming years, especially as IT continues to defer routine maintenance in favor of new 'cost-saving' initiatives, particularly around the cloud."
Re:How is this newsworthy? It's just common sense. (Score:4, Interesting)
Because much of this IT is stuff that affects individuals who have no influence over it.
When a company puts off investing in security, for example, and when they also collect and store my credit card info / medical info / personal demographics / shopping history / etc., they are putting me at risk.
I have to trust that their IT department is on the ball. Something I am beginning to think is never a good idea. But it's impossible to not give companies some info on me and still be a normal modern human, and thus I am forced to trust them all the time.
So if they're further neglecting their IT, it means my data is more vulnerable. Not that's there's a damn thing I can do about it.
Re:How is this newsworthy? It's just common sense. (Score:3, Interesting)
I need to make some friends in the news media.
life of a sysadmin (Score:5, Interesting)
I worked for another company that had a lot of money, but one thing we had to deal with was printing. Print jobs would come into our machines from strange places (IBM mainframe machines, from programs that were written 40 years ago) and go out to strange places (old dot matrix printers in a field office out in some obscure city in India). Thus I was sometimes left to puzzle why some program written in PL/I, coming from a mainframe which I don't have access to, is not printing to some ancient printer in Bangalore which is hooked to some ancient PC's parallel port.
My former company from 2009 had some machines like this. Two very old Ultras running StoryServer and who knows what else. The StoryServer license had long fallen out of use, the machine firmware and Solaris OS had not been upgraded or patched for years. It sent e-mail through, for some reason, four Macintoshes. The Macs did not even run MacOS X, they were previous MacOS versions. E-mail starting with the letters A-F went to Mac1, G-M went through Mac2 etc., if a Mac crashed, mail to those letters would stop going through. The developers did not want to spend the time migrating to a new system, and I don't blame them, the oldest long-time developer there who dealt with such arcana was laid off, while the people building the latest new and shiny that the business wanted had the most secure jobs. Aside from this, we did not ever patch or upgrade our Red Hat Dell servers or firmware, we had no scheduled system downtimes etc. Our major Java application server had had its license run out. As I was leaving, the operations boss (soon to be fired) was considering not re-upping our Red Hat licenses.
If a sysadmin goes on a job interview, and is not desperate, these are the types of questions they should ask, at least on the second round of interviews. Are all of the machines, OSs and applications I'll be responsible for under license? Are they all fully patched and upgraded for firmware, OS and application on a regular basis? What is the oldest machine still under responsibility - is it older than three years? Because all servers should be phased out every three years - at the very least. Try getting Dell/HP to support a 7 year old server decently. Also, do you have scheduled downtime once a week? Meaning do you have the option of rebooting and patching your main database machine, even if it is early Sunday morning? If they want 100% uptime it would necessitate paying for the infrastructure for high availability.
Why should they spend the money when they can just call you in the middle of the night, to continue keeping it running with duct tape? Then they can blame you the next day after it broke. And you get no credit for it continually running either - the time you spend keeping it running is not counted, only time you devote to the latest shiny they want to implement. In fact, too much time devoted to keeping the machines they decided not to spend money on keeping up can cost you your job - if there's a choice between laying off the guy maintaining legacy stuff, and the guy who makes the new shiny for the business group and management and who deals with the
Re:degradation of aging enterprise apps? (Score:5, Interesting)
1) The languages, special hardware, libraries and controls become unsupported on new hardware.
2) The languages, special hardware, libraries and controls become unsupported on new versions of the operating system.
3) The operating system becomes unsupported.
4) The hardware becomes unsupported.
Example: VB6 program uses bar code scanners.
2004? VB6 unsupported as a language.
2008, VB6 unsupported for security patches (so any required security patch could kill VB6 program)
2009 bar code scanners unsupported (change to optical recognition with new software interface)
2009 VB6 Outlook/Word integration fails.
2010 Hardware and operating systems to support VB6 start becoming unavailable. All are unsupported by vendors.
Cost to redevelop VB6 program-- about 1.6 million dollars.
At some point- basically find a new packaged product (cost $100k + $500k user licensing & support + loss of ability to differentiate business) which provides 80% of functionality of the VB6 program and toss it. Can't be changed to match your business - you must change business to match it.
Re:How is this newsworthy? It's just common sense. (Score:5, Interesting)
Well, it is sort of a "duh" story the way it is written, but OTOH the subject is not without merit.
I have been involved with infrastructure assessment of companies prior to acquisition and some stuff is just shocking. Publicly owned companies are driven by return to the shareholders; one way to keep the dividends flowing when the economy is in a downturn or when the business plan isn't working is to reduce operational expense.
Releasing employees is very effective to reduce the spend side but usually that means there is less available effort to work on maintenance. It looks good to have all employee time capitalized on projects but who is keeping stuff working? Also, each person out the door takes expertise with them that is lost to the company. After a while, the company may not even have enough knowledge internally to understand that their boat has holes in it and that patching isn't happening.
This isn't smoke; I've seen it. Data centers with overheating problems and with inadequate standby generators. Power is distributed unwittingly to cause a cascading failure if one breaker trips. Leaking roofs over financial servers (plastic tarp and bucket gave that away). Licensing that has not been kept up to date because no one has a good inventory and no one wants to look-see. So... Oracle enterprise instances running in non-secure network zones and without proper licensing ( potentially million$ in back costs). A database server being used as a network monitoring node and firewall because funds were not available to separate the functions.
Deferred infrastructure investment and maintenance investment happens and it is a ghastly mess to clean up. I am not surprised that more of this is happening.