Forgot your password?
typodupeerror
Sun Microsystems IT

Sun's CIO Talks Internal Experiences 115

Posted by Zonk
from the learn-from-a-man-who's-been-there dept.
daria42 writes "This is an interesting interview with Sun's chief information officer Bill Vass, about his experiences as the CIO of one of the world's best-known high-tech company. In particular, Vass talks about corporate blogging (and frustrated lawyers), problems providing IT support to finicky Sun engineers (who sometimes demand Indian help desk support knows kernel details), Sun's programs testing its software internally on employees before it goes out, and how ultimately, his job is like any other CIO's...just with some cool toys."
This discussion has been archived. No new comments can be posted.

Sun's CIO Talks Internal Experiences

Comments Filter:
  • For example, he said Sun president Jonathan Schwartz -- who keeps a public blog -- was frustrated when April Fool's day came around, because he couldn't use his blog to play a practical joke.

    "A few times, he's said things like 'maybe we should acquire Novell', and it changed the stock price," Vass said of Schwartz's blog. "You have to be careful ... if ever he's writing anything controversial he has to get the lawyers to look at it."


    Sun is buying Novell? Ack! I need to go call my stock broker!
    • by Otter (3800)
      Honestly, if the SEC outlawed April Fools Day altogether I'd consider it a net win.
      • Honestly, if the SEC outlawed April Fools Day altogether I'd consider it a net win.

        Hey, dont go grinching up april fools day. That's one of the few holidays I still celebrate whole heartedly! It has also remained fairly pristine in the face of commercialism.
    • "A few times, he's said things like 'maybe we should acquire Novell', and it changed the stock price," Vass said of Schwartz's blog. "You have to be careful ... if ever he's writing anything controversial he has to get the lawyers to look at it."

      What's that supposed to mean....

      Is this implying that his laywers are pissed because they wanted the hot-stock-tips before the general public?!?

      • It means that Schwartz made a joke, and it had a very real effect on Sun's stock price. Even if Sun really was actually acquiring Novell, the executives have to be very careful about what they say. The wrong thing could be seen as stock manipulation, thus placing the exec in very deep legal trouble.

        Half the point of getting an MBA is to learn how to avoid situations with the SEC and other regulatory commissions.
      • Assuming you meant no sarcasm :)

        Lawyers are always concerned about any officer of the company who may make "Forward looking Statements"

        These kinds of statements often lend themselves to SEC FTC violations, and in more recent history, jail time.

        For a brief history in time of Stupid statments CEO's wish thier CFO's never heard,see;

        http://www.google.com/search?hl=en&q=SEC+investiga tes+forward+looking+statements&spell=1 [google.com]
  • by teiresias (101481) on Thursday July 14, 2005 @11:00AM (#13063117)
    Although Vass had no knowledge of this, he soon discovered the system in question was in fact the desktop machine of an engineer who had recently left the company. The desktop had been reformatted following his departure, cutting off 600 users who had over the last three years depended on it for network services.

    Reminds me of a guy whose leaving our company right now. We're probably not going to delete his homespace since lord knows what will break if the things in there are gone.

    It'll take us awhile to get that stuff into a common place. Probably took Sun a lot of time to get that one system back up and running.
    • Reminds me when I got my first Sun Workstation. One of the things that impressed me most about it was that the machine could run tons of services AND support my regular desktop usage without the two impacting each other. It was hard to resist the temptation to load the machine to bear.

      For awhile I was running nightly Mozilla builds, and even considered voluteering to be the build source for Solaris Sparc binaries. (The Mozilla project had a hard time getting Solaris builds back then.) Sadly, I left the company before I could volunteer. I imagine that if they had that machine plugged into the network and turned on, it would still be building Mozilla every night, automatically. :-)
    • by chill (34294) on Thursday July 14, 2005 @11:33AM (#13063388) Journal
      Reminds me of a guy whose leaving our company right now. We're probably not going to delete his homespace since lord knows what will break if the things in there are gone.

      This is why workstations should be workstations and servers should be servers. Allowing users in a client-server environment to share resources from their workstations is bad network design/policy. Add a cheap server and give them space, but sharing should be disabled and disallowed on workstations.

      -Charles
      • Allowing users in a client-server environment to share resources from their workstations is bad network design/policy.

        He said his homespace not his workstation. It's probably /home/user which is NFS mounted on the workstation.

        But he built scripts that referenced ~user/bin/ all over. Of course, it's not really that hard to grep around for 'user' and s///g with a new system-level shared directory.
      • This is why workstations should be workstations and servers should be servers. Allowing users in a client-server environment to share resources from their workstations is bad network design/policy.

        Maybe it is on a server, but a lot of critical stuff depends on what's under his public_html dir.

      • For a large company, user home directories are usually stored on a communal server, which allows the admins to back everything up from a central point.

        Unfortunately, it also allows project build trees to be reconfigured to search user directories.
      • Bah. I can't tell you the number of places I've worked where "Single point of failure" was the rule. In an ideal world, yes, we'd have all the hardware we need, but in the real world it is difficult to convince the PHBs why you need another machine that does X when you already have one.

        So the desktop becomes the emergency backup system. Every single service I am responsible for is duplicated on my desktop. It functions as my test environment, but, if it hit the fan, I could swap it out with the primary app
        • So the desktop becomes the emergency backup system. Every single service I am responsible for is duplicated on my desktop. It functions as my test environment, but, if it hit the fan, I could swap it out with the primary app server and things would keep running.

          And when the app server hits its max load, and I can't add more stuff to it, and I can't spend my capital budget for 5 more months, where do the new apps go? The one place they can go.


          I've always been lucky enough to be in the position to say "no,
          • Yea, it'd be nice. They do listen to that kind of logic here, thankfully, so I only have to worry when I'm too industrious and make a production-quality application on a crap-quality machine, and then have to hope it holds together until I can spend money to get a replacement.

            The last place I worked however, I came in one day and found a nest of cat5 on the floor in my office, and a sticky note on my monitor that said, "DO NOT REBOOT". My desktop had become the primary mailserver.
    • In an earlier company my desktop workstation was a Sparc server for the other Sparc users in my group (3 or 4 of them). One day I was moving cubicles, and I completely forgot that my machine was a server, so I just shut it down and moved it over. I immediately got a bunch of complaints from the guys who couldn't access their files any more. :)
    • by Lumpy (12016) on Thursday July 14, 2005 @11:46AM (#13063505) Homepage
      this problem is typically management's fault. or caused by management.

      I am the guy who just replaced one of those engineering guru guys. he has crap EVERYWHERE running critical systems and data collection as well as processing. someone powered off his PC and crashed the billing system.

      After digging in his notes and code as well as his old email I discovered that he wanted to do things right, he had a subversion server set up and a development as well as a production server in the server room.

      But, management did not allow him to do his job right. I saw endless emails and messages about needing X Y or Z right now! did he finish Z yet? why is Y not in testing? who told you to stop working on X?

      it was endless so the poor guy had to half ass everything because management refused to hire him any help, refused to accept realistic deadlines or adjust importance... everything was top super critical!

      I was promoted to this position, I was able to find out most of this before accepting the promotion and told them that I work very differently. I use project management, refuse to work on 5 things at the same time as that creates 5 crappy, broken things as well as makes the process 10 times longer. I explained my concerns to the divisional VP that interviewed me and he agreed that that working atmosphere was not acceptable and told me that I have his authority to tell my superiors that they have to sort out priorities themselves and that EVERY new project request will come in at the bottom of the to-do list unless it has been signed off by the VP of operations to deserve to be escalated above everything else.

      The origional mess was cause by management. and until someone in management gives a peon engineer or programmer the authority and protection to tell other management "nope, sorry." it will never get any better.
      • A couple of things:

        The business needs drive IT needs.

        If it takes an act of congress and a 15 email thread for an engineer to get real access to proper server, this kind of things will happen.

        Everything that goes on in a company, good or bad, is management's doing.

        The other thing is that they had backed up his hard drive, before mirroring, after he left. Is that common practice at many companies? My little finger is tingling with the "it's all company data, so yes." Can anyone confirm/deny?

        We're in the
      • If I had real-life modpoints you'd get one. You've avoided a mistake that is so damn easy to make; taking on too much work and/or allowing overlings to saturate your available capacity into oblivion. Saying no (with reasonable arguments) is something that techies are not inclined to do (in my oh-so humble opinion of course).
  • Herding Cats (Score:5, Interesting)

    by DanielMarkham (765899) * on Thursday July 14, 2005 @11:00AM (#13063119) Homepage
    While the CIO is more businesman than technical guy, his job has to be like herding cats. FTA
    In another example, Vass received a mysterious note that a major system had been disabled and had stopped production on a hardware chip. Although Vass had no knowledge of this, he soon discovered the system in question was in fact the desktop machine of an engineer who had recently left the company. The desktop had been reformatted following his departure, cutting off 600 users who had over the last three years depended on it for network services.
    I knew a major financial company, which will remain nameless, that rolled out a new customer product a couple years ago. When I was talking to the architect, it turns out the whole thing was running off a computer under his desk! He said that one day he accidentally kicked the power switch, and the whole place went beserk. I "encouraged" him to perhaps move it to the server farm, where it would be a little safer. He declined, saying he "wanted more control" over the application until it was stable enough. And this was on a production product.
    I do NOT envy the job of CIO. Those guys have a tough row to hoe. BTW, if you ever want to know how the industry is being perceived by business, CIO magazine [cio.com] is a great read.(but expensive) It's real eye-opener to hear things from the other side of the tracks.

    Moore's Law: Not the Only Game in Town [whattofix.com]
    • Its Eleven where I am, so heres the FILM.

      http://www.candlelightdreams.com/videos/funny_cats _1.wmv [candlelightdreams.com]

      Just found this, free(as in freedom) and open ( as in red light district) no DRM :), its good for a few laughs.
    • by jellomizer (103300) * on Thursday July 14, 2005 @11:24AM (#13063321)
      When you are CIO of a technical company it is tempting be lax with policy and give the employees more access then they should have, it seems like a decent policy, first you save money because the desktops that people use anyways are also the servers so you don't need expensive servers, the technical people can administer their own system, and whatever they are serving.

      But being a CIO you need to be a Dick every once in a while and make sure the technical people have the only the access they need to do their work properly. Have the IT department put buisness level servers in the server room and have them properly managed.

      While the first way seems quicker and easier and has less personal conflect. The second way is better to manage and reduces of mission critical mistakes. It also allows for proper upgrading for the future.

      Sure the employess can do the work themselvs but they rairly consider the big picture and end up with a spread of services which are hard to track and manage. It also creates a situration where an employee cannot be moved to a different position because they have the information that others dont.
    • by Anonymous Coward
      When I was talking to the architect, it turns out the whole thing was running off a computer under his desk!

      Posting anon to avoid any affiliations...

      But one of my co-workers (we're an ISP) was running an aggregation router for about 10000 xDSL users - in his room. It was quite hot in there, but he preferred it that way because he could access the console (via serial port) with just a simple cable from his laptop. Suggestion to move the router to a rack with a terminal server was not accepted because it
    • by Anonymous Coward
      Bah, that's nothing. I know a guy who was fired by an IT director because the poor guy accidentally kicked loose the wall-wart transformer for an ISDN router under his desk.

      Turns out the router was supplying a free ISP service to the IT director's wife, who was running a recruiting firm and web site at the company's expense over the ISDN line!

      The transformer kicker asked something like "is that really ethical?" and was fired the next day for some completly trumped up crap. But we all *know* why he was re
    • He said that one day he accidentally kicked the power switch, and the whole place went beserk. I "encouraged" him to perhaps move it to the server farm, where it would be a little safer. He declined, saying he "wanted more control" over the application until it was stable enough. And this was on a production product.

      Why would a CIO have problem ordering this person to move the service? Are they not given enough authority to can people not working in the best interests of the company (within reason)?
      • Jeff,

        Just in case you've been wondering all these years why dutch people keep sniggering at you. It's because your nick translates as 'Jeff The Virgin'. If you ever come over you might want introduce yourself as just Jeff.

        Cheerio!

    • CIO [cio.com] magazine is indeed a good read. It's not expensive, though. It's free. Soul-sucking registration is required.
  • by WarmNoodles (899413) on Thursday July 14, 2005 @11:06AM (#13063171)
    >However, some unusual problems did surface sometimes, he said, citing the example of a Solaris engineer who contacted Sun's IT help desk in India and subsequently sent Vass a note complaining the help desk member who assisted him didn't know intricate kernel settings for the operating system he needed help on.

    Can you imagine that call, "I am so happy to be helping you, however, I am sorry to be informing you that... pause...I am not being the Dammed premier kernel support line! " SLAM!

    lol
    • I am sure this was some lame ass attempt by some engineer who couldn't figure out why he could not open 18000 file descriptors or why his "malloc 2^64" was failing, and thought changing the kernel time slice parameters to reduce context switches would help.

      The poor guy in India probably had a Master's Degree in Solaris Kernel Tuning and pissed off the engineer by telling him he was an idiot.
  • by jellomizer (103300) * on Thursday July 14, 2005 @11:08AM (#13063189)
    I would be angry too if I called up technical support and I couldn't get kernel level knowlege. Most administrators know or at least use to know enough about the platform they are admistering to handle most of the problems and then for other things the search the web and blogs, for more help. If this fails them they have a good question for technical support. And having to go threw level 1 then 2 then 3 technical support is just annoying, and a waist of time. Technical support should be able to quicly figure out the complexity of your problem and move you to the aproprate level. If I am adding a user the person who answers the phone should give me to level 1, but if I am configuring the system kernel options then I should be placed on a higher level support.
    • I would be angry too if I called up technical support and I couldn't get kernel level knowlege.

      Really? I normally don't expect anything more than the most basic support when I call. After all, I assume theres a reason this person is answering phone calls (spending most of his time helping people find the "any" key) and not applying for my job. Which is probably why I almost never call tech support in the first place.

      • This isn't AOL new user tech support. This is commercial technical support that professionals use. Most of their callers know the basic admin stuff. They call tech support when they have an actual problem.
        • True, but at the end of the day, both of their jobs deal with the same technical system and I would expect the caller is making more money than the person he's calling. So it only makes sense to expect useful help when you are exploring an area of the system you are not familiar with, rather than when you are dealing with its intricacies and have come across a problem.
          • They may make more money than the person that they are calling, because the engineer is an 'expert programmer' writing (a subset of) the system in question. When they have a problem with using the system, they call the help desk, which should be 'expert users'.

            I think that asking the programmer that wrote the grammar checker in MS Word about a file configuration problem, and you probably wouldnt get a very good answer. You ask the help desk, and you're damn right you should.
    • by frodo from middle ea (602941) on Thursday July 14, 2005 @11:31AM (#13063376) Homepage
      I remember a few years back when we had problems with one of our 0r@c13 Database. One of the tables had about 10 million records, and we had to run a complicated query that required a range scan.

      The query was quite complicated, but analyzed and tuned to the best of our DBAs ability.

      THe query would hang the listener every once in a while and then no connections could be made to the DB over that listner, the only solution was to bounce the DB all together.

      When we called them for this problem, instead of looking at our query /db schema, all they said was try running the query with a few 100 records.

      This was the response when we were paying them for a level 1 support and the problem was rated severe.

    • by AnalogBoy (51094) on Thursday July 14, 2005 @11:40AM (#13063449) Journal
      This is my newest pet peeve. Tech support, outsourced, gets a crash course it seems in provinding support. They never do it well. I've had the fortune of calling Microsoft's professional line several times, and i always get someone who is 1) difficult to understand, and 2) not any more knowledgable in the app than i am. I don't call tech support for the "Two heads are better than one" approach, i call tech support to hear "Yes, we've seen that before, here's how to fix it." Unfortunately i haven't heard something like that since my Sun days. And another thing - when they start troubleshooting at a step you tried 3 hours ago and say "we have to try this sir"... grrr.. i get angry *instantly* at that. 0 to pissed in one processor tick.

      Disclaimer: I have nothing at all against Indians - however, i do believe if i call tech support i should be able to clearly understand you. I've worked in a call center before and clarity was an important benchmark - i guess if you can get people to work for small wages your standards go down a bit too. Unfortunately it looks like Outsorcing for tech support is here to stay, as unappealing as that may be. Sprint, Dell, Microsoft... grr. I call upon Shiva to bitch slap them all!

      • As much as most people dispise M$, their enterprise support with MSDN registration is actually one of the better ones. I don't know about their low-level homeuser windows xp line etc.

      • however, i do believe if i call tech support i should be able to clearly understand you. [...] I call upon Shiva to bitch slap them all!

        to steal a phrase, shiva H. vishnu!

        that's part of the ploy. keep the customer confused and eventually, he'll stop calling. if he can't understand you, he'll also stop calling. either way, problem 'solved'.

    • Escalations (Score:4, Informative)

      by British (51765) <british1500@gmail.com> on Thursday July 14, 2005 @12:31PM (#13063939) Homepage Journal
      Here's how escalation works...

      1. Customer calls tech suppport(level 1)
      2. Level 1 can't fix it. Fills out an escalation form to level 2. The unseen beings of level 2 are supposed to call back. A "trouble ticket" is made to great detail by level 1 tech, apologizes to customer.
      3. Time passes by
      4. The unseen overlords of level 2(or escalation department) forget about the trouble ticket, hoping the customer and level 1 forget about the trouble ticket
      5. level 1 prays customer never calls back, since he/she heard nothing from level 2 about it, and never will.

      That's at least what happened when I did tech support for an ISP. I think I later checked on the customers with escalations, and they, well, weren't customers anymore.
      • I think I later checked on the customers with escalations, and they, well, weren't customers anymore.

        Well, then, that took care of the problem, I'd say.

    • ...Most administrators know or at least use to know enough about the platform they are admistering ...

      "Hi. I'm having a bit of trouble with what appears to be ambiguous interrupt return addressing in a legacy driver, and I'm looking for a bit of info. Can you help me?"

      "Have you checked to make sure the device is powered on?" -- Nullus stercus, ipi eram.

  • Engineers (Score:1, Funny)

    by mfloy (899187)
    "problems providing IT support to finicky Sun engineers (who sometimes demand Indian help desk support knows kernel details)"
    Engineers making outlandish requests is as common as Microsoft making buggy products. Good enginners and famous rock stars both need to be a little weird to be succesful.
  • Grammar... (Score:4, Funny)

    by Hugonz (20064) <hugonz&gmail,com> on Thursday July 14, 2005 @11:17AM (#13063255) Homepage
    ...who sometimes demand Indian help desk support knows kernel details

    I, for one, welcome our new Indian poor grammar kernel hacker overloards...

  • by mparaz (31980) on Thursday July 14, 2005 @11:42AM (#13063469) Homepage
  • Wow. That was fluff. I want my 30 seconds back.
  • >However, some unusual problems did surface sometimes, he said, citing the example of a Solaris engineer who contacted Sun's IT help desk in India and subsequently sent Vass a note complaining the help desk member who assisted him didn't know intricate kernel settings for the operating system he needed help on.
    well, if a solaris engineer doesn't know anything about the kernel, what can we say about the product..Sun needs a lot of good luck..oh wait..I forgot Microsoft!!
  • Had a chance to meet Mr. Vass at the 2005 TechSouth [techsouth.org] Expo where he gave a keynote. Cool guy, good demonstration of Sun Rays and their mobile desktop technology. Being able to insert a smart card and pull up his SunOS and/or Windows XP desktop in Lafayette, LA from their corporate HQ in a matter of seconds was pretty cool. He also took the opportunity to send some barbs towards Cox and Bellsouth over the proposed Fiber-to-the-Home [lafayettec...gether.org] initiative ("Stuff like this is why it's so great you guys are getting fiber ru

On the Internet, nobody knows you're a dog. -- Cartoon caption

Working...