Forgot your password?
typodupeerror
Businesses Stats IT

Ask Slashdot: Good Metrics For a Small IT Team? 315

Posted by samzenpus
from the measuring-up dept.
First time accepted submitter shibbyj writes "I'm a member of a small 3 person IT team for a medium sized business (approximately 300-350 employees) that has multiple locations internationally. I have been tasked with logging our performance using the statistics from our ticket management system. I've also been tasked with comparing these stats and determining if we are performing above or below what is considered optimal. I'm wondering what people opinions are on what good metrics should be in regards to mttr mtbf etc. I have had trouble finding information on this."
This discussion has been archived. No new comments can be posted.

Ask Slashdot: Good Metrics For a Small IT Team?

Comments Filter:
  • Hahaha (Score:5, Insightful)

    by mewsenews (251487) on Thursday December 15, 2011 @09:26PM (#38392346) Homepage
    One of you is getting fired
    • Re:Hahaha (Score:5, Insightful)

      by ITConsultant (2525166) on Thursday December 15, 2011 @09:36PM (#38392432)
      As a professional IT consultant, I would concur with the parent statement.

      Also, the title is a bit redundant: all IT teams are small.
      • Re:Hahaha (Score:5, Insightful)

        by Anonymous Coward on Friday December 16, 2011 @01:22AM (#38394186)

        As an IT professional who was just fired from a job shortly after management started to ask for all sorts of new metrics from our ticketing system, I also fully agree with the parent statement. Get your resume together and start getting it out to recruiters pronto.

        In other words, this is a sham. There ARE no standard metrics because every single company and department are different. The only real metric that matters is if your customers are happy and satisfied, and that they could find out better than you could. You're being set up.

    • Re:Hahaha (Score:5, Informative)

      by Anonymous Coward on Thursday December 15, 2011 @09:53PM (#38392622)

      Yes, this.

      Management wants to eliminate someone and wants to do so in an "objective" way to hide the fact that they're firing someone while probably giving the CEO a fat Christmas bonus. You're tasked with figuring out which of the three of you gets fired and how you can cloak this in enough "objectivity" that no one can object to it. Your best bet is to make this shit up. Figure out who the weakest link besides yourself is, or who you like the least, and generate a system of metrics that's biased towards eliminating that person. Use lots of acronyms and jargon. Also, make sure no one at work reads Slashdot.

      totally irrelevant CAPTCHA: forgive

      • Re: (Score:3, Insightful)

        by Anonymous Coward
        If there were any justice in the world firings would start at the top. Fuck management and their big salaries. What the fuck are they doing that they have to be paid so much? What value do they bring to a company? They're too happy to fire you to make more money for themselves (the fact that most people are one paycheck away from financial ruin doesn't seem to bother them) but do they ever get fired even if they run the company into the ground? Fuck no. I'd love nothing more than to see some of the bosses
        • Re:Hahaha (Score:5, Funny)

          by Jeremiah Cornelius (137) on Thursday December 15, 2011 @10:11PM (#38392800) Homepage Journal

          The very core of your writing while sounding reasonable initially, did not sit perfectly with me after some time. Somewhere throughout the paragraphs you actually were able to make me a believer but just for a very short while. I however have got a problem with your leaps in logic and one would do nicely to fill in all those gaps. If you can accomplish that, I will surely end up being fascinated.

          • Re:Hahaha (Score:4, Insightful)

            by Anonymous Coward on Thursday December 15, 2011 @11:23PM (#38393468)

            Yet another AC here.

            If there were any justice in the world firings would start at the top. Fuck management and their big salaries. What the fuck are they doing that they have to be paid so much? What value do they bring to a company? They're too happy to fire you to make more money for themselves (the fact that most people are one paycheck away from financial ruin doesn't seem to bother them) but do they ever get fired even if they run the company into the ground? Fuck no. I'd love nothing more than to see some of the bosses I've had in the past be out there doing actual work.

            Is this better?

            • Re:Hahaha (Score:4, Funny)

              by wmelnick (411371) on Thursday December 15, 2011 @11:31PM (#38393528)
              An obvious troll. In order to become management you have had to prove yourself over the course of many years. While you may think those above you are useless, they would not be there had they not done what you did first and for probably longer than you have.
              • Re:Hahaha (Score:4, Insightful)

                by slippyblade (962288) on Thursday December 15, 2011 @11:44PM (#38393616) Homepage

                Do you actually believe this? Seriously? Most upper management is there because they either knew someone in the right position or had enough money/clout to force their way into it. Often time through family. Rarely these days do you see any upper corporate management that actually worked their way up.

                • Re: (Score:3, Insightful)

                  by phantomfive (622387)

                  Do you actually believe this? Seriously? Most upper management is there because they either knew someone in the right position or had enough money/clout to force their way into it. Often time through family. Rarely these days do you see any upper corporate management that actually worked their way up.

                  Do you have statistics or data that shows this? Because my experience has been the opposite, people do become upper management because of skill. Except in companies that are about to fail.

                  My guess is you just made up your facts there, but if you have hard date, I'm willing (and interested!) to change my opinion.

                • Re: (Score:3, Insightful)

                  by DigiShaman (671371)

                  Pretty much. Either you earned an MBA from a respected university and groomed into the role (very quickly I might add), or they hired someone who worked in management from a completely different industry (a lateral move). Management isn't about having the ability to offer an extended work knowledge base to your subordinates. Management is all about leadership and when to crack the whip. This role is also very self serving. They will not go down without sacrificing their underlings first as though they're a

                  • Management isn't about having the ability to offer an extended work knowledge base to your subordinates. Management is all about leadership and when to crack the whip

                    This kind of thinking has caused a number of companies to fail. That's part of management, but the most important part of senior management is providing direction (there's a hint in the job title for a lot of them...). This requires them to have a detailed understanding of the market. A successful manager needs to understand three things:

                    The first is what his subordinates are capable of. When an engineer says 'that's impossible' does he mean 'I can't be bothered?' Or 'we could do that, but only with f

              • Re: (Score:3, Informative)

                by WOOFYGOOFY (1334993)
                Your comment represents a type f fallacy I believe is called, , because it's so, therefore it's so. Essentially you're saying because they are in management they must have earned it.

                First observe that no matter what, someone HAS to be in management. CEO is a position, unlike yours, which cannot by stay empty for long.

                Second, all that has to happen to rise in a company is someone above you promotes you. People who bet on Skill and Hard Work taking them there are a dime a dozen and what's become essenti

            • by rev0lt (1950662)
              I'm management on my own firm and I don't get a big salary and sometimes work 14 hours a day, 6 days a week. What am I doing wrong? Should I slack more often?
        • by sleigher (961421)
          I find your ideas fascinating and would like to subscribe to your newsletter.
    • Done in one (Score:5, Funny)

      by Weaselmancer (533834) on Thursday December 15, 2011 @10:23PM (#38392928)

      Seriously I just have to say that this is the single funniest comment I've ever read on Slashdot. Laughing, pointing at the screen, drug my wife over here to have her read it funny. Brutal. Absolutely brutal.

      From one cynical bastard to another, I salute you.

    • by SpacePunk (17960)

      I was coming in here to say that. LOL

    • Re:Hahaha (Score:5, Insightful)

      by JoeMerchant (803320) on Thursday December 15, 2011 @10:44PM (#38393160)

      One of you is getting fired

      On a three person team, metrics are irrelevant - personality and politics are 100% more important than technical skill.

      If you've got 15-20 people and it's time for a 10% downsizing, or (less realistically), performance based bonus or advancement, then metrics start to be something worth looking at.

      When I had to choose between 2, my boss asked me who was my choice, I told him, he agreed, I started to tell him why, he stopped me and said: "it doesn't matter why, I'm just glad that we came up with the same answer."

      • Re: (Score:3, Interesting)

        by Anonymous Coward

        Concur with the "someone is getting fired". 3 people supporting 300+ people. That's overtaxed, even school teachers only have to deal with 30 potential-assholes for 8 hours a day and have to put in unpaid overtime because you can't just leave things when the shit hits the fan 10 minutes from clocking out.

        As for metrics, serious or otherwise, 3 person team, you should already know who the weakest link is (hint, -you- if you don't come up with a solution.) The boss always asks the least intimidating person, t

    • by bfwebster (90513) on Thursday December 15, 2011 @11:16PM (#38393418) Homepage

      Might as well close the comments now. :-)

      Go look up Robert Austin's book on measurements and management. Read it and recognize that you've been given a task that is at best counterproductive and at worst impossible. Dust off your resume, because it may be more than one of you that are getting fired. ..bruce..

    • Re:Hahaha (Score:5, Interesting)

      by ScuzzMonkey (208981) on Friday December 16, 2011 @12:49AM (#38393992) Homepage

      As I was idly paging through the comments thinking about how many of them were jumping to outlandish, unsubstantiated conclusions about why the poor submitter was being asked to come up with metrics, I also realized that pretty much nobody (in the finest tradition of Slashdot) has bothered to answer the actual question: "I'm wondering what people opinions are on what good metrics should be in regards to mttr mtbf etc." I think I misunderstood this at first as well... in light of what he does say about the goals ("...determining if we are performing above or below what is considered optimal") I don't think he's asking what metrics to measure or whether or not we think doing so is any good, but instead what reliable industry benchmarks are for those metrics, so he can tell his boss what is "optimal" or not about their operation.

      Well, shibby, sorry, but I don't think there are such things, or at least none relevant enough to take back to your management team. The only way to get anything meaningful out of metrics (obviously a lot of other posters are arguing you'll get nothing meaningful out of them; I disagree, but it may not matter either way if those are your orders) is to establish baselines for your organization and track future performance against those. It will take a while and it will have to be viewed in context to be worthwhile... important points to make when you are presenting your findings to management.

      I'm being optimistic and assuming genuine business goals and a desire to understand IT operations on the part of non-technical managers are the point of this request, not some haphazard effort to chop down a three-person department, but it is also worth passing along some of the critiques that are being posted here. On the other hand, if you don't already, you should understand that not all managers are buggers, and that many of the better ones have legitimate reasons for trying to understand what is going on in their IT department. We often forget how mysterious what we do looks to the un-initiated, and I have seen enough poorly run IT departments to sympathize with non-technical managers who are grasping for the tools to understand theirs. The point being, getting defensive and obstructive in the face of these requests isn't always the brightest idea; instead, you can look at it as an opportunity to present some of your perspectives and difficulties to managers who are finally prepared to hear about them (after all, they did ask!). They may not have the time or horsepower to learn everything you do in depth, but it is possible to analyze operations based on the right sort of shorthand.

      The cynics may be right; only you will know. But I've seen companies run down by their IT departments as often as I've seen IT departments run down by the management team, so performance concerns on both sides are well-founded. Anyone who thinks their manager shouldn't ask for a suitably abstracted toolset for judging performance is asking for a stupid manager.

  • Metrics suck (Score:5, Informative)

    by SJ2000 (1128057) on Thursday December 15, 2011 @09:28PM (#38392362) Homepage
    Didn't we just have a story [slashdot.org] about how metrics suck?
    • by JMJimmy (2036122)

      Great read too. I question this:

      determining if we are performing above or below what is considered optimal

      How do you determine 'optimal'? How you determine that determines the metrics you use.

      • Determining optimal comes well after you start collecting metrics--usually months after. If you've never collected them, you are almost guaranteed to find things in the first month (or even the first day) that will look stunningly bad at first, but are quickly correctable. Once you get things balanced out--if you can--then you can figure out what's optimal.

        Getting management to understand this may be a chore if they're new to metrics or not trying to find a reason to fire someone.

    • Re:Metrics suck (Score:4, Interesting)

      by Ethanol-fueled (1125189) on Thursday December 15, 2011 @09:43PM (#38392518) Homepage Journal
      Top-post whore responder advice giver here. As somebody who works in a company with very similar numbers described in the summary, the few IT personnel should take over some of the infrastructure programming duties like databases and internal support software and use their existing knowledge of the corporation to prepare to either take on more work or transfer to a different position or department within the company.

      Don't know how to program? Hit the books - your job may depend on it.
    • Re:Metrics suck (Score:5, Informative)

      by ScuzzMonkey (208981) on Thursday December 15, 2011 @09:44PM (#38392530) Homepage

      Bad metrics suck, good metrics are useful data.

      As folks have already mentioned, time to answer and time to resolve are both important, and I think you have to watch for re-opened as well to curb "how fast can I shove this under the bed?" resolution games.

      My favorite is average tickets per user, though. Particularly on a small team, what you really want to gear your measurements toward is preventing incidents in the first place. It is helpful to know what your overall ticket volume looks like, then, and to aim to decrease it over time in the same way you might try to decrease time to answer and time to resolve. That's important, because as the previous article suggests, if you will get what you measure... and your overall goal should not simply be to answer tickets faster and resolve them more quickly, but to not have as many in the first place*. Every issue represents a waste of somebody's time and therefore corporate resources that could be put to more productive uses. Steadily decreasing mtta and mttr are nothing to cheer about if your ticket count is increasing.

      But you can keep it simple. You can drown yourself in metrics and lose sight of why you're tracking them and what you really want to accomplish. You may not really need any more than these few; better to start small and add what you need when you need it. I know there's always tension over getting a system in place that can capture what you need for historical purposes when you realize you need to know something new down the road, but resist the urge to over-collect. Half the time you won't need it all and are just wasting time getting it in the system.

      * There is a caveat to this; in some organizations, I actually DO want to see an increasing ticket/user count, at least for a time. This is something I shoot for when relations between IT and users have broken down badly enough that users have stopped reporting problems to IT because they feel it's useless, and their issues are never resolved anyway. In those cases, a rising ticket count can represent an increasing trust level, which is good. You generally won't fix issues you don't know about.

      • First call resolution is almost as important as 7 day repeat rate... while you do want to look at things like mttr, you want to be careful with using that as a hire 'em and fire 'em metric, because it will encourage people to cherry pick easy problems, and can penalize the people who get stuck with the hard issues. It takes 5 minutes to configure somebody's e-mail client, but it will take much longer if you have to order parts to replace defective hardware.

      • Re:Metrics suck (Score:5, Informative)

        by CAIMLAS (41445) on Thursday December 15, 2011 @10:20PM (#38392888) Homepage

        Metrics is just a management word for "bad statistics".

        With a distribution of 3, it's not really possible to have statistics of meaningful nature. You've got shared responsibilities and bounce things off of each other. One person may open and close more tickets, have a shorter duration for tickets, etc. - but he's only doing the actual "work". The others may be giving him all the input necessary to complete the task(s).

        Ideally, your ticketing system will reflect, very vaguely, who's doing work and who is not, but even then it's not going to be well representative of what's actually going on.

        People do different types of work, of different levels of difficulty. For instance, I may do one ticket on Monday, three on Tuesday, and one for Wednesday through Friday. Why? Aside from the fact that I'm bad about actually doing tickets for my work (my god, I'd not have the time for work, and then there'd be more things that aren't getting done, making us -all- look bad), there's the reality that my tickets aren't terribly easy, often requiring hours of log perusal and research to try to fix problems. Meanwhile, the guy who knocks out 40 tickets a week - malware disinfection, workstation reinstalls, etc. - has fairly wrote work of a repetitive nature, comparably. Also, he's following instructions or asking for advice on a regular basis, even if I'm not his boss.

        You said so yourself: you're a member of a 3-person IT team. The only use 'metrics' have here aside from what should be plainly obvious in a group of 3 (who's fucking up, who's not getting back to people, who's not doing work) is to keep track of what amounts to customer requests and problems. X workstation needs to be reinstalled, Y server has a crashing whatever, and so on. If you're working on and sharing a ticket queue, you are all mutually responsible for all of the tickets: if something isn't getting done, it's everyone's fault (or nobody's fault). You may consider presenting your metrics in the light of this reality (like statistics, metrics can lie, too).

      • by sjames (1099)

        The problem with metrics is that if you look closely enough at things to actually KNOW if they're good or bad, you know enough to not need them.

        Is the guy with the lowest average tickets slow or is he the guy that takes the hard ones because the rest of the team doesn't stand a chance of solving them? If you know the answer to that question, he is already evaluated, what do the metrics contribute? As you point out, a declining ticket count can be very good or very bad. If you know which it is, you already k

      • Re:Metrics suck (Score:5, Insightful)

        by Anonymous Coward on Friday December 16, 2011 @06:38AM (#38395556)

        Totally agree.

        My small team was taking a beating and getting little support from management. Issues were piling up, we were getting randomized, and I felt like I needed another staff member. Management wouldn't budge though. I needed to prove to them that I could manage my department better. Nobody even cared about IT metrics at our company. Rather than see them as an evil thing, we embraced them to gain credibility in our organization.

        We started by outlining some of our support boundaries, types, and set goals for response times. Then we customized our ticket system and added in some categories and priorities.

        For Support (meaning something is broken and must be fixed)
        High pri: 24 hour solution
        Medium pri: 1 week
        Low pri: 3 months or more, not a critical metric

        We also had a "Request" category (not a fix, but doing standard tasks like toner replacement, adding e-mail alias, help train someone, etc.). These had different goals for solutions. We even had ticket categories for maintenance and projects. Though they junk up the ticket system, it allowed us to track time going into other tasks and paint a picture of a staff member's whole day of work, not just the support end.

        Some of our favorite metrics:

        1. Average time on support OR request tickets aggregated by our team or split down to individual staff members, and divided by priority level.

        2. Total time spent on tickets by user (so I know if we're actually working or lounging around). ---this is a motivator for staff to actually enter tickets and get credit for their work. Some flexibility is necessary here because people eat lunch and go to the bathroom - you can't expect this to add up to 40 hours a week perfectly.

        3. "Most troublesome user or department." I don't advertise the data on this report, but it lets me know who to focus on with either training or nudging their boss a little. Execs get interested in this once in a while, and the users that found out we keep such a log try to keep off of it. Many will joke about it, but still ask "Hey I'm not in the top 10, right?" It encourages users to be a little smarter and not lean so heavily on IT for silly things.

        4. Most troublesome product - THIS IS BY FAR THE BEST METRIC. This has helped me gain support in dropping old junk software, getting better solutions, setting up training for people, or creating general awareness that we need to improve. For example when I was able to quantify how much time it took when people forgot their passwords for our cloud-based e-mail password, I got support for Office 365 and ADFS for single sign-on. In the interim, people stopped took a little more personal responsibility because they realized that password stupidity = preventing us from getting value-added projects done. Stuff like this is beautiful because it takes most of the heat off of IT staff, and informs other managers of our company-wide pain points that we should invest in.

        5. On the same lines as #3, identifying how many high pri tickets are out there vs. medium or low (and possibly categorizing by customer or department) can sometimes sniff out IT abusers that need stuff now now now. Tread on this carefully as you don't want to constantly wage war with your metrics, but with some political prowess and some data to back you up, you can start solving some of these problems.

        #3 and #4 are big for us. Don't just think of metrics in terms of how you will hang yourself. Think about the bigger picture and answer questions that will help motivate your team in a positive way, get credit for what they/you do, and enable you to tell a better story to management. You probably will want to develop two sets of metrics - a more detailed set for yourself so you can manage effectively and proactively identify problems (and solve them in advance before your boss gets to it), and a more general set for your boss so you can deliver a simple and concise picture without getting him too involved with the details you can handle yourse

    • by Sarius64 (880298)
      When help tickets are all that matters, you'd be surprised how many tickets get generated.
      • Re:Metrics suck (Score:4, Interesting)

        by MichaelKristopeit422 (2018884) on Thursday December 15, 2011 @11:21PM (#38393446)
        i'm dealing with a bug detecting team in india right now through a client (their choice to use them)... they use "fixed" bugs as their main metric.... so they file tons of bugs... we waste tons of time explaining why there isn't a bug, then they agree and say "ok we'll mark it fixed" and we say "mark it rejected" and they say "ok we'll mark it fixed".

        it's a nightmare.

        metrics and quotas will only lead to less overall return... unless, of course, you're just trying to create a jobs program.

    • by jginspace (678908)
      I think the main thing the submitter can take away from that article is that you measure the availability of systems rather than trying to painstakingly log activities among the IT staff. What is considered 'optimal' is not covered in the previous story. You'd be working on something like: Internet access only being down 5 mins per day; email for 10 mins; each workstation only suffering 30 mins per month - or whatever. The most important stats probably won't be in your 'ticket management system' - the good
  • Here's a few... (Score:5, Informative)

    by HerculesMO (693085) on Thursday December 15, 2011 @09:29PM (#38392372)

    Time to answer call, time to resolve ticket, abandoned tickets (unresolved).

    If you google a few of those it will bring some more, but that's a simple start.

    • Re:Here's a few... (Score:5, Informative)

      by suutar (1860506) on Thursday December 15, 2011 @09:32PM (#38392400)
      Number of calls back after initial call (measuring, in theory, how often the initial call resolved the issue) Number and duration of system outages (if you're doing sysadmin stuff as well as support stuff)
    • as not all ticket take the same time.

      Do want quick password resets to count the same as doing some think that needs a more time? like setup / reload a pc?

      • by DarthBart (640519) on Thursday December 15, 2011 @11:08PM (#38393358)

        I got written up once because my ticket stats were radically different than the other people on my team. 15% lower "total time on tickets" but 20% more tickets closed. I was apparently fudging numbers and closing unresolved tickets.

        Fortunately, a trip to HR with a ream of printouts from closed tickets proved otherwise.

        Still left the company a few months later.

    • by fsckmnky (2505008)
      Don't forget to correlate any metrics collected internally, with your customers experience ratings.

      Metrics collected internally without correlation can easily be gamed.

      If you data mine the metrics against customer satisfaction, you'll see who is naughty and who is nice.
    • by perpenso (1613749) on Thursday December 15, 2011 @10:58PM (#38393292)

      Time to answer call, time to resolve ticket, abandoned tickets (unresolved).

      In business school it is a common theme in various classes that you get what you reward, not what you ask for, not what is necessarily best for the organization. Here is a highly relevant Dilbert cartoon illustrating this point, http://dilbert.com/strips/comic/1995-11-13/ [dilbert.com].

      The underlying problem is that metrics applied to humans leads to people working towards the metrics, not necessarily doing good work. It is a classic environment for unintended consequences. Its not even that the people are necessarily being opportunistic, there is also a certain amount of practicality. If you are being measured by some metric and keeping your job or getting a raise is dependent upon that metric you may quite rationally decide to act to that metric rather than what is necessarily in the best interest of customers.

      Are you measured by resolved tickets? Then tickets will get resolved quickly. Not necessarily thoroughly, completely, or robustly resolved. Which leads to related followup tickets because of a minimal effort put into resolving the original ticket. I saw this in a programming environment where the tickets consisted of new features or bug fixes.

      Are you measured by abandoned tickets? Then tickets will get resolved, even if they don't reasonably deserve to be considered resolved. You will get things unnecessarily classified as "unable to duplicate", "insufficient information", etc.

      In these two examples, where is the difficulty of the task factored in? Not all task, tickets, are equivalent. Furthermore sometimes there are external dependencies, a part is being shipped, where is this factored in?

      The metrics you offer are reminiscent of stats from call centers. There such metrics are a little more reasonable, not perfect but perhaps OK, given that the calls are somewhat equivalent in the amount of effort required, a small number of minutes not hours, and that they are randomly assigned. Over the period of say a month the large number of calls handled by any operator will resemble a normal curve with respect to effort required. For an IT organization the evaluation period may need to be some number of years to get to a normal curve with respect to effort required.

      • by inviolet (797804) <slashdot&ideasmatter,org> on Friday December 16, 2011 @01:26AM (#38394210) Journal

        Are you measured by abandoned tickets? Then tickets will get resolved, even if they don't reasonably deserve to be considered resolved. You will get things unnecessarily classified as "unable to duplicate", "insufficient information", etc.

        I experienced a form of this phenomenon just this afternoon, albeit not in an IT environment...

        I went to Bank of America to (proudly) close my accounts, having moved to a smaller and thoroughly more moral bank. The customer service rep figured it out right away when he saw that my six ~18-year-old accounts now all had zero balance, and zero activity for the past month. So we start closing. He is doing the keyboarding and mousing while I am watching the screen.

        At a certain point in the process of closing each account, the rep is required to give the reason for account closure. In the popup list of reasons are some very relevant choices: 'Service' and 'Competition' sprang out as the correct choices. He chose 'Misc', and for the sub-reason chose 'Bank of America Consolidation', whatever the hell THAT meant.

        It was then that I knew there had been a memo from headquarters, probably last month, that said "We know people are closing their accounts. Management wants to make sure that the reason is NOT people leaving in disgust, headed to our competitors." And so now the CEO can stand up and say "We've only lost 1% of our accounts to the competition!"

        Just this afternoon that happened, right in front of me. I almost laughed out loud.

  • easy (Score:5, Insightful)

    by Anonymous Coward on Thursday December 15, 2011 @09:30PM (#38392378)

    "what good metrics should be in regards to mttr mtbf etc"

    Easy, there are no good metrics. Metrics don't lead to improved business outcomes, they rarely cover enough variables to tell the whole story, so all they lead to is people gaming the metrics, most likely leading to worse business outcomes.

    Metrics are favoured by lazy management.

    • Re:easy (Score:5, Funny)

      by mewsenews (251487) on Thursday December 15, 2011 @09:34PM (#38392422) Homepage

      Metrics are favoured by lazy management.

      Look, using metrics doesn't indicate lazy management.

      Look, using metrics that you don't have available doesn't indicate lazy management.

      Look, using metrics that you don't have available so you ask your staff to measure their own metrics doesn't indicate lazy management.

      Look, using metrics that you don't have available, so you ask your staff to measure their own metrics, but you don't know what metrics they should measure, so they end up asking Slashdot what some good metrics are, doesn't indicate lazy management.

    • by Lisias (447563)

      Metrics are favoured by lazy management.

      Metrics are also favored by management that does not knows squat about what the heck they are managing.

    • Re:easy (Score:4, Insightful)

      by nine-times (778537) <nine.times@gmail.com> on Friday December 16, 2011 @01:06AM (#38394080) Homepage

      More to the point: There is no good reason to use metrics for a 3 person team. You could sit down and observe the 3 person team for a week and get a very good idea of what's going on, and it would be easier and probably take much less time than designing, implementing, gathering, and analyzing metrics. It's easier and more accurate.

      If you're going to use metrics, use it in situations where you're managing such a large group that you can't possibly know who everyone is. Don't use it in situations where you have the option of doing a simple evaluation.

    • Re:easy (Score:5, Insightful)

      by LS (57954) on Friday December 16, 2011 @03:39AM (#38394806) Homepage

      I don't buy this. I (hate to admit that I) worked at Zynga, and their entire business model is based off of metrics, both internal and customer metrics. They are behaviorists, and brought in financial industry data modeling to design games and make business decisions. It works, and it works fucking well. In the long term this Skinner box model may or may not work for creating a sustainable business, but it has worked for the last couple years. People may not like their games, but they keep playing them.

      The problem is bad or incomplete or misinterpreted metrics. Metrics in and of themselves or not bad. The problem is with the people that use them.

      LS

  • 3 People? (Score:5, Insightful)

    by stevenfuzz (2510476) on Thursday December 15, 2011 @09:32PM (#38392398)
    Simple... if you have a 3 person IT team at a 300 employee company and your site / it infrastructure isn't in nuclear meltdown your probably doing good. Looks like they are going out of house for IT. Welcome to the cloud-future, where your job is dissolved for magic.
    • by hedwards (940851)

      Doesn't mean they aren't going to try to work with 1 fewer. I remember working an understaffed job a while back and they still managed to figure out how to eliminate an extra position. It didn't work well and stressed out the employees, but they were able to cut the position.

      I'm guessing they'll try that here, even if they do have to give up and hire somebody back.

  • I'm afraid if some one is asking for metrics of three people supporting 350 in international terms does not know what the hell they are asking for or what it is that you 3 do. That being said, it can be expected you will be scrutinized over every little detail. Be careful. I won't ask, but I do wonder who you work for. I have been a similar situation and it is completely frustrating. Good luck...
    • Re:Good luck (Score:5, Interesting)

      by ScuzzMonkey (208981) on Thursday December 15, 2011 @09:52PM (#38392594) Homepage

      It's not unusual for management to be clueless about what exactly it is that their IT staff does on a daily basis, nor is it unnatural that they should take an interest. Often, it's a good sign when they actually ask the guys doing the work what the metrics should be... it indicates some degree of trust, and they haven't simply read an "IT Management for Dummies" book over the weekend laying out some arbitrary system that isn't going to fit your organization.

      As a more cynical commenter points out, it also provides the opportunity to create a measurement system that you can game to make you look good. But I think it isn't a terrible sign that the bosses care what their employees are up to. It may represent an opportunity to explain what you think is important that perhaps they hadn't considered previously.

  • by Anonymous Coward

    You're always below optimal, by definition of optimal.

    • Re: (Score:2, Insightful)

      by DWMorse (1816016)

      I think you've confused "peak" with "optimal."

      Peak = best possible output

      Optimal = most satisfactory

      You can tune an engine to run within optimal specs. But if you run it at peak nonstop...

  • optimal? (Score:5, Informative)

    by Anonymous Coward on Thursday December 15, 2011 @09:35PM (#38392428)

    talk about flow, about bottle necks. Visualize workflow. Look at Henrik Kniberg's paper on kanban as applied to IT Ops. My guess is that your ticketing systems will provide low value data on volumes on resolution time - gear up - visualize the pipeline. check http://www.infoq.com/minibooks/priming-kanban-jesper-boeg - turn the conversation around to "business value" - don't get wrapped in the ropes of volumetrics /peace.

    • Re: (Score:3, Informative)

      by bbutton (90403)

      +1 for this answer. Kanban is a great way to generate actually useful metrics for a team, project, or department. You'll be able to calculate things like how long it takes the average ticket to work its way through your processes, where tickets tend to get stuck (cumulative flow diagram), and where the sources of waste are in your processes.

      In addition to the book mentioned above, I also like this one by David Anderson: http://www.amazon.com/Kanban-Successful-Evolutionary-Technology-Business/dp/0984521402/r [amazon.com]

  • No Metrics (Score:4, Insightful)

    by Anonymous Coward on Thursday December 15, 2011 @09:36PM (#38392436)

    There is no metrics system that can't be gamed.

    If you set it for "total tickets fixes" (higher=good): you just encourage people to report trivial problems you can fix easily.
    If you set it for "total tickets" (higher=bad): you refuse to do things, add features etc, or you make it hard to contact IT to log a fault
    If you set it for "time taken per ticket" (higher=bad): you end up pushing kludge solutions
    If you set it for "user rated response" (higher=good): you end up blackmailing the end users to rate you 10/10 otherwise their emails/logs/dirt etc get published and sent to boss/wife/etc

    Ask your manager how their performance is evaluated? Then start suggesting ways they could bust their KPIs, and they should get the drift.

    • by grcumb (781340)

      If you set it for "total tickets" (higher=bad): you refuse to do things, add features etc, or you make it hard to contact IT to log a fault
      If you set it for "time taken per ticket" (higher=bad): you end up pushing kludge solutions
      If you set it for "user rated response" (higher=good): you end up blackmailing the end users to rate you 10/10 otherwise their emails/logs/dirt etc get published and sent to boss/wife/etc

      So... 'user-rated response' it is, then.

      Thanks!

  • by DWMorse (1816016) on Thursday December 15, 2011 @09:37PM (#38392444) Homepage

    Metrics. Excellent, I hate when bosses use the Imperial system.

    All jokes aside: If you care about your job in this economic climate, I suggest you do what your 2 other teammates are doing - picking through the stats that make YOU look the best. The company isn't going to look out for you. IT is an expense to be cut, remember. Boosts the temporary bottom line, promotes "growth" in this fiscal quarter, gets the investors going so the CEO can shuffle another fold into his golden parachute. If non-important metrics are selected that sacrifice your job, it's a brief victory lap straight into the unemployment line.

    We can't answer your question, though. In the end, I recommend you watch a clip from "Office Space" - wherein the Bobs interview the employees:

    Bob: "So tell me, what is it, exactly, that you do here?"

    If you can't answer that question, you probably should be job hunting already. Or should have kept a copy of the job posting from when you applied.

  • I have been tasked with logging our performance using the statistics from our ticket management system. I've also been tasked with comparing these stats and determining if we are performing above or below what is considered optimal.

    Standard ticketing system metrics are no good. Add a post-incident survey to your customer interactions. Your metrics don't measure performance, unless you can actually measure how 'hard' the ticket is intrinsically / what skill is required, and how well the ticket is solved.

    • where people are constantly calling me asking to fix stuff, then my numbers will be awesome!

      god, capitalism is awesome.

      • by mysidia (191772) *

        where people are constantly calling me asking to fix stuff, then my numbers will be awesome!

        I think you missed the part where total IT service revenue for that application is divided by the number of tickets. The more tickets for a specific IT service, the less the issue resolution is worth.

        The fewer total tickets for that service, the more each resolution is worth.

  • by Anonymous Coward

    Is shit broken?
    Does shit get fixed fairly quickly?
    Are your people busy, but not swamped?

  • by colonel (4464) on Thursday December 15, 2011 @09:47PM (#38392552) Homepage

    - Percentage of staff with ITILv3 foundations certification: zero.

    I know this because of your question. Watch thes videos as a start: http://pmit.pl/en/it-management/free-itil-v3-course-collection-of-itil-v3-moviesdarmowe-szkolenie-itil-v3-zbior-filmikow-o-itil-v3/ [pmit.pl] -- and sell a formal training course to your management.

    The people joking about how one of you is getting fired, or you're all getting outsourced. . . probably true. Learning ITIL is all about learning what's important to your business stakeholders, how to monitor/measure these things, and how to make sure you're always making the right decisions based on the business priorities.

      If you can't convince them to pony up for you three to take the certification course, then pay for it out of your own pocket, you'll need it to find a new job.

  • Seriously. Once management fall under the belief that they can have reports automagically generated for them by measuring your working habits, you will never hear the end of it. Why does X have less keystrokes per hour than Y? You only committed 15 lines of code yesterday. You must not be working. Why did this junior employee fix 100 bugs, when this senior only fixed 10? And take a look at yesterday's topic too.
  • Add something (Score:4, Informative)

    by BenEnglishAtHome (449670) on Thursday December 15, 2011 @09:55PM (#38392632)

    Your ticketing system needs or needs to have added an automatic followup to the customers. The system sends out an email after every ticket asking "Did the problem get resolved in a reasonable amount of time? Did the IT staff respond in a way that enabled you to get back to work?" Nothing more complex than that, though you can parse things out by ticket priority (though deciding what's a higher priority than other things is, just by itself, a major undertaking).

    Your goal should be to increase the percentage of positive responses.

    Why this touchy-feely stuff instead of a hard metric? Easy. There are no metrics that work in your situation. It's quite easy to argue that there are no metrics that work, period.

    By adding this email feedback to your ticketing system, you have met the requirement to come up with a metric derived from the ticketing system.

    Selling this to management can be simple, depending on how you handle it. Something along the lines of "Given that the IT staff is so idiotically understaffed, we must be given the agility to solve problems instead of meeting random metrics. Only our customers can know if we met their needs, considering all pertinent factors. Someday, when we actually have enough people and money to divide work more rigidly, we can add metrics like timeliness of ticket closure, etc." Then you hope they never notice that you never divvy up the work rigidly. All of this requires having an IT manager who is dedicated to the inescapable truth - that their function is to keep the MBAs off your ass and let you do your job.

    I've worked where my performance was measured in this way. It can be heaven.

    One more thing - if your upper management doesn't already have faith in you, they'll never go for it. They need to already appreciate your contribution to go along with this. The very fact that they're asking for metrics tends to suggest they don't sufficiently appreciate you now. If that's the case, than all I can say is that I've worked under those circumstances, too, and my heart goes out to you.

    • by Jaime2 (824950)
      This is too easy to game. I used to work at a training center where our most important metric was post-class evaluations. One day, some brainiac decided that merit raises would be decided solely on evaluation scores. From that day on, nobody got under 95% on evaluations and the difference between the best and the worst was statistically insignificant. If you want honest metrics, you have to collect them for their own sake and not actually use them. As soon as you tie incentives and punishments to them, the
  • by lucm (889690) on Thursday December 15, 2011 @09:57PM (#38392644)

    > determining if we are performing above or below what is considered optimal

    Scenario 1: you are below optimal -> you are inefficient so they replace you
    Scenario 2: you are above optimal -> you are overkill so they replace you

    Bottom line, I would rent The Wire and learn how to "juke the stats" because that's the only way you won't get to jump on that grenade.

    Been there, done that - my advice: be just under optimal so you have room to grow and show improvement, but don't be too low so they don't feel the need to consider a business case for outsourcing.

  • by mikem170 (698970)
    I'd try to find out what they really want. Are they concerned about head count? Customer satisfaction? Having a good answer to justify their budget?
  • by Above (100351) on Thursday December 15, 2011 @10:00PM (#38392690)

    I'm not with most IT management on this one, but I always thought the best metric was customer satisfaction. For instance every time I open a ticket with Cisco I get a survey at the end of like 5 questions. Was my problem resolved, was the person polite, etc.

    The other metrics suggested are things to graph and look at trends. Are repair times getting worse or better? Is the average time per ticket going up or down? They are great int he aggregate. They break down quickly when divided. Only one guy on your team knows network devices, so he gets all the network devices which include the 8 hour fiber cuts, so his times always are worse than the guys fixing printer problems, as an example. You have to be very careful as you start to divide them up.

    At the end of the day though you're trying to make the customer happy. Track it, and see how your staff is doing. If people are happy with their IT support, your department will be seen in a more positive light.

  • by decora (1710862) on Thursday December 15, 2011 @10:00PM (#38392692) Journal

    1. make your numbers.

    nobody actually cares what 'the numbers' are, or if they actually mean anything. but you have to make them.

    you might ask yourself - isn't this a huge waste of time? isn't it completely counter productive? doesn't it actually decrease efficiency? aren't the metrics measuring completely the wrong thing? as the slashdot story the other day said, aren't bad metrics actually worse than no metrics, because they cause people to do inane, wasteful things to make their numbers?

    well, your problem is that you are asking yourself. in a corporate environment, do not ask. just do.

    just make the numbers.

    hopefully, if you get good enough at 'making your numbers', you will have time left over to actually do some work.

    2. but what about the theory of capitalism, the free market, efficiency, etc?

    its all bullshit. just like the theory of communism was bullshit. what statistics and 'numbers' were reported to the government were just flat out garbage. people somehow managed to make the system work through personal relationships and working-around the assholse in charge. but most of the theories it was built on have no resemblance to reality. think about it - if efficiency really made for the best corporation, why would you be spending 4 hours a week filling out meaningless statistical performance reports that nobody will ever read, let alone understand?

    the only difference between the soviet union and 'the west' is that 'the west' still hasnt collapsed yet.

    • the only difference between the soviet union and 'the west' is that 'the west' still hasnt collapsed yet.

      It will. People looking at the inner structure have known it for a while. Now the cracks are starting to show on the surface where anyone can see them. It won't be long.

      If you are brave try to look past how you're going to survive the collapse: try to plan how you will help rebuild it better.

  • by jwkane (180726) on Thursday December 15, 2011 @10:08PM (#38392764) Homepage

    The only truly meaningful metric is customer satisfaction. After each ticket is closed send a survey email to the user. If your team plows through enough tickets you get a statistically significant success % per tech that you can compare to the other techs.

    Without knowing a lot more about the nature of tickets it is hard to give a better response. It might be very important to gauge the difficulty (trivial-hard) and novelty (common-rare) of tickets but it could also be a waste of time. Does one tech plow through dozens of tickets a day or a few? Are the techs specialized? Anything email related goes to joe, hardware issues to tom, etc.. Are the tickets auto-generated from emails, called in or do they fill out a web form?

    Given the size of the team the company thinks 1) you aren't getting the job done, 2) one or more people on the team are dead weight or 3) you are overworked and need more people. I wouldn't bet on #3.

  • by LazLong (757) on Thursday December 15, 2011 @10:09PM (#38392770) Homepage

    Falling back to metrics is a lazy manager's way of proving to her superiors that her drones are operating at peak efficiency. The most lazy of all will rely on utterly meaningless metrics such as the number of help tickets closed per day, per individual per day, etc. A metric such as this is completely useless as all tickets don't require an equal amount of effort to complete. Diagnosing a problem due to an intermittent hardware issue doesn't take the same amount of effort as helping a user change their password. Unfortunately these types of issues generally comprise the vast majority of tickets generated and therefore often end up being the ones that are 'measured. ' This often leads to a drop in morale and thereby negatively impacts performance; ironically the opposite of what the whole exercise is attempting to accomplish.

    Trouble ticket data is primarily useful for detecting trends, thereby helping an IT team appropriately focus their human capital on issues that will enable their users to be more efficient. Going back to the password issue above, the speed and alacrity with which the IT staff help users change their passwords isn't a useful metric at all. A more meaningful metric would be the frequency of password change requests before and after the installation of a self-service password reset solution that was put in place in response to the analysis of help ticket data that showed that this was one of the most frequent issues and one that could be easily solved with little effort and financial expenditure. Measuring a sharp drop in password reset requests would show that the solution worked and was therefore beneficial to the organization by enabling users to help themselves, resulting in their having more time to concentrate on their primary tasks, and also by allowing IT staff to allocate their resources on issues that are less amenable to resolution via automation.

    Unfortunately, in my experience, ticket systems get used to determine useless metrics such as the first example mentioned above, and therefore end up being the bane of IT staff, rather than a useful analytical tool.

  • While not a standard metric, consider the amount of time you spend planning and implementing upgrades to your infrastructure, as opposed to the amount of time spent supporting it. In a healthy company, the weight is towards planning and implementing upgrades to your infrastructure, where not a day passes that a new piece of equipment isn't being delivered. Why, you ask? Because it shows that there are enough people to handle the daily problems adequately, freeing up others to plan for several months (or yea

  • If you have to go down that path, I'd go with TAT. Simple to measure even over old data, hard to game consistently, and closely related to customer impact. If you want to, weigh it by customer-perceived severity of each request so that people don't boost their numbers by cherry picking easy tickets.

  • The best answer is to not use tickets as your metric. I worked at a smaller company and our metric was a simple yearly user survey. We would ask what they thought of their equipment, the how the support team did (response time, niceness, knowledge, etc) and even took it so far as to ask for suggestions on what could be improved. In the end, how many tickets you close really doesn't matter. What matters is how happy your users are. If you are doing the job correctly, the majority of your users will at le

  • I can only imagine that most people replying with derision about metrics have never been in the position of having to justify what they are doing, and when they have been they have acted dishonestly.

    People that actually work in the real world, with real companies with real budgets, and that have some self respect, honesty and pride in what they do will have to justify their salaries or rates somehow, and one of the tools used is some kind of metrics.

    A professional will find metrics that are meaningful to bo

    • by russotto (537200)

      I can only imagine that most people replying with derision about metrics have never been in the position of having to justify what they are doing, and when they have been they have acted dishonestly.

      No, just the opposite. The people who have never been in that position probably don't understand why there's so much derision. The people who act dishonestly aren't derisive at all; they're busy writing themselves a new minivan.

      People that actually work in the real world, with real companies with real budgets,

  • Nagios [nagios.org] isn't too difficult to set up to monitor lots of things, and lots of useful uptime metrics for every service, planned and unplanned maintenance, etc. fall out quite naturally from it. And you can kinda just keep adding modules to it into it and grow it until it's full of awesome.

    I haven't personally used Redmine [redmine.org] yet, but have been using Trac and everyone seems to agree that Redmine is the clear successor in terms of lightweight but capable trouble-ticket / project / task management systems.

  • by slashmydots (2189826) on Thursday December 15, 2011 @11:39PM (#38393580)
    Well, when they say small, I'm the sole IT worker and thus the IT Manager at a 4 server, 40 workstation business so my only metrics are: I didn't make them spend a bunch of money, nothing lit on fire, I didn't quit. That's seriously about it and this quarter, I got all but the middle one but I wasn't the one who ordered that HP workstation, nor would I have, so it's sort of a gray area lol.
  • by certain death (947081) on Thursday December 15, 2011 @11:52PM (#38393660)
    Touch up your Resume', go tell your boss to get bent, pound sand, etc. and look for a new place to work. Anyone who needs metrics on a three person team deserves anything they get, up to and including a swift kick in the ass. If the manager can't figure it out on his or her own, they should be the one being sent out the door with boxes.
  • by PatMcGee (710105) on Friday December 16, 2011 @12:02AM (#38393720)
    Before you do anything else, read Robert Austin's book, "Measuring and Managing Performance in Organizations".

    The points I got from the book:
    1) Measuring the wrong thing or in the wrong way makes things much, much worse.
    2) Good measurements are possible but take a lot of hard work.
    3) Measuring things that are easy to measure is almost certainly wrong.

    I also endorse BenEnglishAtHome's comment timestamped 8:55pm.
  • by vinn (4370) on Friday December 16, 2011 @12:20AM (#38393832) Homepage Journal

    I manage a department roughly the same size for a company about the same size.

    I use two main metrics to see how we're doing:
    1. The IT budget should be about 3% of the total company revenue. That's pretty average for companies of this size. If you're significantly under or significantly over, you need to do some soul searching to figure out why.

    2. Every year we put together a survey and ask the exact same questions. Its going on 5 years now so we can compare our performance year over year. We ask about 20 questions and score them on a scale of 1-5. Things like 'hows training?' to 'how well does your cell phone work?'

    Counting trouble tickets is mostly a worthless exercise. Although, you can manipulate it to your advantage. Start closing 120 tickets this month, 140 next month, etc. When you get to 240 tickets a month you can take that graph to upper management and say, "we're working twice as hard as we were 7 months ago and need to hire someone."

    In the end, you have two ways to view this: as a bullshit exercise (and possibly an excuse to fire someone as others have said) or as a way to attempt to objectively evaluate your department.

    • by vinn (4370)

      Regarding revenue, I meant to add - do you have a revenue generating component within IT? I think there's some novel ways of creating one for most businesses. Anyway, actually generating some form of revenue and growing it can be a good thing as long as it isn't a big distraction.

  • "...determining if we are performing above or below what is considered optimal."

    optimal, adj.: most favorable or desirable; best; optimum [Webster's New World Dictionary]

    How can you possibly perform above optimal?

  • by PerformanceDude (1798324) on Friday December 16, 2011 @12:36AM (#38393922)
    Just working backwards from the "one of you will be fired" comment above. Why not try and come up with a metric that shows you are impressively efficient, but drowning under a massive workload? Done right, it might just force management to hire rather than fire.

    There are a number of ways you can do this:
    1) For the next few weeks, only deal with issues in the ticketing system that can be resolved quickly. This shows how responsive you are on the "count of problems solved" and "time to resolution".
    2) Always upgrade easy problems to "Extremely Urgent", so that they get picked up first (as per above).
    3) Do NOT under any circumstances touch a complicated problem that requires consideration or actual work. Find someone to outsource it to. Then blame the outsourcing costs and lack of efficiency (obviously they do not have the same fast response time as you) for the problem.

    Seriously: In a 3 man team, you and your manager should KNOW who is working and who is on facebook all day. If you are all working hard, then it is not time to add more pressure by introducing metrics, it is time to hire more help. If on the other hand you are all on facebook all day - well - then good luck to you in your new job at Walmart....

  • by swell (195815) <jabberwock@@@poetic...com> on Friday December 16, 2011 @04:16AM (#38394954)

    Twice you used the phrase "been tasked with". This is not the English you learned in school. Is your wife 'tasked with' washing the dishes?

    There is a certain kind of lizard who embraces corporate speak, perhaps to kiss the ass of management or impress co-workers. This is a soulless sort of individual who is doomed to a lifetime of servitude.

    If you reclaim your dignity, speak correctly and stand up straight you will either be respected or fired. Either way you will have found a better path in life.

Whoever dies with the most toys wins.

Working...