Ask Slashdot: Good Metrics For a Small IT Team? 315
First time accepted submitter shibbyj writes "I'm a member of a small 3 person IT team for a medium sized business (approximately 300-350 employees) that has multiple locations internationally. I have been tasked with logging our performance using the statistics from our ticket management system. I've also been tasked with comparing these stats and determining if we are performing above or below what is considered optimal. I'm wondering what people opinions are on what good metrics should be in regards to mttr mtbf etc. I have had trouble finding information on this."
Hahaha (Score:5, Insightful)
easy (Score:5, Insightful)
"what good metrics should be in regards to mttr mtbf etc"
Easy, there are no good metrics. Metrics don't lead to improved business outcomes, they rarely cover enough variables to tell the whole story, so all they lead to is people gaming the metrics, most likely leading to worse business outcomes.
Metrics are favoured by lazy management.
3 People? (Score:5, Insightful)
Re:Hahaha (Score:5, Insightful)
Also, the title is a bit redundant: all IT teams are small.
No Metrics (Score:4, Insightful)
There is no metrics system that can't be gamed.
If you set it for "total tickets fixes" (higher=good): you just encourage people to report trivial problems you can fix easily.
If you set it for "total tickets" (higher=bad): you refuse to do things, add features etc, or you make it hard to contact IT to log a fault
If you set it for "time taken per ticket" (higher=bad): you end up pushing kludge solutions
If you set it for "user rated response" (higher=good): you end up blackmailing the end users to rate you 10/10 otherwise their emails/logs/dirt etc get published and sent to boss/wife/etc
Ask your manager how their performance is evaluated? Then start suggesting ways they could bust their KPIs, and they should get the drift.
Ahh, metrics, good. (Score:4, Insightful)
Metrics. Excellent, I hate when bosses use the Imperial system.
All jokes aside: If you care about your job in this economic climate, I suggest you do what your 2 other teammates are doing - picking through the stats that make YOU look the best. The company isn't going to look out for you. IT is an expense to be cut, remember. Boosts the temporary bottom line, promotes "growth" in this fiscal quarter, gets the investors going so the CEO can shuffle another fold into his golden parachute. If non-important metrics are selected that sacrifice your job, it's a brief victory lap straight into the unemployment line.
We can't answer your question, though. In the end, I recommend you watch a clip from "Office Space" - wherein the Bobs interview the employees:
Bob: "So tell me, what is it, exactly, that you do here?"
If you can't answer that question, you probably should be job hunting already. Or should have kept a copy of the job posting from when you applied.
For a team of 3 people? (Score:2, Insightful)
Is shit broken?
Does shit get fixed fairly quickly?
Are your people busy, but not swamped?
Re:Above optimal? (Score:2, Insightful)
I think you've confused "peak" with "optimal."
Peak = best possible output
Optimal = most satisfactory
You can tune an engine to run within optimal specs. But if you run it at peak nonstop...
Doomed from the start (Score:4, Insightful)
> determining if we are performing above or below what is considered optimal
Scenario 1: you are below optimal -> you are inefficient so they replace you
Scenario 2: you are above optimal -> you are overkill so they replace you
Bottom line, I would rent The Wire and learn how to "juke the stats" because that's the only way you won't get to jump on that grenade.
Been there, done that - my advice: be just under optimal so you have room to grow and show improvement, but don't be too low so they don't feel the need to consider a business case for outsourcing.
how to survive in the corporate environment (Score:5, Insightful)
1. make your numbers.
nobody actually cares what 'the numbers' are, or if they actually mean anything. but you have to make them.
you might ask yourself - isn't this a huge waste of time? isn't it completely counter productive? doesn't it actually decrease efficiency? aren't the metrics measuring completely the wrong thing? as the slashdot story the other day said, aren't bad metrics actually worse than no metrics, because they cause people to do inane, wasteful things to make their numbers?
well, your problem is that you are asking yourself. in a corporate environment, do not ask. just do.
just make the numbers.
hopefully, if you get good enough at 'making your numbers', you will have time left over to actually do some work.
2. but what about the theory of capitalism, the free market, efficiency, etc?
its all bullshit. just like the theory of communism was bullshit. what statistics and 'numbers' were reported to the government were just flat out garbage. people somehow managed to make the system work through personal relationships and working-around the assholse in charge. but most of the theories it was built on have no resemblance to reality. think about it - if efficiency really made for the best corporation, why would you be spending 4 hours a week filling out meaningless statistical performance reports that nobody will ever read, let alone understand?
the only difference between the soviet union and 'the west' is that 'the west' still hasnt collapsed yet.
Metrics - A lazy manager's out every time. (Score:4, Insightful)
Falling back to metrics is a lazy manager's way of proving to her superiors that her drones are operating at peak efficiency. The most lazy of all will rely on utterly meaningless metrics such as the number of help tickets closed per day, per individual per day, etc. A metric such as this is completely useless as all tickets don't require an equal amount of effort to complete. Diagnosing a problem due to an intermittent hardware issue doesn't take the same amount of effort as helping a user change their password. Unfortunately these types of issues generally comprise the vast majority of tickets generated and therefore often end up being the ones that are 'measured. ' This often leads to a drop in morale and thereby negatively impacts performance; ironically the opposite of what the whole exercise is attempting to accomplish.
Trouble ticket data is primarily useful for detecting trends, thereby helping an IT team appropriately focus their human capital on issues that will enable their users to be more efficient. Going back to the password issue above, the speed and alacrity with which the IT staff help users change their passwords isn't a useful metric at all. A more meaningful metric would be the frequency of password change requests before and after the installation of a self-service password reset solution that was put in place in response to the analysis of help ticket data that showed that this was one of the most frequent issues and one that could be easily solved with little effort and financial expenditure. Measuring a sharp drop in password reset requests would show that the solution worked and was therefore beneficial to the organization by enabling users to help themselves, resulting in their having more time to concentrate on their primary tasks, and also by allowing IT staff to allocate their resources on issues that are less amenable to resolution via automation.
Unfortunately, in my experience, ticket systems get used to determine useless metrics such as the first example mentioned above, and therefore end up being the bane of IT staff, rather than a useful analytical tool.
Re:Hahaha (Score:3, Insightful)
Re:Hahaha (Score:5, Insightful)
One of you is getting fired
On a three person team, metrics are irrelevant - personality and politics are 100% more important than technical skill.
If you've got 15-20 people and it's time for a 10% downsizing, or (less realistically), performance based bonus or advancement, then metrics start to be something worth looking at.
When I had to choose between 2, my boss asked me who was my choice, I told him, he agreed, I started to tell him why, he stopped me and said: "it doesn't matter why, I'm just glad that we came up with the same answer."
You get what you reward, not good work (Score:4, Insightful)
Time to answer call, time to resolve ticket, abandoned tickets (unresolved).
In business school it is a common theme in various classes that you get what you reward, not what you ask for, not what is necessarily best for the organization. Here is a highly relevant Dilbert cartoon illustrating this point, http://dilbert.com/strips/comic/1995-11-13/ [dilbert.com].
The underlying problem is that metrics applied to humans leads to people working towards the metrics, not necessarily doing good work. It is a classic environment for unintended consequences. Its not even that the people are necessarily being opportunistic, there is also a certain amount of practicality. If you are being measured by some metric and keeping your job or getting a raise is dependent upon that metric you may quite rationally decide to act to that metric rather than what is necessarily in the best interest of customers.
Are you measured by resolved tickets? Then tickets will get resolved quickly. Not necessarily thoroughly, completely, or robustly resolved. Which leads to related followup tickets because of a minimal effort put into resolving the original ticket. I saw this in a programming environment where the tickets consisted of new features or bug fixes.
Are you measured by abandoned tickets? Then tickets will get resolved, even if they don't reasonably deserve to be considered resolved. You will get things unnecessarily classified as "unable to duplicate", "insufficient information", etc.
In these two examples, where is the difficulty of the task factored in? Not all task, tickets, are equivalent. Furthermore sometimes there are external dependencies, a part is being shipped, where is this factored in?
The metrics you offer are reminiscent of stats from call centers. There such metrics are a little more reasonable, not perfect but perhaps OK, given that the calls are somewhat equivalent in the amount of effort required, a small number of minutes not hours, and that they are randomly assigned. Over the period of say a month the large number of calls handled by any operator will resemble a normal curve with respect to effort required. For an IT organization the evaluation period may need to be some number of years to get to a normal curve with respect to effort required.
I just can't add anything to that (Score:5, Insightful)
Might as well close the comments now. :-)
Go look up Robert Austin's book on measurements and management. Read it and recognize that you've been given a task that is at best counterproductive and at worst impossible. Dust off your resume, because it may be more than one of you that are getting fired. ..bruce..
Re:Hahaha (Score:4, Insightful)
Yet another AC here.
If there were any justice in the world firings would start at the top. Fuck management and their big salaries. What the fuck are they doing that they have to be paid so much? What value do they bring to a company? They're too happy to fire you to make more money for themselves (the fact that most people are one paycheck away from financial ruin doesn't seem to bother them) but do they ever get fired even if they run the company into the ground? Fuck no. I'd love nothing more than to see some of the bosses I've had in the past be out there doing actual work.
Is this better?
Re:Hahaha (Score:4, Insightful)
Do you actually believe this? Seriously? Most upper management is there because they either knew someone in the right position or had enough money/clout to force their way into it. Often time through family. Rarely these days do you see any upper corporate management that actually worked their way up.
Re:Hahaha (Score:3, Insightful)
Do you actually believe this? Seriously? Most upper management is there because they either knew someone in the right position or had enough money/clout to force their way into it. Often time through family. Rarely these days do you see any upper corporate management that actually worked their way up.
Do you have statistics or data that shows this? Because my experience has been the opposite, people do become upper management because of skill. Except in companies that are about to fail.
My guess is you just made up your facts there, but if you have hard date, I'm willing (and interested!) to change my opinion.
Re:Hahaha (Score:2, Insightful)
Re:easy (Score:4, Insightful)
More to the point: There is no good reason to use metrics for a 3 person team. You could sit down and observe the 3 person team for a week and get a very good idea of what's going on, and it would be easier and probably take much less time than designing, implementing, gathering, and analyzing metrics. It's easier and more accurate.
If you're going to use metrics, use it in situations where you're managing such a large group that you can't possibly know who everyone is. Don't use it in situations where you have the option of doing a simple evaluation.
Comment removed (Score:3, Insightful)
Re:Hahaha (Score:5, Insightful)
As an IT professional who was just fired from a job shortly after management started to ask for all sorts of new metrics from our ticketing system, I also fully agree with the parent statement. Get your resume together and start getting it out to recruiters pronto.
In other words, this is a sham. There ARE no standard metrics because every single company and department are different. The only real metric that matters is if your customers are happy and satisfied, and that they could find out better than you could. You're being set up.
Re:easy (Score:5, Insightful)
I don't buy this. I (hate to admit that I) worked at Zynga, and their entire business model is based off of metrics, both internal and customer metrics. They are behaviorists, and brought in financial industry data modeling to design games and make business decisions. It works, and it works fucking well. In the long term this Skinner box model may or may not work for creating a sustainable business, but it has worked for the last couple years. People may not like their games, but they keep playing them.
The problem is bad or incomplete or misinterpreted metrics. Metrics in and of themselves or not bad. The problem is with the people that use them.
LS
Re:Metrics suck (Score:5, Insightful)
Totally agree.
My small team was taking a beating and getting little support from management. Issues were piling up, we were getting randomized, and I felt like I needed another staff member. Management wouldn't budge though. I needed to prove to them that I could manage my department better. Nobody even cared about IT metrics at our company. Rather than see them as an evil thing, we embraced them to gain credibility in our organization.
We started by outlining some of our support boundaries, types, and set goals for response times. Then we customized our ticket system and added in some categories and priorities.
For Support (meaning something is broken and must be fixed)
High pri: 24 hour solution
Medium pri: 1 week
Low pri: 3 months or more, not a critical metric
We also had a "Request" category (not a fix, but doing standard tasks like toner replacement, adding e-mail alias, help train someone, etc.). These had different goals for solutions. We even had ticket categories for maintenance and projects. Though they junk up the ticket system, it allowed us to track time going into other tasks and paint a picture of a staff member's whole day of work, not just the support end.
Some of our favorite metrics:
1. Average time on support OR request tickets aggregated by our team or split down to individual staff members, and divided by priority level.
2. Total time spent on tickets by user (so I know if we're actually working or lounging around). ---this is a motivator for staff to actually enter tickets and get credit for their work. Some flexibility is necessary here because people eat lunch and go to the bathroom - you can't expect this to add up to 40 hours a week perfectly.
3. "Most troublesome user or department." I don't advertise the data on this report, but it lets me know who to focus on with either training or nudging their boss a little. Execs get interested in this once in a while, and the users that found out we keep such a log try to keep off of it. Many will joke about it, but still ask "Hey I'm not in the top 10, right?" It encourages users to be a little smarter and not lean so heavily on IT for silly things.
4. Most troublesome product - THIS IS BY FAR THE BEST METRIC. This has helped me gain support in dropping old junk software, getting better solutions, setting up training for people, or creating general awareness that we need to improve. For example when I was able to quantify how much time it took when people forgot their passwords for our cloud-based e-mail password, I got support for Office 365 and ADFS for single sign-on. In the interim, people stopped took a little more personal responsibility because they realized that password stupidity = preventing us from getting value-added projects done. Stuff like this is beautiful because it takes most of the heat off of IT staff, and informs other managers of our company-wide pain points that we should invest in.
5. On the same lines as #3, identifying how many high pri tickets are out there vs. medium or low (and possibly categorizing by customer or department) can sometimes sniff out IT abusers that need stuff now now now. Tread on this carefully as you don't want to constantly wage war with your metrics, but with some political prowess and some data to back you up, you can start solving some of these problems.
#3 and #4 are big for us. Don't just think of metrics in terms of how you will hang yourself. Think about the bigger picture and answer questions that will help motivate your team in a positive way, get credit for what they/you do, and enable you to tell a better story to management. You probably will want to develop two sets of metrics - a more detailed set for yourself so you can manage effectively and proactively identify problems (and solve them in advance before your boss gets to it), and a more general set for your boss so you can deliver a simple and concise picture without getting him too involved with the details you can handle yourse
Re:Hahaha (Score:2, Insightful)
Every single place I've ever worked or been involved in the corporate going-ons. Everywhere. If you drink with the right people, play golf with the right people, sleep with the right people, or have the same rich friends as certain people, you can be upper management. The 'skilled' workers are kept in the trenches exactly because they are skilled and good doing what they do. The few skilled hard workers that make the move to upper management are just the ones that are tired of getting fucked over on salary while they produce everything, so they buy into the management bullshit and mingle with the other wordsmiths in order to make a change in their life. You need more exposure to different situations, try being a consultant for a while, you'll discover the reality real quick if you are even mildly observant.