As VP of Business Applications at SolarWinds, I have the privilege of working with a team of IT professionals dedicated to achieving operational excellence. Our internal metrics for mean time to repair (MTTR), mean time to acknowledge (MTTA), and customer satisfaction (CSAT) all range between 95% and 100%, but what are the stories behind the percentages? I gathered team members to explore their approach to key performance indicators (KPIs) and set out some guidelines to help you on your journey to optimum performance.
Start With a Trial Period to Assess Historical Data and Existing Procedures
As part of a shift-left model designed to reduce the cost of resolving a ticket, Senior IT Manager Jonathan Kenneally (JK) oversees the optimization of support processes by alleviating tier-one and tier-two ticket responsibilities from engineering teams. He highlights the importance of assessment periods, whereby historical data and real-time experience are integrated to find useful performance metrics. “It was a new area and a new team, and we didn't know what to expect. I was judging what kind of tickets had been coming in previously, and what tickets we may be able to target.” Naturally, this takes time, and at first, leaders may find themselves “sandbagging”—setting targets that are too easily achieved. “We originally set out a goal of just 60% of tickets. We looked at that and thought, maybe that's too easy of a target. We had to keep trying to get a more realistic one. Over time, we figured out what was coming in and what we could handle.”
Measure Incoming Workload Against Team Capacity to Refine Your Targets Over Time
In due course, JK determined that his team should handle 80% of the tickets. If they fall short of this target, they must find out why. “Maybe we need more training in this area. We use our KPIs to see how we meet that target and if there are ways we can increase it over time, whether through training or hiring someone with more experience in a certain area.” Above all, JK emphasizes the importance of patience. “There were significant shifts with personnel, creating that team, and evaluating what experience they had. So, it was a metric we had to build up over time. Eventually, we reached a point where it made sense. We're not sandbagging it by making it too easy for us, but we know that reaching 90% isn't feasible with our current personnel or access since it's more project-based.”
Segment Metrics for a Truer Understanding of Performance
IT Manager RJ Steadman’s team runs the access management program to ensure that identities in the IT environment are where they belong. He points out that an overall score can be misleading if it masks significant differences in performance between teams. “If I look at our CSAT score across all of IT, I might come up with something beautiful—99.99% or something of that nature. But if my colleague’s team handles around 80% of the tickets, we need to take their teams out of it and say, "Okay, what about our engineering teams?" What is the CSAT for the group of people who don't do front-tier triage? What happens when something has to be escalated, becomes more difficult, or requires a custom solution? Just because you're at 99.9% doesn’t mean the metric is valid for every segment. So, having that tier matters.” JK adds: “From my side, I've got different functions within my team. An SLA score for one team might be 98%, while another could be 75%. That 75% might sometimes be acceptable, depending on what they're working on. I could produce an overall score that looks great, but if I isolate data from my web ops team, for example, their mean time to recovery might be longer than that of my SolarWinds support team. I split the metrics up to reveal underlying issues we need to address.”
Keep Certain Metrics Consistent Across All Teams
While tiering and granularity are important, establishing two to three consistent metrics across all teams is crucial for enabling comparative assessments. Metrics such as ticket closure, mean time to repair, resolution, and acknowledgment can be standardized across teams to identify staffing issues, training needs, or process flow deficiencies. High customer satisfaction and timely ticket closure should be baseline expectations for IT teams. Leaders can drive motivation by implementing visual dashboards to provide real-time insights into team performance. “When you look at our dashboards, it’s very visual. It’s right in people’s faces”, says JK. “There are green arrows and red arrows, so people can really see how they’re performing. It has helped us, even in terms of monthly trends, to follow that. We also manage open tickets more effectively because we don’t want any tickets left untouched for over 30 days. That kind of visualization has really helped us in that regard.”
Align KPIs With Wider Business Objectives
Every measurement should be shared with an audience that values it. Part of establishing KPIs is ensuring that someone outside your team is interested in seeing these numbers. RJ says: “Any performance indicator could become a vanity metric if it’s not scoped to business needs. If uptime is five nines, that’s great. But do I get to define what uptime means however I want? Because if it’s an average across every one of our VMs (virtual machines) on any infrastructure, I could make five nines pretty quickly—with a lot of useless stuff. But if I can say, “Hey, this service requires these pieces of technology and these network connections, and in that scope, I’ve got five nines,” then we’re aligning with what the business actually needs. So if it’s an external requirement we’ve promised or an internal business need where IT is providing a service so other functions like finance, sales, support, or engineering can meet their commitments, those are the two places where we need to make sure we’re measuring for success.”
Find Your Ugly Spot
Ultimately, KPIs are more than just numbers; they are sophisticated tools to help leaders understand how teams are actually performing. Their true value lies not in highlighting what your team is doing well but in illuminating faults in helping find the “ugly spot”. With this knowledge, you can address the flaw and close the gap. Excellence means following this process again and again. Only with a willingness to face shortcomings can an organization stay ahead in today's hyper-competitive business landscape. In the end, KPIs can be used to tell any story you want, but if you aren’t using them to shine a light under the couch, you’re missing the point.
That’s the lowdown on setting game-changing KPIs. But what about when things go wrong? Check out Brad’s article on the human impact of system failure.
