What Are The Metrics of Artificial Intelligence in Service and Support?

Each month, MetricNet highlights a Key Performance Indicator (KPI) for service and support. We define the KPI, provide recent benchmarking data for the metric, and discuss key correlations and cause-and-effect relationships for the metric. The purpose of the column is to familiarize you with the Key Performance Indicators that really matter to your organization, and to provide you with actionable insight on how to leverage these KPIs to improve your performance

This month, we depart from our usual format. Instead of discussing a single metric, I will discuss the metrics of artificial intelligence, as well as how you can measure the effectiveness of your AI technology.

How Do We Define Artificial Intelligence?

It doesn’t make sense to discuss the metrics of artificial intelligence unless we first define what we mean by AI. Much of what passes for AI today is simply human intelligence dressed up to look like artificial intelligence. Some of the best-known AI tools in the industry do nothing more than read knowledge articles, and then tell you what they have read.

That’s not AI, and it’s certainly not machine learning.

I have personally witnessed more than 100 demonstrations of AI tools in the past few years. Only recently have these demos convinced me that we are at an inflection point with AI. There are now credible demonstrations of AI tools that not only exceed the capability of human experts, but also get smarter over time.

The true test of any AI tool for service and support is the following: Without human intervention, will the tool reduce ticket volumes, resolve problems more quickly, decrease total cost of ownership (TCO), and improve the customer experience? If it checks all these boxes – and gets smarter over time – then it’s true AI, powered by machine learning.

The Metrics of AI

The definition above gives us good insight into the metrics of AI because we can measure ticket volumes, resolution times, TCO, and the customer experience. An effective AI tool will move the needle on all these metrics – without human intervention.

Ticket Volume

Specifically tickets per user per month will decrease with an effective AI tool. In 2018 I wrote a Metric of the Month on Tickets per User per Month. The current average for tickets per user per month is about 1.1 tickets at level 1, and 0.5 tickets for desktop support. The most effective AI tools will reduce these numbers, sometimes quite dramatically. Reducing the tickets per user per month by 50% or more is not uncommon with an effective AI tool.

Resolution Times

Mean Time to Resolve (MTTR) is one common metric for resolution time. My HDI article on MTTR can be viewed here. MTTR is primarily a level 2+ metric. It is commonly tracked for desktop support, level 3 IT, field support, and vendor support, but not at level 1. At level 1 (the service desk), First Contact Resolution Rate (FCR) is the resolution time metric that is in common use. Whether for level 1 support or level 2+ support, effective AI will improve both metrics: MTTR will decrease, and FCR will increase. Depending upon your starting point, the reduction in MTTR and the increase in FCR can once again be quite dramatic. Reductions of 50% or more in MTTR and increases of 20+ percentage points in FCR are not uncommon.

Total Cost of Ownership

AI reduces TCO in two ways: by reducing ticket volumes, and by shifting left. We all know that the best ticket is the ticket that never happens. Effective AI makes this a reality by preemptively detecting and correcting problems before they become tickets. With AI data mining algorithms, it is now possible to operate at Level -2 on the shift left spectrum (illustrated in Figure 1 below). This “search and destroy” capability, which was not possible before the advent of AI, reduces TCO by reducing the volume of tickets, and it saves enterprise customers time and money by preventing many tickets altogether.

Customer Experience

Customers are very sensitive to FCR and MTTR. Improving these metrics with AI – reducing MTTR and increasing FCR – has the effect of producing a higher quality customer experience. You can read more about the customer experience here. Empirical benchmarking data has definitively established the cause-and-effect relationship between MTTR, FCR, and the customer experience. When MTTR decreases and FCR increases, the customer experience improves.

A complete list of metrics that are impacted by AI is shown in Figure 2 below.

Industry Case Study

Since I mentioned Shift Left above, I will share an industry case study from a large insurance company that leveraged AI to shift left and reduce their TCO. Their Shift Left metrics before and after the deployment of AI are summarized in Figure 3 below. As you can see, their AI-fueled performance improved their shift left metrics substantially.

Although this company’s cost per ticket increased at each level, their total cost of ownership decreased dramatically because of the large reduction in ticket volume – from 1.91 to 1.02 tickets per user per month at level 1, and from 0.52 to 0.31 tickets per user per month at desktop support.

Figure 4 below summarizes the insurance company’s TCO before and after the Shift Left initiative. This support organization reduced their TCO from $116 million annually to $79 million annually, a stunning savings of $37 million in just one year!

Key Takeaways

We are in the infancy of AI, and the technology will continue to evolve for many years to come. AI has, and will continue to be, a disruptor in the industry. Initially, it will eliminate the need for agent-based commodity support – think Microsoft Office, Windows, password resets, and other easily resolved problems. Then, as machine learning makes each deployment of AI progressively more intelligent, even the most complex support provided by today’s customer facing agents will be replaced by more intelligent bots. Yet the majority of agents see AI and automation as a good thing because it will transform careers for the better.

Much like today’s auto industry assembly line workers are engineers monitoring computer screens while robots actually build the cars, the support technician of the future will become a support engineer who monitors, coordinates, and directs the efforts of the AI bots. For the best and most talented in the industry, the future of IT service and support has never been brighter!

Please join us for next month’s Metric of the Month: The Metrics of Problem Management, a strategic set of metrics that focuses on ticket prevention rather than ticket resolution.