Start with Reporting, then Measure For Success

What are some common reasons IT commonly measures and reports metrics? Unfortunately, in my experience, the answers are not always the greatest.

It may come down to "we produce this report/measure this indicator because we always have. We measure this indicator because everyone else we know does. We produce this measure because our tools enable us to. We read/heard about this measurement in a magazine/training class, and it sounded like something we should do. Or, my boss expects to see this report, none of which are the best reasons to start measuring. 

The purpose of measurement and reporting is to enable fact-based decision making. But if no one is making any kind of decisions from your reports, then you’re not capturing and providing the right measures to the right people in your reports. Perhaps you should apply outcome-based thinking and start with reporting first, then measurement.

Are Your Measurements and Reports Telling the Right Story?

The fact is that regardless of the quality of your measures and reports, a story is getting told. But is it the story that needs to be told? What you measure, what you report, and the story that is told within those reports is the difference between enabling valuable insights and actions and just noise and confusion. In fact, there are many common mistakes made while measuring and reporting. 

There is mistaking outputs for outcomes, as many reports contain measures that confuse activity - outputs with accomplishments - outcomes. In other words, just because you closed the incident record, or answered the telephone within 60 seconds, doesn’t mean that you’ve accomplished any kind of result. Some only focus on the service desk, while measures that are meaningful and relevant to the service desk have little to no meaning outside of the service desk. No one outside of the service desk really cares about measures like ASA or utilization.

There is also missing business-oriented measures altogether, as in today’s digital enterprise, technology is integrated with business processes – so integrated that businesses cannot function without technology. But IT-produced technology-based measures and reports typically do not reflect the business of the business. And there's the infamous “watermelon SLA” report, the measures that IT reports do not reflect the experience of the consumer. IT hits its self-defined targets, pats itself on its back, but then wonders why customers aren’t happy.

Well… customer satisfaction isn’t about the customer as many IT organizations conflate the terms customer and user. Remember, a customer defines the requirements for a service and signs a service level agreement (SLA). Users do not sign SLAs. Yet IT organizations send out satisfaction surveys to users and call that customer satisfaction.

Also remember that customer satisfaction isn’t about the customer as the surveys you send out to users are often not returned. And if they are returned, it is typically from people who are not happy with the interaction that they just had with IT. Yes, occasionally you get a response from a happy user. Yes, it’s good to know that users have had a poor experience. But regardless of the response, IT continues to send out the same surveys, containing the same questions, reporting the same measures, and nothing changes– good or bad – about user interactions with IT.

Not having consistent, relevant, and meaningful performance measures and reports damages the reputation and value of IT. What’s worse, measurement and reporting are activities often done as an afterthought.

Start with a Reports-First Mindset

Starting with a reports-first state of mind will only work to dramatically improve the usability and impact of your reports. How can you apply outcome-based thinking to measurement and reporting? In other words, start with reporting first. Ask this, then identify the audiences for your reports. At a minimum, your audiences will be consumers, IT management, senior executives, and the service desk team. Keep in mind that each audience will have unique reporting requirements. For example, reports that you may provide to senior executives will be different than reports that you may provide to consumers; reports that you may provide to service desk staff will be different than reports that you may provide to other IT colleagues.

What does each audience want to know? Why do they need to know this information? What will they do with the information? What decisions do they need to make? Do they have performance objectives or targets that rely on this information? How frequently do they need the information? What do you want your audiences to know? Why do you need them to know this information? What would you like them to do with the information? What decisions do you want them to make? All important questions to ask. 

By identifying what your audiences want to know, as well as what you need to tell them, you’ll identify the specific measures that you will need to capture, monitor, and report. But there's still more to ponder does the data to develop these reports exist? Can the data for each measure be captured? And is the data coming from a source that can be controlled? If the answer to the above questions is “yes,” you’re on your way to providing the measures and reports your audience’s need – and value.

But if the answer is “no,” never fear. This just means that you and your audiences will have to do some more work and negotiate what can be done that will meet your audience’s needs.

Make Sure Your Reports 'Measure Up'

Do your reports measure up? While an important question to consider, there is more to do on this front. And here are my suggested steps for getting your approach to measurement and reporting up to speed. You should consider auditing your current reports and asking do you know the audience for each of your reports? Are you missing an audience for reporting? Along with do you know what decisions are made from what is reported?

Next, develop decision maps for each published measure and consider what decisions are enabled by a published measure? Who makes or should be making decisions based on the measure? What may potentially no longer need to be reported? It is also key to compare published measures with the organization’s mission, vision, goals, and objectives (MVGO) and ask are your measures strictly technology or process measures? What measures are needed to relate the activities of your team or department to MVGO is another key consideration. 

Finally, spend time reviewing your reports with your audiences and gather feedback. Yes, meet with the receiver of each report and walk through your current reports with them. Ask questions about their job function, business contributions, and goals for their team. Ask them to tell you what measures they need to take action or make decisions. This does a few things for you. One, it helps you understand how reports are being used. And two, it provides you with insight into another area of the organization. Three, it builds and enhances your reputation as a business-focused not just technology-focused colleague.

Effective measuring and reporting don’t happen by chance - or overnight. So, starting with reporting first will result in better measurements, better decisions, and better overall value.