A Foolproof Guide for Identifying and Selecting Measures

Measures. Perhaps one of the most critical, yet often haphazardly defined concepts within an IT organization. If you don’t measure, you will fail. If you do measure, but you’re measuring the wrong things, you will fail.

But measuring the “right things” steers IT in the right direction. As Dominica DeGrandis states, “Measures help us make good decisions. When we can’t see the problems, it’s hard to analyze them, which in turn makes it difficult to know which direction to take.”

It’s one thing to define and collect measures. It is completely another thing to identify and collect the right measures. But way too often, measures are identified, collected, and published with little thought or reasoning.

So, what should you measure? 

The Too-Typical Scenario

We in IT often collect measures for one (or more) of the following reasons:

• Some book/body said that we should measure it
• The measure is “easy” to capture
• The tools we’ve implemented provide a set of measures
  

It is nice to get the opinion of others. It’s great if you can leverage what your tools are able to provide, if it makes sense.  But identifying and collecting the “wrong” measures is a distraction and a waste of time.

Identify and collect the right measures, and you and your team will have great success. Identifying and selecting the right measures starts with understanding the big picture, then understanding how this big picture affects you and your team.

Why Measures Are So Important

Measures allow us to understand how we are performing, compare our current performance to past performance, predict future performance, identify opportunities for improvement, and determine the effectiveness of implemented improvements or decisions.
ITIL^® describes how measures are used:

• Validate. Is what we are doing supporting vision and strategy?
• Justify. Do we have the right targets and metrics?
• Direct. Based on facts (evidence collected through measures), what guidance do w need to provide to the team?
• Intervene. Based on facts (evidence collected through measures), what corrective actions do we need to take?

Measures help establish context (“Is it better or worse?”), remove ambiguity, and enable fact-based decision-making. Measures are a big deal.

Selecting the Right Measures

Here’s my overarching golden rule for measuring anything IT-related:

The closer you can relate what you’re doing to how it supports or enables your business, the more successful you will be.

The most important question you must be able to answer is “How are we helping our business be successful?”  To answer that question, you first must understand the organization’s mission-vision-goals (MVG) statement.

• Mission. Describes the purpose of the organization; answers the question “Why do we exist?”
• Vision. Describes what the organization aspires to be; it describes the “future state” organization.
• Goals. Articulates what needs to be achieved by when.

MVG provides the organization with a compass, or direction. MVG identifies what is important to the organization and applies to individuals, teams, and departments. Good measures tell us if we are on course.

So, if MVG provides direction and good measures tell us if we are on course, how can you determine what measures from your job, team, or department support MVG? Or even if you have the right measures?

The answer is that these measures must align; that is, individual measures must support team measures. Team measures must support departmental measures. And departmental measures must ultimately support company-wide or enterprise MVG. How then can we select the measures that result in this alignment? 

I would suggest that this is a great use of a strategic framework. As I discussed in a previous blog, a strategic framework is a structured method used to define how a project or initiative supports the key objectives of stakeholders. Measurement is one of the four components of a strategic framework.

To ensure success, the strategic framework must be based on the organization’s MVG. The strategic framework addresses how a project or initiative will help the organization achieve its MVG, by identifying the business objectives that will push the organization toward achieving its goals. As part of defining the approach for achieving those objectives, measurements should be defined. The right measures illustrate how the objective helps the organization meet a goal (or goals).

For example, rather than report on service availability (e.g., “98% availability”), how about report on a business-relevant measure that reflects availability (e.g., “IT provided sufficient availability such that the business was able to exceed its goal of producing over 1000 widgets for the third consecutive month”)? The first example, 98%, has little meaning to the business; but business colleagues can immediately relate to the second measure, 1000+ widgets produced for three consecutive months.

I think the strategic framework is a great tool for identifying and selecting measures, not just for objectives, but for any work—individual, team, or departmental—being done within the IT organization. And it helps put those measures into contexts that are meaningful to the organization and to coworkers.

Now That I’ve Identified Measures, I’m Done…Right?

Uh…no.

Just because you’ve identified measures, doesn’t mean that you’re done. It may sound obvious, but the question that must be answered next is “Can it be measured?”

Ensure that whatever you’ve identified as measures can, in fact, be measured. An effective measure must be reliable, repeatable, and consistently produced. For perhaps an extreme example, let’s say that you and your business colleagues have identified that you want to measure response time of the internet. Obviously, you can’t measure the internet. But the need is likely based on some underlying concern. What is the real measure that answers the concern and indicates success?  Measure that.

Talk to your business colleagues. What measures would they like to see?  Learn what conditions or situations cause them concern, and then determine if that can be measured. Ask what measures would deliver confidence in what we are doing. What matters the most?  Do the measures you’ve identified even matter to the organization or to your colleagues?  These conversations will provide you with great insights into what’s important for your organization and to the colleagues with whom you interact.

Do the measures you’ve identified even matter to the organization or to your colleagues?

And just like a business evolves over time, so will what you measure. There will always be some fundamental measures that reflect what is being done to “keep the lights on” (KTLO)—the operational aspects of your work or team’s work. These fundamental measures help you understand trends and the types and volume of work for KTLO, and there is value in KTLO. But, as the business changes, new objectives will be defined. This means that new measures must be identified and collected to reflect progress toward those objectives. Again, the strategic framework can be used to identify what should be measured, and just as importantly, measures that no longer need to be collected.

Call to Action

Have you identified and selected the right measures?  Here’s my advice for how to identify and select the “right” measures for you and your team:

• Review your organization’s MVG. This helps you understand the organizational big picture and describes what is important to the organization.
• Apply a strategic framework. Use a strategic framework to identify and select measures, then confirm how those measures support MVG. Make sure you’ve identified measures that are meaningful for your organization and measures that are meaningful to the coworkers that interact with you.
• Confirm measurability. This may be the most critical aspect of measurement. If what you’ve identified can’t be (reasonably) measured, look for alternative measures. A good measurement should be part of delivering results, not something done in addition to delivering those results. 
• Talk about your measures with others. Ensure that the measures you’ve identified resonate with your colleagues and reflect MVG from their perspective.
• Capture those measures and tell the story. This is the second most critical aspect of measurement. A good measure is collected consistently and repeatedly. But all that good measurement is meaningless unless they are reported on a regular basis, along with some brief commentary to provide context to the measure. Show and tell the story about how these measures help the organization achieve its MVG!

Remember, the right measures illustrate how you and your team are helping the organization meet its goals. Don’t just haphazardly pick measures; use a strategic framework to help you identify and select the right measures.

Doug Tedder is a strategic, innovative, and solutions-driven IT service management professional with more than 20 years of progressive experience across a variety of industries. He’s a resourceful and hands-on leader with track record of success implementing ITSM and IT governance processes. Doug is a certified ITIL Expert and ISO/IEC 20000 Consultant Manager and holds many other industry certifications. In addition, Doug is an accredited ITIL Foundation trainer and HDI Support Center Analyst and Support Center Manager instructor. Follow Doug on Twitter and connect with him on LinkedIn.