Single Point of Success

Within the information technology realm, our servers, networks, and much else are redundant. We have backups, alternates, and contingencies in place to prevent a single point of failure. However, even with all of this in place, we often overlook the most common point of failure, our people.

We often overlook the most common point of failure, our people.

Even the best run IT departments are rarely starved for work. They spend a lot of time focusing on putting out fire after fire, and if they somehow put them all out, there are always new projects and initiatives lined up and ready. In an ideal world, you would be able to cross-train your teams and make sure your whole team can fulfill any request and solve any incident. But this go-go-go attitude perpetuates an environment where teams and technicians are acquiring new skills and specializing more and more. Specialization creates an ever-widening skill-gap, until one day, the unthinkable happens…someone leaves the organization.

That person leaving magnifies the skill-gap in what can feel like a canyon, an insurmountable chasm of varied, and specific skills that you now need to fill. Your situation may not be as dire as this example. However, there are several techniques and best practices that can help mitigate and even prevent creating a single point of failure on our team.

Spread Institutional Knowledge

Sometimes called tribal knowledge, institutional knowledge is when one person or a team of people horde knowledge about processes, technologies, and service delivery within their group. This problem can be especially risky if contained within one person.

Knowledge management, or the process of documenting solutions, workarounds, internal processes, and instructions, allows for transfer of knowledge between people. Knowledge management and concepts like Knowledge Centered Services (KCS) help us to document how and why we do things and how we can provide better support. Ideas are no longer stuck in one person’s head, but now freely shared.

Of course, this ideal is easier said than done. To enable better knowledge management and documentation, we should look to make the process as effortless as possible. This can be as simple as creating a template in Word, or using a tool like SharePoint, or better leveraging tools you might already own for a knowledge base or your ITSM tool. By making transfer of knowledge painless, you’ll be surprised how quickly your techs will adopt it.

Professional development and training are other areas where a small investment can see huge returns. Much institutional knowledge is acquired on the job in operations. However, by formalizing an employee development and training plan, you can more clearly plan for cross-training and fill existing skills gaps on your teams.

Professional development doesn’t need to be the traditional method of sending one person to a class or bootcamp. It can be helpful to do a flipped master-class-style professional development where you host the trainer on site and have a much larger portion of your team attend. For those times when you send one person off to a conference or training, you can adopt this same format when they return and have them do some knowledge-transfer back to the team.

Be Proactive Versus Reactive

The constant putting out of fires as incident after incident enters our queue is common. This reactive response puts high-stress on our workers and can cause issues like burnout and increased turnover.

A helpful metaphor when looking at becoming more proactive is to look at how we do our work. If our work were a tree, and we were measuring our success by how many times we hit the tree, a worker hitting a tree with a dull axe would take pride in hitting the tree, over-and-over. They might exclaim to others, “My team has hit the tree 800 times today.” But we lose focus on the bigger goal of what it means to hit the tree. What is the end-goal? Had we stopped to sharpen our axe, we may have been able to take down the tree in a few hits.

This is the main issue with a reactive IT culture. We might close 800 tickets. But had we stopped to look at the recurring incidents as a problem, we might have been able to prevent them.

First, stop to sharpen your axe! In IT terms, this means that when we notice trends or see a high volume of the same-old repeat issues, we can take time to do root-cause analysis or look at process improvement. By fixing the root cause, and enhancing our service offerings, we can provide a higher-quality and more robust service to our stakeholders.

Secondly, make sure your metrics and key performance indicators (KPIs) are designed with the right end goal in mind. A metric like “ticket volume” sounds good, but as a KPI, it incentivizes the wrong behavior. When looking at our metrics, it can be valuable to also look at numbers like prevented incidents and self-service utilization. Or for those metrics we do want to keep (like ticket volume) we should make sure to look at them within context, with an eye for improvement.

Create a More Resilient Organization

A single point of failure can be scary. But with the right applications of knowledge management, professional development, proactivity, and context-aware KPIs, we can start to combat the risk and create a more resilient organization. Beyond enhancing our processes and our teams, many of these mitigation techniques have far-reaching benefits that will only serve to augment our organization and enable delivery of better IT services.