Governance29

What You Measure Matters

"What You Measure Matters" | Mike Carey

Mike Carey

April 14, 2020

OverviewRelatedHighlight

The funny thing about metrics and HR policies is that they work... sometimes they way they're intended. Qualitative data can help measure progress in your business agility transformation--but, where do you start? What is the best way to measure success?

About Mike Carey

Photo of Mike Carey

Mike Carey shared his story about the importance of metrics as "part success story, part cautionary tale" at Business Agility 2020. He is a software engineer, tech manager, and Agile coach who has seen the power of metrics for good and for ill.

Photo Gallery

Presentation Slides

Summary Transcript

So glad to be here! Before I begin, let me get the clicker sorted.

Currently, I’m with Freddie Mac, but I’m not going to be talking about Freddie Mac today. This story precedes that employment. I live in the DC area, in Northern Virginia, and one of the things I love about it is all the incredible things available in the area.

This is my lovely family at the Capitol building—some pictures we took last fall. Don’t we look so normal? We had a great photographer who did a really good job of tricking people into thinking we are a well-adjusted family.

If you’ve played around with photography at all, you might know a trick: if a picture isn’t quite good enough, make it black and white, and suddenly it looks artsy and intentional. Here are a couple of those—this is what my family actually looks like. I have four kids, and they are, how do you say… extra.

There are times when I’m not at my best. Sometimes I’m trying to have a conversation or a thought, and I find myself struggling to hear myself think. In those moments, I’ll turn to my kids and yell, “Stop yelling!”

That’s when my beautiful, patient, caring, thoughtful wife will pull me aside and say, “You know, that’s not the best way to actually get those results. You’re not communicating what you think you are—you’re sending mixed messages.”

And that’s a little bit of what I want to talk about today: metrics. I’m not here to argue which metrics matter and which don’t. I’m here to tell you that if you’re measuring something, it matters.

Why Do Metrics Matter?

1. Metrics Signal What You Value

The first reason why what you measure matters is because it signals what your values are—to the people being measured and to the organization as a whole.

I started my career as a software engineer in a very siloed organization. We had the engineers over here, the testers over there, the ops folks somewhere else, and all of IT was separate from the business. The things we measured were technical—how up-to-date our tech stack was, whether we followed proper coding techniques. What we weren’t measured against were business values or outcomes.

In fact, as an engineer, I was expressly forbidden from communicating with end-users. Their time was too valuable, and my outlandish engineering ideas would only distract them.

Ironically, I was doing well—I got promoted quickly, received awards. But looking back, I see a disconnect. On multiple occasions, my project teams would be out celebrating the successful delivery of a project, while at the exact same time, the business team was drafting a project charter to fix all the things they didn’t like about what we had just delivered.

What I understood to be valuable—compliance with technical standards, reducing coding violations—was what my leadership measured. And what gets measured drives behavior.

2. Metrics Drive Behavior

Beyond signaling values, metrics actively shape behavior. They don’t just drive actions—they influence attitudes and culture.

At one point, I became an enterprise-wide agile coach. When our CIO decided we were moving toward agility, we needed a way to track progress. We chose a simple metric: the percentage of projects classified as agile in our project tracking system.

At the beginning of the year, fewer than 10% of projects were agile. By the end of the year, less than 10% were not agile. We had done it! We had transformed the entire company in less than a year!

Except… we hadn’t. Our outcomes didn’t change. The behaviors we drove were people updating their project classification in the tracking system—not actual agile transformation. That’s what our metric had signaled to them as important.

Take a moment to think about this: What are some of the behaviors that the things you measure are currently driving in your team or organization?

3. Metrics Align Values with Behaviors

After working as an agile coach, I was moved to Virginia to help build an office for a company we had acquired. Our goal was to build something great, so the leadership team sat down and asked: What do we want this office to look like? What values do we want? What actions will reinforce those values?

Over the next three years, we tripled the number of engineers, had less than 5% attrition, and achieved 80% engagement across all teams. The measures we put in place ensured that we not only liked the people we were hiring, but that they liked working with us too.

One of our biggest projects involved replacing four legacy systems with a single application for retail employees. We didn’t force them to use it. Instead, we said, “Here’s the new system. You can try it out. You have until a set date to fully transition.”

Within the first week, we had 94% adoption. When the rollout was complete, the financial impact was estimated at $4 million per week. How? Because we aligned our measures with the behaviors we wanted to drive.

The Danger of Bad Metrics

There’s a quote from James Wilbanks, an Army advisor: “The problem, as it often is, is the metrics. If you can’t count what’s important, you make what you can count important.”

Large organizations tend to pick the easiest thing to measure, even when it doesn’t reflect what truly matters. And that can have unintended consequences.

At one point, my company introduced a new annual evaluation process. We were required to meet a force distribution—40% of employees had to be ranked “below meets,” meaning no bonus and no cost-of-living increase.

We had built an amazing team over three years. We had great results. But suddenly, I had to choose people to rank lower than they deserved. It was demoralizing.

One of my values is simple: When I come home, do I feel like I’m the man my kids think I am? In that moment, I realized I couldn’t stay in that position and still hold my head high. So, I left the company. I left a team I loved. I left an organization I still respect. But I couldn’t be a part of a system that forced me to make choices that didn’t align with my values.

Final Thoughts

At the end of the day, what you measure signals your values. It drives behaviors. If your metrics aren’t reinforcing the values and behaviors you want, you need to rethink them. And they should always be treated as experiments—never as law.

Whatever role you play in your organization, keep this in mind: What you measure matters.

Thank you.

Download Materials

Share

Are You Ready to Uncover Your Agility?

The Business Agility Profile™ is a detailed, research-based snapshot of your organization’s Business Agility capabilities & behaviors.

Based on years of research and trusted insight, it delivers data-driven analysis highlighting what’s pushing your organization forward — and what’s pulling you back.

  • Understand where your organization is on its Business Agility journey today

  • See how your organization compares to a benchmark of 1300+ other companies

  • Know the most important next steps to further develop and grow

The component MostRecentArticles has not been created yet.
The component LibraryHighlightsSmall has not been created yet.

You have NaN out of 5 free articles to read

Please subscribe and become a member to access the entire Business Agility Library without restriction.