Punching through the target

Don't let the Measure become the Goal

I took Taekwondo as a class in college. One of the few things I remember the sabom telling us is: Punch through the target. The goal in a real fight wasn’t just to connect, punching the opponent’s skin. It was to push them back. Knock ‘em down. In other words: punch through the target.

Measurements are important. We have to somehow track observable progress in order to course correct. And measuring the actual outcome is a lagging indicator. By the time you know you’ve missed your annual sales goals, it’s too late. What we need is something that predicts success or movement toward the goal. It’s a compromise between ease of measurement and predictive ability.

What’s your magic number?

Norm Brodsky, a lawyer-turned entreprenuer, wrote a monthly column in Inc Magazine, unpacking wisdom acquired running his storage business. He called his leading indicators Magic Numbers.

The new-box count is one of my magic numbers -- a reliable indicator of what my overall sales for the week actually were. Although new boxes represent only one element of my total revenue, I've learned over the past 13 years that our sales rise in direct proportion to the number of new boxes we add in any given period. Tell me on September 1 how many new boxes came in during August, and I can tell you our overall sales for August within 1% or 2% of the actual figure. The 70% decline in new boxes suggested that we could be looking at a significant slowdown.

-One of Norm Brodsky’s Magic Numbers

Peter Drucker is famously (although incorrectly) quoted as saying “What gets measured gets managed”. Unfortunately that’s true even when it’s pointless to measure and manage it, and even if it harms the purpose of the organization to do so.

The problem is when the Measure becomes the Goal. In a Taekwondo competition you measure someone on how many times they hit the opponent because it’s easier than measuring just how hard they hit the opponent. You could probably win by just coming into contact enough times. But in a real fight, you don’t want to be lying on the ground, a bloody mess, thinking as you fade to black… I really scored a lot of points...

This also reminds me of the story of why there is so much useless academic research. It’s called the Streetlight Effect

A policeman sees a drunk man searching for something under a streetlight and asks what the drunk has lost. He says he lost his keys and they both look under the streetlight together. After a few minutes the policeman asks if he is sure he lost them here, and the drunk replies, no, and that he lost them in the park. The policeman asks why he is searching here, and the drunk replies, "this is where the light is".[2]

PhD students demonstrate their ability to research by…researching. It doesn’t matter how valuable the question is, so many aspiring PhD’s research what’s easy to measure instead what’s valuable to know. They favor easy measurement over gaining knowledge.

We create metrics because they are…

1. Easier to measure than our goal

2. Leading indicators of progress

3. A stepping-stone along the way to the goal.

Norm’s boxes were already being counted. And he had a history of cause-and-effect observations that they were a leading indicator. He also didn’t use a single measurement.

But what if moving the needle on your metric doesn’t actually move you toward your goal? What if what you thought was your Magic Number was really just Magical Thinking?

Risky Metrics

Not everything that can be counted matters. And not everything that matters can be counted. It’s all too easy to measure the wrong thing for the wrong reasons:

1. Vanity Metrics - measuring something that makes you feel or look successful in an attempt to fool ourselves of the world. This is all too common in companies. I knew someone in healthcare who’s hospital simply deleted the survey questions they scored low on. I know a university researcher who cherry picked her own research results an and then multiplied them to get to the volume of responses she needed.

2. Control Metrics - choosing a metric we can control.

I know a Personal Trainer who wants all of his trainers to schedule their clients out 5 visits. His intent, I think, is to have a profitable business. Having his trainers billing out for a full week’s work would ensure revenue. Scheduling all your clients out 5 visits means the business will be great for at least 5 weeks, right? I think he chose 5 visits because it’s easy to measure and gives his trainers time to “solve the problem”. But that assumes the clients will actually stick to the schedule.

But in focusing obsessively on the 5-visit metric, he’s missing something.

Can you spot the problem?

1. Scheduling 5 visits does not guarantee completing 5 visits.

2. His no-show rate (cancellations within 24 hours) has increased.

3. To fix that, he instituted a policy to charge for no-shows..

4. Now he’s had many clients complain about the cancellation charge and completely cancel their accounts.

5. So his trainers are working hard to schedule out multiple visits and yet he’s losing business.

What he should, in my experience, be measuring is whether the cancellation policy is reducing no-shows. But that’d be a lot harder to measure. What’s easy to measure gets measured. And he’s managing the measures, and little else.

And he's wondering: How can I get more clients scheduled out 6 weeks? He’s like a dog chasing his tail. In a gym. That dog is getting a dopamine rush. It’s fun & exciting. But may not be getting him what he thinks.

Takeaways

1. Be clear in your intention or goal.

2. When you pick a metric, remember that it’s a just proxy for the intention you want.

3. Consider having several Metrics.

4. Test the metric(s). Does moving the needle on that move you toward your intended outcome?