(R)Evolutionary Agility: The metrics enigma

This is the third post I host from Mauro Bagnato: my friend, if you keep producing at this rate in 2014, I will be gladly forced to open a special column for you!

A hot issue is touched this time: organizational metrics. The ones among you who are constantly reading (R)Evolutionary Agility know that I wrote already an article about this subject and I have plans to write more about it.

Whether you celebrate Christmas or not, I wish you all restful and peaceful season’s holidays!

I’ve been struggling against organizational metrics for almost two years.

I must say that it’s a bit frustrating to look for a reasonable answer to the question: “which are the right metrics that an Agile organization should have?” and get the feeling to be very close to the goal, but without being able to hit the target!

These days I recalled a famous quote from Charles Kettering saying: “A problem well stated is a problem half solved”. It made me reflecting.

Couldn’t it be that it’s so hard to find the solution of the metric enigma just because of the wrong question? Got convinced about it, I decided to start from scratch and to ask myself: Why do we need metrics?

Here you are the answers which came up in my mind.

We might need metrics because we want to:

find the evidence that the Agile transformation is worth the investment
understand if we’re improving
understand if we’re getting closer to our vision
show our stakeholders we’re a fantastic organization
etc. (I could go on…)

This approach looked promising. Clarifying the purpose traces the direction and helps identifying the metric we really need, the ones linked to our goal. So far so good I’d say!

And now? How to move further? How to turn this goal into metrics? How to measure something so vague and undefined like stakeholders’ perception, improvement, ROI of the agile transformation etc.? In other words, how to measure intangibles?

The answer, or better, the method to measure this kind of things is described in the book “How to measure anything” from Douglas Hubbard. This very interesting book proposes a three steps way to measure intangibles:

Concept of measurement. What does measurement means?

Usually measurement is thought as the process aiming at quantifying something or giving something an exact value. This common conception implies that measurement means reaching a sort of certainty (numbers or values are expression of certainty).

This idea fits very well with the measurement of lengths, widths, etc. but when it comes to the intangibles (like happiness, empowerment or motivation) it doesn’t help a lot.

Scientists use a different definition: A measurement is a quantitatively expressed reduction of uncertainty based on one or more observations.

This definition introduces two interesting points:

· Measurement is no more connected to the idea of certainty but is defined as reduction of uncertainty.

· It’s possible to measure anything (intangible or not) observing the impacts, the effects, the results.

2. Object of measurement. What do we have to measure?

Even using the new definition, something could still seem immeasurable simply because the object of measurement is not clear or is not clear enough. Very often clarifying the meaning of what we want to measure makes the problem much easier. What does it mean? That’s the fundamental question to answer.

How to measure happiness? One million dollar if someone is able to find the answer!
Let’s try to change perspective and decompose the problem by answering the question: What does happiness mean? For me happiness means having enough time to dedicate to my family and to my hobbies. Measuring the time spent that way gives me an indication of how happy I am!

I found something measurable to observe reducing the uncertainty related to the measurement of happiness!

It might happen that the answer to the fundamental question is still something intangible. In this case the trick is to repeat the question till the answer is become something measurable.

D. Hubbard suggests also a different way to clarify the object looking at the effect or the consequences: Let’s suppose to be able to clone someone and to be able to instill a massive dose of happiness in the cloned guy, holding the amount constant in the original one. What do you imagine you would actually observe that would change for the cloned guy?

Whatever the method, once the object of measurement is clear, it’s possible to find the way to measure it.

3. Methods of measurement. How do we measure?

I’ll not dig too much into this point but D. Hubbard suggests to make use of whatever methods of measurement based on:

· Sampling. It simplifies a lot the process of measurement. We need much less data than we think to get a valid measurement.

· Experiment. Anyone can develop an intuitive method for measurement and the best approach is to try it and learn from it.

Let’s come back to the starting problem and see if the method works. Assuming that we need metrics because we want to know if we are improving, next step is understanding what improvement means. Assuming that improvement means delivering more value, now we need to understand what delivering more value means. Assuming that delivering more value means delivering what customers really need, what happens when customer have what they need? Assuming that when customers have what they need, they simply use it, we could start observing how many customers really use what we deliver.
We got the object of measurement! Now the point is how to measure it? How many data are we going to collect? Making a sampling of the customers could simplify a lot the measurement itself without compromising the results (see “The rule of five” proposed by D. Hubbard).

It seems we got the point but…could we get there without wasting our time with all this stuff?

Before answering the question, we should consider that the outcome of this method is not only metric itself, but mainly the path to get there. Questioning the need (what we want to achieve), decomposing it reflecting on its meaning/effect and then finding what to measure, builds a sort of picture communicating: