Measuring The Wrong Thing

Any goal that is tracked is tracked through some proxy, because most things worth pursuing cannot be counted directly. The proxy then begins to shape behaviour.

Measures are always approximations

Understanding a subject is measured by pages read or exercises completed. Fitness is measured by sessions attended or distance covered.

Each is related to the goal and none is the goal. The relationship holds under normal conditions and weakens under pressure to move the number.

The substitution is not a mistake; the proxy is necessary. The mistake is forgetting the difference once the number becomes the object of attention.

Attention migrates towards what is counted

Once a measure exists, decisions begin to favour it, usually without deliberate intent. It provides the feedback, so it receives the effort.

Where the measure captures most of the goal, the effect is benign. Where it captures a narrow part, the remainder is quietly deprioritised.

The person continues to believe they are pursuing the goal, because the measure is still improving. The divergence is invisible from inside.

Easily countable things are not the important things

Measures are selected for availability rather than relevance. Countable quantities are chosen because counting is possible, not because they matter most.

The important components are frequently the least measurable: the quality of judgement, the depth of understanding, the durability of a relationship.

A measurement system built only from the countable therefore systematically underweights exactly what the goal was about.

Several measures resist distortion better than one

A single measure can be advanced at the expense of everything else. Multiple measures that would move in opposite directions under distortion make that harder.

Pairing a quantity measure with a quality one is the common form. Advancing one while the other declines is a visible signal rather than an unnoticed drift.

The set should stay small, since a long list of indicators is not consulted and therefore constrains nothing.

Occasional direct inspection is the check

Periodically examining the actual goal rather than the proxy catches divergence that no arrangement of measures will show.

Asking whether the underlying thing has genuinely improved is a judgement rather than a reading. It is less precise than a number and considerably harder to distort.

Doing this at long intervals is enough. The purpose is to detect drift rather than to track progress, and drift accumulates slowly enough to be caught a few times a year.