The advice industry and how to read it

This site is part of a genre with a well-documented tendency toward overclaiming, which makes it worth setting out how to read it — including this.

The structural incentives

Certainty sells. "Three habits that will transform your mornings" outperforms "some evidence suggests modest effects under specific conditions."

Which means the market selects for confidence rather than for accuracy, and the most cited writers are not the most careful ones.

A single mechanism is more compelling than several. One explanation for everything is memorable. The accurate picture — several interacting factors with modest effects — is not.

Novelty is required. The genre must keep producing new frameworks, and there is a limited amount of new knowledge.

The result is repackaging: the same findings under new names, presented as discoveries.

The successful case is visible. Advice is illustrated with people it worked for, and the people it did not work for do not write about it.

The specific patterns to notice

A single striking study carrying an entire argument. Frequently the study is small, old, and has not replicated well.

The marshmallow experiment, power posing, ego depletion, and several priming effects have all served this function long after the literature moved.

Mechanism presented as evidence. A plausible physiological explanation for why something should work is not evidence that it does.

Popular writing frequently substitutes neuroscience-flavoured explanation for outcome data, and the explanation is more persuasive than it should be.

Correlation reported as cause. Particularly common in writing about habits of successful people.

The fact that successful people rise early may reflect that people in senior roles have control over their schedules, not that rising early produces seniority.

Survivorship. Studying only those who succeeded tells you nothing, because the same behaviours may be equally common among those who did not.

Effect size omitted. A statistically significant effect can be too small to notice. Popular accounts almost never report the size.

The tests worth applying

What would this predict that the opposite would not? If a claim is compatible with any outcome, it is not saying anything.

Has it replicated, by other people? Preferably preregistered.

What is the effect size? Not whether an effect exists, but whether it is large enough to act on.

Who was studied? A great deal of psychological research uses university students in wealthy countries, and generalisation beyond that population is frequently assumed rather than demonstrated.

What is the comparison? Better than nothing is a weak claim. Better than the obvious alternative is a strong one.

Does the writer state limits? Careful writing says where a finding does not apply. Its absence is informative.

What the genre gets right

Worth saying, because relentless scepticism is its own failure mode.

Some findings are robust and useful. Implementation intentions. The testing effect. Spacing. Sleep. Environmental design and friction. Social support. The value of feedback.

These have survived scrutiny, have reasonable effect sizes, and are worth acting on.

The problem is not that self-improvement writing contains nothing. It is the ratio, and the difficulty of telling which is which from the confidence of the presentation.

The honest position for a reader

Treat advice as a hypothesis about yourself rather than as a finding about people.

Try it, for long enough to see, and observe whether it works for you. Individual variation in almost everything discussed is large, and population averages predict individuals poorly.

Keep what works and discard what does not, without concluding that the discarded thing was a personal failure.

And be particularly sceptical of anything that explains everything, arrived recently, and is delivered with certainty — including on this site.

The test that costs nothing

One more, and it is the most practical.

Ask whether the writer describes anything that did not work for them.

Honest accounts of behaviour change include failures, reversals and things abandoned, because that is what the process consists of.

An account containing only successes is either unusually fortunate or edited, and the second is far more common than the first.

General information about evidence and behaviour, not psychological advice. Consult a qualified professional about your own circumstances.