Clever people always want an explanation of why things happen. But that may not always be necessary. We should always want to encourage curiosity about how the world works: why the sun shines, why plants grow or why brains produce thought. This hunger drives knowledge.

But knowing why or how something works isn’t always necessary for effectiveness. This is one of the odder features of knowledge, and wholly counter-intuitive. But it’s true in many domains.

Psychology is one of the most striking. Over the decades it has discovered many treatments that work. But almost without exception these treat symptoms rather than being based on any reliable understanding of the causal mechanisms. It would be preferable if there was some stronger theory to underlie a discipline that has only ever replicated 1% of its trials. But the field gets by all the same.

In our own daily lives we are not too troubled by a parallel situation. We find a spouse or partner or buy a house without much in the way of theory to guide us (and if we do find a theory it’s as likely to mislead us as to help us). Does it matter? Probably not.

Drugs like penicillin were discovered and manufactured long before there was a detailed understanding of how they worked at a cellular level. Again it mattered more that it worked than that we knew how it worked.

In computing, a whole array of theories have grown up to justify machine learning methods that are satisfactory – sufficiently generalisable – but don’t attempt to construct comprehensive causal models (such as Lesley Valiant’s ‘probably approximately correct” algorithms which aim to be adaptive mechanisms that try to do better at probabilistic terms).

Milton Friedman in economics was a strong advocate for parsimonious but strongly predictive models that didn’t need to be plausible as explanations for why the economy works the way it does but were nevertheless useful.

What is the implication? It’s helpful to have logic models, and theories of change (which my next blog covers). But it’s best not to fetishise explanations or to expect too much of them. Much of life is about trial and error, incremental improvement and adaptation, and given the choice it’s better to have a rough but effective approach, than a beautifully comprehensive theory that can’t be acted on.