Decision-Making Biases

Congruence Bias

Definition

Congruence bias is the tendency to test a hypothesis only by looking for observations that would confirm it – without checking whether alternative hypotheses fit at least as well. You test 'does it work?' instead of 'what would distinguish this from the alternative?'

Real-world example

Peter Wason's famous 2-4-6 task (1960) is the clearest example. Participants were given the sequence 2-4-6 and asked to find the rule. Most assumed 'even numbers ascending by 2' and tested 8-10-12, 14-16-18 – all confirming. Few thought to try 1-2-3 or 5-4-3, which would have revealed the actual rule was just 'three ascending numbers.'

The same pattern runs through product development: the team believes feature X will lift conversion, tests feature X against no feature, and sees an increase. They never test feature Y – which might have delivered more impact for less work.

Supplementary perspective

Congruence bias is closely related to confirmation bias but more specific: it's about test strategy, not just interpretation. Even if you're willing to update on disconfirming evidence, you can structure your tests so disconfirming evidence almost never appears.

Practical advice

Recognize

—Ask: 'What alternative explanation could also produce this result?'
—Be skeptical when every test you run confirms what you already believed.
—Notice if you only test A vs. nothing, never A vs. B.

Counteract

—Formulate at least one rival hypothesis before designing the test.
—Design tests that could falsify – not just confirm.
—Use A/B testing with real alternatives, not just on/off.

Ethical use

—In debugging: test whether the same symptom could have other causes.
—In strategy: ask 'what would show we're wrong?'
—In research: preregister multiple competing predictions.