Alexander Bird (King’s College, London), Understanding the Replication Crisis as a Base Rate Fallacy

Abstract

The replication (replicability, reproducibility) crisis in social psychology and clinical medicine arises from the fact that many apparently well-confirmed ex- perimental results are subsequently overturned by studies that aim to replicate the original study. The culprit is widely held to be poor science: questionable research practices, failure to publish negative results, bad incentives, and even fraud. In this paper I argue that the high rate of failed replications is consistent with high quality science. We would expect this outcome if the field of science in question produces a high proportion of false hypotheses prior to testing. If most of the hypotheses under test are false, then there will be many false hypotheses that are apparently supported by the outcomes of well conducted experiments and null-hypothesis significance tests with a type-I error rate (α) of 5%. Failure to recognize this is to commit the fallacy of ignoring the base rate. I argue that this is a plausible diagnosis of the replication crisis and examine what lessons we thereby learn for the future conduct of science.

Alexander Bird spent several years as a Professor of Philosophy at the University of Bristol. Since January 2018, he is the new Peter Sowerby Chair in Philosophy and Medicine at King’s College London.

Video of the talk:

Slides of the talk:

Understanding the Replication Crisis as a Base Rate Fallacy