In 1919 a young mathematician took a job at the Rothamsted Experimental Station in Hertfordshire, an agricultural research institute that had been running field trials on fertilisers since 1843. Nearly seventy years of records were sitting in the archive, and Ronald Fisher was hired, more or less, to see whether anything could be extracted from them.
He described the task as raking over the muck heap. Out of it came most of the apparatus that experimental science still uses to decide whether a result means anything.
The Problem With Seventy Years of Data
The Rothamsted trials compared manures and fertilisers on plots of land over decades. The difficulty is that a field is not uniform. Drainage, soil depth, exposure and the history of each strip all vary, and they vary in patterns that are invisible.
So when plot A outyields plot B, the difference might be the treatment, or it might be that plot A sits on better ground. With one field and one season there is no way to tell, and no amount of statistical cleverness applied afterwards can separate two effects that were confounded when the seed went in.
Fisher’s conclusion was that the analysis of a badly designed experiment is largely a waste of time. His line about it is the most quoted thing he ever said: to consult the statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination, and he can perhaps say what the experiment died of.
Three Principles
The answer, set out in Statistical Methods for Research Workers in 1925 and fully in The Design of Experiments in 1935, is that the experiment must be built so that the question can be answered.
- Randomisation. Assign treatments to plots by a genuinely random procedure. This is the crucial one and it was resisted, since assigning treatments systematically feels more orderly. Randomisation guarantees that unknown sources of variation, including ones nobody has thought of, are distributed by chance rather than in step with the treatments. It is also what makes the probability calculations valid, because it supplies the physical basis for saying what would have happened by chance alone
- Replication. Repeat each treatment on several plots. Replication provides an estimate of how much variation occurs between plots given the same treatment, which is the yardstick any treatment effect has to be measured against
- Blocking. Group plots into blocks that are as similar as possible, and randomise within each block. Variation between blocks is then removed from the comparison instead of contaminating it
He added a fourth idea that overturned standard practice. The received wisdom, going back to the chemists, was to vary one factor at a time. Fisher showed that factorial designs, varying several factors simultaneously in a structured pattern, extract more information from the same number of plots, and are the only way to detect interactions, where the effect of one factor depends on the level of another. In agriculture, and in most of biology, interactions are the interesting part.
The Lady Tasting Tea
The most famous illustration of his method came from an afternoon tea at Rothamsted. An algologist named Muriel Bristol declined a cup because the milk had been poured first, and said she could tell the difference.
The company was sceptical. Fisher’s response was to ask how one would find out, and the answer he constructed is the opening chapter of The Design of Experiments.
Present eight cups, four poured each way, in random order, and tell the taster that there are four of each. Then ask how likely it is that someone with no ability at all would sort them correctly by guessing. There are seventy possible ways of choosing four cups from eight, so the chance of getting all four right by luck is one in seventy, which is about 1.4 percent.
The reasoning contains the whole logic of significance testing. Set up the hypothesis that there is no effect, work out the probability of a result at least as extreme under that hypothesis, and if that probability is small enough, treat the no effect hypothesis as implausible. Fisher named it the null hypothesis.
Accounts differ on how she did. The version generally repeated is that she identified all eight correctly.
Where 0.05 Came From
Fisher suggested a probability of one in twenty as a convenient point at which to regard a result as worth attention, and said explicitly that it was a matter of convenience rather than a rule. In other passages he used one in a hundred, and he argued that a researcher should report the actual probability and let readers judge.
The 0.05 threshold hardened into a convention anyway, and then into a publication requirement, which is roughly the opposite of what he intended. Much of the replication crisis in the biomedical and social sciences a century later comes from treating a rough guide as a boundary between truth and nothing, and from analysing data flexibly until something crosses it.
Fisher also fought a long and bitter dispute with Jerzy Neyman and Egon Pearson, whose framework of accepting and rejecting hypotheses with fixed error rates was designed for repeated industrial decisions rather than scientific inference. What most textbooks teach is an uneasy hybrid of the two, which neither side would have endorsed.
Analysis of Variance
His other great tool, the analysis of variance, follows directly from the designs. Total variation in a set of results is partitioned into components: how much is attributable to each factor, how much to interactions between them, and how much remains unexplained. Each component is compared against the unexplained residual.
ANOVA is what makes a factorial experiment readable, and it is now the standard analysis across agriculture, medicine, psychology and industrial quality control. The distribution used to make the comparison is named the F distribution after him.
The mathematical foundations of much of this rest on the normal distribution and the theory of errors developed by Gauss a century earlier for astronomical measurements. What Fisher supplied was the machinery for small samples, where the older large sample approximations fail, which is the situation any real experimenter is in.
The Other Career
Fisher was simultaneously one of the founders of modern evolutionary biology. The Genetical Theory of Natural Selection, published in 1930, showed mathematically that Mendelian inheritance in discrete units is not merely compatible with Darwinian selection but is exactly what selection requires in order to work.
That reconciliation, developed alongside J B S Haldane and Sewall Wright, is the basis of the modern evolutionary synthesis, and it removed the objection that had troubled Darwin: that blending inheritance would dilute any advantageous variation before selection could act on it.
In 1936 Fisher applied his own methods to Mendel’s published pea data and found the fit to expectation implausibly good, with a probability of about one in fourteen thousand of arising by chance. The suggestion that Mendel’s results had been tidied, whether by him or by an assistant, has been argued about ever since.
The Part That Has Not Aged
Fisher held views that do not survive contact with the present. He was an active eugenicist throughout his career, served on eugenics committees, and treated the differential fertility of social classes as a serious problem requiring intervention. That commitment was not incidental to his statistics: much of the biometric tradition he worked in had been built for exactly that purpose.
In the 1950s he also argued publicly and repeatedly against the conclusion that smoking causes lung cancer, proposing that a genetic predisposition might cause both, while receiving fees as a consultant to the tobacco industry. He was correct in principle that correlation does not establish causation, which makes the episode a genuinely awkward one, since he was misapplying his own strongest argument in a case where the evidence was accumulating against him.
Both facts belong in an account of him. So does the other one: essentially every controlled trial run since, including the ones that established the link he denied, uses the design principles he wrote down at an agricultural station in Hertfordshire.
What Changed
Before Fisher, an experiment was a demonstration. You did the thing, showed the result, and argued about it. After him, an experiment is a structure built in advance so that the data collected can distinguish between the possibilities, with the analysis specified before the first plot is sown.
That is the same shift Florence Nightingale had been pushing for from the other direction seventy years earlier, when she insisted that army mortality was a measurable quantity rather than a matter of opinion. Fisher supplied the part she did not have: a way of knowing, before you collect anything, whether the numbers you are about to gather can possibly answer the question you are asking.
No products in the cart. 