On 6 July 1885 Joseph Meister, a 9 year old boy who had been severely bitten 2 days before by a rabid dog, was treated in Paris with the rabies vaccine developed in Louis Pasteur’s (1822–95) laboratory after years of brilliant scientific research and experimentation on animals.^1,2 Before this, no one who had developed the symptoms of rabies had survived. Meister was at the highest risk of developing symptoms but, after 10 days of vaccinations, he was fine and lived for many years. The results of this and a second case were so dramatic compared with previous experience that, by October, speakers at the French Academy of Sciences stated that it was “necessary to organize this treatment for everyone” and “This is a memorable day in the history of medicine”. Philanthropic contributions poured in and by 1888 the Pasteur Institute was founded. By then about 1200 patients had been vaccinated with a mortality rate of 1%.³

RABIES

Transmitted to humans by animal bites, rabies has always been a rare event. A biting animal may not have rabies. Verification that an animal is rabid is not always possible if the animal cannot be caught and watched. A rabid animal that bites does not always transmit the rabies virus. If the person is infected, the incubation period is about 20–60 days before symptoms develop leading to a painful and certain death.⁴ Today there are occasional reports of survival but these are so rare as to make headlines.⁵ The rarity of rabies and the long incubation period led to Pasteur’s novel approach of not vaccinating everyone preventively. The incubation period allowed time for his 10 days of vaccination to build immunity.

Would you—having been infected by a rabid dog—be willing to participate in a randomized controlled trial (RCT) when being in the control group had a certainty of a “most awful death”? If volunteers could be found, the trial would have to be small and would therefore have low statistical power. If one wished to show that vaccination was effective for men and women—young, middle aged, and old—the sample size would have to be 264, condemning half those people to a certain death. In this example the application of statistical process control (SPC) makes more sense. SPC avoids the ethical issues, saves lives, builds on prior experience to control confounding variables, gives an answer more rapidly, and has much more statistical power.

In order to demonstrate the application of statistical process control (SPC) by the use of control charts, we have simulated pre and post 1885 rabies mortality using data from the literature. This simulation assumes a mean (SD) survival of 20 (10) days before 1885. Patients were grouped into blocks of five so that each point on the control chart represents five patients recorded sequentially over time. This would be equivalent to an average of five patients bitten in a day. The survival in days is plotted for 500 groups of five patients sequentially. Survival in days after receiving Pasteur’s treatment is simulated for a similar number of patients based on a mean survival of 45 years combined with a mortality rate of 1% from the treatment (figs 1 and 2).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

 Shewhart control chart for mean survival after onset of rabies symptoms. This chart indicates when there is a shift in the mean of the process. In this case the mean of the process was set to 20 days (the mean survival after onset of rabies symptoms before July 1885).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

 Shewhart control chart for mean log survival days after onset of rabies symptoms. This chart indicates when there is a shift in the mean of the process. In this case the mean of the process was set to 1.3 days in the log scale (the mean survival after onset of rabies symptoms before July 1885). This graph allows us to identify more clearly the control limits of the process (LCL and UCL) which contains all the points for people presenting with symptoms before July 1885.

• Download figure
• Open in new tab
• Download powerpoint

Figure 3

 Structure of an Xbar (mean) control chart. The center line indicates the grand mean of the process when in control. The upper (UCL) and the lower (LCL) dashed lines represent the upper and lower control limits calculated as 3-sigma. Points in the graph correspond to average for blocks of size five observations each. All points are contained within the 3-sigma limit.

The results are so dramatic that we have presented them in two formats. In fig 1 the vertical axis is measured in number of days of survival. In this presentation the variation before 1885 is so small as to be unobservable. Even the 3-sigma upper and lower control limits are too close together to be seen. The shorter survival points in the post 1885 treatment experience reflect one death in that group of five patients. In order to show more clearly the pre 1885 variation, fig 2 presents exactly the same data but with the vertical axis transformed into a logarithmic scale. The pre 1885 data show a very stable process without special cause variation. Every one with rabies symptoms soon died. The serious scholar may disagree with our simple approach and assumptions about survival, but we think that the before and after differences were so great that this simulation is plausible.

The statistical power of this evidence is overwhelming. Based on these pre 1885 control limits, the probability of living 4000 days is above the 894-sigma level yet the results were even more dramatic than that. When little Joseph Meister had lived to October (or 90 days), his survival was at the 20-sigma level (p = 6.4×10⁻¹³) and the French academicians were right in declaring Pasteur’s treatment a great victory. Thus, SPC methods can demonstrate a dramatic difference as a result of the outcome from one patient and the new treatment can be started immediately, rather than waiting for the results of a prospective controlled trial.

PARACHUTE JUMPING AND OTHER EXAMPLES

Gordon Smith and Jill Pell wrote a fine satirical article in 2003 pointing out that the use of parachutes has never been subject to a randomized controlled trial.⁶ A careful literature review found rare examples of people falling from great heights and living. Olympic ski jumpers survive their falls. The authors report that parachutes can fail to open resulting in death. We all learn at an early age the unvarying power of gravity and do not need to be convinced that falling from a great height is likely to be fatal. SPC evaluation using past experience which we applied to rabies vaccination may be relevant here. Even more dramatic control charts could be simulated, particularly if ski jumpers are excluded.

There are other examples. The progress made in the treatment of acute lymphoblastic leukemia (ALL) of childhood is one of the true success stories of modern medicine.⁷ Incremental advances over 50 years mean that ALL has gone from a uniformly fatal disease to one with an overall cure rate of more than 75%.

Another example was the dramatic introduction of ether as an anaesthetic during surgery. In this case the outcome measure would be pain rather than mortality.

RCT VERSUS SPC

For the four dramatic improvements described here, we propose that information from prior experience using SPC is to be preferred to RCTs for six reasons (table 1). In these circumstances SPC has greater statistical power to exclude chance as an explanation. The RCT is designed to control for unknown confounding variables. Perhaps the treatment only works for young boys and not for older women. In the case of symptomatic rabies before 1885, men and women (young and old) all died without variation. If there were any unknown confounding variables they would appear as special cause variation in a long series of prior observations.

SPC can give a very rapid answer in these circumstances. There needs to be a plausible process (treatment) change associated with the astonishing outcome that is replicable for the next patients. Without this scientifically based replicable treatment, Joseph Meister’s survival could be declared a miracle and his cure associated with the intervention of a saint—the statistical analysis of miracles. One test of an ethical randomized trial is whether you yourself and others would volunteer for random assignment to control or experimental groups.

When the expected differences are small, when unknown confounding variables are likely to overwhelm the treatment effect, where the casual model is weak, where prior information is thin, when there is a single intervention and single end point, and results are not urgent—then RCTs are more useful.

Appendix: Technical Note

This section describes the simulation we performed to demonstrate the applicability of control charts to detect extraordinary causes in the rabies case study.

We generated two samples to represent the survival of two populations of individuals after onset of rabies symptoms. They are before and after July 1885, the date in which Pasteur incidentally initiated the test of the rabies vaccine in humans. We modeled each population’s survival assuming that the survival time presents a Weibull distribution, a common probability distribution used to model time-to-an-event variables.⁸ This distribution allows for a dependence of the hazard on time. In this case, this would represent a potential change in the risk of death with time since presentation of rabies symptoms. Also, in practical terms, this distribution has more flexibility for modeling than a simpler exponential distribution, also commonly used in survival analysis. Following is the survival function of a random variable exhibiting a Weibull distribution with parameters λ and γ representing the scale and shape parameters of this distribution:

We selected for the first sample, before July 1885, the parameters λ and γ such that the mean and standard deviation of the survival time were close to 20 and 10 days, respectively. For the second sample, after July 1885, we chose the parameters λ and γ in order to represent survival time with mean and standard deviation of 45 and 5 years, respectively. This was the average survival time in France for 1885.⁹ For this second sample we incorporated individuals in whom the vaccine did not work with a frequency of 1/100 individuals. We assumed for these particular patients a survival time identical to those for people before July 1885. For both samples we assumed a period of 15 months each. This was the interval reported in the study by Moulin¹⁰ in which 2500 people had received the rabies vaccine. We assumed this rate of approximately five bitten people/day for both the before and after July 1885 samples. Table 2 shows the summary of the parameter values we used to generate both samples.

Control charts are a key component of the area of statistical process control (SPC) initially developed and more disseminated in quality control of industrial processes. Several types of control charts are available depending on the nature of the measurement to control and the particular statistical control scheme. One of the purposes of control charts is to identify instances in which a particular process monitored through a particular measurement goes “out of control” and the potential causes for such situation. Another purpose is to improve the process itself. One of the most popular ones is the mean control chart developed by Shewart.¹¹ As its name indicates, the mean control chart displays the time evolution of the mean of a continuous variable of interest (fig 3). The chart has a distinctive pattern marked by three reference lines. One in the center, designated the center line (CL), set at the average value of the characteristic to monitor while “in control”. This value can be obtained from previous experience or assigned as a milestone. The other two reference lines assigned upper and lower control limits (UCL and LCL respectively) corresponding to the boundaries beyond which the process will be signaled as out of control. For mean control charts these limits are commonly set at three standard error of the process mean of the process in control or what is known as the 3-sigma limit in the SPC jargon.

We performed all the computations for the simulation in SAS for Windows Version 9.1.