• Quality improvement
• Quality improvement methodologies
• Continuous quality improvement

Multiple problems arise when the logic of the randomised controlled trial (RCT) is used to dismiss the success of an intervention when measurable improvement occurs in the intervention groups but an equivalent improvement also occurs in the control groups.

When groups both inside and outside a programme improve over the same period to the same degree according to an assigned metric, there is a secular trend that may exhibit the features of what Chen et al1 label a ‘rising tide’.

But when organisations in the control groups improve just as much as those that are the subject of targeted interventions, how do we know any intervention was necessary?

The rising tide phenomenon presents a confusion, a contradiction and a conundrum for logic model of the RCT, since it is constructed to eliminate bias by blinding both the researcher and subjects as to who has been randomised to the treatment arm and who the control arm, and assumes as well that the control group's data will not be contaminated by the intervention.

Quality improvement programmes, as Chen et al point out, are very different. They are rarely introduced into a vacuum; rather, they are introduced into fields where there is already a flurry of ongoing interventions. In practice, any specific intervention is merely one of many interventions, occurring at multiple levels—professional associations, workers at the sharp end delivering care, and payers at the backend refusing to reimburse for care that fails to meet established standards—all working to improve targeted outcomes.

At another level entirely, quality improvement is a field of activity in which different stakeholders with competing visions of the mission of healthcare, diverse metrics for assessing quality and different theories of organisational change compete for scarce funding to implement interventions.

Accordingly, in a study, both the treatment and the control arms may be exposed to similar internal and external incentives to perform better. The control arm may not be blinded to the bundle of procedures and changes that are being used in the treatment arm. In fact, they may have adopted on their own the very same procedures as those participating in the intervention.

When data that are ‘uncontaminated’ from a biostatistician or epidemiologist's point of view are virtually non-existent, separating signal from noise is hard. Separating programme effects from the political context that created them is a daunting task.

By calling attention once again to the difficulty of attributing improvements in quality or patient safety to a single cause, Chen et al demonstrate how poorly the statistical logic underlying experimental designs aligns with the social logic needed to explain interventions. The idea that the impact of a ‘rising tide’ and a specific intervention might be isolated from one another is scientifically appealing but substantively suspect.

In reality, both the secular trend, and the specific programme or intervention(s), share a parent—the broader campaign or movement to reduce the harm that has received public attention. The reduction of that harm in the delivery system as a whole and in individual units of that system serves as evidence—concrete and symbolic—that the system is capable of improving when challenged to do so.

Rather than ask how to discriminate programme effects from ‘rising tides’, the proper question is: how do we create and sustain the phenomenon of ‘rising tides?’ Indirectly, the authors provide an answer when they state the four conditions that lead researchers to suspect a rising tide might be at work: public concern, improvement prior to the start of an intervention in programme and control groups, qualitative evidence of strong motivation to improve among frontline workers in the intervention and control groups and circumstantial evidence that the target of an intervention was a topic of pervasive social concern.

However, the authors state these conditions as if achieving them were unproblematic. This, of course, is not the case. Merely because publicity has been directed at harm does not guarantee that interventions to reduce it will be undertaken. Both a belief that targeted harms are ‘preventable’ rather than a ‘normal, non-eliminable random risk of care’, and evidence that a bundle of steps will significantly reduce the risk of those harms are necessary for designing an intervention. Once an intervention is designed, frontline workers need to buy into or be compelled to adopt whatever new work processes are involved. Finally, in order for rising tides not to recede as quickly as they rose, the additional resources that are devoted to implementing specific interventions need to remain in place once the programmatic intervention has ended.

The important thing to remember if we wish to use a tidal metaphor for change is that tides do not control themselves. Rather, external forces determine how high they rise or fall and when they do so. Presumably, this is not the case for planned intervention to improve the quality of care or patient safety. Rather than ask whether improvement from a specific intervention exceeds the secular trend, we need to see that those interventions are a part of what creates the secular trend. Well-publicised efforts to improve quality have the capacity to create ‘bandwagon effects’. Without pervasive concern, without formal and informal interventions, without public reporting and without cooperation within and among clinical communities working to solve a common problem, there is no rising tide—organisations do not improve effortlessly. Nor do they improve performance without public pressures that require new methods for tracking and reporting specific outcomes.

The point is not so much to isolate the contribution of each programme as to recognise the synergies that are created when attention is focused on pervasive system problems. The absence of a rising tide in the face of powerful system-wide incentives to improve is a reason for public alarm. Explaining the coincidence of a rising tide and multiple interventions, and providing various stakeholders their due, is a delightful problem for biostatistician and epidemiological methodologists to have.