Health improvement programmes: really too complex to evaluate?

Imagine an intervention whose effects vary within and between individuals and depend on subtle interactions between deliverers and recipients, and in which exposure is uncertain. Given this complexity, who would contemplate conducting a randomised controlled trial? In fact, all these issues must be dealt with in drug or therapeutic trials, as well as in more obviously complex interventions.1

Mackenzie and colleagues describe the problems of a specific intervention—Keep Well—to argue that randomised controlled trials are inappropriate or impossible for evaluating most health improvement interventions.2 In doing so, they ignore the many successful randomised controlled trials of health improvement interventions that suggest this intervention type is not a special case,3 4 and misrepresent the MRC guidance for the development and evaluation of complex interventions.5

Keep Well has many features that make evaluation difficult, but shifting the focus of evaluation from effectiveness towards implementation is a mistake. Why should decision makers want to know about implementation, populations reached, or impact on practice unless they know whether the intervention is effective? Such questions are highly pertinent, but only in the context of good information about outcomes.

The MRC guidance is pragmatic in recommending randomised controlled trials. It warns researchers to beware of blanket statements prescribing specific methods for settings, and draws attention to successful and useful applications of a range of non-experimental approaches. The guidance also recognises that rigid protocols are often impractical, and emphasises the need for theoretically informed process evaluations. It is misleading and unhelpful to class it as part of a powerful “lobby for controlled trial designs.”1