Table 1

Issues to be considered for assessing a rising tide phenomenon and results of assessment for the four case studies*

SPI2Critical pathwaysEQHIVMERIT
Positive evidence
Direct evidence
 Improvement in process and/or outcome measures observed in external sites:YesYesYesYes
  Timing: before or during evaluation studyBefore and duringBefore and duringBeforeDuring
  System-wide or specific external site(s)System-wideSpecific external sitesSystem-wideSystem-wide (but 30% participation)
 Qualitative evidence showing behaviour changes driven by external factors in both study groupsYesYesYesYes
Suggestive evidence
 Baseline measures better than expected, or already showing high standards or improving trendYesYesYesYes
Circumstantial evidence
 Heightened awareness of the problemsYesYesYesYes
Negative evidence
 Contamination within studyNoNoUnlikelyUnlikely
 Other potential sources of biases‡Not apparentNot apparentAttrition bias cannot be ruled outNot apparent
  • *Improvement in process and/or outcome measures were observed in both intervention and control groups in these studies during the evaluation period.

  • †Factors of which the impact on study findings could resemble a rising tide phenomenon.

  • ‡Including selection bias (eg, control group being a selective sample of highly motivated units or having more headroom for improvement), bias in outcome assessment (eg, changes in methods of data collection or coding over time) and attrition bias (eg, poor-performing units dropping out and being excluded from analysis).