Issues to be considered for assessing a rising tide phenomenon and results of assessment for the four case studies*
| SPI2 | Critical pathways | EQHIV | MERIT | |
|---|---|---|---|---|
| Positive evidence | ||||
| Direct evidence | ||||
| Improvement in process and/or outcome measures observed in external sites: | Yes | Yes | Yes | Yes |
| Timing: before or during evaluation study | Before and during | Before and during | Before | During |
| System-wide or specific external site(s) | System-wide | Specific external sites | System-wide | System-wide (but 30% participation) |
| Qualitative evidence showing behaviour changes driven by external factors in both study groups | Yes | Yes | Yes | Yes |
| Suggestive evidence | ||||
| Baseline measures better than expected, or already showing high standards or improving trend | Yes | Yes | Yes | Yes |
| Circumstantial evidence | ||||
| Heightened awareness of the problems | Yes | Yes | Yes | Yes |
| Negative evidence† | ||||
| Contamination within study | No | No | Unlikely | Unlikely |
| Other potential sources of biases‡ | Not apparent | Not apparent | Attrition bias cannot be ruled out | Not apparent |
*Improvement in process and/or outcome measures were observed in both intervention and control groups in these studies during the evaluation period.
†Factors of which the impact on study findings could resemble a rising tide phenomenon.
‡Including selection bias (eg, control group being a selective sample of highly motivated units or having more headroom for improvement), bias in outcome assessment (eg, changes in methods of data collection or coding over time) and attrition bias (eg, poor-performing units dropping out and being excluded from analysis).