We know that every improvement is a change—but how do we know that a change is an improvement?1 Improvement experts and patients, each in their own terms, ask this question with appropriate impatience.23 For example, how can valid conclusions be drawn when an improvement initiative employs no control population, or if the study employs predominantly qualitative measures? Indeed, some argue that if the RCT is the gold standard, how can improvement scientists settle for less?4 Others counter that this frame actually constrains the options for knowing.56 Flyvbjerg describes these debates in the larger scientific community—at their most vehement—as the Science Wars.7 That goes far beyond where I intend to take this modest essay.

ROLE FOR QSHC AS A TEST BED FOR HOW WE KNOW

Increasingly there are calls for a rich discussion and appraisal of new ways of knowing while striving to improve existing methodologies in healthcare improvement and patient safety.23 I propose that this journal, given its unique subject matter, provides an appropriate test bed to assess the rigour and utility of scholarly reports in these fields.

While doing so, we must also be mindful that there are emerging media that exist outside both print and online scholarly journals that could also contribute to validation of improvement strategies. The emerging roles for discussion venues such as wikis, networking sites and chat rooms are also ways of exploring methodology, results and their application to patients and systems. While outside the purview of this editorial, they must inevitably be addressed as we move to new ways of knowing.

A FOCUS ON CONTEXT

Batalden and Davidoff have defined improvement as the sum of generalizable scientific evidence plus context.1 Scientific evidence in this equation is frequently developed by RCTs. It is the task of improvement scholars, however, to ask the critically defining questions of context as it relates to implementation of that evidence. Study of context requires candour and transparency. A critical test of such a report, for example, would focus on whether the authors are mindful of whether the proposed improvement could be implemented in contexts other than that in which the study was performed.8

For example, when the Wagner Chronic Care Model (CCM) appears to work well in some residency training practices—achieving lower HgbA1c levels, lower blood pressure and therapeutic levels of LDL—but not in all such practices, the question is not whether the components of the CCM are valid. That evidence exists.9 The critical question for a successful improvement initiative is to explore and define the context of such a study in a way that leads to improvement in other residency teaching practices.10 It dictates that a variety of settings be studied to determine what is characteristic of each—both those that demonstrate successful improvement and those that do not.11 Such study of culture, leadership and effective teams requires application of appropriate scholarly disciplines, for example, sociology, psychology or organisational research.

WHAT IS THE OPPORTUNITY FOR QSHC’S AUTHORS, REVIEWERS AND EDITORS?

The task of authors, reviewers and readers of QSHC remains unchanged—to ask the discerning questions about submitted papers that reflect critically on the evidence that has been generated. Reviewers must insist on clarity of theory;6 explicit attention to context;1 application of validated tools and methods; and mindfulness of bias as well as the limitations of the study’s methods and interpretation. But this opportunity goes further: how can QSHC provide a test bed that takes the process beyond traditional review and revision? The editors of QSHC offer the following three strategies.

First, provide a platform for discussion and debate by publishing critical reports that explore the epistemology of healthcare improvement and patient safety. Brown and colleagues have provided an excellent example of this in their series of four papers published in this issue that address the epistemology of patient safety (see page 158).12^–15

Second, actively promulgate and test publication guidelines for scholarly publication in healthcare improvement. Guidelines for Quality Improvement Reports (QIRs) were originally proposed in 199916 and republished in QSHC in 2004.17 Over 50 QIRs have used this format. Draft publication guidelines for improvement research (now called SQUIRE—Standards for Quality Improvement Reporting Excellence) were advanced in QSHC in 20058 and have been the topic of a series of commentaries. These guidelines place strong emphasis on context and its role in providing reliable evidence for improvement. How might the scholarly utility of these guidelines be tested? One approach would be for surveys of authors and readers to assess their usefulness. In addition, retrospective analyses could address the impact of these guidelines on both the rigour and utility of published reports. Prospective comparisons could be conducted to test alternative formats for the same report, similar to the recent application of a Cochrane review18 and Realist review19 to examine the same cohort of studies.

Another application for publication guidelines is their potential use in the publication of abstracts from scientific meetings. In this issue, QSHC publishes abstracts from the 2007 IHI Forum Scientific Symposium. We offered the authors a choice of both QIR and SQUIRE formats for their abstracts as they prepared them for formal publication. Assessment of the guidelines’ utility in this instance will serve to craft instructions for authors who contribute to future compendia of abstracts.

Third, implement studies that develop a practical typology for the range of reporting formats. When is a case report appropriate? When do the QIR or SQUIRE guidelines apply? When will only an RCT serve? In the final analysis, the ultimate test of a scholarly report for healthcare improvement is its contribution to diffusion of effective strategies for better health and healthcare for our patients.

We invite your comments, advice and reports of studies that address these questions.