Examples of elements and criteria to be considered in reporting guideline items
| Title and abstract |
| • What was improved? |
| • What was the name of the process that was changed? |
| • What outcome was affected? |
| • Have the key words been identified so that future search strategies will be able to identify the contributions of this paper? |
| Background |
| • What is generally known about this clinical area? About the limits of knowledge? About cause and effect in this condition? |
| • What is known about this area locally (and elsewhere) based on earlier efforts to change it? |
| • Who was in a position to know about this locally? Why? |
| • Was there an advantage to the local setting to address this problem now? |
| Problem |
| • What was the history of performance dysfunction, and its degree, over time? |
| • What made the current situation unacceptable? For whom and how was it unacceptable? What contributed to the local awareness of unacceptability? |
| Purpose of change(s) |
| • What was the specific aim of the proposed change/improvement? |
| • What question(s) was this test of change trying to answer? |
| Setting |
| • What were the structures, processes, and patterns of the setting? |
| • What were the relevant habits and traditions in the setting? |
| • Was the setting tightly or loosely coupled? |
| • Did people in this setting know their own work as a process? Were they included in the change? |
| • Did people in this setting have a history of working to change their own work processes? If so, what had contributed to successful change in the past? |
| Function |
| • What was the aim/role of the clinical unit being changed? |
| • Did the unit understand itself as a functioning, interdependent system? As a system, did it include the patient beneficiaries of care as an integral part of the system? |
| • What activities/processes do the staff and patients regularly engage in? |
| • How were the leadership needs of the unit met? What was most valued about the unit’s leadership? |
| • How had this system changed in the past? How did that change contribute to the participants’ understanding of themselves as a system? |
| Intervention |
| • What was the nature of the initial process change planned (who, what, when, where, how, how much)? |
| • Who were the people connected to the process that was to be changed? Was there a “natural” work group connected to the daily operations of the process and the intervention intended to change it? Did they have successful experiences changing in the past? |
| • What were the anticipated outcomes? |
| • Who was expected to lead the effort? What was their prior experience leading change? What made them curious about this situation? What had made them successful in their prior change leadership efforts? |
| • What about the local setting was conducive to addressing the problem? |
| • What resources were locally available to facilitate the learning and the testing of change? |
| Measures |
| • What balanced (biological, function, satisfaction, cost) measures of the existing situation—process and outcome—were available prior to the change? Which measures had to be developed de novo for the change effort? |
| • Were the conceptual and operational definitions of the measures and the measurement process available to all involved in conducting the measurement process? |
| • What processes were used to assure that the measures accurately represented the phenomenon under study? |
| • Were the different perspectives of patients, staff, and payers taken into account in the measures used? |
| Analytical methods |
| • Were the methods for longitudinal measurement appropriately standardized and validated? |
| • Was the software used (if any) well documented and tested? |
| Situation analysis |
| • What were the initial findings concerning the specific dysfunction, lack of effectiveness or efficiency of processes, patterns, outcomes? Were the problems assessed from the point of view of both patients and professional staff? |
| • What was the history of efforts to change the particular problems being addressed? Had change been successful in the past? |
| • Were there artifacts of the helpfulness of the organizational culture that enabled the change? |
| Outcomes |
| • How did the initial plans for improvement evolve during the course of the improvement effort and what contributed to that evolution? For example, as the initial change efforts took shape, how did the organization respond—signalling the way the system would process the change, and allow modifications to be made in the change designs? |
| • What did leaders do to encourage the improvements and modifications thereof? |
| • What alternative change strategies were considered, and why were they not selected? |
| • What were the major effects of the change(s) on the process and outcomes of clinical care for patients? The effects on staff? |
| • What harms or system/process failures were considered and looked for? |
| • Did the change process or the changes themselves introduce burdens or harms? |
| • Were there any unexpected changes, either beneficial or harmful? Why do you think they were unexpected? |
| Summary |
| • What were the most important lessons learned? From the sequence of changes that occurred? From the outcomes achieved? |
| • What contributed most importantly to the successful change(s)? On what basis do you think so? |
| • What might be necessary to sustain these gains and/or test a similar change elsewhere? |
| • What new possibilities emerged as these (successful, intended changes) were reflected upon? |
| Context |
| • How does what you found in your setting compare with other experiences in dealing with the problem (published and unpublished), and efforts to change? Similarities and differences in other settings reporting success/failure? |
| • How did you search the literature? What limited your review? |
| Interpretation |
| • What are the implications of these findings for the leadership of change? The degree of agreement regarding the desirability of or need for change? |
| • Does this work help understand the predictability of cause and effect in improvements of this type? |
| • What did the improvement process reveal about perceptions of “competing commitments” that might have surfaced during the project, and the way those were best managed? |
| • Did the improvement effort provide insight into the roles of process owners/operators? |
| • What are the inferences from this work regarding the means necessary to sustain and spread the new levels of performance? |
| • What are the implications of this work for future professional and staff development? |
| Limitations |
| • Were there aspects of the process of change or the changes themselves that were dependent on locally distinctive/unique characteristics? |
| • Were there any “artifacts” (things you could see or directly observe/witness) that would illustrate the relevant culture-in-use of the setting? Did locally held assumptions about daily work and care affect the interpretation of results? |
| • Did changes other than the specific improvement intervention occur during the period of measurement that could have confounded the interpretation of its apparent efficacy? If so, what were they? What did you do to mitigate any such confounding? How do any such confounders influence your interpretation of the observed outcomes? |
| • Were there failures in, or harms from, the change process that occurred because of special local circumstances? |
| • Did the characteristics and application of the measurement process and analytical techniques create any problems? |
| Conclusions |
| • Why does this experience make sense? How does it fit into the context of prior experiences? |
| • How does this experience invite changes in the rationale underlying the patient care involved? |
| • What are the implications from this work for patient care, the improvement process, future research, and the development and formation of health professionals—either initially or in mid career? |