Intended for healthcare professionals

CCBYNC Open access
Research

Effect of monitoring surgical outcomes using control charts to reduce major adverse events in patients: cluster randomised trial

BMJ 2020; 371 doi: https://doi.org/10.1136/bmj.m3840 (Published 04 November 2020) Cite this as: BMJ 2020;371:m3840

Linked Editorial

Improving outcomes for surgical patients

  1. Antoine Duclos, professor of public health123,
  2. François Chollet, data analyst2,
  3. Léa Pascal, statistician2,
  4. Hector Ormando, associate professor of management4,
  5. Matthew J Carty, associate professor of surgery3,
  6. Stéphanie Polazzi, data analyst12,
  7. Jean-Christophe Lifante, professor of surgery15
  8. on behalf of the SHEWHART Trial Group
    1. 1Health Services and Performance Research lab (HeSPeR, EA 7425), Université Claude Bernard Lyon 1, France
    2. 2Health Data Department, Hospices Civils de Lyon, France
    3. 3Center for Surgery and Public Health, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA
    4. 4Human Resource Department, EM Lyon Business School, France
    5. 5Department of Digestive and Endocrine Surgery, Hospices Civils de Lyon, France
    1. Correspondence to: A Duclos, Service des Données de Santé des Hospices Civils de Lyon 162 avenue Lacassagne 69424 Lyon cedex 03, France antoineduclos{at}yahoo.fr
    • Accepted 3 September 2020

    Abstract

    Objective To determine the effect of introducing prospective monitoring of outcomes using control charts and regular feedback on indicators to surgical teams on major adverse events in patients.

    Design National, parallel, cluster randomised trial embedding a difference-in-differences analysis.

    Setting 40 surgical departments of hospitals across France.

    Participants 155 362 adults who underwent digestive tract surgery. 20 of the surgical departments were randomised to prospective monitoring of outcomes using control charts with regular feedback on indicators (intervention group) and 20 to usual care only (control group).

    Interventions Prospective monitoring of outcomes using control charts, provided in sets quarterly, with regular feedback on indicators (intervention hospitals). To facilitate implementation of the programme, study champion partnerships were established at each site, comprising a surgeon and another member of the surgical team (surgeon, anaesthetist, or nurse), and were trained to conduct team meetings, display posters in operating rooms, maintain a logbook, and devise an improvement plan.

    Main outcome measures The primary outcome was a composite of major adverse events (inpatient death, intensive care stay, reoperation, and severe complications) within 30 days after surgery. Changes in surgical outcomes were compared before and after implementation of the programme between intervention and control hospitals, with adjustment for patient mix and clustering.

    Results 75 047 patients were analysed in the intervention hospitals (37 579 before and 37 468 after programme implementation) versus 80 315 in the control hospitals (41 548 and 38 767). After introduction of the control chart, the absolute risk of a major adverse event was reduced by 0.9% (95% confidence interval 0.4% to 1.4%) in intervention compared with control hospitals, corresponding to 114 patients (70 to 280) who needed to receive the intervention to prevent one major adverse event. A significant decrease in major adverse events (adjusted ratio of odds ratios 0.89, 95% confidence interval 0.83 to 0.96), patient death (0.84, 0.71 to 0.99), and intensive care stay (0.85, 0.76 to 0.94) was found in intervention compared with control hospitals. The same trend was observed for reoperation (0.91, 0.82 to 1.00), whereas severe complications remained unchanged (0.96, 0.87 to 1.07). Among the intervention hospitals, the effect size was proportional to the degree of control chart implementation witnessed. Highly compliant hospitals experienced a more important reduction in major adverse events (0.84, 0.77 to 0.92), patient death (0.78, 0.63 to 0.97), intensive care stay (0.76, 0.67 to 0.87), and reoperation (0.84, 0.74 to 0.96).

    Conclusions The implementation of control charts with feedback on indicators to surgical teams was associated with concomitant reductions in major adverse events in patients. Understanding variations in surgical outcomes and how to provide safe surgery is imperative for improvements.

    Trial registration ClinicalTrials.gov NCT02569450.

    Introduction

    Healthcare related adverse events are a leading cause of mortality.1 Worldwide, the large numbers and complexity of surgeries expose patients to a risk of substantial harm.2 One in 10 patients who undergo surgery experiences a preventable complication.3 As most cases of surgical morbidity and mortality seem to be avoidable, improving surgical safety is a priority.4

    Monitoring indicators with the intent of improving surgical outcomes is becoming increasingly important. To provide safer care, surgical teams must engage in quality measurements and understand how their performance fluctuates over time, which requires a strong foundation for fostering discussion on data and feedback within teams. The ideal method for considering the huge amount of data available would be to allow the identification of safety issues in a timely and accurate fashion. Such a methodology has already been developed within the industrial world. In his theory of variation, the American physicist Walter Shewhart postulated that quality is inversely proportional to variability in production processes and that understanding the variation of some indicators could teach the operator when and how to reduce variation. To categorise variation according to the action needed to reduce it, Shewhart designed a graphical tool known as the control chart.5 This decision support tool combines a time series analysis with a visual presentation of data, plotting successive indicator measurements in chronological order, with control limits demarcating the expected variations.6

    By converting data into knowledge, the control chart offers a way to establish prospective outcome monitoring to guide continuous quality improvement initiatives in surgery.78 The chart has been validated through more than 50 years of usage in industry and has proved effective at improving the quality of manufactured products or services.9 Previous experience suggested its transferability to healthcare10; however, rigorously designed studies with a low risk of bias are needed to determine whether this affordable intervention has the potential to benefit surgical care.11 In this cluster randomised trial, we evaluated the impact of introducing prospective monitoring of outcomes with regular feedback of indicators to surgical teams. We hypothesised that the control chart randomly implemented at the hospital level would reduce the rate of major adverse events (inpatient death, intensive care stay, reoperation, and severe complications) after digestive tract surgery.

    Methods

    Study design and participants

    We prospectively conducted a nationwide parallel cluster randomised trial. After a pre-implementation period of two years (1 January 2014 to 31 December 2015), the surgical departments in 40 participating hospitals were randomised into two cluster groups: intervention and control hospitals. We then introduced the monitoring of surgical outcomes using control charts in hospitals allocated to the intervention group (n=20) over the next two year implementation period (1 January 2017 to 31 December 2018). The control hospitals continued with usual care (n=20). We compared the observed levels in surgical outcomes from the pre-implementation to implementation period between the intervention and control hospitals to determine any improvement attributable to implementation of the control chart.

    The study was conducted in the surgical departments of 40 hospitals across France (supplementary figure S1). We screened eligible departments in all public and private hospitals performing digestive tract surgery and combined the criteria related to volume of inpatient stays (≥600 per year), rate of major adverse events (≥3.5%), and data coding quality (≥2 secondary diagnoses on average for each hospital stay). Among 134 eligible surgical departments, we enrolled the first 40 responders. All adults who underwent one of the following operative procedures in those departments were considered for inclusion: hernia repair, cholecystectomy, appendectomy, bariatric, colorectal, hepatopancreatic, or oesophageal and gastric surgery (see operations and procedures codes in supplementary appendix). We excluded patients if they were younger than 18 years, underwent ambulatory surgery, or were admitted for invasive peritoneal cancer, pre-existing adverse events, polytrauma, palliative care, or organ transplantation or retrieval.

    The study was conducted according to the study protocol and data were analysed according to the statistical analysis plan (for the protocol see http://shewhart.univ-lyon1.fr).

    Randomisation and masking

    After the pre-implementation period, the health data department of Hospices Civils de Lyon used a computer generated randomisation schedule to assign 20 hospitals to the intervention group and 20 to the control group. To achieve comparability between the groups, randomisation was stratified on the median proportion of the primary outcome (rate of major adverse events (inpatient death, intensive care stay, reoperation, and severe complications) after digestive tract surgery) recorded in each hospital during the pre-implementation period (≤8.5% v >8.5%) with a difference for the overall primary outcome of 0.5% or less, and the number of inpatients between the hospital groups (difference ≤5000 patients).12 Because this trial concerned an open label intervention and involved local investigators, it was not possible to mask hospital staff, although patients were masked to study group allocation.

    Intervention

    Hospitals allocated to the intervention group implemented the monitoring of surgical outcomes. A set of Shewhart p-control charts was provided for each operative procedure with indicators of postoperative death, intensive care stay, reoperation, and severe complications. The p-control chart, where p stands for proportion, is useful for the routine monitoring of a binary outcome, such as the occurrence of an adverse event. The data points on the charts depict variation in the indicators for each quarter, and the central line represents the mean indicator value for each hospital. Control and warning limits were set at 99.7% (3 SD) and 95.5% (2 SD) around the central line, respectively, based on binomial distribution.6 Variation in a special cause was defined as a single point outside the control limits or 2 of 3 successive points outside the warning limits. Therefore, special cause was characterised by substantial changes in patient outcomes (eg, clustering of several complications) caused by unanticipated phenomena within care delivery that deserved further investigation. The control charts were displayed each quarter as wall posters in the operating room, and variations in surgical outcomes were discussed during team meetings (see supplementary figure). In cases of worsening outcomes, special attention was paid to the identification and resolution of the causes, and actions for improvements in care were tested and implemented.

    To facilitate implementation of the control chart, study champion partnerships were established at each site, comprising a surgeon and another member of the surgical team (surgeon, anaesthetist, or nurse). Each of these duos was responsible for conducting meetings to review the control chart and maintaining a logbook in which changes in care processes were recorded. In addition, the duos from each hospital met during three one day training sessions held at intervals of eight months. Simulated role play and feedback from participants at these sessions were aimed at providing the skills needed to use the control charts appropriately, leading review meetings for effective cooperation and decision making, identifying variations in special causes, and devising plans for improvement.

    In parallel, the control hospitals continued with usual care—that is, no specific intervention was implemented within the surgical departments as part of the study.

    Intervention is described in more detail in our protocol and in the supplementary material (see components of control chart based programme). A tutorial to reproduce the control chart programme, with slideshows, videos, and logbook is also available (http://shewhart.univ-lyon1.fr) as well as key elements on how to develop and interpret a p-chart for clinical practice and how to successfully integrate this tool within a comprehensive approach (https://academic.oup.com/intqhc/article/22/5/402/1786749).

    Outcomes

    The primary outcome measure was a composite of major adverse events occurring at any hospital within the 30 days after each surgical procedure. The composite outcome comprised inpatient death, intensive care stay (at least two nights in intensive care or five nights in critical care), reoperation (open or laparoscopic digestive tract procedure), or severe complications (cardiac arrest, pulmonary embolism, sepsis, or surgical site infection). The composite was based on the Clavien-Dindo classification, which ranks a surgical complication in an objective and reproducible manner.13 For secondary endpoints, we considered each of these outcomes separately. We also assessed the frequency of signal detection related to deterioration or improvement in surgical outcomes on the control charts, and we measured the compliance of each hospital with implementation of the programme based on a previously designed six item scoring method: formation of duo partnerships, participation in training sessions, maintenance of the logbook, display of the poster, meetings of the control chart team, and the implementation of an improvement plan. Compliance with those items was mainly determined by external review of information recorded in the logbook, in addition to pictures of team meetings or posters in operating rooms and formal presentation of the improvement plan by on-site leaders during training sessions.

    We obtained data from the French Medical Information System, Programme de Médicalisation des Systèmes d’Information (PMSI). The PMSI is a nationwide database routinely used for care reimbursement and updated weekly with data from all hospitals in France. The data are prospectively collected, and the database relies on a coding system with strict definitions for variables. A subset of records is audited regularly to avoid coding errors. Because of the accuracy and exhaustive data collection of the PMSI database, no patients were lost to follow-up during the study period. Inpatient stays were recorded as standard discharge abstracts containing compulsory information about patients and their primary or secondary diagnoses using ICD-10 (international classification of diseases, 10th revision) codes as well as detailed procedural codes associated with the care provided. From the PMSI database, we extracted the personal characteristics of the patients; comorbidities according to the Elixhauser algorithm, which has acceptable validity14; emergency admission; date and type of operative procedure; main diagnosis; complexity of the surgical procedure; and median household income based on residential codes.

    Statistical analysis

    According to data from the pre-implementation period, a statistical power of 80%, an α value of 0.05, an average of 1978 patients per hospital at each period, 20 hospitals in each group, an intracluster correlation coefficient of 0.056, and observed rate of 10.9% for the primary outcome, we expected a ratio of odds ratios of 0.91 between the intervention and control hospitals from the pre-implementation to implementation period.15

    For the main analysis, we computed mixed effect logistic regression models to estimate the impact of implementing the control chart on surgical outcomes while accounting for patient clustering within hospitals. Odds ratios were used to compare surgical outcomes between pre-implementation (2014-15) and implementation periods (2017-18) in intervention and control hospitals. Utilising a difference-in-differences approach, we used the interaction between hospital groups and period to estimate the ratio of odds ratios and corresponding 95% confidence intervals that compared changes in outcomes from the pre-implementation to implementation period between intervention and control hospitals. Using estimated parameters obtained from these models and a marginal standardisation method, we determined a difference of absolute risk difference and difference of relative risks difference for every outcome, along with corresponding 95% confidence intervals computed from non-parametric bootstrap based on 1000 replications.16 The number of avoided cases among surgical patients in intervention hospitals during the implementation period was estimated from the difference of relative risks difference. The models were adjusted for potential confounders from a patient risk score that predicted the probability of adverse events and was previously developed from patient data derived during the randomisation period (2016). We established a specific risk score separately for each outcome and operative procedure considering all variables extracted from PMSI and hospital status in multivariable logistic regression.

    In the secondary analyses, we first split the intervention hospitals into a high compliance group and a moderate to poor compliance group to investigate whether compliance with the control chart implementation programme was associated with better outcomes. Then we evaluated the impact of the intervention on signal frequency for detection of variation in special causes, considering all control charts of participating hospitals whatever the outcome and procedure type. Deterioration or improvement in surgical outcomes was counted as the number of upward or downward signals, and this was modelled using mixed effect Poisson regression models with ratios of rate ratios estimations.

    Sensitivity analyses included patients who underwent ambulatory surgery. Operative procedures were stratified into minor (hernia, cholecystectomy, appendectomy, bariatric) and major (colorectal, hepatopancreatic, oesophageal and gastric) surgery. Missing household incomes in the dataset were imputed based on the mean value for each study group and period.

    Analysts were not blinded as a result of the modelling of intervention impact according to its level of implementation, which deliberately introduced a distinction between surgical departments among study groups. All reported P values were two sided and we considered a value of less than 0.05 to be significant. Data were analysed using SAS version 9.4 (SAS Institute, Cary, NC).

    Patient and public involvement

    In this intervention study focusing on healthcare professionals, it was not appropriate or possible to involve patients or the public in the design, conduct, or reporting of our research. However, we disseminated the control chart utilisation, methodology, and results of the research to healthcare professionals and the relevant patient community based on a dedicated website (http://shewhart.univ-lyon1.fr).

    Results

    Of the 159 688 patients who underwent surgery in the 40 participating hospitals during the pre-implementation and implementation periods, 156 133 (97.8%) were eligible for inclusion in the trial, of whom 155 362 (99.5%) were analysed (fig 1). After the hospitals had been randomised, 75 047 patients were analysed in the intervention hospitals and 80 315 in the control hospitals. Table 1 and supplementary material (tables S1 to S3) present the hospital and patient characteristics between study groups and periods.

    Fig 1
    Fig 1

    Trial profile. Of the 155 362 included patients, 79 127 were assigned to the 2014-15 pre-implementation period (37 579 patients in intervention hospitals, 41 548 in control hospitals) and 76 235 to the 2017-18 implementation period (37 468 and 38 767, respectively)

    Table 1

    Hospital and patient characteristics by study group: prospective monitoring of outcomes using control charts with regular feedback on indicators (intervention group) or to usual care (control group). Values are numbers (percentages) unless stated otherwise

    View this table:

    Partnership duos were established in all intervention hospitals; of these, 90% (18/20) participated in all training sessions. During the implementation period, each duo reported an average of 20 changes in care process in the logbook, five posters displayed in the operating room, seven team meetings (mean duration, 54 minutes; mean number of participants, 9); 95% (19/20) introduced at least one improvement plan. Nine hospitals were considered highly compliant with implementation of the control chart (implementation score 5-6), nine were considered moderately compliant (score 3-4), and two were considered poorly compliant (score 2) (see supplementary appendix for details).

    During the study period, 17 469 (11.2%) patients experienced a major adverse event, 2407 (1.5%) died, 8814 (5.7%) experienced intensive care stay, 7023 (4.5%) underwent reoperation, and 6575 (4.2%) had a severe complication. Table 2 shows changes in surgical outcomes from the pre-implementation to implementation period by hospital group. A significant decrease in major adverse events (adjusted ratio of odds ratios 0.89, 95% confidence interval 0.83 to 0.96; P=0.001), patient death (0.84, 0.71 to 0.99; P=0.04), and intensive care stay (0.85, 0.76 to 0.94; P=0.001) was found in intervention compared with control hospitals. The same trend was observed for reoperation (0.91, 0.82 to 1.00; P=0.06). Those results were even more noticeable in hospitals highly compliant with implementation of a control chart based programme (fig 2), whether for patient major adverse events (0.84, 0.77 to 0.92; P<0.001), death (0.78, 0.63 to 0.97; P=0.02), intensive care stay (0.76, 0.67 to 0.87; P<0.001), or reoperation (0.84, 0.74 to 0.96; P=0.009). Similar results were found in the sensitivity analysis including ambulatory surgery (supplementary table S4 and figure S2) and patients with missing household income (supplementary table S5).

    Table 2

    Comparison of primary and secondary outcomes by hospital group between pre-implementation period (2014-15) and implementation period (2017-18) in hospitals assigned to prospective monitoring of outcomes using control charts with regular feedback on indicators (intervention) or to usual care (control). Values are numbers (percentages) unless stated otherwise

    View this table:
    Fig 2
    Fig 2

    Primary and secondary outcomes by implementation of control charts. Highly compliant with implementation of control charts=scores 5-6, moderately compliant=scores 3-4, and poorly compliant=score 2. The adjusted ratio of odds ratios (ROR) captured the effect of the control chart on outcomes from the pre-implementation to implementation period between highly compliant intervention and control hospitals, and between moderate to poor compliance intervention and control hospitals. A ROR less than unity indicated an improvement caused by control chart use. Bars denote 95% confidence intervals that considered patient risk score and clustering at the hospital level

    The absolute risk of a major adverse event was reduced by 0.9% (95% confidence interval 0.4% to 1.4%) in intervention hospitals compared with control hospitals, corresponding to 114 patients (70 to 280) who needed to receive the intervention to prevent one major adverse event (supplementary table S6). Also, after implementation of the control chart, 362 (95% confidence interval 141 to 573) major adverse events and 93 (2 to 183) deaths were avoided in the intervention hospitals (supplementary table S7). Supplementary figures S3 to S6 present the rates of surgical outcomes during the pre-implementation and implementation periods by study group and hospital.

    The ratio of rates ratios for variations in surgical outcomes on control charts showed a significant reduction in the frequency of deterioration signals (0.60, 95% confidence interval 0.37 to 0.96; P=0.03) and a significant increase in the frequency of improvement signals (3.89, 1.40 to 10.83; P=0.009) within intervention versus control hospitals (fig 3). Among intervention hospitals, a median number of 3 (range 1-11) variations in special causes in each hospital were detected over eight consecutive quarters of the implementation period.

    Fig 3
    Fig 3

    Signal detection on control charts between hospitals groups. A total of 640 quarters corresponding to 16 quarters for each of the 40 hospitals were included in the analysis. Rates of signal detection were calculated as the total number of signals detected for all quarters divided by the number of interpretable indicator variations on control charts provided for all hospitals. Indicator variation was considered interpretable when the warning or control limits were not equal to 0% or 100%. The signal detection of variation in a special cause was defined as a single point outside the control limits or two of three successive points outside the warning limits. Deterioration and improvement signals were studied separately. Deterioration (improvement) signals were counted as the number of upward (downward) signals regardless of the surgical outcome and operative procedure. P values are for rates ratios estimated using mixed effect Poisson regression models to compare rates of signals between the pre-implementation and implementation periods in intervention and control hospitals

    Discussion

    This national cluster randomised trial showed that introducing prospective monitoring of outcomes with regular feedback of indicators to surgical teams using control charts was associated with a noticeable reduction in major adverse events after digestive tract surgery. The effect size was robust on the sensitivity analyses and even more noticeable in highly compliant hospitals that had implemented control charts. Furthermore, safety issues decreased while improvement signals increased on control charts in the intervention compared with control hospitals. These findings provide evidence that the routine use of control charts has a favourable effect on outcomes in surgical patients.

    Comparison with other studies

    For decades, control charts have been tested to monitor adverse events in a wide range of settings and specialties, suggesting a broad applicability to healthcare.10 They have also been used for other purposes, such as public health surveillance,17 evaluating hospital performance,18 and monitoring individual patient variables,19 with heterogeneous adherence to methodological principles in their construction.2021 Although these quality control tools could be key to enhancing surgical safety synergistically with checklists22 or process improvement engineering,23 tangible evidence of the benefits for care management was lacking.11

    In the broader context of improvement in health services performance based on strategies to feedback on indicators, inconsistent findings resulted from non-randomised studies. Pay-for-performance models that reward or penalise hospitals for meeting predefined indicator targets showed no or at most modest improvement in patient outcomes.24 Although hospital enrolment in a national surgical outcomes benchmarking programme did not result in performance improvement over time,25 alerts notification from a surveillance system using control charts might be associated with a reduction in inpatient mortality.26 A plausible explanation is that cross sectional comparison of aggregated outcomes between institutions is not ideally suited to assess hospital performance prospectively or to highlight the sudden occurrence of improving or worsening outcomes.7 Instead, performance monitoring using control charts lays the foundation for a more dynamic approach to interpreting variations in surgical outcomes over time.27 Performance monitoring also permits the identification of appropriate ways to improve patient safety, as iterative assessment more readily permits the identification of aberrant outcome patterns, potentially triggering more timely and routine investigations and interventions to correct them. The consideration of each hospital as its own performance benchmark also reduces methodological concerns about confounding variables that might compromise a fair and transparent interpretation of performance between hospitals.2829

    Strengths and limitations of this study

    Because the cluster randomisation design aims to reduce contamination, and the same institution (identified as a separated geographical entity) could not commit more than one participating surgical department to the study, we assume the risk of crossover among control hospitals is unlikely. If such a scenario had occurred, the true impact of our intervention might potentially be greater.30 Although a stepped wedge design or complier average causal effects might represent convenient solutions to reduce crossover, methodological concerns remain for adequately controlling secular trends and heterogeneous duration of exposure to the intervention among clusters.31

    The comparability between study groups might turn out to be uncertain with a limited number of clusters. By embedding a difference-in-differences analysis within a cluster randomised trial, we sought to balance both observed and unobserved patient and hospital characteristics across study groups and to ensure hospital comparability between the pre-implementation and implementation periods. To account for underlying differences in performance among hospitals, the randomisation was stratified according to baseline postoperative outcome rates. In addition, we used risk adjustment to control for potential differences in patient populations.

    Despite the careful study design, several limitations remain. Although the potential influence of a Hawthorne effect on study findings was largely compensated by the presence of a contemporary control group, feedback on indicators in the intervention group might have reinforced vigilance among surgical teams under observation. However, the attention paid to quality improvement rather than the use of charts in themselves could have improved outcomes. Also, owing to potential inaccuracies inherent in medico-administrative data, we cannot rule out the possibility that residual confounders influenced our findings. Of greater concern would be the presence of bias, in which hospital behaviour about coding accuracy of inpatient comorbidities or complications would change between study groups and periods.32 Because that information is critical for billing purposes, inpatient deaths, intensive care stays, and reoperations are accurately collected in hospitals claims databases. This is not necessarily the case for identifying specific adverse events based on diagnoses codes, the validity of which is debatable and might explain the absence of differences observed for severe complications between hospitals groups. Study outcome measurements within one month after surgery were also limited to occurrences during in-patient stay and might reflect what happened in the operating theatre and the quality of perioperative care, including the success or failure to rescue during a postoperative stay in the intensive care unit.33 Furthermore, implementing the intervention in a specific country limits the generalisability of our findings. We only included the 40 first responders among eligible French departments of digestive surgery in this study. Thus, the feasibility and impact of implementing the control chart programme in a different context remain unknown.

    Policy implications and conclusions

    Modern surgery still has a high incidence of adverse outcomes, sometimes resulting in important consequences for patients.3 Understanding variations in surgical outcomes and how to deliver surgery safely is imperative for improvements in this area. It requires the utilisation of a tool for tracking, interpreting, and controlling outcome indicators. The control chart methodology captures knowledge as a product of care and integrates related evidence in the delivery process of a learning health system. Its implementation in routine practice might encourage surgical teams to continuously critically examine the care they deliver. Regular feedback on performance can motivate behavioural and organisational changes, leading to safer surgery. In addition to computerised decision support systems based on clinical practice guidelines, the integration of control charts into electronic health records for triggering monitoring alerts for surgical outcomes could change the way surgeons manage patients. Considering that the probability of adverse events might vary across patients undergoing a particular surgical procedure, incorporation of risk adjustment or stratification represents an important contribution for further improving the tool’s performance to enable correct interpretation of variations in outcome indicators from heterogeneous populations.3435 Furthermore, the cumulative sum chart might be helpful in overcoming the limitations related to sensitivity of control charts in routine practice, as it performs fairly well in detecting small changes in surgical outcomes.36 Instead of indicators measured over a long aggregation period, which can mask and delay reactions to some process changes, the cumulative sum chart allows real time monitoring of activity, procedure after procedure, to detect any defects in surgical safety as soon as possible.37

    Measurement alone does not, however, result in improvement; rather, the reduction of adverse events relies on dedicated champions, regular interdisciplinary meetings, and the pursuit of every cause in issues regarding patient safety. Despite the satisfactory compliance of participating hospitals with programme implementation, those were especially motivated in following the trial protocol and we cannot exclude potential barriers to widespread dissemination of this intervention. To strengthen compliance in refractory teams, solutions might come from active engagement of surgical staff for leading the process of implementation.38 To be effective, the implementation of control charts requires constant interactions among healthcare professionals. Beyond enabling awareness of patient outcomes, this quality control tool promotes commitment and communication for sharing better practices. For this reason, our programme was not focused on surgeons alone but aimed to involve medical and non-medical team members in the operating room.

    Conclusions

    The value of control charts and sharing ideas within surgical teams designed to eliminate patient harm has been mostly underappreciated. In this study, the implementation of a control chart based programme was associated with a statistically significant reduction in major adverse events in patients after surgery. Such a finding shows that prospective monitoring of indicators using available data sources is feasible and can improve surgical outcomes. This methodology can be reproduced worldwide based on inpatient abstracts using a common set of data that are routinely collected in many countries.39 A method of investigating variations in patient outcomes over time within hospitals, might augment the capacity of surgical teams to improve performance and prevent major adverse events.

    What is already known on this topic

    • Modern surgery still has a high incidence of adverse outcomes, with important consequences for patients

    • Control charts to monitor outcomes have been implemented in a wide range of settings and specialties, suggesting a broad applicability to healthcare

    • Tangible evidence of the impact of a nationwide system for monitoring outcomes using control charts to reduce inpatient adverse events occurrence is lacking

    What this study adds

    • A statistically significant reduction in major adverse events and patient death after surgery was found after implementation of a programme using control charts with regular feedback on indicators to surgical teams

    • The findings support the routine use of control charts to monitor variations in surgical outcomes over time to help prevent major adverse events

    • This affordable tool based on commonly available hospital data can be a cornerstone in the continuous improvement of patient safety

    Acknowledgments

    We thank the members of the SHEWHART Trial Group.

    SHEWHART Trial Group

    Françoise Bourgoin, Housseyni Holla, and Sandra Steunou (Agence Technique de l’Information sur l’Hospitalisation); Clotilde Naudot (Assistance Publique Hôpitaux Paris - Louis Mourier); Isabelle Lacombe and Jérémie Lefevre (Assistance Publique Hôpitaux Paris - Saint Antoine); Matthew J Carty (Brigham and Women’s Hospital); Jean-Marc Arimont and Charles Foulkes (Centre Hospitalier Alpes-Leman); Patrice David and Laurence Neyer (Centre Hospitalier Colmar - Louis Pasteur); Clément Gayet, Sandrine Hemet, and Loïc Le Menn (Centre Hospitalier Cornouaille - Laennec); Valérie Serra-Maudet (Centre Hospitalier Le Mans); Emeric Abet (Centre Hospitalier La Roche-sur-Yon); Matthieu Poussier (Centre Hospitalier Montélimar); Jérôme Broli, Domenico Papaleo, and Jan Martin Proske (Centre Hospitalier Sud Francilien - Jean Jaurès); Valérie Filippi and Davide Mazza (Centre Hospitalier Toulon - Sainte Musse); Bénédicte Fraleu Louër, Dominique Gratien, and Hélène Poirier (Centre Hospitalier Versailles - André Mignot); Béatrice Alves-Neto, Kévin Fixot and Matthieu Hournau (Centre Hospitalier Régional Metz Thionville - Bel Air); Jean-Marc Regimbeau (Centre Hospitalier Universitaire Amiens Picardie - Nord); Nicolas Bouviez (Centre Hospitalier Universitaire Besançon - Jean Minjoz); Yoann Marion (Centre Hospitalier Universitaire Caen Normandie - Côte de Nacre); Anne Dubois, Sophie Perret-Boire, and Denis Pezet (Centre Hospitalier Universitaire Clermont Ferrand - Estaing); Christophe Mariette (Centre Hospitalier Universitaire Lille - Huriez); Laurent Brunaud and Adeline Germain (Centre Hospitalier Universitaire Nancy - Brabois); Juliette Podevin and Edwige Riegler (Centre Hospitalier Universitaire Nantes - Hôtel Dieu); Tarek Debs (Centre Hospitalier Universitaire Nice - Archet); Riccardo Gauzolino (Centre Hospitalier Universitaire Poitiers - La Miletrie); Reza Kianmanesh (Centre Hospitalier Universitaire Reims - Robert Debré); Amine Brek (Centre Hospitalier Universitaire Saint Etienne - Nord); Sylvain Kirzin (Centre Hospitalier Universitaire Toulouse - Purpan); Benoît Bourdet and Bertrand Suc (Centre Hospitalier Universitaire Toulouse - Rangueil); Dorothée Brachet and Dumitru Cojocarasu (Centre Médico Chirurgical du Mans - Pôle Santé Sud); Philippe Granger (Clinique Belledonne); Serban Bageacu and Michel Bourbon (Clinique Mutualiste Chirurgicale – Saint Etienne); Walter Bertolaso (Clinique Pont de Chaume); Pierre Caillon (Clinique Tonkin); Hector Ormando (EM Lyon Business School); Renato Lupinacci and Olivier Oberlin (Groupe Hospitalier Diaconesses - Croix Saint Simon); Axèle Champault and Monique Sigismond (Groupe Hospitalier Paris - Saint Joseph); Pascal Frileux and Alexandre Rault (Hôpital Foch - Suresnes); Raphaël Bourdariat (Hôpital Privé Jean Mermoz - Lyon); Antoine Lamblin and Christine Leclercq (Hôpital Privé La Louviere - Lille); Bernard Pol (Hôpital Saint Joseph - Marseille); Mathilde Adam, Gilles Poncet, and Catherine Valette-Lagnel (Hospices Civils Lyon - Edouard Herriot); François Chollet, Cyrille Colin, Antoine Duclos, Keitly Mensah, Philippe Michel, Léa Pascal, Cécile Payet, and Stéphanie Polazzi (Hospices Civils Lyon - Lacassagne); Sébastien Couraud, Jean-Christophe Lifante, Guillaume Passot, Jean-Louis Peix, and Vincent Piriou (Hospices Civils Lyon - Lyon Sud); Cédric Beau (Polyclinique Jean Villar - Bruges); Philippe Benand and Benjamin Brugiere (Polyclinique Poitiers); Dine Koriche (Polyclinique Vauban - Valenciennes).

    Footnotes

    • Contributors: AD and JCL conceived and designed the study. MJC, FC, AD, JCL, HO, LP, and SP acquired, analysed, and interpreted the data. FC, AD, LP, and SP drafted the manuscript. MJC, FC, AD, JCL, HO, LP, and SP critically revised the manuscript for important intellectual content. AD and LP did the statistical analysis. AD and JCL obtained funding. AD and SP provided administrative, technical, or material support. FC, AD, JCL, and HO supervised the study. AD is the guarantor. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.

    • Funding: This study was supported by a grant from the French Ministry of Health and Hospices Civils de Lyon (Programme de Recherche sur la Performance du Système des Soins (PREPS 2013-84)). The funder of the study had no role in considering the study design or in the collection, analysis, and interpretation of the data, writing of the report, or decision to submit the article for publication.

    • Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: no support from any organisation for the submitted work beyond the grant funding; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

    • Ethical approval: This trial was approved by institutional review board 11 263 (Sud-Est II ethical research committee) and the French Data Protection Authority (CNIL DR-2015-309). The requirement for written informed consent was waived because all data were blinded and made available to investigators from the Secured Data Access Platform of the French Technical Agency of Hospital Information under formal convention. The leadership of surgical departments provided facility level consent without incentives for participation and permission for trial staff to investigate anonymous data on every eligible patient from their hospital.

    • Data sharing: Anonymised participant data extracted from the nationwide hospital data warehouse are available from the ATIH Institutional Data Access Platform for researchers who meet the legal and ethical criteria for access to confidential data by the French national commission governing the application of data privacy laws. To obtain this dataset for an international researcher, email demande_base{at}atih.sante.fr. All trial materials, including the study protocol and statistical analysis plan, are freely available on the trial website (http://shewhart.univ-lyon1.fr).

    • The manuscript’s guarantor (AD) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned and registered have been explained.

    • Dissemination to participants and related patient and public communities: We plan to disseminate the control chart utilisation, methodology, and results of the research to healthcare professionals and the relevant patient community based on a dedicated website available at http://shewhart.univ-lyon1.fr.

    http://creativecommons.org/licenses/by-nc/4.0/

    This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

    References