You are here

Article Text

Article menu

Download PDFPDF

Original research
Quality gaps identified through mortality review
  1. Daniel M Kobewka1,2,
  2. Carl van Walraven1,3,
  3. Jeffrey Turnbull4,
  4. James Worthington4,
  5. Lisa Calder5,6,
  6. Alan Forster1,6
  1. 1Department of Medicine, The Ottawa Hospital, Ottawa, Ontario, Canada
  2. 2Department of Epidemiology and Community Medicine, The University of Ottawa, Ottawa, Ontario, Canada
  3. 3Institute for Clinical Evaluative Sciences, Toronto, Ontario, Canada
  4. 4The Ottawa Hospital, Ottawa, Ontario, Canada
  5. 5Department of Epidemiology and Community Medicine, University of Ottawa, Ottawa, Ontario, Canada
  6. 6Department of Clinical Epidemiology, The Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
  1. Correspondence to Dr Daniel Michael Kobewka, Department of Medicine, The Ottawa Hospital, 1053 Carling Ave, Ottawa, ON K1Y 4E9, Canada; dkobewka{at}toh.on.ca

Abstract

Background Hospital mortality rate is a common measure of healthcare quality. Morbidity and mortality meetings are common but there are few reports of hospital-wide mortality-review processes to provide understanding of quality-of-care problems associated with patient deaths.

Objective To describe the implementation and results from an institution-wide mortality-review process.

Design A nurse and a physician independently reviewed every death that occurred at our multisite teaching institution over a 3-month period. Deaths judged by either reviewer to be unanticipated or to have any opportunity for improvement were reviewed by a multidisciplinary committee. We report characteristics of patients with unanticipated death or opportunity for improved care and summarise the opportunities for improved care.

Results Over a 3-month period, we reviewed all 427 deaths in our hospital in detail; 33 deaths (7.7%) were deemed unanticipated and 100 (23.4%) were deemed to be associated with an opportunity for improvement. We identified 97 opportunities to improve care. The most common gap in care was: ‘goals of care not discussed or the discussion was inadequate’ (n=25 (25.8%)) and ‘delay or failure to achieve a timely diagnosis’ (n=8 (8.3%)). Patients who had opportunities for improvement had longer length of stay and a lower baseline predicted risk of death in hospital. Nurse and physician reviewers spent approximately 142 h reviewing cases outside of committee meetings.

Conclusions Our institution-wide mortality review found many quality gaps among decedents, in particular inadequate discussion of goals of care.

  • Chart review methodologies
  • Hospital medicine
  • Healthcare quality improvement
  • Quality measurement

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjqs-2015-004735

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Hospital mortality has been a key quality measure since Nightingale created league tables comparing mortality rates for London hospitals in the mid-19th century,1 ,2 Mortality rates are reported publicly in many jurisdictions as indicators of hospital quality but there continues to be legitimate criticisms of using mortality rates as a measure of quality.3 ,4 These criticisms relate to the effectiveness of risk adjustment to adequately control for differences between hospital and patient characteristics5; and the failure of the death rate to inform providers about the nature of quality issues contributing to increased mortality. Despite these concerns, mortality rates will remain an important quality indicator for several reasons: death is a highly visible and usually undesirable outcome, most deaths occur in hospitals6 ,7 and increased mortality rate can be caused by poor-quality care.8–13

    A natural step for hospitals tracking mortality rates is to create processes to investigate deaths and determine if care could be improved. While departmental mortality-review programmes are widespread and have been previously evaluated,14–19 there are few reports of institution-wide mortality-review programmes.20 An institution-wide process may facilitate understanding of system-wide challenges less visible at the departmental level. However, there are significant obstacles to implementing such a programme including case identification, review methodology, stakeholder engagement and opportunity costs. In order for hospital mortality rates to be a useful metric for improvement, these obstacles must be overcome.

    We recently implemented a hospital-wide mortality-review process. In this paper, we report the findings of our review and our experience implementing the review process. This information will be helpful to hospitals that are tracking mortality since quality gaps identified at our hospital likely exist elsewhere. Also, information about how we implemented our review process can guide others interested in building a mortality-review programme.

    Methods

    Setting and sample selection

    The study site was a tertiary-care academic teaching hospital with two campuses. We included every death that occurred at our hospital between 5 September 2013 and 16 December 2013. Patients admitted for day surgery, elective and emergency admissions were all included if they died during the study period. We excluded stillbirths and patients who were admitted as cadaveric donors.

    Detection of deaths and screening for quality issues

    We built a computer program that used administrative data to detect deaths as they occurred and then created a record in our hospital's Patient Safety Learning System (Datix). Each record included data extracted from our electronic health data repository including patient demographic information, encounter details and patient treatment history at our institution.

    The record was then sent to a physician and a nurse from the same admitting service as the patient. For example, if the patient was admitted under general surgery then the nurse and the physician were from general surgery. They independently reviewed each case and created a case vignette describing the patient's care leading up to death. In cases where there was uncertainty about whether care could have been improved, the nurse interviewed the physicians and nurses who were caring for the patient prior to their death to get more details. The nurse and physician reviewers each implicitly judged whether or not:

    1. There were any opportunities for quality improvement? We defined a quality-improvement opportunity as any situation where the patient's care was adversely influenced by gaps in standard care processes. This definition was intentionally broad and reviewers were encouraged to include cases if they were uncertain so that all cases with opportunities for improvement would be brought to the next step of the review. The gap did not have to be the cause of death.

    2. The death was anticipated? We instructed reviewers to label a death as unanticipated if it was not foreseeable at the time of admission or throughout the hospital stay. Evidence that a death was unanticipated could include active discharge planning, lack of documentation of end-of-life wishes or scheduling elective surgery prior to the death. Unanticipated deaths were all classified as having an opportunity for improvement. Although there are instances of unanticipated death with no opportunity for improvement, we wanted to capture cases where death was a reasonably probable outcome but the medical team did not acknowledge it. When death was judged to be a likely outcome, we considered failure to prepare for it as an opportunity for improvement.

    A registered nurse performed oversight of the reviews ensuring that reviews were properly assigned to the correct service. In addition, the oversight nurse conducted some of the reviews for services with a large numbers of deaths (eg, the intensive care unit). If one reviewer thought that the death was unanticipated or the case had an opportunity for quality improvement then the case was taken to the corporate quality review committee (figure 1).

    Figure 1
    Figure 1

    Review process workflow.

    Corporate quality review committee

    The committee's mandate was to perform a final classification of whether the death was unanticipated and/or associated with a quality-improvement opportunity and describe the opportunity for improvement. Each decision was reached by group consensus after a presentation of the case summary that had been created by the reviewers. Discussion of each case continued until consensus was reached.

    The mortality-review committee met monthly and consisted of nurses and physicians from internal medicine, critical care, psychiatry, emergency medicine and surgery. The institution's Chief Quality and Performance Officer chaired the meetings and all nurses and physicians who work at our institution were invited to attend. We defined the committee as subordinate to our institutional quality-of-care committee, which ensured that all discussions reflecting provider opinions would be protected from freedom of information requests as per local law.

    Data collection

    We recorded demographic and admission characteristics for all patients in the decedent cohort including age, gender, admission type, number of admissions in the last 6 months, inpatient length of stay and admitting service. We also recorded the risk of death in hospital using a validated risk score described by Escobar et al.21

    Analysis

    We reported the proportion of deaths that were unanticipated, the proportion with quality-improvement opportunities and the frequency of the various types of quality-improvement opportunities. Opportunities for quality improvement were grouped by theme to determine the most common ones. We tested for associations between patient and hospitalisation characteristics and the presence of an opportunity for improvement or unanticipated death. We report p values from t tests for normally distributed variables, Kruskal–Wallis test for variables with highly skewed distributions and χ2 tests for categorical variables.

    Measures of process success

    To assess the implementation of our corporate mortality-review process, we tracked time from case identification to completion of the review by the nurse and the physician, time from case identification to final classification and attendance at review committee meetings. We also documented whether there was consensus in describing the quality-improvement opportunity for each case. Lastly, we estimated time spent by study personnel to implement and maintain our mortality-review system. Our hospital research ethics board approved this study.

    Results

    Our mortality review included 427 consecutive deaths over a 3-month period. During this time, the hospital admitted 12 819 patients (overall mortality risk=3.3%). The patients who died during the study period were older, more likely to be male, had more admissions in the previous 6 months and had a longer median length of stay than those who did not die (table 1).

    View this table:
    Table 1

    Characteristics of patients admitted to hospital during the study period by dead and live status on discharge

    Results of the mortality-review process

    The nurse and physician reviewers rated 66 and 89 cases as having an opportunity for quality improvement, respectively while 23 and 24 cases were rated as unanticipated deaths, respectively. Thirty-three deaths (7.7%, 95% CI 5.2 to 10.3%) were rated as unanticipated by at least one reviewer. Figure 1 shows the number of cases rated as having an opportunity for quality improvement or being unanticipated while figure 2 shows the overlap in the reviewer's ratings. In total, 100 (23.4%, 95% CI 19.2 to 27.6%) deaths were judged as having an opportunity for improvement by at least one reviewer. The corporate mortality-review committee reached consensus for all 100 deaths and concluded that 80 cases had opportunities to improve care while 20 were expected deaths with no opportunity to improve care. We held seven committee meetings over 6 months to review these cases. Attendance at the meetings declined over time with 51 in attendance at the first meeting and 14 in attendance at the seventh meeting.

    Figure 2
    Figure 2

    Proportional Venn diagrams of deaths categorised as having an opportunity for improvement and as unanticipated by the nurse and physician reviewers.

    Opportunities for improvement and unanticipated deaths

    Of the 100 deaths reviewed at the corporate review committee, 80 (80/427=18.7%, 95% CI 15.3 to 22.7%) were judged to have one or more opportunities for improvement. Deaths with a quality problem had a lower baseline probability of death during their hospitalisation (median 0.24 IQR 0.08–0.40 vs 0.31 IQR 0.15–0.48 p=0.01), a lower probability of being classified as an urgent or emergent case (90% vs 95%, p<0.01) and a longer total length of stay (median 15 days IQR 5–31.5 days vs 6 days IQR 2–15 days p<0.001) compared with deaths with no quality problem (table 2).

    View this table:
    Table 2

    Characteristics of decedents with and without opportunities for quality improvement

    Within the 80 cases, there were 97 opportunities for improvement. The most frequent opportunity was: goals of care were not discussed or the discussion was deemed inadequate (26/97, 26.8%, 95% CI 19.0 to 36.4%) followed by delay or failure to achieve a timely diagnosis (8/97, 8.3%, 95% CI 4.2 to 15.4%) and then delay in transfer to long-term care/hospice and uncontrolled pain both with equal frequency (7/97, 7.2%, 95% CI 3.5 to 14.2%) (table 3). Table 4 contains pseudonymised case examples of opportunities for improvement along with processes that were put in place to prevent such errors in the future.

    View this table:
    Table 3

    Opportunities for improvement as classified by the corporate mortality-review committee

    Of the 427 deaths reviewed, 20 (20/427=4.7%, 95% CI 3.1 to 7.1%) were classified as unanticipated. These cases also had a lower median predicted probability of death in hospital (median 0.08 IQR 0.02–0.18 vs 0.30 IQR 0.15–0.47 p<0.001), a lower probability of being classified as an urgent or emergent case (80% vs 95%, p<0.01) and had a longer median total length of stay (19 days IQR 3.5–37 days vs 7 days IQR 7–13 days p=0.01) (table 5).

    View this table:
    Table 5

    Characteristics of decedents with unanticipated and anticipated deaths

    Timeliness of review process

    The median time between death and completion of the review by the nurse and the physician was 55 days (IQR 26–80 days). The median time from death to completion of review by the committee was 98 days (IQR 75–118 days).

    Resource use

    Coordinating the review took a registered nurse 0.2 full-time equivalents during the 4-month study period. Approximately two-thirds of the cases took 5 min for each reviewer to complete while the other one-third of cases took 20 min. The estimated mean time per case per reviewer was 10 min. In total, reviewers spent 142 h reviewing cases, outside of committee meetings.

    Discussion

    Our mortality-review process found that death was unanticipated at the beginning of the hospitalisation for 1 in 20 deaths and important quality issues were present for 1 in 5 deaths. The most common opportunity for improvement during the study period was goals of care were not being discussed or the discussion was inadequate. We also found that patients with unanticipated deaths or opportunities for quality improvement had lower baseline risk of mortality and had longer total lengths of stay and acute length of stay.

    Our finding that the most common opportunity for improvement is absent or inadequate discussion about goals of care implies that medical errors leading to death are relatively less common. This is important, as detecting ‘preventable’ deaths is one of the strong motivating factors for monitoring death rates. On the other hand, it is somewhat predictable given that only 13% of the population in Canada has completed some form of advance care planning.22 In addition, recent research has shown that physicians are not adequately discussing or documenting goals of care in the hospital.23–26 Our data highlight the need for action on this important issue. Aside from discussing goals of care, our review found numerous other common opportunities for improvement. We have used this data to motivate several quality-improvement projects (table 4).

    View this table:
    Table 4

    Cases illustrating system issues and processes implemented to mitigate recurrence of the issue

    We found that patients classified as having an unanticipated death or opportunity for quality improvement had a lower baseline risk of dying in hospital and longer length of stay. These findings increase the face validity of our work. Patients classified by clinicians as having ‘unanticipated’ deaths would be expected to have a lower baseline risk by objective measures. Also, the longer a patient is in hospital, the greater the opportunity for errors, which in turn could be amenable to quality improvement. Given that dying patients commonly lack adequately documented goals of care discussions and that these patients tend to remain in the hospital longer suggests that there may be significant opportunity to reduce patient suffering during this difficult time. This has been shown previously.23 ,27 Our results validate this prior work and suggest that continued efforts to improve this aspect of care are required.

    Several aspects of our review process helped shift the focus of discussion around mortality review from individual provider behaviour to system issues. A review method, which focuses on assigning responsibility for an unanticipated outcome to an individual, promotes a culture of blame and secrecy that is counterproductive.28 ,29 Our review process helped identify system issues by involving both nurses and physicians from all medical specialties in detecting and evaluating opportunities for quality improvement. Our open door, multidisciplinary approach to mortality review emphasised patient care as a team activity instead of a single individual's responsibility. We argue that the identification of improvement opportunities is enhanced when performed by front-line staff directly involved in service delivery. Furthermore, from a staff engagement perspective, we observed, as others have, that the review process itself contributed to desired behaviours.30 ,31 Using a team approach to quality improvement has been shown to positively affect patient-safety culture, which in turn is linked to patient-safety outcomes.32–34

    Our consensus-driven, multidisciplinary review process required time from clinicians. The estimated mean time per case reviewed was 10 min meaning that 71 h of physician time and 71 h of nurse time were spent reviewing cases outside of committee meetings. This is a significant time commitment from clinicians who have competing demands. The investment must be balanced against the information obtained. Resource usage may be reduced if predictive algorithms can be used to exclude cases with low likelihood of quality problems.

    Declining attendance at committee meetings and the long time between death and completion of the review was concerning. Consensus on a particular opportunity for improvement has less weight when only a small number of staff is present. The low attendance may have been the result of poor feedback to the committee about initiatives motivated their review or perhaps the frequent meetings became onerous. The relatively long time between death and review completion was due to inadequate recruitment of clinical reviewers early on in the review process. The future iterations of our review must maintain engagement by giving timely feedback to the committee on actions motivated by their work, and reducing the time lag from death to review completion.

    Our study is unique because we examined care gaps that may have contributed to death and care gaps that decreased the quality of death. In this way, our study is unique and important. However, there are some limitations of our work. Our review was conducted in a single teaching hospital. This limits the generalisability of our findings. There were several possible sources of bias. First, it is possible that a reviewer was involved in a case they were reviewing. Considering the size of our hospital, this was an unlikely occurrence and using two reviewers to screen each case ensured at least one objective review in the event that one was involved in the case. Hindsight bias may have occurred because reviews often occurred weeks after the death and further information such as biopsy results may point to a missed diagnosis that was not evident at the time of death. Finally, bias may have been introduced by the subjective nature of assessments. We addressed this concern by having a multistage review. While imperfect, subjective assessment by multiple reviewers is often used in research evaluating quality of care.8 ,9 ,35–37

    In conclusion, our mortality-review process engaged front-line clinical staff in reviewing consecutive deaths. The process shed light on systemic issues, giving us a broad overview of trends in quality-improvement opportunities and providing an evidence base to guide quality improvement. Our findings related to discussions of ‘goals of care’ have been documented elsewhere. Efforts should be made across the health system to improve this aspect of care. Future iterations of our mortality-review process need to focus on keeping clinical staff engaged by ensuring that discussions about cases occur soon after each death and by taking explicit and visible action to address the quality gaps that are uncovered.

    References

    1. ↵
      1. Bird W
      . Thunder Bay hostpial death rates trending up: CIHI report. CBC News 2014 December 4th 2014.
    2. ↵
      1. Press TC
      . Saskatoon hospital has the highest rate of in-facility deaths in Canada. The Gobe and Mail 2014 December 3rd..
    3. ↵
      1. Bottle A,
      2. Jarman B,
      3. Aylin P
      . Strengths and weaknesses of hospital standardised mortality ratios. BMJ 2011;342:c7116. doi:10.1136/bmj.c7116
      OpenUrl
    4. ↵
      1. Brown C
      . Value of hospital standardized mortality ratio unclear, administrators say. CMAJ 2011;183:E23–4. doi:10.1503/cmaj.109-3728
      OpenUrlFREE Full Text
    5. ↵
      1. van Gestel YR,
      2. Lemmens VE,
      3. Lingsma HF, et al
      . The hospital standardized mortality ratio fallacy: a narrative review. Med Care 2012;50:662–7. doi:10.1097/MLR.0b013e31824ebd9f
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      Deaths in hospital and elsewhere, Canada, provinces and territories. In: Canada S, ed., 2011.
    7. ↵
      1. Hall MJ,
      2. Levant S,
      3. DeFrances CJ
      . Trends in inpatient hospital deaths: national hospital discharge survey, 2000–2010. In: Services UDoHaH, ed. National Center for Healthcare Statistics, 2013.
    8. ↵
      1. Baker GR,
      2. Norton PG,
      3. Flintoft V, et al
      . The Canadian Adverse Events Study: the incidence of adverse events among hospital patients in Canada. CMAJ 2004;170:1678–86. doi:10.1503/cmaj.1040498
      OpenUrlAbstract/FREE Full Text
    9. ↵
      1. Brennan TA,
      2. Leape LL,
      3. Laird NM, et al
      . Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med 1991;324:370–6. doi:10.1056/NEJM199102073240604
      OpenUrlCrossRefPubMedWeb of Science
    10. ↵
      1. Dubois RW,
      2. Brook RH
      . Preventable deaths: who, how often, and why? Ann Intern Med 1988;109:582–9. doi:10.7326/0003-4819-109-7-582
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      1. Hayward RA,
      2. Hofer TP
      . Estimating hospital deaths due to medical errors: preventability is in the eye of the reviewer. JAMA 2001;286:415–20. doi:10.1001/jama.286.4.415
      OpenUrlCrossRefPubMedWeb of Science
    12. ↵
      1. Hogan H,
      2. Healey F,
      3. Neale G, et al
      . Preventable deaths due to problems in care in English acute hospitals: a retrospective case record review study. BMJ Qual Saf 2012;21:737–45. doi:10.1136/bmjqs-2011-001159
      OpenUrlAbstract/FREE Full Text
    13. ↵
      1. Zegers M,
      2. de Bruijne MC,
      3. Wagner C, et al
      . Adverse events and potentially preventable deaths in Dutch hospitals: results of a retrospective patient record review study. Qual Saf Health Care 2009;18:297–302. doi:10.1136/qshc.2007.025924
      OpenUrlAbstract/FREE Full Text
    14. ↵
      1. Aboumatar HJ,
      2. Blackledge CG Jr.,
      3. Dickson C, et al
      . A descriptive study of morbidity and mortality conferences and their conformity to medical incident analysis models: results of the morbidity and mortality conference improvement study, phase 1. Am J Med Qual 2007;22:232–8. doi:10.1177/1062860607303292
      OpenUrlAbstract/FREE Full Text
    15. ↵
      1. Bechtold ML,
      2. Scott S,
      3. Nelson K, et al
      . Educational quality improvement report: outcomes from a revised morbidity and mortality format that emphasised patient safety. Qual Saf Health Care 2007;16:422–7. doi:10.1136/qshc.2006.021139
      OpenUrlAbstract/FREE Full Text
    16. ↵
      1. Donaldson LJ,
      2. Panesar SS,
      3. Darzi A
      . Patient-safety-related hospital deaths in England: thematic analysis of incidents reported to a national database, 2010–2012. PLoS Med 2014;11:e1001667. doi:10.1371/journal.pmed.1001667
      OpenUrlCrossRefPubMed
    17. ↵
      1. Friedman JN,
      2. Pinard MS,
      3. Laxer RM
      . The morbidity and mortality conference in university-affiliated pediatric departments in Canada. J Pediatr 2005;146:1–2. doi:10.1016/j.jpeds.2004.10.053
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      1. Gore DC
      . National survey of surgical morbidity and mortality conferences. Am J Surg 2006;191:708–14.
      OpenUrlCrossRefPubMed
    19. ↵
      1. Calder LA,
      2. Kwok ES,
      3. Adam Cwinn A, et al
      . Enhancing the quality of morbidity and mortality rounds: the Ottawa M&M model. Acad Emerg Med 2014;21:314–21. doi:10.1111/acem.12330
      OpenUrlCrossRefPubMed
    20. ↵
      1. Provenzano A,
      2. Rohan S,
      3. Trevejo E, et al
      . Evaluating inpatient mortality: a new electronic review process that gathers information from front-line providers. BMJ Qual Saf 2015;24:31–7. doi:10.1136/bmjqs-2014-003120
      OpenUrlAbstract/FREE Full Text
    21. ↵
      1. Escobar GJ,
      2. Greene JD,
      3. Scheirer P, et al
      . Risk-adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care 2008;46:232–9. doi:10.1097/MLR.0b013e3181589bb6
      OpenUrlCrossRefPubMedWeb of Science
    22. ↵
      Harrisdecima. What Canadians say: the way forward survey report. Ottawa, ON: Canadian Hospice Palliative Care Association, 2013.
    23. ↵
      1. Heyland DK,
      2. Barwich D,
      3. Pichora D, et al
      . Failure to engage hospitalized elderly patients and their families in advance care planning. JAMA Intern Med 2013;173:778–87. doi:10.1001/jamainternmed.2013.180
      OpenUrlCrossRefPubMed
    24. ↵
      1. Meeussen K,
      2. Van den Block L,
      3. Echteld M, et al
      . Advance care planning in Belgium and The Netherlands: a nationwide retrospective study via sentinel networks of general practitioners. J Pain Symptom Manage 2011;42:565–77. doi:10.1016/j.jpainsymman.2011.01.011
      OpenUrlCrossRefPubMedWeb of Science
    25. ↵
      1. Silveira MJ,
      2. Kim SY,
      3. Langa KM
      . Advance directives and outcomes of surrogate decision making before death. N Engl J Med 2010;362:1211–18. doi:10.1056/NEJMsa0907901
      OpenUrlCrossRefPubMedWeb of Science
    26. ↵
      1. Street M,
      2. Ottmann G,
      3. Johnstone MJ, et al
      . Advance care planning for older people in Australia presenting to the emergency department from the community or residential aged care facilities. Health Soc Care Community 2015;23:513–22. doi:10.1111/hsc.12162
      OpenUrlCrossRefPubMed
    27. ↵
      1. Detering KM,
      2. Hancock AD,
      3. Reade MC, et al
      . The impact of advance care planning on end of life care in elderly patients: randomised controlled trial. BMJ 2010;340:c1345. doi:10.1136/bmj.c1345
      OpenUrlAbstract/FREE Full Text
    28. ↵
      1. Berwick DM
      . Not again! BMJ 2001;322:247–8. doi:10.1136/bmj.322.7281.247
      OpenUrlFREE Full Text
    29. ↵
      1. McDonnell C,
      2. Laxer RM,
      3. Roy WL
      . Redesigning a morbidity and mortality program in a university-affiliated pediatric anesthesia department. Jt Comm J Qual Patient Saf 2010;36:117–25.
      OpenUrlPubMed
    30. ↵
      1. Berkowitz RE,
      2. Schreiber R,
      3. Paasche-Orlow MK
      . Team improvement and patient safety conferences: culture change and slowing the revolving door between skilled nursing facility and the hospital. J Nurs Care Qual 2012;27:258–65.
      OpenUrlCrossRefPubMed
    31. ↵
      1. Thomas L,
      2. Galla C
      . Building a culture of safety through team training and engagement. BMJ Qual Saf 2013;22:425–34. doi:10.1136/bmjqs-2012-001011
      OpenUrlAbstract/FREE Full Text
    32. ↵
      1. Morello RT,
      2. Lowthian JA,
      3. Barker AL, et al
      . Strategies for improving patient safety culture in hospitals: a systematic review. BMJ Qual Saf 2013;22:11–8. doi:10.1136/bmjqs-2011-000582
      OpenUrlAbstract/FREE Full Text
    33. ↵
      1. O'Leary KJ,
      2. Wayne DB,
      3. Haviley C, et al
      . Improving teamwork: impact of structured interdisciplinary rounds on a medical teaching unit. J Gen Intern Med 2010;25:826–32. doi:10.1007/s11606-010-1345-6
      OpenUrlCrossRefPubMedWeb of Science
    34. ↵
      1. Weaver SJ,
      2. Rosen MA,
      3. DiazGranados D, et al
      . Does teamwork improve performance in the operating room? A multilevel evaluation. Jt Comm J Qual Patient Saf 2010;36:133–42.
      OpenUrlPubMed
    35. ↵
      1. Thomas EJ,
      2. Studdert DM,
      3. Burstin HR, et al
      . Incidence and types of adverse events and negligent care in Utah and Colorado. Med Care 2000;38:261–71. doi:10.1097/00005650-200003000-00003
      OpenUrlCrossRefPubMedWeb of Science
    36. ↵
      1. Vincent C,
      2. Neale G,
      3. Woloshynowych M
      . Adverse events in British hospitals: preliminary retrospective record review. BMJ 2001;322:517–19. doi:10.1136/bmj.322.7285.517
      OpenUrlAbstract/FREE Full Text
    37. ↵
      1. Wilson RM,
      2. Runciman WB,
      3. Gibberd RW, et al
      . The Quality in Australian Health Care Study. Med J Aust 1995;163:458–71.
      OpenUrlPubMedWeb of Science

    Footnotes

    • Contributors DMK contributed to data acquisition, analysis and interpretation. He also drafted the article. CvW participated in data analysis, interpretation and critical appraisal of the manuscript. JT participated in project design, interpretation of data and critical appraisal of the manuscript. JW participated in interpretation of data and critical appraisal of the manuscript. LC participated in project design, data acquisition and interpretation of data. She also critical appraised the manuscript for intellectual content. AF designed the project, performed data acquisition, analysis and interpretation. He also critically appraised and revised the manuscript.

    • Funding The Ottawa Hospital Academic Medical Organization.

    • Competing interests None declared.

    • Ethics approval The Ottawa Hospital Research Ethics Board.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement Forms used for data collection and chart review are available from DMK (dkobewka@toh.on.ca) on request.