Article Text
PDF
Abstract
Importance Death due to preventable medical error is a leading cause of death, with varying estimates of preventable death rates (14%–56% of total deaths based on national extrapolated estimates, 3%–11% based on single-centre estimates). Yet, how best to reduce preventable mortality in hospitals remains unknown.
Objective In this article, we detail lessons learnt from implementing a hospital-wide, automated, real-time, electronic mortality reporting system that relies on the opinions of front-line clinicians to identify opportunities for improvement. We also summarise data obtained regarding possible preventability, systems issues identified and addressed, and challenges with implementation. We outline our process of survey, evaluation, escalation and tracking of opportunities identified through the review process.
Methods We aggregated and analysed 7 years of review data regarding deaths, review responses categorised by ratings of possible preventability and inter-rater reliability of possible preventability. A qualitative analysis of reviews was performed to identify care delivery opportunities and institutional response.
Results Over the course of 7 years, 7856 inpatient deaths occurred, and 91% had at least one review completed. 5.2% were rated by front-line clinicians as potentially being preventable (likely or possibly), and this rate was consistent over time. However, there was only slight inter-rater agreement regarding potential preventability (Cohen’s kappa=0.185). Nevertheless, several major systems-level opportunities were identified that facilitated care delivery improvements, such as communication challenges, need for improved end-of-life care and interhospital transfer safety.
Conclusions Through implementation, we found that a hospital-wide mortality review process that elicits feedback from front-line providers is feasible, and provides valuable insights regarding potential preventable mortality and prioritising actionable opportunities for care delivery improvements.
- healthcare quality improvement
- hospital medicine
- incident reporting
- quality improvement
- safety culture
Statistics from Altmetric.com
Introduction
Since the Institute of Medicine (IOM) published its seminal paper ‘To Err is Human’ estimating that up to 98 000 inpatients die due to medical errors, there has been increased focus on reducing preventable deaths.1 Recent US national extrapolated estimates describe between 210 000 and 400 000 preventable inpatient deaths annually (14%–56% preventable death rate of total deaths).2 However, single-centre retrospective chart-based reviews have published preventable death rates in the 3%–11% range.3 Safety science experts suggest a three-tiered approach to address preventable deaths: (1) increase the visibility of errors, (2) rapidly respond to errors, and (3) prevent errors from occurring; tiers 2 and 3 are contingent on better identification of errors.2 Given the variability of national estimates, limited generalisability of single-centre retrospective reviews and the importance of identifying opportunities to prevent loss of life, evidence-based strategies to identify preventable death remain paramount.
A range of strategies have been employed to improve the recognition of medical errors that result in death.4–16 Most published US mortality review studies do not incorporate front-line providers—a crucial source of information that is often underappreciated—to identify potential errors. A study of US surgeons illustrated that 8.9% of surgeons surveyed reported that they had made a major medical error within the last 3 months, and 1.5% believed that their error resulted in death.12 Notably there are efforts in the UK to implement a standardised process to facilitate review and learning of deaths by providers.13 14 Understanding and addressing barriers to provider reporting is essential; the IOM identified several potential barriers, which include confidentiality concerns, lack of belief that information will be used, lack of definitions for reporting and lack of education and training with respect to recognising errors.1 Design and implementation of mortality review systems aimed at overcoming barriers to front-line provider reporting is necessary to adequately capture data about preventable deaths.
In June 2011, Brigham and Women’s Hospital (BWH) launched a hospital-wide electronic mortality reporting system (Brigham Mortality Review System [BMRS]) to capture real-time information about patient deaths directly from the front-line providers. Details of the development and first year evaluation of the system have been published.13 To our knowledge there has not been a study published regarding the implementation of a real-time, comprehensive mortality review system over several years. In this paper, we detail lessons learnt from our BMRS experience which include: (1) a summary of data obtained related to potential preventability and associated complications; (2) systems issues identified and addressed; and (3) challenges with implementation. We attempt to answer the inevitable question: is this type of mortality review system worth the investment in time and resources to implement?
Methods
Setting
BWH is a 793-bed tertiary care academic hospital in Boston, Massachusetts. On average, 1100 patient deaths occur annually, with an observed mortality of 1.9%. In 2015, BWH transitioned to a common electronic health record (EHR; EPIC, Partners version) system. The BMRS interfaces with EPIC, which provides death and provider information, through an intermediary database. After a patient’s inpatient death is entered into the EHR, the BMRS sends emails requesting reviews to physicians and nurses providing care at the time of a patient’s death. If a review is not completed, reminder emails are sent to facilitate completion.
BMRS survey
Our current inpatient physician survey form is illustrated in online supplementary figure 1. We have previously discussed our development process in detail.15 There have been updates to questions on the form based on feedback from providers. For example, additional questions related to discussions about patient code status have been added.
Supplemental material
The electronic survey captures provider opinions about complications, timeliness of interventions, communication issues, end-of-life information, presence of a medical error or systems issue and suggestions for quality improvement. Potential preventability is rated using a 5-point scale: 1–2 non-preventable, 3 non-preventable with presence of a medical error, 4 possibly preventable and 5 likely preventable. Reviewers can also request peer support or contact from the Department of Quality and Safety through the form. A shorter survey for emergency department (ED) physicians focuses on the interventions, treatments, consultations and handoffs occurring in the ED. Importantly, these forms are designed to capture our physicians, physician assistants and nurses’ opinions or concerns. For example, when they rate a death as potentially preventable, the rating serves as a trigger for investigation (as opposed to establishing the death as preventable as a matter of fact).
Mortality review process
Box 1 illustrates the four-step process for a mortality review: Survey, Evaluate, Escalate and Keep track.
Schematic of the Brigham Mortality Review System
Survey
Immediately after patient death, mortality review request generated for physicians, physician assistants and nurses.
Recipients complete review, can identify other providers and/or decline review.
Evaluate
Initial review by Quality and Safety Department medical director.
If indicated:
Additional information from respondents.*
Secondary review by local department quality leaders.
If indicated:
Collaborative case review†.
Escalate
If indicated:
Presentation to multidisciplinary quality committee.
Systems changes implemented.
Presentation to executive safety committee and board.
Keep track
Central database used to track all cases.
Feedback given to all mortality review respondents.
*Outreach by email and/or phone.
†Group meeting with care team members involved in patient’s care and local leadership.
Survey
After a patient’s inpatient death is entered into the EHR, an electronic mortality review survey link is automatically emailed to the attending physician and responding clinician (trainee physician or physician assistant) caring for the patient at the time of death. Nurses also receive review requests (nursing was recently added to the mortality review process; see online supplementary figure 2 for a screenshot of the email addressed to nurses). ED physicians receive a review request if they cared for the patient at the time of death or in the 48 hours before the patient’s death. The review requests are titled ‘mandatory’ for physicians and physician assistants, and they can select the survey link and complete a review, decline to review if they feel it was assigned in error and confidentially suggest another provider also submit a mortality review. If a review is not completed and not declined, a reminder email is sent every 4 days for 1 month. The median time to complete the survey in 2018 was 2 min and 57 s.
Evaluate
Completed mortality reviews in which any concerns are raised are evaluated the same day, or the next weekday, by a medical director in the Department of Quality and Safety (MLM). If the death is rated as potentially preventable, or a potential medical error or systems issue is raised, the medical record is reviewed and, if required, additional questions are posed to the reviewer. A determination is then made regarding whether additional review is needed by local clinical leaders (physicians, nurses, staff for a specific department or unit). After local clinical review, cases may be closed and feedback is provided to the reviewer, escalated as outlined below or presented as a collaborative case review (CCR). CCRs involve multiple providers involved in the patient’s care to establish the sequence of events and opportunities for improvement. After a CCR, cases may be closed and feedback is provided to the respondent. There is variability in the amount of time needed to conduct this further investigation into submitted reviews. The mortality review process is integrated with the department’s review of patient safety events as shown in online supplementary figure 3. Criteria are applied to all reviews to stratify as low, medium or high risk, and risk/safety managers track cases through the evaluation process. Similarly, departmental morbidity and mortality reviews are triggered by mortality and safety reviews, and findings are shared with the central Department of Quality and Safety.
Escalate
Some CCRs and cases after local clinical review are presented to a peer-review quality assurance and risk management committee. This group determines action items to address the systems issues and medical errors identified in the review. The committee is interdisciplinary, comprised quality and safety directors of all clinical divisions, nursing leadership and pharmacy leadership. As needed, cases can be escalated to the hospital board quality committee or other hospital leaders (eg, chief medical officer or chief quality officer).
Keep track
Reviews, results of investigations, presentations and action items are tracked in the Department of Quality and Safety. All respondents receive feedback about the details of the investigation and, as applicable, clarification about concerns and/or actions implemented to address deficiencies. Mortality data and a summary of reviews are reported monthly to the respective departments and units in the hospital. In addition to the central database to track mortality cases, data are abstracted from the BMRS and loaded to the Balanced Scorecard platform to present data on frequency of deaths by department and unit, number of complications, timeliness of interventions, communication issues, end-of-life information, preventability and presence of a medical error or systems issue. Monthly reports are generated for each department that provide department-specific mortality statistics and a summary of reviews completed.
Evaluation of our process
We evaluated response data from systems launch, 1 June 2011 to 1 June 2018. A death was classified as potentially preventable if at least one respondent scored the death as potentially preventable. We calculated the total number of deaths, mortality review response rates categorised by preventability and inter-rater reliability for cases with multiple reviewers (data from 2013 and 2014 were excluded because of inconsistent review identification due to programming glitches during this time period). Data on contributing factors were aggregated from the reviews. Qualitative analysis of provider reviews was performed to identify care delivery opportunities highlighted through mortality reviews, and subsequent institutional response to address gaps in care.
Statistical analyses
Clinical and patient characteristics were described using percentages, mean and SD, and median and IQR for length of stay. Wilcoxon rank-sum tests were calculated for non-parametric comparisons. When dichotomising cases to potential preventability, deaths without a preventability score (n=12) were excluded from the analysis. Inter-rater reliability of agreement for multiple potentially preventable death reviews was calculated using Cohen’s kappa coefficient statistic. All p values were two sided, and alpha values <0.05 were considered statistically significant. Data were analysed using SAS V.9.3 (SAS Institute).
Results
Deaths and completed surveys
From 1 July 2011 to 1 July 2018, there were 7856 inpatient deaths and a total of 14 761 submitted mortality reviews. Of the 7856 deaths, 677 (9%) did not have a mortality review and 410 (5.2%) were rated as potentially preventable (likely or possibly). In figure 1, trends in the number of deaths and completed surveys are shown for the 7-year period. Mortality review completion rates were ≥90% for the first six quarters and for the last five quarters of our evaluation. This rate decreased to 72% in quarter 3 of 2015 when BWH transitioned to a new EHR system. Over time, the number of deaths and the proportion of deaths rated potentially preventable largely remained stable. Similarly, our institution’s observed to expected (O/E) mortality rate, based on Vizient data, has varied between 0.9 and 1.0 during this time period. We did see in 2018 an increase in O/E to 1.1, which we attribute to a significant increase in our oncology service volume and accordingly oncology-related observed deaths. Our expected mortality has largely been stable despite this increase in oncology patient volume, likely reflecting opportunities to improve our severity of illness capture.
Aggregate deaths and mortality reviews completed by quarter. 2015 quarter 2 and quarter 3 were periods during which Brigham Mortality Review System function was intermittent due to integration with new electronic health record. Completed review—patient died, review was sent and completed. Multiple reviews could be sent for one death. Potentially p reventable review—review in which reviewer rated the death as potentially preventable with a score of 4 or 5 (scale of 1–5, with 1 and 2 being non-preventable, 3 representing a non-preventable death with the presence of a medical error, and 4 and 5 being a preventable death (possibly and likely, respectively)). Potentially preventable death—at least one reviewer rated the death as potentially preventable with a score of 4 or 5 (scale of 1–5, with 1 and 2 being non-preventable, 3 representing a non-preventable death with the presence of a medical error, and 4 and 5 being a preventable death (possibly and likely, respectively)).
Patient demographics and clinical characteristics
Demographic characteristics, number of admissions in prior 6 months and length of stay were similar for the potentially preventable and potentially non-preventable death groups (table 1). The cohort had a mean age (SD) of 65 (18) years, 74.1% (n=5323) were white and 44.9% had at least one admission in the 6 months prior to their death. Most patients had Medicare (54.4%) and were cared for on a medical service (68%). Oncology was the most common discharge service for non-preventable deaths (24.6%), but accounted for only 8.7% of preventable deaths. Pulmonary (which staffs the Medical Intensive Care Unit) was the most common discharge service for preventable deaths (23.2%).
Patient and clinical characteristics by potentially non-preventable and potentially preventable status
Inter-rater reliability
A total of 189 attending and responding clinician reviews were submitted for related potentially preventable death cases and 11% (n=20) agreed on preventability (figure 2). Online supplementary table 1 shows that inter-rater reliability varied over the 7-year time period, with an average Cohen's kappa statistic of 0.181, indicative of only slight inter-rater agreement.
Inter-rater agreement among mortality reviewers.
Factors contributing to preventable deaths
Online supplementary table 2 reports the potential complications, interventions and communication deficits identified by survey respondents in mortality reviews. Complications were the most prevalent of the factors (8.5% of deaths), and venous thromboembolism was the most prevalent of complications (1.7% of deaths).
Systems issues identified and addressed
Several major themes that we addressed emerged from the submitted mortality review cases (table 2). One theme was that poor communication between primary teams, consultants and ancillary care team members potentially played a role in several deaths, prompting changes in training and notification systems. Another theme was end-of-life care, particularly barriers in determining and honouring a patient’s goals of care, which led to a hospital-wide education initiative regarding standardised content for end-of-life discussions. A third theme was medication errors, which led to changes in EHR alerts for high-risk situations, such as prescribing of QTc-prolonging medications and optimal dosing for obese patients. Some challenges have required large-scale approaches. Transfers from outside hospitals have been associated with multiple safety issues, including notification of the accepting teams and the transfer of important documentation and imaging. These challenges are being addressed by a multidisciplinary committee working on improvements such as templated expect notes and resident notification algorithms. Similarly, aspiration of oral contrast in high-risk patients during transport or radiology exam was identified as a potential opportunity through the BMRS. Systemic changes have been implemented such as an ‘aspiration flag’, for patients at high risk of aspiration as determined by caregivers, in the EHR visible to transport and radiology staff, and training of staff to tailor workflow accordingly (eg, transport raising the head of the bed 30°).
Care delivery opportunities illustrated by Brigham Mortality Review System and responses
Discussion
Through evaluating and describing our institution’s longitudinal experience with implementing a provider-based electronic mortality review system, we found that high clinician engagement provided the impetus to address a number of systems issues. Our completion rates have been high, and most recently, greater than 98% of deaths have been reviewed and greater than 90% of reviews have been completed. Analysis of mortality reviews revealed key factors associated with potentially preventable deaths. This analysis has directly informed potentially lifesaving interventions in the domains of interhospital transfer, EHR decision support and care team communications.
Most previously published mortality review implementation studies describe retrospective reviews conducted by nurses and physicians of a random sample of hospital deaths.4–6 Other institutions detail a comprehensive retrospective review using trained patient safety specialist reviewers.7 Department-specific morbidity and mortality conferences are often used to review select cases.8 9 A number of institutions reported on the Institute of Healthcare Improvement’s Global Trigger Tool, an algorithm with select questions applied to small random samples of patient records, to facilitate identification and further review of adverse events retrospectively.10 11
Three reports describe systems that elicit timely front-line feedback once a death has occurred. The Ottawa Hospital in Canada detected deaths in real time and had a nurse and physician from the admitting service, not necessarily involved in the patients’ care, review cases.4 Inadequate goals of care discussions were an identified opportunity for improvement, similar to our evaluation. The Mayo Clinic has adopted an approach with local physician and nurse leaders reviewing all deaths occurring from the area of the hospital providing care to the patient at the time of death, and conclude that adoption of their mortality review system has led to an increased focus on the culture of safety.17 The Hospital of the University of Pennsylvania implemented a real-time review system, similar to ours, in its querying of providers within 48 hours of death as an intervention trial over 1 year. The authors found that information from the front-line providers captured important data that otherwise would have been missed with a more traditional review, and pointed to poor inter-rater agreement with respect to potential preventability, highlighting the need for feedback across disparate care provider groups.18 The dearth of literature on this topic, despite the enormity of the problem, may reflect the challenge of resource investment to develop and launch a hospital-wide mortality review system.
The BMRS is distinct from previously described mortality review systems given its comprehensive, real-time, multireviewer facing, longitudinal functionality. A distinguishing feature is that we are obtaining feedback from front-line providers caring for the patient at the time of death, as opposed to retrospective reviews by those not involved in the patient’s care. These are subjective assessments of potential preventability and opportunities for improvement based on the provider’s perspective. However, these reviews frequently add information that is rarely present in the patient’s chart. For example, the patient’s experience, communication challenges and systems-level issues that require front-line clinician feedback are captured.
With subjective assessments, a range of viewpoints is crucial. The low agreement of potential preventability between attendings and responding clinicians demonstrates (1) varied perspectives based on years of experience and scope of practice, and (2) the value of obtaining input from multiple types of reviewers to increase the likelihood of identifying care delivery opportunities. This finding has implications for our institution and others—namely, it is vital to aggregate mortality data from multiple care team members involved. We are in the seminal stages of nursing implementation, and in the future will analyse agreement between attendings, responding clinicians and nurses. As a result of this finding we are assessing the feasibility of obtaining feedback from other care team members, such as respiratory technicians and physical therapists.
The data collected regarding specific care delivery factors, such as adverse drug events, delayed diagnoses and poor communication between various providers, reveal the ability of the BMRS to identify tangible and actionable system-level issues. We have outlined small-scale and large-scale improvements in care delivery that have arisen from our mortality review process. Implementation of these initiatives has the potential to reduce preventable mortality, and may have profound effects on safety culture and front-line provider empowerment. Every reviewer that raises concern about a death receives closed loop feedback on changes made in the processes of care because of his or her submitted review, which is a powerful means of empowering staff and promoting transparency—potentially fostering the organisation’s culture of safety.
There are a number of strengths of our mortality review system implementation worth considering. This is the largest study of mortality reviews captured in real time from the front-line providers caring for the patient at or near the time of death. The BMRS addressed a number of recognised barriers to provider error identification such as quick, clearly defined review requests to improve recall, and there is an emphasis on peer support, just culture and feedback about completed investigations. The high completion rates reflect adoption among clinicians across the hospital and automated reminders better incorporate the system into the workflow. BWH has restructured its risk and safety teams, administrative committees and quality improvement representatives to ensure adequate response, investigation and feedback of all mortality review concerns.
As with any provider facing review system, there are limitations due to biases, under-reporting due to fear of repercussions and lack of perspective on the patient’s entire hospital course. However, we believe these have been mitigated over time as the BMRS has been widely adopted, and it has been engrained that all deaths are reviewed, information shared is confidential and multiple providers submit reviews. We encountered challenges in implementing the BMRS. First, when a new EHR was implemented there were difficulties correctly mapping necessary variables. Due to these technical issues, we had interruptions in our mortality review process and data collected. Second, communicating the importance of completing the reviews required a significant amount of effort initially, and in subsequent years with turnover of staff. We have leveraged our local quality improvement leaders to emphasise the importance of the mortality reviews. We have also had to refine our process of investigating, presenting and resolving systems issues identified through mortality review. We arrived at the process illustrated in box 1 after several years of trialling and learning.
Finally, it is important to note that observed mortality and the percentage of preventable deaths has not changed in the past 7 years. Similarly, our O/E mortality rates have not significantly changed over time. We believe this is reflective of several factors: a patient population that is likely getting more complex over time, the slow pace of affecting change with this type of initiative and the need for a more robust implementation and follow-up process. In the past 2 years we have focused on streamlining follow-up, and continue to address the systems issues raised in reviews with an aim of reducing observed mortality. It is important to note that with over 1200 deaths a year, mortality review efforts that prevent, for example, 20 deaths a year or 2% of observed deaths, will not have a statistically significant impact on O/E ratios.
As we examine how to improve our current process, two key elements stand out: data tracking and resolving issues raised. As the aggregate volume of mortality review data and need to compare with other data sources (eg, safety reporting, patient complaints) grows, the challenge of tracking becomes exponential. Thus, we are exploring the creation of a risk registry that could integrate data from mortality reviews and other sources, enable longitudinal tracking of investigation and follow-up, and facilitate prospective identification of vulnerabilities. We recognise that we need to take a similar approach as other high-risk industries, like aviation, that have long-standing systems in place to capture and predict vulnerabilities.
Another major focus area is a consistent accountability plan to evaluate, escalate and resolve systems issues raised during reviews. The process of investigating and implementing process changes is demanding, and it is challenging to weigh against competing priorities. However, creating a safer, more patient-centred environment and mitigating the potential for inpatient death should be high priority. The National Institutes of Health (NIH) funds nearly $5 billion in cancer research and nearly $2 billion in cardiovascular disease research, the top two causes of death.19 Yet there are no direct NIH funds allocated to research and implement solutions for another leading cause of death—medical errors. We would advocate for more resources at a federal, state and local level to implement programmes like BMRS to address preventable inpatient deaths.
Our institution’s automated, real-time, mortality review system and defined process for evaluating concerns raised by providers has helped identify many clear opportunities for improving care. Additional study is needed to test the hypothesis that leveraging prospective, real-time, front-line, clinician-based feedback to address systems issues contributing to preventable deaths will significantly impact rates of observed deaths. We believe that this type of mortality review has been an important facet of our overall approach to providing safer care to patients and engaging staff in safety efforts.
References
Footnotes
Contributors MLM and AK take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: MLM, AK. Study implementation: MLM, JB, KF, AK. Acquisition of data: MLM, MGT. Analysis and interpretation of data: MLM, MGT, JB, YL, AP, KF, AK. Drafting of the manuscript: MLM, MGT, YL, AP, KF, JB, AK. Critical revision of the manuscript for important intellectual content: MLM, AK. Statistical analysis: MLM, MGT. Administrative, technical or material support: AK. Study supervision: AK.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available upon reasonable request.