Importance There is a paucity of literature on the quality and effectiveness of institutional morbidity & mortality (M&M) rounds processes.
Objective We sought to implement and evaluate the effectiveness of a hospital-wide structured M&M rounds model at improving the quality of M&M rounds across multiple specialties.
Design, setting, participants We conducted a prospective interventional study involving 24 clinical groups (1584 physicians) at a tertiary care teaching hospital from January 2013 to June 2015.
Intervention We implemented the published Ottowa M&M Model (OM3): appropriate case selection, cognitive/system issues analyses, interprofessional participation, dissemination of lessons and effector mechanisms.
Main outcomes and measures We created an OM3 scoring index reflecting these elements to measure the quality of M&M rounds. Secondary outcomes include explicit discussions of cognitive/system issues and resultant action items.
Results OM3 scores for all participating groups improved significantly from a median of 12.0/24 (95% CI 10 to 14) to 20.0/24 (95% CI 18 to 21). An increased frequency of in-rounds discussion around cognitive biases (pre 154/417 (37%), post 256/466 (55%); p<0.05) and system issues (pre 175/417 (42%), post 259/466 (62%); p<0.05) were reported by participants via online surveys postintervention, while in-person surveys throughout the intervention period demonstrated even higher frequencies (cognitive biases 1222/1437 (85%); system issues 1250/1437 (87%)). We found 45 action items resulting directly from M&M rounds postintervention, compared with none preintervention.
Conclusions and relevance Implementation of a structured model enhanced the quality of M&M rounds with demonstrable policy improvements hospital wide. The OM3 can be feasibly implemented at other hospitals to effectively improve quality of M&M rounds across different specialties.
- Morbidity and mortality rounds
- Hospital medicine
- Healthcare quality improvement
Statistics from Altmetric.com
Patient safety has become a top priority for healthcare institutions across the world ever since landmark reports at the turn of this century such as the Institute of Medicine's To Err Is Human, Building a Safer Health Care System and the WHO's Background Paper Quality of Care: Patient Safety.1 ,2 Yet, there has since been growing literature to suggest that preventable medical errors continue to contribute a significant threat to patient safety at many hospitals.3–5
Morbidity and mortality (M&M) rounds have long been recognised as a potential forum at which healthcare professionals can identify and review potentially preventable medical errors. However, there exist very few published guidelines or recommendations on how such M&M rounds should be structured, and even less evidence on their effectiveness. This represents a unique opportunity to link advances in patient safety science (such as new understanding of cognitive and system biases) to the already well-established M&M educational processes across different specialties.
In our previous work, we described the development of a novel structured model for M&M rounds, the Ottowa M&M Model (OM3).6 To our knowledge, there has not been any published evidence on the effectiveness of implementing a standardised structured approach to M&M rounds across multiple specialties in a hospital setting. We hypothesise that the implementation of such a structured approach can lead to more meaningful discussions of critical factors around M&M cases as well as mechanisms to act on those discussions.
We implemented a structured hospital-wide M&M rounds model at a tertiary care academic teaching hospital and evaluated the effect on improving M&M rounds quality. Specifically, we hypothesised that the implementation of the OM3 would lead to (1) improvement in overall quality of M&M rounds across the hospital, as defined by the OM3 scoring index (includes five key elements of effective M&M rounds based on previous literature); (2) more appropriate M&M rounds case selection, analysis, cognitive and system issue identification; (3) enhanced M&M rounds summary generation and distribution; and (4) increased number of policy changes and action items to address issues arising directly out of M&M rounds. We also sought to assess existing structural, cultural and process barriers to implementation of such a hospital-wide quality improvement initiative.
This was a prospective before-and-after interventional study with approval by the Research Ethics Board. We implemented the OM3 intervention between January 2013 and June 2014, and collected postintervention data up to June 2015.
The Ottowa Hospital is a 1117-bed acute tertiary care centre and academic teaching hospital encompassing three physical campuses, and serving as the Canadian National Capital region's only Level 1 Trauma Centre with subspecialties in cardiac, vascular and neurosurgery. The Ottowa Hospital has a catchment area of over 1.2 million residents and sees more than 165 000 Emergency Department visits a year and over 34 000 surgical cases a year.
We approached 30 clinical groups (out of a total of 42) within the Ottowa Hospital that hold (or were planning to hold) M&M rounds to participate in the study. Selection of these M&M groups were made based on the following principles: (1) having a representative and varied sample of ‘core’ specialties that would be similar and applicable to other tertiary care hospitals; (2) including ‘high-impact’ clinical groups in terms of patient volumes and/or number of active physicians; (3) achieving a balance between acute (inpatient) versus less-acute (ambulatory care clinics), as well as surgical versus non-surgical services; and (4) engaging those groups with willingness and readiness to modify established M&M processes.
Of the 30 groups identified and approached, 24 participated as part of the intervention group (8 surgical and 16 medical) and 6 participated as the reference groups (those that have chosen not to adopt OM3 at the time of the study but were willing to participate in current-state analysis and interviews). The department of emergency medicine and the division of trauma services were considered ineligible, as they were the groups on which the OM3 was originally modelled and piloted successfully.6
We have described the full OM3 approach to M&M rounds in a previous publication.6 In brief (box 1), key features of the OM3 include guidelines on appropriate case selection (a case must have an adverse event, preventability and lessons to be learnt), structured case analysis to specifically identify cognitive biases and system issues, discussion of possible action items, creation and dissemination of the summaries of each discussion, development of explicit administrative pathways to action on identified issues and encouragement of interprofessional and multidisciplinary involvement. We provided a formal OM3 guiding document (with an accompanying explanatory PowerPoint slide set), as well as presentation template slides to all M&M rounds presenters. We included in the OM3 package a case analysis toolkit containing educational materials about cognitive biases and system issues, as well as a simplified worksheet to help guide presenters to analyse their M&M rounds cases. The study investigators also provided regular coaching workshops to support education around quality improvement and patient safety principles. We encouraged each participating group to assign a dedicated ‘facilitator’ at each M&M rounds to help moderate and promote productive discussions in a blame-free environment (which turned out to be the M&M Champions themselves within our study). Finally, we supported each M&M group to formally establish a quality committee (if not pre-existing) where issues arising out of M&M rounds could be addressed; a sample quality committee terms of reference template was also provided when necessary to help expedite this process. Typically quality committees were to meet monthly and consisted of physician and other health professional members, with the goal of addressing quality of care and patient safety issues arising within each group.
In conjunction with the relevant administrative head, one or two champions were identified within each group to help spearhead the change in their M&M rounds process. An in-person interview was done with each champion prior to implementation for a needs-assessment and current-state analysis, as well as to gauge an understanding of the group's readiness for change. Together with the study investigators, they collaboratively developed an OM3 implementation plan specific to each M&M group's immediate needs and potential challenges. The full OM3 implementation period was flexible and individualised to each clinical group's existing and/or new M&M rounds frequency and so on; overall, we aimed for roughly a 1-year intervention period for each group, without specifying a set frequency of rounds over the year or the total number of cases to be presented at each individual rounds. If requested, the study investigators also provided introductory presentations at a Grand Rounds or department meeting to help provide background knowledge and context for the OM3. The M&M champions were also invited to attend and observe rounds where the OM3 was already well established.
Throughout the implementation period, we conducted regular educational workshops for the champions, which consisted of 1-hour long sessions, providing a summary review of the OM3, advanced training in case analysis around cognitive and system errors, and updates on logistical topics such as medico-legal issues. These workshops also served as a forum for shared learning between champions, where common challenges and barriers associated with their implementation were discussed and where potential strategies to address them were proposed. The study investigators were also available throughout the study period on an ad hoc basis via email and unscheduled meetings.
We conducted structured interviews with each administrative head and/or individuals identified as M&M champions within each group and collected information about current M&M rounds practices, effectiveness, perceived gaps and future needs, and readiness for change. Preintervention online surveys were distributed through SurveyMonkey to all M&M rounds presenters as well as attendees (see online supplementary appendix A). A policy audit was also performed for each group, by review of quality committee minutes or documentation from a similar quality assurance process, to identify any actions that had taken place directly in response to issues arising out of M&M rounds in the preceding 12 months.
Assessment during intervention
Throughout the implementation period, a research assistant attended all M&M rounds to distribute paper surveys to all attendees (see online supplementary appendix B). In cases where the research assistant could not attend (eg, conflicting M&M rounds schedules for different groups), a substitute survey distributor from within the group ensured survey completion.
We conducted structured exit interviews with each M&M champion to capture their reflections on the OM3 implementation experience. Specifically, we assessed any positive or negative changes relative to each group's original preintervention M&M rounds processes, ongoing gaps and future needs, and perceptions of the OM3's sustainability. Postintervention online surveys were distributed through Survey Money to all M&M presenters and invited attendees.
Our primary outcome was the quality of M&M rounds after the implementation of a structured M&M model as measured by the overall OM3 scoring index (see online supplementary appendix C), which is defined by graduated stages of implementation of the various key elements of the OM3. A working group comprising experts in patient safety, health administration and medical education devised this scoring index pragmatically since there were no previous similar measures published in the literature. Escalated grading was given for various degrees of achieving what were deemed as high-quality M&M rounds: (1) regular and frequent dedicated M&M rounds; (2) formal group learning with a facilitator to moderate discussions during rounds; (3) robust case-finding processes from different sources; (4) appropriate case selections; (5) explicit case analyses around cognitive or system issues; (6) interprofessional participation; (7) regular dissemination of lessons and (8) formal mechanisms for addressing issues arising out of M&M rounds. Each OM3 key element was given equal weight in contributing to the overall OM3 score (total out of 24) and were calculated at the beginning and end of each clinical group's intervention period.
Our secondary outcomes included improvement in awareness by M&M rounds attendees of key principles outlined in the OM3 model; changes in clinical group policy and procedures as a direct result of M&M rounds; perception of effectiveness of M&M rounds on quality of care; perception of impact of the OM3 in the context of each group's environment, culture and existing processes; and identification of success factors as well as barriers to implementation.
We conducted statistical analyses to assess change on both an individual clinical group level, as well as a hospital-wide level. We used descriptive statistics to describe survey respondents, and postintervention comparisons were performed using the Student's t-test or the Mann-Whitney U test where appropriate. A p value of <0.05 were considered statistically significant. Results from in person interviews with individual champions were summarised using content theme analysis.
We approached 30/42 (71.4%) distinct clinical groups within the Ottowa Hospital. Of those, we recruited 24/30 (80.0%) clinical groups that held, or planned to hold, their own M&M rounds process. Eight of these were surgical specialties and 16 were non-surgical specialties. These services represented 1584 physicians (925 attending staff and 659 resident trainees) who were collectively responsible for approximately 38 597 inpatient visits a year. The average length of the intervention implementation period was 22.1 months for the surgical groups and 17.9 months for the non-surgical groups (see online supplementary appendix D). M&M rounds presentations were universally the responsibility of resident trainees within the surgical groups, compared with a more equal distribution among staff physicians, resident trainees and other multidisciplinary or interprofessional team members within the non-surgical groups.
Twenty per cent (6/30) of the recruited clinical groups that held their own M&M rounds process were agreeable to having their current M&M rounds process evaluated but declined to participate in implementing the OM3 due a variety of reasons, including but not limited to: satisfaction with status quo, anticipated lack of engagement within their groups and inopportune timing.
Overall OM3 scores
We found that implementation of the OM3 model led to a significant improvement in the median overall OM3 score from 12/24 (95% CI 10 to 14) to 20/24 (95% CI 18 to 21) (figure 1). The surgical groups had a higher baseline OM3 score preintervention at 14.5/24 (95% CI 12 to 17) and improved to 20/24 (95% CI 17 to 23); the non-surgical groups demonstrated an improvement from a median OM3 score of 11/24(95% CI 1 to 13) to 20.5/24 (95% CI 18 to 22). The two non-surgical groups that did not initially hold any regular M&M rounds activity at all (OM3 scores of 0, preintervention) were both able to quickly adopt the OM3 structure successfully (OM3 scores of 17/24 and 18/24, postintervention). Of the six reference groups that did not undergo OM3 implementation, the median overall OM3 score was 12/24 and remained unchanged throughout the study period.
Analysis of the individual components of the OM3 scoring index showed that, overall, all components had significant improvement postintervention with the exception of ‘Activity Frequency’, which remained the same. Subgroup analysis of the non-surgical groups revealed the same pattern, where all components except for ‘Activity Frequency’ demonstrated significant improvement postintervention. The surgical groups, however, only showed significant improvements in ‘Case Selection’, ‘Case Analysis’ and ‘Outcomes’, while the remaining components of the OM3 scoring index remained unchanged.
Online survey results
A total of 883 M&M rounds participants over the course of the study responded to the surveys (table 1). The majority of them were either staff physicians or trainees, followed by registered nurses and allied health professionals. When asked to rate as a percentage of frequency, based on their recollection of the previous 12 months, whether cognitive and system errors were explicitly addressed during M&M case discussions, we found a consistent improvement after implementation of the OM3 across specialties, from 37% and 42% of the time to 55% and 62% of the time, respectively (figure 2). Similarly, respondents from different specialties reported an improvement in the appropriateness of cases being presented at M&M rounds, as well as overall impact on their own individual practice based on learnings around the new structured M&M process. In addition, participants reported a noticeable impact on clinical group administrative changes in response to issues arising out of M&M rounds discussions, in the form of policy changes and other systemic improvements (see online supplementary appendix E).
In-person paper survey results
A total of 1184 M&M rounds participants over the OM3 intervention period responded to in-person paper surveys distributed at the end of each M&M rounds (table 1). The relative distribution of respondents by profession was similar to that of the web survey results. When asked whether the M&M rounds they just attended had explicitly addressed cognitive and system errors, participants reported that the majority of time both cognitive and system issues (85% and 87%, respectively) were adequately identified and addressed at the majority of rounds (figure 2). This exceeded the frequency reported by participants based on recall via the online surveys.
Exit interviews and actions review
Structured exit interviews with each participating group's champion(s) showed that participants viewed the overall efficacy of the OM3 in addressing their individual M&M rounds needs as ‘effective’ (13/24 (54.2%) groups) or ‘very effective’ (10/24 (41.7%) groups). The majority (21; 87.5%) of the participating groups reported that they would ‘definitely recommend’ the OM3 to other clinical groups who are considering improving their own M&M rounds processes, while the remaining three participating groups (12.5%) reported that they would ‘recommend’ it to others.
Table 2 summarises the key themes, in rank order of frequency of responses, that emerged from the qualitative analysis of M&M rounds participant surveys and champions' exit interviews. Among the most commonly cited benefits of the OM3 implementation was the explicit structured approach prescribed by the model, which led to more effective M&M rounds discussions, and greater participation and involvement from different members of the healthcare team. Some of the more common barriers reported included time and scheduling restraints, lack of formal training in patient safety and persistent cultural resistance within individual groups.
Of note, there were 45 policy changes reported by the M&M Champions across the hospital as a direct result of issues arising out of M&M rounds documented since OM3 implementation, compared with 0 action items documented prior to OM3 implementation. Examples of these improvements included implementation of lab specimen labelling protocols, nursing training sessions for infusion pumps, development of preprinted order sets and implementation of standard operating procedures for nosocomial respiratory viral outbreaks.
Summary of results
Initial current state analysis of our hospital before the intervention revealed that there were varied M&M rounds practices among different specialties. Anecdotally, many of the groups identified M&M rounds as a potentially powerful mechanism for improving patient safety and quality of care and self-identified the need and room for improvement. Using the OM3 approach, we were able to successfully implement a hospital-wide structured M&M rounds model across multiple specialties and demonstrated marked overall improvement in key elements of effective M&M rounds practices. Specifically, we found a significant improvement in appropriate case selection and analysis, greater focus on discussion of cognitive biases and system issues that contributed to errors, establishment of formal effector mechanisms in many groups such as quality committees and documented increase in the number of actual policy changes addressing issues that arose out of M&M rounds discussions. It is interesting to note that participants' recall of what was discussed during M&M rounds may be inaccurate, as demonstrated by the inconsistent frequency of reported explicit cognitive or system errors discussion shown in online surveys at the end of the intervention period, versus those immediately following each M&M rounds. Part of this may be driven by the fact that there may be individuals who responded to online surveys but did not actually physical attend M&M rounds. This further supports the importance of having a method of summarising, recording and disseminating key summaries after each M&M rounds. In addition, formalising the development of quality committees appear to be an important component in translating M&M discussions and action items into actual policy improvements.
Feedback from participating groups repeatedly reinforced the sensitivities in implementing any cultural change around how clinicians discuss M&M cases. While the OM3 bundle provided a useful structure to guide improvements in the actual M&M rounds process, other elements related to willingness to change, administrative scheduling, timing and duration of implementation, alignment with senior leaders and management, and educational support from content experts were cited as important for success.
Current state of research
Despite a history stretching back more than a hundred years, M&M rounds remained largely unexamined in the literature until the early twentieth century, with most of the publications focused narrowly on error recognition among surgical departments.7–20 After the Institute of Medicine's To Err Is Human,1 M&M rounds became an attractive potential process for addressing patient safety issues and preventable medical errors. Since that time, 6 national surveys, 17 observational studies21–44 and 34 pilot studies of innovative models14 have been described in the literature. In almost every case, innovations targeting M&M rounds were viewed upon positively, but few studies formally measured participant acceptance or the actual utilisation of these proposed models.
There is also a paucity of hospital-wide studies evaluating structured M&M rounds processes on a system level. Aboumatar et al20 described a prospective observational study on current practice at John Hopkin's Hospital that included 12 clinical divisions. Similarly, Bal et al21 and Sellier et al30 reported on a census of 13 clinical centres, 41 clinical departments and 140 wards at the Grenoble University Hospital in France, where they collected observational data on 24 of the 27 different departments that held M&M rounds. Of the innovative models piloted on a “hospital-wide” level, Higginson et al45 provided the largest experience to date with only five clinical groups across two hospital divisions; Schwarz et al46 also described a hospital-wide study, which involved a 28-staff rural hospital in Nepal.
In contrast, our study is the first to evaluate a structured M&M rounds model implemented by the majority of a tertiary care teaching hospital. Unlike previous studies, the OM3 model was put forward in the form of an intervention as opposed to an observational or current state analysis. With a study population that made up more than 80% of the hospital's staff population, this is also the largest and most comprehensive evaluation of an innovative M&M rounds model currently in publication.
We recognise this study took place at a single centre in Canada, which may have limited generalisability to other institutions. Although we were unable to recruit all clinical groups within the whole organisation during our study period, we believe we have recruited a representative sample of the institution, including a variety of medical and surgical specialties that were responsible for over 93% of inpatient visits at our institution. Since fundamental changes to how clinicians perform M&M rounds require a significant cultural shift, trying to apply a strict randomisation approach would potentially result in adverse and toxic resistance; instead, we chose to apply a more pragmatic and real-world approach of capitalising on those clinical groups that were ready for change. While there is the potential for selection bias, our results also support that the OM3 is a useful bundled tool to help further enhance existing M&M rounds in clinical groups that are eager to change and improve.
It was difficult to estimate an accurate response rate to our online surveys, given that we sent out invitations based on each clinical group's email distribution lists for their own M&M rounds; this often included redundant and/or inactive accounts. Almost all groups did not keep accurate attendance of specific individuals within those mailing lists; thus, it was impossible to report a robust response rate of those who actually attended M&M rounds throughout the year. This limitation was better mitigated for the in-person surveys, where the research assistant physically ensured practically 100% response rate in the majority of rounds attended.
Finally, we were also faced with the difficulty of objectively measuring the quality of M&M rounds in the absence of a validated scoring system. Since the OM3 was a novel structured system with concrete elements, we felt that the most pragmatic approach in evaluating successful implementation was to build on our previous experience in its derivation and explicitly score each of the key OM3 elements.6 While we are unable to directly link the implementation of the OM3 to measurable clinical outcomes, the significant increase in documented action items and policy changes serve as a strong proxy for potential improvement in patient safety. Future work will need to focus on validating surrogate measures of M&M rounds processes in relation to clinical outcomes.
We observed a number of important differences between medical and surgical groups within our organisation. The higher overall OM3 scores preintervention for surgical groups speaks to the fact that most already had established M&M rounds processes in place; however, there was still room for improvement to effectively conduct those rounds. While there was a relatively even distribution of who presented M&M rounds cases within the medical groups (ie, staff physicians vs trainees vs other healthcare professionals), it was almost exclusively the resident trainees who presented at surgical M&M rounds across our institution. This may have implications on future change management strategies as well as educational initiatives to address potential barriers within these subgroups, including targeted approaches for resident trainees in addition to practicing staff physicians.
It is interesting to note that although the OM3 encouraged expanding interprofessional involvement in M&M processes, the relative proportion of participation from non-physicians remained the same throughout the intervention period. While the majority of groups who did not have M&M rounds to begin with did indeed include interprofessional participation through the OM3 intervention, most of the groups that already had established M&M rounds processes prior to this study did not end up significantly expanding beyond traditional interprofessional involvement. While the champions all expressed future plans to promote greater participation by non-physicians, many cited cultural barriers within their respective groups relating to confidentiality, fears of admission of errors in front of peers or co-workers, medico-legal issues and logistical challenges such as scheduling. Future efforts need to explicitly examine and address these factors to improve greater interprofessional participation in M&M rounds processes.
Overall feelings and opinions of both M&M rounds champions and M&M rounds attendees were very positive. Across the board, participants felt the OM3 implementation was effective and would recommend it to other clinical groups seeking to improve their own M&M rounds processes. Providing multiple supports in various formats, such as workshops, educational lectures, individual coaching sessions, PowerPoint templates and visiting Grand Rounds lectures, helped ease the transition to a new model for the majority of the groups. The most common barrier to implementation cited by M&M rounds champions was time limitation; while some reported the 1 year targeted interventional period was adequate, others felt that a change in this magnitude would be easier over 2 years. These sentiments were relative to how established individual groups' original M&M rounds processes were preintervention, and once again reinforces the importance of paying special attention to the timing and overall sensitivity around individual groups' culture around M&M rounds.
We have demonstrated that the OM3 improved the quality of M&M rounds on an institutional-wide level, which serves as a primary mechanism through which hospital clinical groups address quality of care and patient safety issues. Successful implementation led to better case selection, more effective structured discussion of M&M rounds cases, identification of actionable items and demonstrable policy improvements hospital wide. We believe that the OM3 can be feasibly adapted and implemented in other large academic teaching hospitals across a variety of surgical and non-surgical specialties.
Contributors All authors contributed to study concept and design, data collection/analysis/interpretation, manuscript review and final approval of version submitted.
Funding The Ottawa Hospital Academic Medical Organization (TOHAMO) Innovation Fund.
Competing interests None declared.
Ethics approval Ottawa Health Science Network Research Ethics Board.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.