Article Text

Effects of a refined evidence-based toolkit and mentored implementation on medication reconciliation at 18 hospitals: results of the MARQUIS2 study
  1. Jeffrey L Schnipper1,2,3,
  2. Harry Reyes Nieva2,
  3. Meghan Mallouk4,
  4. Amanda Mixon5,6,
  5. Stephanie Rennke7,
  6. Eugene Chu8,
  7. Stephanie Mueller1,2,3,
  8. Gregory (Randy) R Smith Jr9,
  9. Mark V Williams10,
  10. Tosha B Wetterneck11,
  11. Jason Stein12,
  12. Anuj Dalal1,2,3,
  13. Stephanie Labonville13,
  14. Anirudh Sridharan14,
  15. Deonni P Stolldorf15,
  16. E John Orav2,16,
  17. Brian Levin2,
  18. Marcus Gresham2,
  19. Cathy Yoon2,
  20. Jenna Goldstein4,
  21. Sara Platt4,
  22. Christopher Tugbéh Nyenpan4,
  23. Eric Howell17,18,
  24. Sunil Kripalani5,19,
  25. and MARQUIS2 Site Leaders
  26. for the MARQUIS2 Study Group
      1. 1 Hospital Medicine Unit, Brigham Health, Boston, MA, USA
      2. 2 Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, Boston, MA, USA
      3. 3 Harvard Medical School, Boston, MA, USA
      4. 4 Center for Quality Improvement, Society of Hospital Medicine, Philadelphia, PA, USA
      5. 5 Section of Hospital Medicine, Division of General Internal Medicine and Public Health, Department of Medicine, Vanderbilt University Medical Center, Nashville, TN, USA
      6. 6 Geriatric Research, Education, and Clinical Center, VA Tennessee Valley Healthcare System, Nashville, TN, USA
      7. 7 Division of Hospital Medicine, Department of Medicine, University of California San Francisco Medical Center, San Francisco, CA, USA
      8. 8 Division of Hospital Medicine, Parkland Health and Hospital System and Department of Internal Medicine, University of Texas Southwestern School of Medicine, Dallas, TX, USA
      9. 9 Division of Hospital Medicine, Department of Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
      10. 10 Division of Hospital Medicine, Department of Internal Medicine, University of Kentucky Medical Center, Lexington, KY, USA
      11. 11 Division of General Internal Medicine, Department of Medicine, School of Medicine and Public Health, University of Wisconsin, Madison, WI, USA
      12. 12 1Unit, Atlanta, GA, USA
      13. 13 Department of Pharmacy Services, Brigham and Women's Hospital, Boston, MA, USA
      14. 14 Howard County General Hospital Inc, Columbia, MD, USA
      15. 15 School of Nursing, Vanderbilt University, Nashville, TN, USA
      16. 16 Department of Health Policy and Management, Harvard School of Public Health, Boston, MA, USA
      17. 17 Society of Hospital Medicine, Philadelphia, PA, USA
      18. 18 Johns Hopkins Bayview Medical Center, Baltimore, MD, USA
      19. 19 Vanderbilt Center for Clinical Quality and Implementation Research, Nashville, TN, USA
      1. Correspondence to Dr Jeffrey L Schnipper, Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, Boston, MA 02115, USA; jschnipper{at}bwh.harvard.edu

      Abstract

      Background The first Multicenter Medication Reconciliation Quality Improvement (QI) Study (MARQUIS1) demonstrated that mentored implementation of a medication reconciliation best practices toolkit decreased total unintentional medication discrepancies in five hospitals, but results varied by site. The objective of this study was to determine the effects of a refined toolkit on a larger group of hospitals.

      Methods We conducted a pragmatic quality improvement study (MARQUIS2) at 18 North American hospitals or hospital systems from 2016 to 2018. Incorporating lessons learnt from MARQUIS1, we implemented a refined toolkit, offering 17 system-level and 6 patient-level interventions. One of eight physician mentors coached each site via monthly calls and performed one to two site visits. The primary outcome was number of unintentional medication discrepancies in admission or discharge orders per patient. Time series analysis used multivariable Poisson regression.

      Results A total of 4947 patients were sampled, including 1229 patients preimplementation and 3718 patients postimplementation. Both the number of system-level interventions adopted per site and the proportion of patients receiving patient-level interventions increased over time. During the intervention, patients experienced a steady decline in their medication discrepancy rate from 2.85 discrepancies per patient to 0.98 discrepancies per patient. An interrupted time series analysis of the 17 sites with sufficient data for analysis showed the intervention was associated with a 5% relative decrease in discrepancies per month over baseline temporal trends (adjusted incidence rate ratio: 0.95, 95% CI 0.93 to 0.97, p<0.001). Receipt of patient-level interventions was associated with decreased discrepancy rates, and these associations increased over time as sites adopted more system-level interventions.

      Conclusion A multicentre medication reconciliation QI initiative using mentored implementation of a refined best practices toolkit, including patient-level and system-level interventions, was associated with a substantial decrease in unintentional medication discrepancies over time. Future efforts should focus on sustainability and spread.

      • medication reconciliation
      • patient safety
      • healthcare quality improvement
      • transitions in care
      • pharmacists

      Data availability statement

      Data are available on reasonable request.

      Statistics from Altmetric.com

      Request Permissions

      If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

      Introduction

      Transitions of care, especially into and out of hospitals, are vulnerable times for patients due to several factors, including care discontinuity, changes to medication regimens, inadequate patient education, expedited discharge processes and lack of follow-up.1–5 Medication discrepancies, defined as unintentional differences among documented regimens across different sites of care, represent an important hazard that patients frequently encounter during care transitions.6 Up to 67% of inpatients have at least one unintentional discrepancy in their medication history at the time of hospital admission,7 and some studies report general medical inpatients experience on average more than one potentially harmful discrepancy during care transitions.5 Previous studies have shown that approximately 25% of medication discrepancies are potentially harmful, with consequences including serious adverse drug events, prolonged hospital length of stay and readmissions.5 7

      Medication reconciliation is ‘a process of identifying the most accurate list of all medications a patient is taking… and using this list to provide correct medications for patients anywhere within the health system’.8 Inpatient medication reconciliation involves several steps: (1) developing an accurate list of each patient’s medications, (2) validating each medication, dose and frequency at the time of writing orders, (3) identifying and correcting discrepancies, (4) documenting intentional changes to the medication regimen and (5) communicating the new list (eg, at discharge) to the patient, caregiver and next provider(s) of care.9 Implementing medication reconciliation interventions has proven more challenging than expected for many organisations, and there are reports of pro forma compliance with regulatory requirements (eg, checking a box that medications have been reconciled) without actual improvements in patient safety.10 Furthermore, while several studies have demonstrated the benefits of various medication reconciliation interventions,11–13 a broad implementation gap remains.10

      To address the above issues, in 2011 we conducted the first Multicenter Medication Reconciliation Quality Improvement Study (MARQUIS1) at five hospitals in the USA, including two university medical centres, one community teaching hospital, one community non-teaching hospital and one Veterans Affairs hospital.14 From this work, we developed a toolkit of best practices in medication reconciliation15 and mentored its implementation.16 Overall, the intervention was associated with an 8% relative decrease in medication discrepancies per month over baseline temporal trends.17 However, improvement was not consistent, in part due to vendor electronic health record (EHR) implementation at two sites, which was unexpectedly associated with an increased rate of discrepancies. As part of the study, we iteratively refined the toolkit and conducted qualitative and mixed methods analyses to determine barriers and facilitators of implementation.18 19

      Based on these findings, we undertook MARQUIS2 using a mentored implementation quality improvement (QI) model at 18 more diverse hospitals using a refined evidence-based medication reconciliation toolkit and lessons learnt from MARQUIS1.20 We report here the main quantitative results.

      Methods

      Study design

      The methods of MARQUIS2 have been described previously.20 Briefly, this multisite pragmatic quality improvement study used interrupted time series methodology to quantify the effects of implementation on outcomes, relative to baseline temporal trends.

      Settings and participants

      Eighteen sites (hospitals and hospital systems) were selected from 72 applicants via a formal application process to participate. Sites were chosen based on several criteria including: institutional support, a local site leader with QI experience and dedicated project time, an interdisciplinary QI team, institutional track record of successful patient safety initiatives, support and resources to collect data on discrepancies and intention to implement one or more toolkit components. Sites were also chosen for heterogeneity in size, academic affiliation, region and urban location. Sites that had already implemented two or more interventions from the MARQUIS1 toolkit were excluded. Sites varied in size from 88 to 1541 beds and included sites from across the USA and one in Canada, with representation from urban, suburban and rural locations, university medical centres and community teaching and non-teaching hospitals (online supplemental appendix table 1).20 Sites were divided into three waves of six sites each, with staggered implementation start times, based on their planned implementation timelines. The first sites started mentored implementation16 in April 2016, and the last sites ended mentored implementation in April 2018 (online supplemental appendix table 2). Participants in the study at each site assembled QI teams to implement the intervention as well as clinical personnel (attending physicians, residents, advanced practice providers, nurses, pharmacists, pharmacy students and pharmacy technicians) involved in the medication reconciliation process. Each site selected the hospital units to be evaluated.

      Supplemental material

      Interventions

      The MARQUIS2 toolkit consisted of 17 system-level interventions grouped into domains (box 1)21: (1) taking the best possible medication history (BPMH)22; (2) discharge medication reconciliation and counselling; (3) clarifying roles and responsibilities among physicians, nurses and pharmacy staff; (4) patient risk stratification (eg, by creating EHR reports to identify high-risk patients based on number and types of medications, comorbidities and age); (5) health information technology improvements (eg, creating standardised medication history note templates, allowing personnel trained in taking a BPMH to modify medication lists); (6) improving access to medication sources (eg, health coaching and educating patients to keep an up-to-date medication list with them at all times and working with community pharmacies and outpatient practices to streamline obtaining patient medication lists); (7) ‘measure-vention’,23 that is, measuring then intervening to correct discrepancies in real time; and (8) stakeholder engagement (eg, social marketing to patients and inpatient clinicians).24 All sites were provided an implementation manual, instructional videos, presentations and return-on-investment (ROI) calculators (ie, for investing in medication reconciliation personnel). Several changes were made to the toolkit in response to lessons learnt from MARQUIS120 including: (1) addition of simulated cases to train and certify competency in BPMH taking; (2) increased emphasis on pharmacy technicians as BPMH takers and taking the BPMH as early in the hospitalisation as possible; (3) further refinement of our ROI calculations based on the best available estimates for the benefits of medication reconciliation on reduction in hospital length of stay and readmission rates2 25–29 and the development of talking points when advocating for more resources; (4) focus on relatively simple changes to vendor EHRs’ medication reconciliation modules; and (5) refinement of patient/caregiver discharge education materials with input from a Patient Family Advisory Council (PFAC).

      Box 1

      Patient-level and system-level interventions

      Patient-level interventions

      Best possible medication history (BPMH) in emergency department.

      BPMH outside emergency department.

      Admission medication reconciliation by trained personnel.

      Discharge medication reconciliation by trained personnel.

      Patient medication counselling at discharge by trained personnel.

      Other intensive interventions in high-risk patients (eg, health coaching and counselling with teach back).

      System-level interventions

      BPMH taking

      • Train existing staff.

      • Reallocate staff.

      • Hire new staff.

      • Communicate when additional work needed.

      Discharge medication reconciliation and patient counselling

      • Train existing staff.

      • Reallocate staff.

      • Hire new staff.

      Roles and responsibilities

      • Clarify and assign roles.

      • Provide audit and feedback on role.

      Risk stratification

      • Identify high-risk patients.

      • Implement intensive interventions if high-risk (eg, pharmacist driven BMPH and intensive discharge counselling by a pharmacist).

      Health information technology (HIT)

      • Improve medication reconciliation HIT.

      Improve access to medication sources

      • Community medication sources.

      • Patient-owned medication lists.

      Measure-vention

      • Identify and correct defects in real time.

      Stakeholder engagement

      • Social marketing.

      • Community engagement.

      Experience from MARQUIS1 also prompted multiple improvements to our implementation approach,20 including: (1) addition of four regional in-person workshops (attended by 14 sites) for hospitalist and pharmacist leaders to receive hands-on training from the study investigators regarding BPMH-taking and discharge counselling (and how to teach these skills to others) and in addressing barriers to programme implementation; (2) conducting site visits earlier (within the first 6 months of implementation) to establish mentor–mentee rapport and gain institutional support as early as possible; (3) establishment of a PFAC and their engagement in all aspects of the study; and (4) addition of peer-to-peer webinars where sites could share their implementation experiences.

      Each site’s QI team received mentoring from one of eight experienced hospitalists via monthly calls and one or two site visits. The mentors possessed experience in QI and medication safety and underwent an all-day orientation and ‘Mentor University’ training.30 Mentors also participated in ‘mentor council’ calls with each other, the principal investigator and with senior mentors from MARQUIS1 to discuss challenges and share best practices. Throughout the 18-month implementation period, each site’s QI team met internally, assessed local policies and practices, identified improvement opportunities and implemented, refined and spread one or more toolkit components to designated study units. Sites, with mentor input, decided which interventions to implement based on creating process maps of the current and ideal state, conducting hazard analyses to determine the biggest gaps in current practices and using ‘priority-payoff’ matrices to decide which intervention components would provide the highest yield with the lowest effort.

      Outcomes

      The prespecified primary outcome was the total number of unintentional medication discrepancies in admission orders and discharge orders, determined in the same manner as measured in the MARQUIS1 study, other previous studies conducted by our team and as measured by the Leapfrog Group.5 14 20 31 A study pharmacist at each site took a ‘gold standard’ medication history on a random sample of patients (goal of 22 per month), using all available data sources, typically the day after admission. After discharge, this history was then compared with the admission and discharge orders. If there were differences, the pharmacist reviewed the medical record for a clinical explanation. If necessary, the pharmacist communicated with the medical team to distinguish unintentional from intentional discrepancies. Unintentional discrepancies were recorded and categorised with respect to timing (admission or discharge), type (omission, additional medication, change in dose, route, frequency, formulation or other) and reason (history error or reconciliation error).5 Study pharmacists also recorded which patient-level interventions were provided to each patient based on medical record documentation. Study pharmacists could not be blinded to the intervention given the pragmatic design of the study, but they did collect the ‘gold standard’ medication history before evaluating any discrepancies in medication orders. Unlike in MARQUIS1, we did not perform physician adjudication to assess potential for harm for each discrepancy, as the relationship between discrepancies and potential for harm is well documented.5 32

      Sample size

      The Cochrane group recommends 20–30 observations per data point for an interrupted time series analysis, with at least three data points preintervention and postintervention.33 Accordingly, we asked each site to collect data on approximately 22 patients per month (ie, one patient per weekday) and collect baseline data for at least 3 months before implementing any interventions.

      Because of our study design, we did not know how many sites would adopt a particular intervention, nor at what time they would implement that particular intervention (ie, how many months of postintervention data would be available for analysis). However, our target sample size was 7128 patients across all sites (22 patients per month per site for 18 months), with one-third of the data anticipated to be from the preintervention period. This would yield, on average, 132 patients per site preintervention and 264 patients per site postintervention. Based on MARQUIS1 data, we predicted an average 2.9 medication discrepancies per patient with a Poisson distribution,17 allowing for detection of a reduction to 2.4 discrepancies per patient during the intervention at a single site with 82% power and 5% type 1 error; implementation of an intervention at two sites allowed for detection of changes from 2.9 to 2.6 discrepancies per patient.

      Random selection of patient sample

      The statistician used a random number generator in Microsoft Excel (Redmond, Washington, USA) to create the random sample of patients for outcome assessment. Each date in the study period was associated with a random string of numbers from 1 to 30, different for each site, which was then used to determine which patients admitted that day would be evaluated for discrepancies.

      Statistical methods

      Primary analysis: effect of the intervention on discrepancy rates

      We used interrupted time series analysis to determine the effects of the intervention over time, adjusting for baseline temporal trends. We included the first 3 months of the intervention time period (the ‘wash-in’ period) as part of the baseline to provide stable estimates for the baseline trend, as some sites had very brief preintervention time periods of data collection, and interventions were implemented slowly initially. We modelled the number of discrepancies per patient using multivariable Poisson regression adjusted for patient covariates, with total number of medications as a model offset and clustered by site. This approach measures both improvement with implementation of the intervention as a whole (change in y-intercept) and change in the baseline temporal trend (change in slope). Covariates were captured manually by each study pharmacist and included patient age, service (medical, surgical, and other), whether medication information had to be provided by a caregiver (a previously identified risk factor for discrepancies)5 and patient understanding of their medications (high, medium or low), as determined by study pharmacists taking the gold standard medication history.5 17

      To understand discrepancy outcomes over time, we created statistical process control charts34 and conducted a restricted cubic spline analysis35 using standard methods. The latter specifies a best-fit, smooth and continuous curve (with linear, quadratic and cubic terms) between a specified number of ‘knots’.

      Ancillary analyses: determination of mechanisms of action

      To understand how interventions improved discrepancy rates over time, we performed two secondary, post hoc analyses to distinguish between the effects of system-level interventions and patient-level interventions (see box 1). We first measured the mean number of system-level interventions implemented by each site over time. We then analysed discrepancy rates in the first versus last 6 months of the implementation period (ie, under conditions of low vs high implementation of system-level interventions) among patients who did not receive any patient-level interventions as a measure of the effects of system-level interventions by themselves. Second, we measured the proportion of patients who received at least one patient-level intervention over time based on medical record documentation. We then analysed discrepancy rates between those who did and did not receive any patient-level interventions in the first, middle and last 6 months of the study (ie, under conditions of low, medium and high implementation of system-level interventions, respectively) as a measure of the effects of patient-level interventions under different system conditions. To measure possible synergy between system-level and patient-level interventions, we added an interaction term to our model (ie, time period (first, second or third 6-month period)*(receipt of patient-level interventions)).

      Unless otherwise stated, two-sided p values <0.05 were used to determine significance, and all analyses were conducted using SAS V.9.4 (SAS Institute, Cary, North Carolina, USA).

      Results

      Participant flow and baseline data

      The timeline for the study for each of the three waves is shown in online supplemental appendix table 2. One site did not collect enough data on discrepancies to be included in the analyses. During the preimplementation period, the remaining 17 sites randomly sampled 1229 patients for outcome assessment. During the postimplementation period, sites sampled 3718 patients (total sample size 4947). Patient characteristics are shown in table 1. Patients in the postimplementation period were slightly older, more often on medicine services and had less understanding of their medications.

      Table 1

      Patientcharacteristics: preimplementation versus postimplementation

      Effect of the intervention on discrepancy rates

      During the period of data collection, evaluated patients experienced a steady decline in their medication discrepancy rate from 2.85 discrepancies per patient to 0.98 discrepancies per patient (figure 1). Grouped by time period of the study, mean discrepancies per patient were 2.47 in the first 6 months of the study, 2.05 in months 7–12 and 1.62 in months 13–18. In the restricted cubic spline analysis (online supplemental appendix figure 1), the only significant terms were linear terms, with a slope of −0.054 (ie, 5.4% decrease in discrepancies per month) between the first two knots. While visually there is a slightly steeper decline in discrepancy rates after the second knot (month 8), this change in slope was not statistically significant. There were no statistically significant quadratic or cubic terms.

      Figure 1

      Statistical process control chart demonstrating special cause variation in total unintentional medication discrepancies per patient over time. The dotted line represents the baseline mean, and the solid red lines represent the statistical process control limits (3 SDs from the mean).

      The interrupted time series analysis showed a 5% relative decrease in discrepancies per month over baseline temporal trends (ie, change in slope; adjusted incidence rate ratio 0.95 (95% CI 0.93 to 0.97), p<0.001). There was no significant change in the y-intercept (ie, no sudden improvement at the beginning of implementation), adjusted IRR 0.98 (95% CI 0.91 to 1.05, p=0.60). Patients on surgical services had more discrepancies than patients on medicine services (IRR 1.45, 95% CI 1.35 to 1.55, p<0.001).

      Ancillary analyses

      System-level interventions

      Over the course of the implementation period, the mean number of system-level interventions cumulatively implemented by sites increased from approximately two per site to eight per site, with plateau after approximately 12 months (figure 2A). Common interventions (online supplemental appendix table 3) included training existing staff (16 of 18 sites (89%)) and reallocating staff (11 sites (61%)) to take BPMHs, clarifying roles and responsibilities and assigning roles (15 sites (83%)), and identifying high-risk patients (12 sites (67%)). Each of the 17 system-level interventions were implemented by at least one site, and six sites hired new staff (usually pharmacy technicians) to take BPMHs. Patients who only received system-level interventions (ie, without any patient-level interventions)) experienced discrepancy rates of 2.50 per patient (95% CI 2.05 to 3.06) during the first 6 months of the study and 2.39 (95% CI 1.95 to 2.93) during the last 6 months (adjusted rate ratio 0.96, 95% CI 0.89 to 1.02, p=0.11).

      Figure 2

      (A) Mean cumulative number of system-level interventions implemented per site by month. (B) Proportion of patients who received patient-level interventions per month.

      Patient-level interventions

      During the implementation period, the proportion of patients who received patient-level interventions increased from approximately 20% to 60%, which plateaued after approximately 9 months (figure 2B). Patients who received any patient-level intervention had significantly fewer discrepancies than those who did not receive any (table 2). Moreover, this effect grew larger over time: during the last 6 months of the intervention, patients who received patient-level interventions had approximately 40% the number of discrepancies of those who did not (2.50 vs 0.96 discrepancies per patient, adjusted rate ratio 0.40, 95% CI 0.35 to 0.45, p<0.001). The interaction between time period and receipt of patient-level interventions was statistically significant in the fully adjusted model (p<0.001).

      Table 2

      Effects of patient-level interventions over time

      Discussion

      Mentored implementation of a refined, evidence-based toolkit of medication reconciliation best practices was associated with a substantial, statistically significant and steady decline in medication discrepancy rates across 17 diverse, implementation-ready North American hospitals. By the end of the study period, discrepancy rates per patient dropped to almost one-third the level seen at study onset. Implementation of both system-level and patient-level interventions increased over time. System-level interventions by themselves, without patient-level interventions, did not have a significant effect on discrepancy rates. In contrast, patient-level interventions, such as receipt of a BPMH or admission or discharge medication reconciliation performed by trained personnel, had a large effect. Moreover, this effect grew over time, suggesting a synergistic effect of patient-level and system-level interventions. For example, once a site hired and properly trained staff to take a BPMH, improved their health information technology, clarified roles and responsibilities and improved their access to preadmission medication sources, then when that staff took a BPMH, the benefits to patients were increased.

      To estimate the potential clinical impact of these improvements, with a baseline of 2.85 discrepancies per patient, a 5% relative decrease per month would correspond to a decrease to 1.13 discrepancies after 18 months or an absolute reduction of 1.72 discrepancies per patient. Since, based on our prior work, we estimate that approximately 25% of all discrepancies have potential for patient harm,5 this would mean a reduction of 0.43 potentially harmful discrepancies per patient. For a medium-sized hospital with 20 000 admissions per year, that would extrapolate to preventing 8600 potentially harmful medication orders annually.

      One possible reason for the success of this study was its ability to build on the previous work from MARQUIS1, both in terms of refining the intervention and the approach to implementation. This suggests the benefits of conducting mixed methods programme evaluation, applying implementation science methods and learning from those results.36 Another possible reason for improved outcomes relates to careful site selection, identifying sites ready and willing to implement interventions and collect study data.

      Much research documents the benefits of hospital-based medication reconciliation interventions. A recent systematic review of 19 studies focused on pharmacist-led interventions found a 66% decrease in patients with medication discrepancies,11 driven mostly by one large before–after study of 8959 patients in an integrated healthcare system,29 with only one other study containing a sample size of over a thousand patients.37 A systematic review of electronic medication reconciliation tools12 identified 13 studies and demonstrated a significant decrease in the number of medications with unintended discrepancies. To our knowledge, our study is unique in the number and variety of sites involved, the scope of the toolkit and its ability to be adapted to each site’s particular circumstances and the mentored implementation approach, all of which increase its generalisability and its ability to be widely implemented.

      The study is particularly relevant given the well-known difficulty of improving medication reconciliation in the ‘real world’ (as opposed to research studies) due to the intensity of resources needed (eg, to conduct a BPMH, reconcile medications at discharge and counsel patients) and the complex multidisciplinary workflows of medication reconciliation that vary across and within institutions.10 This study demonstrates the ability for a large number of varied hospitals, without the provision of externally provided resources (eg, grant funding), to achieve substantial improvements when given the right tools and guidance.

      The results of the study should be viewed in light of its limitations. Unlike in previous studies, we did not adjudicate unintentional discrepancies for potential for harm, nor did we measure actual patient harm (eg, adverse drug events). Instead, we prioritised resources to scale the study to more sites. As noted above, based on previous studies,5 32 we know that reductions in total discrepancies track with reductions in potentially harmful discrepancies. Second, this was not a randomised controlled trial. However, our interrupted time series methodology adjusted for temporal trends, taking into account each site’s baseline performance. Third, not every site contributed equal amounts of data, but sampling of patients within each site was random, and the ability to collect data on large numbers of patients did not correlate with implementation success, thus making it unlikely that this biased our results. Also, the sample size was less than our original target, but we still had sufficient power for our analyses due to the number of sites that implemented interventions and the overall effect size. Fourth, the analysis of the effects of patient-level interventions could have been confounded by who was judged to be most and least likely to benefit from these interventions and thus which patients received them. If anything, this would likely have biased our results against our interventions because patients at higher risk of discrepancies may have been more likely to receive them. Also, results to date do not explain any site-level differences or which intervention components were associated with the greatest improvements; these analyses are ongoing. Work to date has shown important differences in contextual factors (eg, organisational capacity, goal alignment, staff involvement, patient safety climate and perceived team cohesion) among sites that may help explain differences in implementation success.38 In MARQUIS1, the most effective intervention components were clarifying roles and responsibilities among clinical staff and hiring and training personnel in discharge medication reconciliation and patient counselling,18 but these need to be confirmed in this larger cohort of hospitals. Fifth, our use of the total number of medications as a model offset implies a linear association with the number of discrepancies, but the relationship may be non-linear, which could have led to overestimates of effect in patients with fewer medications and an underestimate in patients taking more medications. Fortunately, the mean total number of medications was similar preimplementation and postimplementation, so we do not believe this issue should have biased our results. Sixth, we did not conduct fidelity testing of implemented interventions, for example, by conducting direct observations of the interventions carried out at each site (however, supervising pharmacists at some sites did observe and provide feedback to their own personnel taking medication histories). Last, in terms of generalisability, sites were selected and therefore may not be representative of hospitals in general. However, the sites were diverse using a variety of criteria. Moreover, based on our experiences to date, only sites committed to improvement in a specific clinical area are most likely to succeed in that domain; it would be a poor use of resources, now and in the future, to foist a mentored QI project on sites lacking capacity to do the work required for process change. One challenge for future efforts is to bring as many hospitals as possible to this level of readiness to implement.

      Next steps include helping sites to sustain their gains19 and spreading the interventions to as many interested sites as possible. The American Society of Health-System Pharmacists and the Pharmacy Technician Certification Board have developed curricula to train and certify an increased workforce of pharmacy technicians in BPMH taking. As noted above, Leapfrog is measuring medication discrepancies among the hundreds of its participating hospitals, providing much-needed data to measure the quality of medication reconciliation and drive further improvement at individual sites. There is also an ongoing need to work with EHR vendors on their medication reconciliation modules39 and to directly demonstrate the effects of medication reconciliation interventions on healthcare utilisation outcomes, which often drive resource allocation decisions.

      Conclusions

      Mentored implementation of a medication reconciliation best practices toolkit resulted in a significant decrease in unintentional medication discrepancies in admission and discharge orders across multiple, diverse hospitals. Future work should focus on sustainability and spread of these interventions.

      Data availability statement

      Data are available on reasonable request.

      Ethics statements

      Patient consent for publication

      Ethics approval

      The study was approved by the applicable institutional review board (IRB); participating sites either deferred to this IRB or waived IRB as a quality improvement project at the local level.

      References

      Supplementary materials

      • Supplementary Data

        This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Footnotes

      • Twitter @drjschnip, @harryreyesnieva, @DrStephMueller, @research_dee, @chrisnyenpan

      • Collaborators MARQUIS2 site leaders: Sanchita Sen, Samer Badr, Michelle Murphy, Corrie Vasilopoulos, Tara Vlasimsky, Christine Roussel, Olugbenga Arole, Loredana Diana Berescu, Arif Arifuddowla, Hattie Main, Susan Pickle, Cristy Singleton, Brenda Asplund, Andrea Delrue, Andrea Forgione, Colleen Shipman, Luigi Brunetti, Hina Ahmed, Adrian Gonzales, Mithu Molla, Sarah Bojerek, Andrea Nguyen, Robert El-Kareh, Kyle Koenig, Loutfi Succari, Scott Kincaid, Pamela Proctor, Robert Pendleton, Amy Baughman, Kimberly Boothe, Katarzyna Szablowski, Olukemi Akande, Eric Tichy.

        MARQUIS2 Study Group: Chi Zheng, Ryan Centafont, Regina Jahrstorfer, Lisa Jaser, Isha John, Margaret Curtin, Jenna Swindler, Joe Marcus, Robert Osten, Tian Yaw, Zainulabdeen Al-Jammali, Nancy Doherty, Brandi Hamilton, Magdee Hugais, Samson Lee, Paul Sabatini, Eddie Eabisa, Jennifer Mello, Julianna Burton, Edward Fink, Anthony Biondo, Trina Huynh, Ken Kormorny, Adonice Khoury, Kathryn Ruf, Dwayne Pierce, Chadrick Lowther, Karli Edholm, Shantel Mullin, Nicole Murphy, Jeni Norstrom, Laura Driscoll, Maribeth Cabie, Andrew Cadorette, Sara John, Amy D’Silva and Lionel Picot-Vierra.

      • Contributors All authors listed have contributed sufficiently to the project to be included as authors, and all those who are qualified to be authors are listed in the author byline, individually or as a group.

      • Funding This study was funded by Agency for Healthcare Research and Quality (K01HS025486, R18 HS023757); National Center for Research Resources (UL1 RR024975-01); US Department of Veterans Affairs (12-168); and National Center for Advancing Translational Sciences (2 UL1 TR000445-06).

      • Competing interests JLS has received funding from Mallinckrodt Pharmaceuticals for an investigated-initiated study of opioid-related adverse drug events. JLS and AM received remuneration from American Society of Health-System Pharmacists (ASHP) to develop their best possible medication history training curriculum.

      • Provenance and peer review Not commissioned; externally peer reviewed.

      • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

      Linked Articles