Article Text

PDF
Enhancing medication use safety: benefits of learning from your peers
  1. V A Kazandjian1,
  2. S Ogunbo2,
  3. K G Wicker3,
  4. A J Vaida4,
  5. F Pipesh3
  1. 1
    Information Systems and Analysis, Maryland Patient Safety Center (MPSC), Elkridge, Maryland, USA
  2. 2
    Maryland Quality Indicator Project, Elkridge, Maryland, USA
  3. 3
    Center for Performance Sciences, Elkridge, Maryland, USA
  4. 4
    Institute for Safe Medication Practices, Pennsylvania, USA
  1. Correspondence to Dr Vahé A Kazandjian, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21205, USA; vkazandjian{at}mhaonline.org

Abstract

Background: Maryland hospitals have been improving the safety of medication use practices since 2000. A retrospective analysis of 35 hospitals was conducted for 2005–2007 to determine the changes in medication use practices, communication methods within hospitals, patient education and changes in medical record management.

Methods: Thirty-five Maryland hospitals completed the Institute for Safe Medication Practices Medication Safety Self-Assessment for Hospitals, a voluntary initiative to improve the safety of medication use. A weighting structure is applied to calculate key element scores, core characteristic scores and overall self-assessment scores that were used in ANOVA and regression analyses.

Findings: The state-wide aggregate score significantly increased from 74.2% in 2005 to 81.2% in 2007 (p<0.05). The 35 hospitals scored highest in the following key areas in 2007: drug standardisation, storage and distribution (90.2%); drug labelling, packaging and nomenclature (88.1%); and environmental factors (84.3%). Results indicated that hospitals scored lowest in the key element area related to accessibility of patient information (72.5%) and in the core characteristics pertaining to redundancies and independent double checks (64.2%) in 2007. A substantial number of hospitals had positive and significant (p<0.05) changes in certain key elements and/or core characteristics. Few hospitals showed significant (p<0.05) decreases in their scores.

Conclusion: MEDSAFE has directly assisted Maryland hospitals in improving medication use safety. The strategies and tools of MEDSAFE have been used in Maryland since 2000 and Singapore and Austria since 2006.

Statistics from Altmetric.com

The Maryland Hospital Association (MHA) has been a pioneer in the development and application of performance improvement strategies. Indeed, 25 years ago marked the start of the Maryland Quality Indicator Project (QI Project),123 which continues to be the leading measurement, educational and performance improvement indicator –based model in the USA and 12 countries worldwide.4 When the QI Project was started, a distinction was made between “quality” and “performance,” namely that “Quality is the evaluation of performance.”5 Implicit in this definition was the placement of a value upon what is measured. In doing so, and since values are often locally contextualised, it was proposed that indicators work best when they measure performance and allow the interpreters of the statistics to transform performance into quality. A number of recent developments show that quality and safety are growing closer to each other.6789

In that definition, the QI Project made no distinction between quality and safety. Rather, it was accepted that doing the right thing covered the safety concept along with the more encompassing concept of performance. MHA’s Council on Clinical and Quality Issues realised that a number of the QI Project’s root cause analyses on patient falls, readmissions and surgical wound infections seemed to suggest potential unsafe medication use practices as possible causes in the events. It was thus recommended that a new and focused initiative be launched to learn more about medication use practices across Maryland hospitals.

The purpose of this paper is to describe the MEDSAFE Project, one of the earliest state-wide medication safety projects in the USA, its strategies, findings and demonstrated impact on improving safer medication use practices.

Impetus and collaboration

The MEDSAFE Project (MEDSAFE) started in 1999, a year before the Institute of Medicine’s landmark report on patient safety.10 The impetus for Maryland hospitals was to understand key QI Project indicator rate changes for medications through investigative stepwise-analyses such as root cause analyses. Since medication dosage, timing and potential interactions due to polypharmacy were frequently encountered during the investigative process, MEDSAFE was designed to first describe what structural, technology-assisted, internal communication and oversight there were among Maryland hospitals for prescribing, dispensing and administering medications. The mediations most frequently involved in actual or potential untoward events and outcomes were insulin, anticoagulants, narcotics and antibiotics.

At the time, there was no consistent ongoing data collection on medication use in the state of Maryland; hence all knowledge of hospital practices was based on anecdotal reporting. Further, since MEDSAFE followed the principles of the QI Project, voluntary participation by all Maryland hospitals was expected and sought. Given the years of participation in the QI Project and its demonstrated usefulness,11 all Maryland hospitals voluntarily participated in MEDSAFE along with funding the initiative.

The MEDSAFE strategy

Maryland Hospitals embarked on a self-assessment of medication use safety without external mandates or requirements. Voluntarily, hospitals first asked the MHA and then the Maryland Patient Safety Center (MPSC) along with Institute for Safe Medication Practices (ISMP), to survey, analyse and interpret patterns of safe medication use processes with technology adoption, organisational communication and other key elements in the ISMP Medication Safety Self-Assessment (MSA) to explain the safety of care profile of each hospital. This use of the ISMP MSA has been reported before.12

Communication of the findings through state-wide educational programmes, publications, discussions with individual hospitals and professional societies (physicians, nurses, pharmacists and risk managers) define the MEDSAFE strategy. Targeted, topic-specific assistance is provided to each hospital and lessons learnt are shared by MPSC with all participants. Individual hospital data have been kept confidential since the start as MEDSAFE is seen as an internal quality/safety improvement initiative. The parallels and overlaps between traditional quality improvement strategies and enhancements in safety of care are recognised not only in Maryland, but also in healthcare systems within and outside the USA.13 The confidentiality of individual hospital data has helped unbiased, complete and reliable reporting by each hospital. Based on observations from the QI Project, it is believed that performance improvement passes through stages starting with the willingness of organisations to learn about themselves, to trusting the measurement tool and its intent, to improving the quality of data as requisites for improving the quality of care.14

Methods

Hypothesis

This retrospective cohort study was designed to test four hypotheses:

  1. The cohort of hospitals will show no overall improvement in their MEDSAFE scores between 2005 and 2007.

  2. The cohort of hospitals will show no overall improvement across the 10 Key Elements of the survey.

  3. The cohort of hospitals will show no overall improvement across the 20 Core Characteristics of the survey.

  4. There will be no improvement in each hospital’s overall performance scores.

These hypotheses are based on the proposal that continuous participation in a performance improvement project is a necessary condition to engage hospitals in quality and safety improvement.15

Data and sample

Each of the 35 participating acute hospitals was asked to convene an interdisciplinary team to complete the ISMP MSA. Results of the assessment were then entered into a secure website using a unique login. The website and database are maintained by MPSC. For this study, 2005, 2006 and 2007 data were analysed.

This study includes 35 acute care Maryland hospitals that have continuously participated in MEDSAFE between 2005 and 2007. Data for the study pertain to 241 ISMP MSA assessments items organised into 10 key elements and 20 core characteristics. Although participation in MEDSAFE and use of the ISMP MSA began in 2002, the 2005–2007 period was chosen since the ISMP MSA was slightly modified in 2004 and comparisons were not possible across all the assessment items with data before 2004.

A normal distribution plot (shown in fig 1) was used to determine if the data set approximated normal levels and to compare the distribution over the study period. These plots were used to inform the decision not to transform the data prior to conducting the ANOVA and regression analysis since the data set looks approximately normal. The normal plots are also useful for a visual assessment of any shifts in the mean and the median during the study period.

Figure 1

Distribution of mean hospital score during study period, 2005–2007.

In order to check for any outliers in the domains pertaining to the 10 key elements and 20 core characteristic, box plots were used to show the distribution of these domains over the study period. The box plots were also useful in determining whether the variabilities, as measured by the inter-quartile range, were about the same for the respective key elements or core characteristic.

The weighting structure used by ISMP was used to calculate key elements scores, core characteristics scores and overall self-assessment scores. A breakdown of ISMP weighting structure for the ISMP MSA tool has been described elsewhere.1617 The key elements, core characteristics and overall aggregate scores were translated into percentage scores for ease of interpretation using the maximum possible score for the 10 key elements, 20 core characteristics or total maximum possible score as denominator. Analysis of variance and regression methods were used to test the four hypotheses and also to identify hospitals with significant or lack of significant improvement in key elements, core characteristics and overall aggregate scores. Additional analysis were performed by regressing overall scores on hospital characteristics that include number of beds, hospital type, ICU status, setting and service type. All analyses were conducted using SAS statistical software (V.9.1; SAS Inc, Cary, NC).

Results

The distribution of mean hospital scores is near-normally distributed for 2005 and 2007 with a shift to the right in 2007, indicating a gradual increase in overall mean scores (fig 1). Compared with the baseline in 2005, the 2007 overall improvement in the mean score was significant (p<0.05) (74.2% vs 81.2%) (table 1).

Table 1

Mean hospital score during study period

Over the study period, the 35 participating hospitals scored highest in the following three key element areas: drug standardisation, storage and distribution (key element 5); drug labelling, packaging and nomenclature (key element 4); and environmental factors (key element 7). The hospitals scored lowest in key element areas related to: accessible patient information (key element 1); communication of medication orders (key element 3); and patient education (key element 9) (fig 2). Scores on all of the key elements significantly increased (p<0.05) between 2005 and 2007 with the exception of key element 7 (environmental factors), which has slowly increased 3.2% over the 2005 score (table 2).

Figure 2

Distribution of mean key element scores during study period.

Table 2

Mean key element scores

Figure 3 provides an overview of hospital core characteristics distribution over the study period. Although the distribution varies greatly between the core characteristics, the results indicate core characteristic 10 (hazardous chemicals secured) has the highest mean and smallest variation in comparison with the other 19 core characteristics during the study period. Additionally, while there is evidence of statistically significant improvement in the scores of several core characteristics, hospitals did not make significant improvement in the following areas: closed drug formulary system (core characteristic 3); unit-based floor stock restricted (core characteristic 9); hazardous chemicals secured (core characteristic 10); adequate space, lighting, where practitioners are free of distractions (core characteristic 12); qualified, well-rested practitioners match work load (core characteristic 13); and redundancies and double checks performed (core characteristic 19) (table 3).

Figure 3

Distribution of mean core characteristic scores during study period.

Table 3

Mean core characteristic scores

It is worth noting that even though no statistically significant score improvements were noticed in the above mentioned core characteristics, hospitals have maintained progress in these areas with an individual average score that exceeds 80%. Nonetheless, hospitals scored highest in the following three core characteristics: hazardous chemicals secured (core characteristic 10); unit-based floor stock restricted (core characteristic 9); and minimise problems with look/sound-alike drugs (core characteristic 5); while performing poorly in redundancies and independent double checks (core characteristic 19); obtaining essential patient information (core characteristic 1); and establishing a closed drug formulary system (core characteristic 3) (table 3).

The regression analysis found no significant relationships between hospital characteristics and hospital overall aggregate scores (data not shown). However, it is interesting to note that while some hospitals have positive and significant (p<0.05) change in scores for key elements and/or core characteristics, there were five hospitals with negative but significant (p<0.05) change in these metrics during the study period.

Discussion

This study of 35 Maryland hospitals provides new insights into the patterns of improvement in safety of care when a uniform assessment tool is used and comparative analyses are shared in open learning sessions.

First, the continuous participation in the project, the use of the ISMP MSA, the completion of the assessment by an interdisciplinary team and the comparative analysis provided by the MPSC has yielded quantifiable and statistically significant results indicating improvements in the safety of medication use practices. These improvements can be seen for the cohort of 35 hospitals as well as at the individual hospital level.

Second, and similar to findings of numerous “quality” improvement initiatives,1819202122 this study did not identify a single hospital showing improvement in every area covered by the 10 key elements or 20 core characteristics in the ISMP MSA. Rather, hospitals do improve variably across these areas and an overall hospital safety profile cannot be described. Prioritisation and perhaps a longer observation period may be necessary to identify changes in each area of practice, hence the need for considering safety improvement a raison d’être of the provision of care instead of a “project.”23

Third, when we compared the overall hospital scores between 2005 and 2007, all the variation seemed to be because of “common cause” variation in the 241 questions of the survey about safe medication use processes. When a statistical process control analysis of the aggregate score is constructed, we note that the variation in the aggregated process scores for these hospitals are “in control,” with the exception of two hospitals (data not shown). Using that finding as a “flag,” the two hospitals have initiated an ongoing internal monitoring of targeted processes. Better practice models will be shared with these two hospitals based on their peer’s experiences.

Fourth, the methods and strategies of MEDSAFE are generic and its findings are proposed to be replicable across and among hospitals outside of Maryland. Indeed, hospitals in Vienna, Austria and in Singapore have been participating in MEDSAFE for more than two years and results from these hospitals are being analysed.24

Limitations and caveats

Participation in MEDSAFE has been voluntary in Maryland, Austria and Singapore. As such, self-selection may predispose to certain bias and challenge the generalisability of the results. This is not believed to be the case in Maryland since 35 of the 47 hospitals have continuously participated in MEDSAFE. Further, the goal of MEDSAFE is to help identify local trends in medication use safety, rather than comparing scores, trends and changes across regions. A commonly asked question is “If our goal is to have zero-percent error in prescribing, does it matter knowing if others are at two or three percent?” However, while comparative analysis may not be of immediate benefit to MEDSAFE participants, better practice models and learning from their peers is of cardinal interest.

The collection and reporting of assessment responses are achieved through consensus among members of cross-disciplinary teams in each hospital. It has been reported that, in some hospitals, the members of those disciplinary teams have changed during the study period. Although no systematic intra-year difference in responses has been identified for these hospitals, it remains possible that the responses may have been influenced by the variability in cross-disciplinary teams.

Maryland hospitals are no different from others in the slow rate of adopting fully electronic medical record systems or computerised physician order entry networks. There is convincing literature on the benefits of computerised information and order entry systems on preventing or even eliminating prescribing errors.252627 The data submitted for MEDSAFE are collected from hybrid sources ranging from fully functional electronic medical record and computerised physician order entry to manual abstraction of at least some of the data. Human error during the abstraction of data to respond to the questionnaire could possibly influence the scores, especially if the coordinator of the multidisciplinary team changed over the years.

Since the improvement in scores is used as the initial “pointer”28 to improvement in the process leading to safer medication use processes, the ongoing refinements to the tool between MPSC and ISMP staff is expected to result in a better assignment of maximum achievable scores adjusted to the services provided by each hospital. This may be a factor in the findings of the present study, although we found no significant relationships between hospital characteristics and hospital overall aggregate scores.

Next steps

A follow-up study is underway to collect both qualitative and quantitative information from hospitals to map the genome of process improvement in select areas of medication use processes. The findings will help these hospitals design real-time safety audits in targeted areas29 and their experiences will be classified into better practice models to be used in educational programmes via the MPSC and shared with all participants in MEDSAFE.

Acknowledgments

The authors wish to thank Ms Dana Bonistalli, executive assistant for the Center for Performance Sciences, for her assistance with this manuscript.

View Abstract

Footnotes

  • Competing interests None.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

REFERENCES