Medication errors are principal contributors to adverse events causing significant harm to hospitalised patients.^1–4 These errors may occur during any phase of the drug delivery process from prescribing to drug administration, with errors related to the ordering process being most common.⁵ Medication errors probably occur more frequently in intensive care units (ICU) because patients in an ICU typically receive a greater number of medications, medications are administered in a more pressured environment, and patients are less able to defend themselves because of barriers to cognition or communication.^6–8 Medication errors are frequently attributed to the failures of front line staff. However, the frequency of failure, the complexity of the drug delivery process, and the number of individuals participating in the process suggest that it is more appropriate to consider medication errors to be failures of a drug delivery system.

Continuous infusion of intravenous drugs represents a particular type of drug delivery process with distinct features that may influence the development or detection of medication errors and their seriousness. Dose calculations are a common contributor to medication errors in general,^9,10 with 10 fold errors being one of the most common calculation errors.^11,12 Many of these errors occur not because of mathematical errors, but because of using erroneous equations.¹¹ The calculations involved in delivering continuous infusions are more complex and may be more error prone than calculations for intermittent dosing. Errors, once committed, may be perpetuated over the duration of an infusion and detection of errors may be delayed. Moreover, the types of medications administered by continuous infusion include vasoactive substances (such as dopamine or epinephrine), sedatives, and narcotics for which medication errors may have a higher likelihood of harm. Continuous infusion may also differ from other drug delivery processes as infusions may be formulated at the point of care with high frequency because of their common use in emergency situations.

One approach to improving the safety of a complex process such as drug delivery is to identify the individual points of failure and implement remedial countermeasures. For example, one might consider errors in dose calculation and implement training or redundancy to reduce that single point failure rate. Alternatively, the process may be considered in its entirety and redesigned to lower the overall failure rate.

Failure mode effects analysis (FMEA) is an industrial tool developed by reliability engineers to evaluate systematically a complex process, identify elements that have a risk of causing harm, and prioritize remedial measures. It is based on the premise that risk is related not only to the likelihood of failure occurring, but also to the severity of a failure should one occur and the ease with which failure might be detected and intercepted before causing harm.^13,14 For example, a process element that fails commonly but which has little chance of causing harm or is easily detected is less risky than an element which fails less commonly but has greater consequences and is undetectable. FMEA takes information regarding failure rates from a variety of sources including the literature, measurement, and perceptions based on experience. It then uses that information to predict the behaviour of a system.

The drug delivery process influences safety in other ways. For example, hospital policies or drug stabilities may dictate the duration of time a prepared IV solution may continue to infuse before being replaced. This “hang time” for IV infusions dictates the number of times an infusion needs to be remade over the course of treatment. Each replacement presents a new opportunity for error and an opportunity for interruptions, albeit brief, that could result in physiological instability. Resource consumption is similarly affected with shorter hang times, increasing the overall work required to maintain an infusion. Thus, redesign of the infusion delivery process offers an opportunity to enhance safety, reduce resource consumption, and improve physiological stability.

We formulated a multidisciplinary team with the objectives of redesigning the process of delivering continuous drug infusions in the pediatric ICU to improve (1) patient safety; (2) efficiency in staff workflow; (3) hemodynamic stability during infusions changes; and (4) efficient use of resources. In addition, we sought to compare the reliability of the redesigned and original processes in order to provide objective evidence that the changes were beneficial. This evidence was thought to be important for the diffusion of these changes across our organization. Accordingly, we compared the original drug infusion process with the redesigned process using FMEA and evaluated the impact of the process change on resource consumption and staff satisfaction with the task of delivering infusions. We report here the results of this analysis as well as a description of the improved infusion process.

METHODS

Setting and period of study

The setting of our study is an 11 bed multidisciplinary pediatric ICU in the children’s hospital of an academic medical center. Characteristics of the ICU are shown in box 1. The ICU is directed and staffed by board certified pediatric intensivists. All medication orders are entered by the pediatric house staff using computerized physician order entry (CPOE). The process redesign and implementation of process changes occurred between May and August 2001. Analyses including measurements of resource consumption and nursing staff satisfaction occurred in September 2002, approximately 1 year after full implementation of the redesigned process. The satisfaction of pediatric residents was evaluated in October 2003 before full implementation of these process changes in other parts of the Children’s Hospital. Because this study was part of an ongoing quality improvement program, approval was not required by our institutional review board.

Team composition and goals

The drug delivery process was refined by a multidisciplinary team consisting of a pediatric intensivist, pharmacist, nurse specialists, hospital epidemiologist, and a quality management administrator. The team had worked together for several years on a range of quality improvement projects. It was led by the intensivist (MA) who has formal training in quality management and is also the medical co-director of the ICU. The goals of the team were to (1) increase patient safety; (2) increase efficiency in staff workflow; (3) reduce hemodynamic instability associated with changes in infusions; and (4) reduce resource consumption.

Analysis of the drug infusion process: FMEA

The procedure for delivering continuous drug infusions was characterized by identifying key elements of the process as well as the class of healthcare worker (physicians, nurses, pharmacists) participating in that element. Following the approach of FMEA (box 2), we first identified the ways in which each element of the process might fail. Values between 1 and 10 were assigned to three characteristics of each failure mode: the risk of injury should a failure result (severity, “S”); the frequency with which failures occur (occurrence, “O”); and the likelihood that a failure goes undetected before an injury results (detection, “D”).

This process incorporates an assessment based on the literature and subjective considerations with the numbers being assigned according to a prespecified framework. Values of 1 reflected failure effects that were unlikely to cause any injury, likelihoods of failure that were very low (imaginable but not reported in the literature), and failures that had a nearly certain probability of being detected before causing an effect. Values of 10 reflected failure effects being death, likelihoods of failure that were nearly certain, and failures that had virtually no chance of being detected before causing an effect. An example of score assignments for the likelihood of occurrence is shown in table 1.

Although the literature was reviewed by the team as a whole, four team members assigned values independently and the average of the four individual assessments for the S, O, and D scores was used in the analysis. The riskiness of each element can be expressed as a risk priority number (RPN) which is calculated as the product of the severity, occurrence, and detectability scores (RPN  =  S × O × D). The RPN identifies those elements that are the most likely contributors to medically serious failures. The maximum RPN is 1000.

Development of computerized decision support

Drug infusion calculators were developed for the internet (fig 1) and for handheld personal digital assistants. Both calculators draw from a database of defined drug formulations and generate dose infusion rate pairs given a patient’s weight. The calculators also specify the amount of drug and the volume of diluent for the standard formulation. The internet calculator was an active server page application developed in Visual Interdev™ (Microsoft, Redmond, WA, USA) drawing from a Microsoft Access™ database. The palm application was developed for the PalmOS™ operating system (Palm Inc, Milpitas, CA, USA) using AppForge™ (Appforge Inc, Atlanta, GA, USA) and Microsoft Visual Basic™.

All medication orders were entered into an Eclipsys 7000™ (Eclipsys Technologies, Boca Raton, FL, USA) electronic information system. Order entry screens were redesigned to force ordering of a standard drug formulation rather than unrestricted entry of drug amount, diluent volume, and infusion rates.

Resource consumption

The hospital activity based cost accounting system allows accounting of most purchased products including medications used in the care of patients in a defined location (such as the ICU) in the hospital. We used this accounting system to determine the number of premanufactured infusion solutions delivered in the ICU during the calendar year following the changes in the infusion process. The number of pharmacy prepared solutions was determined for a similar time period from logs maintained by the pharmacy technicians for the purpose of quality assurance.

The impact of extending infusion hang times was estimated by examining a set of drug infusion orders from our CPOE system, determining the duration for each of the infusions, and then determining the number of infusion preparations that would have had to be formulated using 24 and 72 hour hang times.

Staff satisfaction

All pediatric ICU nurses received a written survey asking them to rate anonymously the strengths of their preference for the original and revised process by using a single 5 point Likert scale to indicate agreement with the statement: “overall, I prefer the revised, standardized approach to the delivery of continuous drug infusions”. On this scale, 1 indicated a strong preference for the original process whereas 5 indicated a strong preference for the revised process. In addition, the survey asked a series of yes/no questions (table 2) to gauge whether staff perceived improvement in specific aspects of the drug delivery process. The survey was administered approximately 1 year after full implementation of the process changes.

A convenience sample of 15 pediatric residents received a similar survey limited to the assessment of preference. Only those residents stating familiarity with both the original and revised processes were asked to rate their preference. Residents were surveyed in 2003 at a time when they continued to have occasion to use the original process in locations besides the ICU.

RESULTS

Characterization of the original process

The original process was characterized by six elements: (1) selecting the drug; (2) selecting a dose; (3) selecting an infusion rate; (4) calculating and ordering the infusion; (5) preparing the infusion; and (6) programming the infusion pump and delivering the infusion (fig 2). Formulations were often calculated according to the “rule of sixes” although this rule was frequently modified to avoid overhydration. Just as often, however, an infusion rate was determined first and the formulation was calculated to allow a particular dose/rate combination. According to existing policy at the time we began this project, infusion solutions hung for a maximum of 24 hours before new solutions were prepared. Solutions were prepared at the bedside or in a medication room at the point of care by registered nurses.

Design of an improved process

The team instituted seven critical changes in the infusion delivery process (box 3). Firstly, we instituted a policy that all infusions would be made with standard formulations rather than leaving the formulations unconstrained. This required a second step to develop a database of standard formulae (metric weight of drug and volume/composition of diluent). One to three formulae were specified per medication to ensure reasonable infusion volumes over a wide range of patient weights.

Thirdly, to support accurate calculations in prescribing, calculators were developed for multiple platforms in order to provide ready access. These calculators drew from the database of standard formulations. The user specified the drug, selected a weight range for the patient, and entered the patient’s weight. The calculators then displayed and printed a spreadsheet with infusion rate/dose combinations over the therapeutic range. Although the hospital has CPOE, the current implementation does not support calculation at the time of ordering. Thus, a fourth change in the process involved redesign of the order entry screens to provide pick lists that more tightly constrained the ordering process. Because our CPOE system does not perform calculations during the order entry process, the calculated dose/rate combinations generated by the infusion calculators are transferred manually into the appropriate field on the order entry screens.

Fifthly, the team specified that drug solutions could infuse for 72 hours without replacement for those medications with sufficient stability. There is evidence that this practice does not increase the risk of bacterial contamination for the infusion solution.^15,16 Extending the time between solution replacements from 24 to 72 hours had the direct effect of reducing the number of times an infusion need be replaced, thereby leading to decreases in the opportunity for errors and reducing resource consumption. For medications with insufficient stability such as amiodorone, the team specified appropriately shorter hang times. The four most commonly prescribed infusions in our ICU are dopamine, furosemide, morphine, and lorazepam. Of these, only lorazepam is not sufficiently stable to hang for 72 hours.

Sixthly, in the revised process we elected to purchase as many commercially available drug infusions as possible, in either premixed form or as “easy to prepare” infusion systems (ADD-Vantage system, Abbott). This practice change was designed to reduce the number of opportunities for error. The seventh change shifted the task of drug admixture from the bedside to the pharmacy. As pharmacists and pharmacy technicians are less likely to commit errors in preparation than bedside nurses,^17–19 this was designed to reduce the likelihood of error.

Comparison of the two processes

Process flow

Flow charts of the original and revised processes are compared in fig 2. The revised process was similarly characterized except that, rather than selecting an infusion rate before calculations, caregivers selected a standard formulation from the database (step 3). In the original process the formulation was dependent on the selected infusion rate whereas the infusion rate in the revised process is dependent on the formulation. Although we considered other sub-processes such as connecting the infusion to the intravenous catheter, failures here had not been detected at our institution to our knowledge and were considered rare enough or so easily detected that they were not considered further in the analysis.

Failure mode effects analysis (FMEA)

FMEA was used to assess the riskiness of each element of the process. The modes of failure for each element were considered and three primary effects of failure were identified: (1) delivering the wrong drug; (2) delivering the right dose but using too dilute a formulation resulting in an excessive infusion rate; and (3) delivering the wrong dose of the drug. The latter could be caused by errors in calculation, formulation, or setting the infusion device. As an example, one medication variance report to our pharmacy described a patient receiving an infusion of furosemide. The initial prescription and preparation provided an appropriate dose. However, after dose adjustments and infusion over a period of several days, the infusion rate for which the formulation was calculated was determined to provide excessive amounts of fluid. The solution was reformulated to correct this but, in that process, a 10 fold calculation error resulted in the wrong drug dose being administered. This case illustrates sequentially the last two of the three error types described above.

After defining the modes of failure, we assigned values for the severity, likelihood of occurrence, and detectability scores (RPN  =  S × O × D). The last three elements of the original process had RPNs >225 whereas the revised process had no elements with RPNs >100 (table 3). An RPN of 150 separated the three riskiest elements of the original process from the remaining elements. A dominant factor in the calculation of the RPNs was the assignment of the severity of failure for each element. We considered an error leading to infusion of the wrong drug concentration and thus the wrong dose being in the most serious category given the potential risks of both underdosing and overdosing. The severity for selecting the wrong rate in the original process is considered much lower because the right drug dose would in fact be delivered, although a greater or lesser volume of solution would be infused to deliver that dose. Overhydration and underhydration were felt to be less severe and more easily detectable. Although the assignments for the risk factors are subjective, the risky elements of the original process continue to have RPNs of >150 even if the value for each factor is reduced by 0.7–1.4 units. In contrast, the revised process continues to have no elements with a high RPN unless the value for each factor is increased by more than 0.7 units.

Resource consumption

Review of purchasing and pharmacy records for the calendar year after full implementation of the revised process identified 304 preformulated infusions used in the pediatric ICU. In addition, 548 infusions were prepared in the pharmacy rather than at the point of care. This suggests that nurses prepared 852 fewer infusions than they would have if all infusions were prepared at the point of care. Given that most infusions hung for 72 hours rather than 24 hours (as in the original process), the savings in labour and supplies are even greater. A review of computerized order entry records showed that the longer hang time resulted in 1.8 fold fewer infusions being prepared—the savings is not threefold because not every infusion is administered for 3 days. The net result of these factors is that more than 1500 fewer infusions were prepared by nurses during the year.

Staff satisfaction with process changes

Overall, nursing staff expressed a significant preference for the revised process (mean score 4.9; n = 10) and expressed a unanimous belief that the revised process decreased the time to prepare infusions, increased the safety of the ordering and preparation processes, and led to increased accuracy in dosing calculations as well as enhancing their overall ability to do the job well (table 2). Six of the 15 pediatric residents surveyed had experience with both the original and revised processes and all six strongly preferred the revised process (score 5 of 5).

DISCUSSION

A comprehensive redesign of the continuous drug infusion delivery process has allowed us to develop a standardized approach that is safer, reduces waste, and is more efficient than the original approach relying on individualized decision making. The success of this process redesign required coordination of a number of important system changes. Standardization of drug formulations was a central component but standardization alone could have led to more frequent calculation errors because of increasing the complexity of calculations and rendering common calculation tools such as the “rule of sixes” unusable. (The “rule of sixes” is a set of equations used to calculate the amount of drug (a tenfold multiple of 6 mg × body weight in kg) to add to 100 ml solution in order for a standard infusion rate to provide a standard μg/kg/min dose.) Making calculators readily available on a number of computing platforms should lead to a decrease in calculation errors so long as the calculators are used.

Critics during planning and implementation of these process changes charged that reliance on computerized calculations might compromise the education of trainees and their ability to perform calculations correctly in environments where calculators are not available. Although this concern may indeed be valid, we find the argument against the new process weak for several reasons. Firstly, data suggest that accuracy in calculations is a function of the frequency with which they are performed by an individual.¹⁹ For trainees rotating through the ICU on a yearly basis, calculations are performed infrequently and with large gaps between opportunities to practise. Secondly, the use of computer calculators in no way precludes teaching approaches to drug dose calculations by hand or to make simplifying formulae readily available. In fact, it is possible to introduce calculations by hand as part of the process if desired to serve as a check on the system. Thirdly, the very nature of progress is one of supplanting old methods and approaches with newer ones that are more efficient, more accurate, or can be reliably performed by less skilled individuals. Nevertheless, it is possible that redesigning a process for enhanced efficiency or safety creates unintended consequences that are not well captured by FMEA. These unintended consequences should be considered and explored as part of the redesign process.

Although widely used in industrial design and quality management, particularly in high reliability industries, FMEA has not been widely used in health care. It has been advocated as a useful tool for proactive risk assessment as recommended by the Joint Commission on Accreditation of Healthcare Organizations,¹³ but there have been few published applications of this tool in health care. Burgmeier reported the use of FMEA in improving the blood transfusion process at Good Samaritan Hospital but recognized the significant organizational commitment to completing the process.²⁰ A major advantage of FMEA over other approaches in quality improvement is that it can be used both prospectively and retrospectively to characterize the safety of a process under conditions in which experimental characterization of that process is either not possible or requires long periods of time compared with the desired speed of change.

One limitation of our study is that we have not measured actual failure rates. In our setting, failures in the drug delivery process are probably underreported and happen rarely enough that statistically significant demonstration of improvement with process changes would require observation over years. This time period was considered too long to rely on experimentation as a guide to process improvement. In the absence of such data, the information gleaned through FMEA has been instrumental in educating our organization and providing a rationale for diffusion of these practice changes to other settings. For example, similar changes in infusion policy have recently been instituted in our Newborn Special Care Unit and there are plans to adopt this practice throughout the hospital.

A second limitation of our study is that we have not compared fully the resources used to support the two processes. For example, we have not directly compared the resources used by nurses to prepare solutions at the bedside with resources used in the pharmacy. In many ways this is not a straightforward comparison given that time spent by staff may not fully capture the opportunity costs of not spending time in other activities such as other aspects of patient care. We have also not directly considered the resources needed to support the decision support technology. Most importantly, we have not directly captured the value of reduced medication errors.

We consider these process changes and analysis to be one stage in a process of continuous improvement and anticipate continued diffusion of these changes to other areas. Diffusion to locations with which we share patients is particularly important as it avoids changing from one approach to another as patients are transferred between nursing units. Furthermore, there are still opportunities for improving the reliability and reducing the risk of the process. For example, calculations are currently performed using a different system than the CPOE system, necessitating transfer of calculation results by hand from one system to another. There is also no forcing function to require computerized calculations. We believe there are opportunities for improvement by integrating the calculating and ordering tasks into the next generation of our CPOE system.

CONCLUSION

It is important to note that we do not believe that our process redesign has resulted in a “fail safe” system. We do, however, believe that reduction in the frequency with which unreliable components of the process need be completed and reduction in the failure rates of the components themselves both contribute to a safer system. FMEA is a useful tool for describing the reliability of a system, comparing alternative designs, and guiding the improvement process.