Medication administration errors (MAEs) occur in 3–8% of all non-intravenous drug doses given^1–8 in UK hospitals; much higher rates have been observed for intravenous drugs.⁹ Many of these MAEs are unlikely to result in adverse clinical outcomes, but others are more serious.¹⁰ All are likely to affect the patients’ confidence in their healthcare. Various strategies have been suggested to reduce the incidence of these errors, but few have been formally evaluated.

The problem

Educational strategies are often advocated as one method of reducing drug errors,^11–13 and we wanted to introduce consistent educational messages for staff responsible for drug administration. We have been running trust-wide medication incidents briefing seminars for some time,¹⁴ but only small numbers of staff from each ward are able to attend. We have also tried running smaller ward-based teaching sessions on drug safety, but concluded that finding time for clinical staff to attend group teaching sessions was a major barrier.

Purpose of change

Our objectives were to develop an internet-based educational package on drug safety, to explore the practicalities of this approach, to find out whether completion of this package led to an improvement in patient safety on the study ward and to explore participants’ views towards this approach to learning.

METHODS

Setting

The evaluation took place in a 29-bed mixed medical ward in a London teaching hospital. The ward received about 100 admissions each month, from many different clinical specialties. A wide range of drugs was used and nursing staff were of all grades. Standard UK paper drug charts were used to prescribe drugs and record their administration.¹ The ward pharmacist visited the ward daily from Monday to Friday to check that all drug orders were clear, legal and clinically appropriate. This included annotating instructions on administration where required. Patients’ own and individually dispensed drugs were stored in bedside medicine cabinets; a drug trolley was used for stock medication. Scheduled drug rounds were conducted four times each day, each split into two or three parts, so that one nurse gave drugs to patients on one third or one half of the ward at each time. The local research ethics committee approved the evaluation.

The intervention

A project group was set up, consisting of a medical consultant, a senior nurse manager and a senior clinical pharmacist. This group agreed with the scope of the project, the evaluation strategy and the key educational topics on which we wanted to focus.

We used a commercial software package originally developed for the financial sector, which has now been adapted for healthcare (The Learning Clinic, London, UK). This delivers modules, each of which includes educational material and multiple-choice questions, via the internet. The software tracks and benchmarks the time spent reviewing the material as well as the test scores achieved. We initially developed 11 modules (table 1) on the subject of drug safety for delivery using this software, aimed at nursing and pharmacy staff in secondary care. Each took about 30 min to complete. The material was written in the context of practice within our trust, and included both general drug-safety issues and specific high-risk areas. We asked relevant local experts to write the modules, which were then reviewed by the project group. The pharmacist acted as editor and ensured that modules had a consistent style. Each module was then tested by a newly qualified nurse to test understanding and ensure that the material was pitched at an appropriate level.

The ward manager was very enthusiastic about the project, and had a key role in encouraging nursing staff to complete the modules. Staff were allowed to complete modules in work time when clinical duties permitted, such as during quieter night shifts. Others chose to complete modules in their lunch hour, at the end of the shift or at home. A certificate of completion was given to each nurse who completed all 11 modules; no pass mark was set for the test questions for the period of this study.

Measures

The expected outcome was a reduction in MAEs. Observation is generally accepted as the gold standard for identifying MAEs.¹⁵ Two pharmacist observers therefore used this approach, using validated methods.¹⁶

An MAE was defined as any dose of drug given (or omitted) that deviated from the patient’s current drug orders. This included the omission of drugs due to unavailability on the ward, but not the omission of drugs for clinical reasons. Administration of drugs in relation to food was not assessed, and failure to follow hospital procedures was not in itself considered an error. The time at which doses were given was not considered a source of error, and documentation errors were excluded. Errors prevented by the observer or patient were included as MAEs; those prevented by other healthcare professionals were not. The different categories of MAE are listed and defined in appendices A and B; it was assumed that each dose could be associated with only one MAE.

The denominator used to calculate the MAE rate was the number of opportunities for error. An opportunity for error was defined as any dose of drug that the researcher observed being given (or omitted) and could classify with certainty as being either correct or incorrect. Administration was taken to include leaving a dose at a patient’s bedside for subsequent self-administration. Doses given outside scheduled drug rounds were not opportunities for error as they could not be observed.

The sample size was 906 opportunities for error in each phase of the study, based on being able to identify a difference in error rates between 5% and 2.5% using an α of 0.05 and a β of 0.2. On the basis of previous work, we estimated that this would require observation of 56 drug rounds in each phase, where each drug round served one third or one half of the inpatients on the ward. A 4-week data collection period was selected both before (2–29 June 2004) and after (18 October to 14 November 2004) asking nursing staff to use the educational package. Nurses completed modules throughout the period July to September 2004. About equal numbers of drug rounds were observed for each section of the ward, time of day and day of the week. Nursing staff were informed that we were studying problems with drug administration.

During each observed drug round, the pharmacist observer accompanied nurses preparing and giving drugs, and compared the drug given with that prescribed. Intravenous drugs were generally given separately from the main oral drug round, but were also observed wherever possible.

The observers tactfully intervened if they were in a position to prevent a drug error that could result in harm to the patient. Such cases were included as MAEs. We did not give any other feedback on the errors observed until after all pre-intervention and post-intervention data had been collected. The identities of the nurses observed were recorded to assist sampling, but were coded anonymously before data analysis.

To explore staff views on the educational package, an anonymous questionnaire was sent to participating nursing staff, comprising a series of five-point scales assessing different aspects of the entire package plus usefulness of each module.

Analytical methods

We calculated error rates and 95% confidence intervals (CIs) for all opportunities for error, and for all non-intravenous opportunities for error; this was to allow comparison with previous studies, most of which exclude intravenous drugs. For each MAE identified, the principal investigator (BDF), a senior clinical pharmacist, also made a judgement as to whether the error was “minor” or “potentially serious”. Minor errors were those considered very unlikely to affect the patient; potentially serious errors were those which could result in patient harm or delays to treatment.

RESULTS

All 19 nurses who administered drugs on the study ward agreed to participate. Of these, 12 (63%) nurses completed all 11 modules. The average number of modules completed by each nurse was 9.3. Only one nurse did not complete at least half of the modules. The ward pharmacist responsible for the study ward also completed the majority. Staff generally enjoyed completing the modules and liked being able to file the certificate of completion in their professional portfolio. Some staff completed modules several times. Mean scores for each module ranged from 73.8% to 88.3%.

Outcomes

A total of 53 drug rounds were observed during the pre-education data collection period, and 55 afterwards. We found no difference between the pre-intervention and post-intervention periods in terms of time of day (p = 0.80, χ² test) or day of the week (p = 0.96, χ² test) on which rounds were observed. A total of 1188 opportunities for error were observed pre-education, and 1397 afterwards. However, 89 of the post-education opportunities for error related to staff who had recently joined the ward and had not participated in the study. These opportunities for error were therefore excluded, and the remaining 1308 were used for analysis.

Before use of the educational package, 82 (6.9%) errors were identified in 1188 opportunities for error. Afterwards, 66 (5%) errors were identified in 1308 opportunities for errors. The reduction of 1.9% does not reach significance (95% CI for the difference −3.8% to 0%). Table 2 shows the types of MAE. The most common MAEs were omission, wrong dose, extra dose and fast intravenous bolus. Almost all errors with regard to intravenous doses were due to fast administration of intravenous bolus doses.

The MAE rates for non-intravenous drugs were 6.1% pre-education (71 MAE in 1165 opportunities for error) and 4.1% (52 MAE in 1282 opportunities for error) afterwards. This reduction of 2% is significant (CI −0.2 to −3.8%).

Of the 82 pre-education MAEs, 16 were judged to be potentially serious (1.4% of all opportunities for error). Post-education, 8 of 66 (0.6% of all opportunities for error) MAEs were potentially serious. The reduction of 0.8% is not significant (CI −1.6% to 0.0%). The potentially serious errors are listed in appendix C.

In all, 19 different nurses were observed pre-education; 13 of these were also observed post-education. Pre-education, the median error rate per nurse for the nurses observed was 6.8% (range 0%–14%); post-education, this was 4.6% (range 0%–11.8%). A paired t test for the 13 nurses for whom we had pre-education and post-education observations shows no significant difference (p = 0.1). Analysis of MAE rates for only these 13 nurses results in a reduction in all MAEs of 2.1% (CI 0% to −4.2%) and a reduction in non-intravenous MAEs of 2.2% (CI −0.1% to −4.2%).

Staff views

Ten questionnaires were returned (response rate 53%). Table 3 gives the scores for statements relating to the whole package.

Median scores for each module ranged between 4 and 5, where 1 was “poor” and 5 “excellent”. Two modules (warfarin, and vancomycin and gentamicin) were given median scores of 5.

DISCUSSION

Summary

We developed a series of educational modules based on drug safety. Nursing staff found this approach to be useful and enjoyed completing the modules. Although we had completed questionnaires from only 10 nurses, this suggested that participants found this approach to learning to be useful and practical. The high uptake of the package suggested that staff were keen to learn about the areas covered. We believe that the ward manager’s encouragement was vital in the success of the intervention on the study ward, and recommend that a key senior member of staff be identified for any ward to which this approach is extended.

Context

To our knowledge, this is the first observational study of the effect of an educational intervention on MAEs outside the USA. A recent US study¹⁷ reported increased adherence to safe administration practices after use of an interactive CD-ROM programme, but no effect on MAEs.

In our study, the overall MAE rate before the use of the educational package was 6.9%; afterwards, it was 5.0%. The non-intravenous error rate was 6.1% beforehand and 4.1% afterwards; both are in line with previous observational studies of non-intravenous MAEs, which report error rates between 3% and 8%.^1–8

The most common MAEs identified were omission of drug due to unavailability, other types of omission (either because the nurse could not find the drug concerned or because it was overlooked or forgotten), wrong dose errors (eg, selection of the incorrect strength of nebule or tablet or selection of the wrong number of tablets) and fast administration of intravenous bolus injections. Again, these findings are in line with those previously reported.

Interpretation

The reduction in the overall error rate failed to reach significance. However, these error rates include many errors due to the fast administration of intravenous bolus injections. Such injections were rarely given over the recommended minimum time of 3 min. Arguably, this reflects standard practice and should not be included as an MAE, although other studies of intravenous MAEs have included this as an error.^9,18,19 If the non-intravenous error rates are compared, the results suggest a reduction in non-intravenous MAEs after this educational intervention.

The lack of a more dramatic effect of an educational intervention emphasises that errors are the result of a complex system, of which human error is only one part. It was noted that some errors that were specifically targeted by the package, such as omissions, also occurred post-intervention.

Limitations

Firstly, we had no control ward for the evaluation. Selecting an appropriate control ward in a complex organisation is notoriously difficult, as no two wards are alike. We therefore believed that this would not be helpful. However, there is no evidence that observation itself reduces MAEs,^6,16 and we are not aware of any external influences that may have affected MAEs. The same ward pharmacist was responsible for the study ward throughout and there was no known seasonal variation in workload. Secondly, the time-consuming nature of observational data collection meant that a formal evaluation could be conducted only on one ward; generalisability to different wards, hospitals and periods is therefore limited. Thirdly, we did not observe equal numbers of opportunities for error for each participant pre-intervention and post-intervention. Finally, assessment of error severity was based on the principal investigator’s judgement. A more robust approach would have been to use a validated method for assessing the clinical significance of the errors identified.²⁰

Conclusions

We developed an interactive educational package focusing on patient safety. There was a high rate of uptake of the package among nursing staff on the study ward; we believe that the role of the ward manager was important in achieving this. We identified a considerable reduction in non-intravenous MAEs after the use of the package, but no marked change in the overall error rate. This approach to education is practical and may result in modest improvements in patient safety. We have since extended the package to all nursing staff and some groups of pharmacy staff in our trust, and are developing additional modules on pain, heparin and insulin.

APPENDICES

APPENDIX A: CATEGORIES OF MEDICATION ADMINISTRATION ERROR, WITH THEIR DEFINITIONS

The following categories of medication administration error (MAE) are designed to be exhaustive and mutually exclusive; each opportunity for error can be associated with only one MAE.

• Omission: A dose of drug that has not been given by the time of the next scheduled dose (does not include doses omitted according to doctor’s instructions, nurse’s clinical judgement or if patient not on ward). Includes omissions due to unavailability or lack of venous access. Subdivided into omissions due to unavailability and other omissions.

• Unordered drug: The administration of a drug that was not prescribed for the patient concerned (classified as a wrong drug error if drug X prescribed but drug Y given instead).

• Extra dose: The administration of an additional dose of a prescribed drug (includes giving a drug more times in the day than prescribed and giving a dose of drug after it has been discontinued).

• Wrong drug: A dose of a drug given that is not the drug prescribed (does not include generic substitution).

• Wrong route: The administration of the correct drug by a route or site that was not that prescribed.

• Wrong dose: The administration of the correct drug by the correct route but in a quantity that was not that prescribed (includes administration of incorrect number of dose units, selection of the wrong strength and the measurement of an incorrect volume of an oral liquid). Where liquid preparations are not measured but instead poured into ungraduated medicine cups, a wrong dose error was counted only when the observer judged that the wrong volume had been given. Failure to shake a bottle of suspension before giving was considered a wrong dose error; failure to shake a solution was not. Includes incomplete dissolution of powdered drug before giving.

• Wrong pharmaceutical form: Giving a correct dose of the drug by the correct route but in a formulation that was not prescribed (includes giving of a modified release when non-modified prescribed and vice versa). This does not include appropriate purposeful change, such as substituting tablets with an equivalent soluble form or liquid to help administration.

• Deteriorated drug: Giving a drug that has exceeded its expiry date, or a drug with its physical or chemical integrity compromised, where none of the above error types occurred.

• Other: Any other event considered to be an MAE that does not fit into any of the above.

APPENDIX B: ADDITIONAL CATEGORIES OF INTRAVENOUS MEDICATION ADMINISTRATION ERROR, WITH THEIR DEFINITIONS

• Wrong solvent: Use of a solvent to dissolve a powered drug that is different from that specified in the hospital intravenous guide or in the package insert.

• Wrong diluent: Use of a diluent to further dilute a drug that is different from that specified in the prescription, hospital intravenous guide or in the package insert.

• Wrong volume of solvent: Use of a volume of solvent to dissolve a powered drug that is different from that specified in the hospital intravenous guide or in the package insert.

• Wrong volume of diluent: Use of a volume of diluent to dilute a drug that is different from that specified in the hospital intravenous guide or in the package insert.

• Fast administration of intravenous bolus: Given faster than that specified on the prescription, hospital intravenous guide or package insert.

• Wrong infusion rate: Given at an infusion rate faster than that specified on the prescription, hospital intravenous guide or package insert.

• Incompatibility errors: Giving a drug via the same intravenous line as another drug or solution with which it is documented to be incompatible.

APPENDIX C: POTENTIALLY SERIOUS MEDICATION ADMINISTRATION ERRORS IDENTIFIED

The observers intervened in most cases to prevent the dose from reaching the patient (see table).