Article Text
PDF
Abstract
Background Double-checking the administration of medications has been standard practice in paediatric hospitals around the world for decades. While the practice is widespread, evidence of its effectiveness in reducing errors or harm is scarce.
Objectives To measure the association between double-checking, and the occurrence and potential severity of medication administration errors (MAEs); check duration; and factors associated with double-checking adherence.
Methods Direct observational study of 298 nurses, administering 5140 medication doses to 1523 patients, across nine wards, in a paediatric hospital. Independent observers recorded details of administrations and double-checking (independent; primed—one nurse shares information which may influence the checking nurse; incomplete; or none) in real time during weekdays and weekends between 07:00 and 22:00. Observational medication data were compared with patients’ medical records by a reviewer (blinded to checking-status), to identify MAEs. MAEs were rated for potential severity. Observations included administrations where double-checking was mandated, or optional. Multivariable regression examined the association between double-checking, MAEs and potential severity; and factors associated with policy adherence.
Results For 3563 administrations double-checking was mandated. Of these, 36 (1·0%) received independent double-checks, 3296 (92·5%) primed and 231 (6·5%) no/incomplete double-checks. For 1577 administrations double-checking was not mandatory, but in 26·3% (n=416) nurses chose to double-check. Where double-checking was mandated there was no significant association between double-checking and MAEs (OR 0·89 (0·65–1·21); p=0·44), or potential MAE severity (OR 0·86 (0·65–1·15); p=0·31). Where double-checking was not mandated, but performed, MAEs were less likely to occur (OR 0·71 (0·54–0·95); p=0·02) and had lower potential severity (OR 0·75 (0·57–0·99); p=0·04). Each double-check took an average of 6·4 min (107 hours/1000 administrations).
Conclusions Compliance with mandated double-checking was very high, but rarely independent. Primed double-checking was highly prevalent but compared with single-checking conferred no benefit in terms of reduced errors or severity. Our findings raise questions about if, when and how double-checking policies deliver safety benefits and warrant the considerable resource investments required in modern clinical settings.
- medication safety
- paediatrics
- human factors
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Introduction
Systematic reviews continue to demonstrate the high prevalence of medication errors in paediatric inpatients worldwide.1 2 Double-checking medication administrations has been embedded in nursing practice for decades as an intervention to safeguard against errors and associated harm. In paediatric hospitals, double-checking is recommended3 and widely applied given the complexity of paediatric medication management, and the vulnerable nature of patients who have reduced physiological reserves to buffer the effects of errors and a limited ability to step in and prevent obvious errors.3
The independent nature of the double-checking process, whereby both the requesting and checking nurse separately perform a check without sharing information, has been identified as a crucial element in ensuring the effectiveness of the process. In contrast, primed double-checking, in which one nurse shares information with the checking nurse, such as the name of the drug to be checked, may lead to confirmation bias.4 As the Institute for Safe Medication Practices states: ‘Two people working independently are less likely to make the same mistake; if they work together or suggest what the checker should find, both could follow the same path to error’. 5 The requirement to have two nurses involved in the double-checking process consumes considerable resources, yet these have rarely been quantified.6
Despite the widespread adoption and support for double-checking, evidence of an association between its use and reduced medication errors or harm is very limited. A recent systematic review7 of the association between double-checking and medication administration errors (MAEs) identified seven studies. Only two were rated of good quality.8 9 Common study limitations were: insufficient sample sizes; inadequate control of bias; reliance on self-report or incident reports of MAEs; and unclear definitions of double-checking.7 No study has reported on the association between double-checking and medication-related harm. Most studies failed to distinguish primed from independent double-checking.
Of the two studies given a higher quality rating, the first9 was a controlled simulation trial involving 43 pairs of intensive care and emergency department nurses randomised to double-check or single-check groups. The simulation scenario included intentional errors and the evaluator recorded whether nurses detected these errors. Overall, 33% (n=7 pairs) of the 21 nurse pairs in the double-checking group identified errors, compared with 9% (n=2 pairs) in the single-check group. However, in both groups the majority of nurses failed to detect the errors planted.
The second good quality study8 was a direct observational study of 32 nurses administering 1058 doses to 122 patients on four wards in an adult hospital in Finland in order to identify factors associated with MAEs. Applying a stepwise logistic regression model, double-checking (not defined in the study) was one of several factors found as significantly associated with fewer MAEs (OR 0·44 (0·27–0·72); p=0·001).
Of the five poor/fair quality studies,10–14 two12 14 reported a positive association between double-checking and reduced MAEs, and three showed no evidence of effect. Only one study10 was conducted in a paediatric setting and involved a quality improvement programme to increase compliance with two-person verification when setting an infusion pump within an anaesthetic radiology imaging service. However, limited preintervention/postintervention MAE data were reported. Thus, there is no evidence of the effectiveness of double-checking procedures to reduce MAEs in children to date, despite the extensive use of the policy in paediatric hospitals worldwide.
Given the extent of the published evidence base, critical evaluation of the value of double-checking in error reduction, relative to other safety interventions, is not possible. However, double-checking continues to be accepted practice. The Institute for Safe Medication Practices recommends ‘… the selective and proper use of independent double-checks…’, 5 particularly among vulnerable populations such as children and for high-risk medications, but acknowledges the need for more robust evidence of its effectiveness in practice.
Our aim was to undertake a direct observational study to measure the association between double-checking, and the presence, and potential severity of MAEs. We also sought to identify factors associated with double-checking use when mandated or voluntary, and to estimate the time and associated costs consumed by double-checking.
Methods
Patient sample
We conducted a prospective direct observational study of medication administrations to 1523 children within a 340-bed tertiary paediatric hospital in Sydney, Australia. The study was nested within a larger study investigating the effectiveness of an electronic medication management (eMM) system to reduce medication errors.15 Patients were sampled from nine medical and surgical wards during the hours of 07:00–22:00 over 22 weeks (weekdays and weekends) between April and September 2016.
Procedures
All nurses on the study wards were invited to participate via information sessions followed by direct approach. In total 298 nurses consented (representing >95% of nursing staff on those wards) and their demographic information (eg, age, gender, years of nursing experience) was recorded. Observers attended wards to conduct observations at the key medication administration times and randomly selected, from available consented nurses, a nurse to shadow. We have applied this technique in a previous study in adult hospitals.16
Observers (n=7) had nursing or pharmacy qualifications, were employed by the research team and underwent extensive training over 8 weeks which involved workshops, simulated cases and infield practice. During this training process observers spent several weeks on the wards which allowed them to become familiar with the ward organisation and for nurses to become accustomed to their presence. Observers recorded details (eg, drug name, strength, dose, route, infusion rate for intravenous (IV) administrations) of medications observed to be administered and entered these data using specialised software, the Precise Observation System for the Safe Use of Medicines (POSSUM)17 on a handheld electronic device. Figure 1 shows a research nurse undertaking observations using POSSUM.17 Observers did not have access to patient medication charts. Observers recorded the number of interruptions to nurses (defined as an external stimulus which required the nurse to switch tasks),16 18 if the nurse multitasked (defined as conducting two tasks in parallel, eg, responding to a question while also drawing up a drug), or if a parent was present at the patient’s bedside at the time of medication administration. Observers were instructed not to intervene unless they witnessed an administration error which was potentially dangerous (online supplementary appendix 1). One observer was chosen as the gold standard data collector. Inter-rater reliability was assessed on multiple occasions until all the other observers reached substantial to perfect consistency with the gold standard observer (kappa scores >0·83 for medication strength, >0·93 for medication form, 1 for route and >0·76 for double-checking).
Supplemental material
Observer using a handheld device with Precise Observation System for the Safe Use of Medicines (POSSUM) software to record details of medications being prepared and administered to patients.
Double-checking
Hospital policy required independent double-checking by registered nurses (RNs) for all medication administrations except for a select group (online supplementary appendix 2). Enrolled nurses (ENs), who are nurses who have completed a minimum of a diploma of nursing and provide care under the direction of RNs, are required to have all administrations double-checked with an RN.
Hospital policy defined double-checks as an independent process in which a second nurse verifies in the presence of the first nurse the: medication order, correct dose for patient weight, time of last dose administration, medication and solvents/diluents when applicable (eg, amount in syringe, number of tablets), dose calculation, preparation and patient identification. Independent checking required nurses to not tell each other details about the medication to be checked prior to or during the actual check, so as not to prime the checker with potentially incorrect information. The hospital policy provided considerable narrative detail about how the double-check process should be conducted emphasising the importance of the independence of the checking. In addition, a flow chart outlining the roles of the requesting and checking nurse is included in the policy (online supplementary appendix 2).
For each dose administration, observers recorded whether nurses performed independent, primed, incomplete or no double-check. Independent double-checking required all steps of the process that could be viewed by the observer to be completed independently. This included independent checking: of medication against the patient’s chart; amount drawn up in syringes; and the final prepared medication. For this study we did not include independent verification by two nurses of the patient’s identification as part of the double-checking process. Primed double-checking occurred when at least one step involved priming the other nurse with details to be checked. Incomplete double-checks occurred when at least one step was not completed. To estimate the time and associated costs of double-checking, observers recorded when the checking nurse joined and completed the check for a subset of administrations (n=803). POSSUM provided time stamps for these variables. Time spent looking for a nurse to check was not assessed. We applied the average daily number of administrations in the hospital (n=1800) and average hourly nurse rates with oncosts ($A55·21).
Medication administration errors
MAEs were defined as administrations which deviated from: the prescriber’s medication order documented in the patient’s chart; the manufacturers’ preparation/administration instructions; or relevant hospital medication administration policies. To identify MAEs, observational data of medications administered were compared with medications recorded on patients’ medication charts by a reviewer blinded to information about double-checking status. Fifteen MAE categories were applied (online supplementary appendix 3) including: incorrect drug, strength, formulation, dose, route, IV rate. These categories have been defined and applied in previous studies.16 Medication timing errors were excluded.
The potential harm severity of each error identified was rated by the reviewer. If multiple errors occurred in the same dose administration, potential harm severity was based on the cumulative effect of all errors. A severity scale, adapted from the National Coordinating Council for Medication Error Reporting and Prevention19 was used (online supplementary appendix 3).
Statistical analysis
Analysis was conducted for two groups of medications, where: (1) Double-checking was mandated. (2) Double-checking was optional. Descriptive statistics were presented by characteristics of patients, nurses, dose administrations and context (table 1). Of 5140 dose administrations observed only 0·8% (n=43; 36 in the mandated group, 7 in the optional group) were independently double-checked. Thus, both independent and primed double-checks were combined. As 2·0% (n=104) of administrations had an incomplete double-check, these administrations were considered the same as having had no double-check.
Characteristics of medication administrations by double-checking status (n=5140)
Four generalised linear mixed models (GLMMs) were applied to the two groups (mandatory double-checking and optional double-checking) to determine the association between double-checking and: (1) The occurrence of MAEs (yes/no) using logistic regression. (2) Their potential severity (no error, minimal or minor, moderate or greater) using multinomial ordinal logistic regression. These GLMMs considered correlation of administrations conducted by the same nurse and adjusted for 13 variables related to characteristics of patients, nurses and contextual factors listed in table 1.
We used generalised linear models to identify factors related to the use of double-checking. The initial models included the 13 variables in table 1. The backwards stepwise elimination process was adopted (final variables in tables 2 and 3). Results were considered significant at a value of p=0·05. All analyses were conducted using SAS software, V.9·4.
Factors associated with adherence to mandatory double-checking (n=3563 administrations)
Factors associated with the use of optional double-checking (n=1577 administrations)
Results
Table 1 reports the characteristics and distribution of medication administrations in relation to double-checking status, MAEs, and nurse, patient and contextual factors. The MAE rate for medications where double-checking was mandatory was 71·6/100 administrations, and 34·7/100 administrations among medications where double-checking was optional (table 1). MAEs by category and double-checking status are reported in online supplementary appendix 4.
Among all 5140 medication administrations observed, 3563 (69·3%) required double-checking according to hospital policy. Time taken (for the checking nurse alone) to double-check averaged 6·4 min (7·9 min for IV (n=295) and 5·5 min for non-IV administrations (n=508)). With ~1800 administrations/day across the hospital (69.3% requiring double-checks), the process consumed ~133 nurse-hours/day (expended by checking nurses) at an estimated cost of $A7344/day (~$2·7 M annually).
Association between mandated double-checking and MAEs
We found no association between double-checking and the occurrence of a MAE (OR 0·89 (95% CI 0·65 to 1·21)) or the potential severity of MAEs (OR 0·86 (0·65–1·15)) (table 4; figure 2). For double-checked administrations the error rate was 72/100 administrations and for those not double-checked 71/100 (table 1).
ORs of the association between double-checking and any MAEs and potential MAE severity*
Flowchart of study results.
Association between optional double-checking and MAEs
When double-checking was optional (n=1577), and applied (n=416), we found there was a significantly lower odds of the occurrence of a MAE (OR:0·71 (0·54–0·95)) and MAE severity (OR: 0·75 (0·57–0·99)) (table 4; figure 2). For double-checked administrations the error rate was 29/100 and for those not double-checked 37/100 administrations (table 1). Dose errors were the most frequent category of MAE and a lower rate of dose errors in the optional double-check group appeared to drive the overall difference between the double-checked and single-checked groups (17.3 dose errors/100 administrations vs 29.0/100 in the single-checked group) (online supplementary appendix 4).
Compliance and factors associated with mandated double-checking
Among the medication administrations where double-checking was mandated (n=3563), 36 (1·0%) were independently double-checked, 3296 (92·5%) were primed double-checked and 231 (6·5%) received an incomplete or no double-check (figure 1).
We examined factors associated with mandated double-checking and found compliance was significantly higher: on one ward (orthopaedics) (OR=2·1 (1·08–4·11) relative to the reference ward; for RNs compared with ENs (OR=1·58 (1·07–2·32)), and on weekends compared with weekdays (OR=1·50 (1·02–2·21)). Administrations occurring using the eMM were 28% less likely to be double-checked (OR=0·72 (0·53–0·98)) compared with administrations using paper medication charts (table 2).
Compliance and factors associated with optional double-checking
Double-checking was optional for 1577 administrations but applied in 416 (26·4%) of these. In only seven (1·7%) administrations was an independent double-check performed. Nurses who chose to double-check when it was optional were: more likely to have fewer than 2 years clinical experience; not multitasking at the time of administration; using a paper-based medication chart; administering medications after 18:00 and by a route other than oral (eg, inhalation) (table 3).
Discussion
We found that mandatory double-checking, as currently performed in paediatrics, conferred no additional safety benefit compared with single-checking. Nurses were highly compliant, but rarely performed independent double-checks, relying almost exclusively on primed double-checking. When mandatory double-checking was required, we found no significant association between double-checking and the occurrence of MAEs or the potential severity of those errors. The question as to whether the use of ‘independent’ double-checking improves safety remains unanswered as this process was so infrequently performed, as others have found,20 we could not assess its effectiveness.
The high level of compliance with primed double-checking, at over 90% of administrations, suggests nurses believe it is a necessary step, consistent with surveys of nurses’ beliefs.21 Previous studies in paediatrics have also reported high adherence at 75%–90% of administrations.7 Failure to adhere to ‘independence’ in the checking process has been attributed to poorly described policies,22 but this was not the case in our study. The hospital’s policy provided a step-by-step process for the requesting and checking nurses. However, it is possible that while the hospital policy was explicit about independence, the 298 nurses in our study may not have read or fully understood the policy. Calls for greater training and nurse education is a common response when policies are not followed as intended. However, an alternative response is to question whether independent double-checking on a large scale is practically possible in busy clinical environments. As such, increased investments in training and education may not reap improvements.23 Further, there is currently no good evidence that independent double-checking will reduce MAEs. From our data we were unable to investigate whether the extent of priming made a difference, and this is a question worthy of investigation in the future.
In many instances when nurses were not required by hospital policy to double-check medications, they chose to do so. In these cases, we found a small but significant outcome with a reduced chance of error and lower potential severity. Overall, medications where double-checking is optional (eg, topical creams, vitamins, oral antibiotics, inhaled medications) are viewed as presenting lower safety risks to patients in relation to both the likelihood and consequences of any administration errors. Our results confirm this assumption with MAE rates in this group less than half the MAE rate observed in the mandatory double-check group. While complex dose calculations are usually not required of the medications in this optional double-check group, we found dose errors were the most prevalent error category and occurred at a rate similar to that reported in other studies of paediatric dose errors.24 When nurses chose to double-check, the dose error rate was lower than for the single-checked group. Thus, investigation of the underlying factors leading to dose errors may be beneficial in future studies to understand the potential value of double-checking this step when not mandated by hospital policy.
The use of optional double-checking may be a consequence of habit, or because nurses prepare a range of medications at the same time and apply the same checking process to all. However, we found that less experienced nurses were more likely to invoke such double-checks, which suggests that their choice may be related to clinical uncertainty due to inexperience. In their simulation study, Douglass et al 9 also found that 4 of 22 nurse pairs randomised to the single-check group still chose to double-check. These and others’25 findings support the notion that even when not compelled by policy, nurses will use their clinical judgement as to when a double-check may be warranted, and in such situations the process may be more likely to confer a benefit.
We found that mandatory double-checking was more prevalent on weekends, as Alsulami et al 20 did in their observational study of medication administration in a UK paediatric hospital. Lack of time and ability to locate nurses during busy periods are likely barriers to double-checking compliance and may explain greater weekend adherence.
A 2019 statement reinforcing the use of double-checking by the Institute for Safe Medication Practices stated that ‘…there is enough evidence today to suggest that conducting a manual independent double check is worth the time and effort if this strategy is used judiciously…’.26 A critical appraisal of existing evidence,7 including the results for our study, indicate that there is not sufficient evidence to substantiate this statement. A high-quality, multisite, randomised controlled trial, incorporating a cost-effectiveness analysis, is needed to answer the fundamental question, is independent double-checking worth the time and effort in medication administration, and further in what circumstances is benefit likely? This trial should address the question of whether ‘independent’ double-checking is effective in reducing MAEs relative to possible alternative strategies. As Pfeiffer et al 27 argue, consideration should be given to reconceptualising the double-check process to include a wider range of options, such as, single-person double-checking (eg, one nurse checks each step twice). They also suggest a review of the cognitive processes required by different steps in the checking process. For example, the cognitive resources needed to check a drug calculation are different to those required to compare the drug name on a vial against that on a medication chart and may warrant different types of checks in order to prevent errors.
At face value, the logic of checking a medication twice before administration is hard to argue against, and thus it is not surprising that the policy has been adopted widely and is strongly supported by nurses. The potential value of double-checking is often reinforced when retrospective audits of medication incidents point to failures in the checking process as a potential cause.28 29 Evidence from psychological studies and theories30 provide some insights into reasons why double-checking may be no superior to single-checking. Factors include that the double-checking process diffuses responsibility so that neither participant takes full responsibility.21 31–33 As a high frequency procedure, which may be undertaken many times each day, double-checking can become ‘ritualised’ and automatic, rendering it ineffective.4 Double-checking may also be influenced by the relative roles and experience of nurses participating, with one nurse deferring to a more senior or experienced colleague.4 Douglass et al 9 found in their randomised simulation trial, that in some cases within the double-check group, the second nurse falsely reassured the first nurse who expressed concerns, or they rushed the first nurse in the process.
Our results add to the very limited information available6 on the potential workforce costs consumed by double-checking. A small time and motion study on a geriatric ward estimated 17 hours/1000 administrations could be saved by moving to single-checking.12 Our results suggest nurse-time costs may be considerably higher in paediatrics with double-checking consuming ~107 hours/1000 administrations, at an annual cost ~$A2.7 M (US$ 1.5 M). This may partly reflect the greater complexity of medication checking in paediatrics. The opportunity costs of applying an extensive double-checking policy, in terms of how, even a proportion, of these staff resources could be redirected to other safety strategies, are rarely discussed.
As new medication administration technologies are now routinely implemented in hospitals, the double-checking process has become encoded into practice. Two nurses must log onto a computer to acknowledge and sign off the checking process, which takes more time than signing paper medication charts. Our finding that nurses were significantly less likely to double-check medications when using an eMM may reflect this additional time burden. Several qualitative studies4 22 have further indicated substantial disruption to nurses’ work in both seeking out and responding to requests for double-checking which can contribute to task errors and potentially increased safety risks.16 18
Two decades ago, Lucian Leape reportedly described double-checking as one of the ‘sacred cows’ of nursing practice that ‘…saps time and is ineffective’.4 Changing or de-implementing such ingrained practices is very difficult, even in the face of evidence indicating they may be ineffective.34 In the absence of compelling evidence to support double-checking practices, a small number of healthcare organisations11 23 have changed from a policy of mandated to optional double-checking. Chua et al 23 reported on the removal of mandatory double-checking procedures in an ambulatory cancer centre in Singapore following concerns that the process had become a ‘ritualistic chant’ and an attempt to reinforce independent double-checking had failed. They reported no increase in errors (based on a prepost study using incident reports; six to eight errors/year vs four/year post). Unfortunately, without a more comprehensive assessment of MAEs, it is difficult to draw clear conclusions from that study.
Our study had some limitations. We used direct observations of staff which may have increased their compliance with policy, and nurses may have been more careful during the administration process. These factors would result in an overestimation of any beneficial effect of double-checking (due to greater policy compliance) and an underestimation of the true MAE rate (due to increased vigilance). The long period over which observations took place (5·5 months plus 2 months when observers were practising) reduced the chance of sustained behaviour change by nurses on busy clinical wards. Our time and cost estimates were based on a subsample and are intended only as broad indicators of the magnitude of resource use.
Conclusions
Our results show nurses were highly compliant with mandatory double-checking but failed to ‘independently’ check, despite a clear hospital policy. When mandatory double-checking was required, we found that primed double-checking, compared with single-checking, conferred no additional safety benefit to paediatric patients. It is questionable that continuing to promote the adoption of mandated, independent double-checking in its traditional form will reap substantial change in practice, and the opportunity costs are high. As independent double-checking was so infrequently performed, we were unable to determine whether independent double-checking is effective and this important question remains unanswered. The value of clinical judgement rather than policy-mandated double-checking warrants further exploration.
There have been enormous changes over the past 50 years in the nature of medicines, their physical packaging and presentation and technological developments (eg, smart IV pumps, automatic dispensing cabinets, bar coding and computerised medication ordering and administration systems), all of which place different cognitive demands on nurses responsible for their preparation and administration. Yet, we have not seen a commensurate review or transformation of the checking processes required. Our results indicate that the current application of double-checking policy may no longer be fit-for-purpose in modern clinical settings. It is time for a critical reappraisal of double-checking policies and innovation is required.
Acknowledgments
The authors thank the hospital staff who generously gave their time to participate in this study. The authors also thank the trained observers who were critical to the success of the study, for their expertise.
References
Footnotes
Twitter @JWestbrook91, @VLichtner
Correction notice The article has been corrected since it was published online first. The co-author Madlen Gazarian's affiliation has been updated to Faculty of Medicine, University of New South Wales, Sydney, Australia.
Contributors JIW, LL, MZR, MTB conceived of and designed the study. AW, MZR, MP, MTB, PG, JIW provided specific input into the observational techniques and co-ordinated data collection. LL and AK analysed the data. CM, ROD, VM, MG, LD, PB, AG, LW supported nurse recruitment and provided specialist expertise in relation to nursing, medical and pharmacy practices, medications and hospital policies. VL and MW provided expertise in human factors. JIW drafted the manuscript. All authors contributed to the interpretation of results and provided review and approval of the final manuscript.
Funding This study was funded by European Union Horizon 2020 and grant number: Marie Sklodowska-Curie Fellowship (VL), National Health and Medical Research Council (http://dx.doi.org/10.13039/501100000925) and grant number: Early Career Researcher Fellowship (MZR)1143941, Elizabeth Blackburn Leadership Fellowship (JIW)117, Partnership Project Grant (1094878) With Sydney Ch.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval Ethics approval was granted by Sydney Children’s Hospital Network Human Research Ethics Committee HREC/15/SCHN/370.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement No data are available. Study data cannot be obtained by a third party as a requirement of ethics approval.