Article Text
PDF
Abstract
Objective: To test the hypothesis that the prescribing behaviour of doctors would improve after having experience with a computerised rule based prescribing system.
Design: A prospective observational study of changes in prescribing habits resulting from the use of a computerised prescribing system in (1) a cohort of experienced users compared with a new cohort, and (2) a single cohort at the beginning and after 3 weeks of computer aided prescribing.
Setting: 64 bed renal unit in a teaching hospital.
Intervention: Routine use of a computerised prescribing system by doctors and nurses on a renal unit from 1 July to 31 August 2001.
Main outcome measures: Number of warning messages generated by the system; proportion of warning messages overridden; comparison between doctors of different grades; comparison by doctors’ familiarity with the system.
Results: A total of 51 612 records relating to 5995 prescriptions made by 103 users, of whom 42 were doctors, were analysed. The prescriptions generated 15 853 messages, of which 6592 were warning messages indicating prescribing errors or problems. Doctors new to the system generated fewer warning messages after using the system for 3 weeks (0.81 warning messages per prescription v 0.42 after 3 weeks, p = 0.03). Doctors with more experience of the system were less likely to generate a warning message (Spearman’s ρ = −0.90, p = 0.04) but were more likely to disregard one (Spearman’s ρ = −1, p<0.01). Senior doctors were more likely than junior doctors to ignore a warning message.
Conclusions: Doctors are influenced by the experience of using a computerised prescribing system. When judged by the number of warning messages generated per prescription, their prescribing improves with time and number of prescriptions written. Consultants and registrars are more likely to use their clinical judgement to override warning messages regarding prescribed drugs.
- prescribing
- computerised system
- medication errors
Statistics from Altmetric.com
Difficulties can arise at any part of the prescription process from the moment the prescriber makes the choice of drug treatment to the time the patient receives that treatment.1,2 However they are defined,3 medication errors are very common and, in many instances, avoidable.4 Illegible prescriptions are one cause of avoidable medication error5,6 and electronic prescription systems are increasingly being introduced to remove this danger.7,8 Electronic prescribing systems can act as “expert” systems, preventing other drug errors such as drug interactions.9 They can also enforce local prescribing rules. These systems have been shown to reduce drug errors.10
However, a well intended action can often have the unintended consequence of introducing unforeseen errors.11 The sheer number of warning messages generated by the system regarding potential interactions and laboratory results may overwhelm doctors. Not all of these warnings will be clinically important, and doctors may unconsciously override a warning that indicates a potentially serious problem.9,12
We have used the data from one electronic prescribing system in use in the renal unit at the Queen Elizabeth Hospital in Birmingham7 to test the following hypotheses:
-
an intervention by an expert computer prescribing system improves the prescriber’s future prescribing and so doctors will learn over time to avoid errors;
-
more senior doctors are more likely to disregard warnings;
-
warning messages following a prescription are less common for more senior doctors and more common as the workload of the unit (measured by the number of patients) increases.
METHODS
Description of electronic prescribing system
Details of the renal unit at the Queen Elizabeth Hospital, Birmingham and its electronic prescribing system are shown in box 1.
Box 1 Renal unit at Queen Elizabeth Hospital, Birmingham and its prescribing system
-
The renal unit at the Queen Elizabeth Hospital has 64 beds. Currently the unit cares for a total population of more than 500 patients with end stage renal failure and 500 patients who have had renal transplants; over 100 renal transplants are performed annually.
-
The prescribing system was developed by Wolfson Computer Laboratory and the renal unit at the Queen Elizabeth Hospital. The system’s extensive “drug dictionary”, which includes details of drug interactions and contraindications, was created and is maintained locally. Although the design of the system enables the import of most of these data from a commercial drug database, for a number of reasons this has not yet happened. The system was introduced in January 1998 and was modified and extended in response to comments from users over the following months. However, during the period of this study there were no significant changes to the system.
-
New doctors and nurses are trained when they arrive in the unit. The training sessions last about 2 hours for doctors and 45 minutes for nurses. The system can be used to review drug treatment, laboratory results, and radiology reports; to prescribe and record administration of drugs and intravenous fluids; and to request laboratory investigations.
-
The renal unit is particularly suited to carrying out this research as the patients are on complicated drug regimens, have problems relating to renal function and excretion of drugs, and may be at risk from a large number of potentially serious interactions.
The computerised system stores each prescription or administration as a separate record which is linked to a unique patient number. New users receive up to 2 hours of introductory training on the use of the system. Each user on the system has a unique identifier number. Individual doctors’ grades are identified using this number, but not the individual users’ identities.
The system generates warnings using the rules designed into it and maintains a record of every occasion a message is displayed. Each message can be linked to the user whose action generated it, to the individual prescription (using a unique prescription key), and to the outcome of the warning—that is, whether the prescription was abandoned or followed through. If the message is “disregarded” the doctor is making the decision to note the content of the message, to disregard any warning, and to go ahead and prescribe. Otherwise, the prescription process stops and the drug is not prescribed.
The severity of the messages is graded as follows:
-
0: “Information”—no action required.
-
10: “Red information”—no action required.
-
20: “Warning”—must be ticked off before prescription can be completed.
-
25: “PopUp information”—must press OK (no alternative).
-
30: “Password”—must be ticked off and password entered before prescription can be completed.
-
35: “System”—where the drug dosage or interactions were not held by the computer system.
-
40: “Disallow”—cannot be disregarded.
Some procedures will generate multiple messages and some of these will be unavoidable. For example, when changing the dose of vancomycin from 1 g to 500 mg, if the number is changed first a message will be generated because “gram” is the default unit of vancomycin. Also some are “nurse messages”—that is, standard messages for a particular form of a drug which are displayed to the nurse administering the drug but can also be viewed by pressing a button on the screen (in other words, you do not have to be prescribing or administering a drug to see these messages).
Some messages will appear in certain circumstances—for example, “There are no daily dose limits defined”, or “There are no single dose limits defined”, or which start with “There are no dose limits defined”, “Total of doses must be”, or “First administration due”. We did not consider further those messages that were solely informational rather than warning. The remaining messages are warnings that relate to potential medication errors and we have used these as a proxy for the potential rate of errors in the unit.
The study was performed retrospectively and the doctors using the system were not informed that the audit was taking place.
Data analysis
The data recorded by the electronic prescribing system from 1 July to 31 August 2001 were imported into a Microsoft Access database as flat files by one of us (CA) who was unaware of the identity of any of the doctors using the system. Each new prescription in the system has a unique prescription key and this key can be used to link all the future administrations and warning messages which arise from it. Access query tools were used to analyse the data initially, and tables were generated listing the individual prescriptions and all the warning messages produced during the 9 week period; these tables also contained the user’s (doctor or nurse) code. Statistical tests were done using Minitab (Version 13.1, Minitab Inc, PA, USA). The statistical tests used were comparison of two proportions, Spearman’s rank correlation, χ2 test, Wilcoxon matched pairs signed rank test, and Mann-Whitney U test.
The hypotheses were tested in the following ways:
-
by comparing a cohort of doctors who were experienced in the use of the system and who finished on the unit in July 2001 with a cohort who started on the unit in August 2001 and had no prior experience of the unit;
-
by comparing the rate of warning messages and the proportion that were “disregarded” for the August cohort during their first week on the unit with their fourth week at the end of August;
-
by examining the rate of warning messages and the proportion that were “disregarded” by grade of doctor for the entire sample over the 2 months;
-
by examining the rate of warning messages and the proportion that were “disregarded” by the number of patients on the unit as a surrogate of workload; and
-
by looking at the most commonly occurring warning messages and interaction warnings.
RESULTS
There were 5995 individual new prescriptions in 257 patients. Of these, 5518 were prescribed by 42 individual doctors and the distribution by grade is shown in table 1. Twenty eight of the doctors used the system during both months, eight used the system in July only, and six started in August. Together, these prescriptions generated 51 612 records of transactions requiring computer access (for prescribing and administering the drug). The process is illustrated in fig 1.
Numbers and types of doctors and other users who used the system during July and August 2001
Flowchart showing the prescription process using the Queen Elizabeth renal unit prescribing system. *Numbers in parentheses refer to messages for nurses.
Twenty one of the doctors had made over 100 prescriptions in the system. Sixty one users who were not doctors were predominantly nurses who were allowed to prescribe single doses of drugs such as paracetamol or Gaviscon.
During the 2 month study period 15 853 messages were generated. We excluded from analysis all informational messages—that is, those considered “Information” (n = 4111) or “PopUp information” (n = 10). We also excluded “System” messages (n = 754) which relate to the prescribing system rather than to the drugs being prescribed (for example, “The details for this drug have not been checked by pharmacy. The system will provide little or no validation of prescriptions. Do you wish to proceed with this prescription?”) and all the other systems messages which will inevitably appear (n = 4386). This left 6592 warning messages, of which 6159 were generated by doctors’ prescribing; 3434 of these messages were disregarded (doctors 3192) but 3158 (doctors 2967) led to the prescription being abandoned. There were 1498 distinct warning messages in the 2 month study period, 226 of which concerned drug interactions or prescribing in renal failure. The most common are shown in table 2. Warning messages were disregarded in 52% of cases overall, but messages warning of interactions were disregarded in 85% of cases. The number of warning messages received and acknowledged by each group of users is shown in table 1.
Warnings disregarded by the prescriber resulting in the prescription being administered to the patient
Experience
Six doctors—four pre-registration house officers (PRHOs) and two senior house officers (SHOs)—started using the system regularly in the first week of August 2001, and eight doctors—five PRHOs and three SHOs—stopped using the system by the beginning of August 2001. The results for these two groups are summarised in table 3.
Comparison of experienced doctors who stopped using the system in July with inexperienced doctors who started using the system in August
The July cohort generated fewer warning messages per prescription than the August cohort, although the difference was not statistically significant (median (range) 1.03 (0.47–1.70) v 1.43 (1.13–1.79), Mann-Whitney test, p = 0.22). There was no difference between the two cohorts with respect to the proportion of warning messages disregarded (July: 676/1417 (48%), August: 817/1643 (50%), comparison of two proportions, p = 0.26). The warning message rates for the first and fourth week in August for the six doctors who started using the system in August 2001 are shown in table 4.
Warning message rates (calculated as the number of warning messages generated per prescription) for weeks 1 and 4 for six doctors who started in the unit in August 2001
By the start of week 4, each doctor (apart from C) had had over 3 weeks’ familiarity with the system. In all cases the number of warning messages generated per interaction had decreased as the doctors gained familiarity with the system. Doctors new to the system generated fewer warning messages after 3 weeks’ experience (Wilcoxon matched pairs rank sign test, p = 0.031).
Ranking all the doctors by the number of prescriptions entered in the system showed an inverse correlation with the number of warning messages per prescription (table 5). The doctors with least familiarity with the system generated the greatest number of warning messages per prescription (Spearman’s ρ = −0.90, p = 0.04) and disregarded the smallest proportion of them (Spearman’s ρ = −1, p<0.01).
Doctors ranked by the number of prescriptions entered into the system during the 2 months
Grade of staff
The percentage of all warning messages and interaction warning messages disregarded according to grade of staff is shown in table 1. There was a significant association between the grade of doctor and the number of warning messages disregarded (p<0.001, χ2 test) which persisted even when warning messages relating to renal failure or drug interactions only were analysed (p<0.001, χ2 test). The consultants (who used the system the least) generated the greatest number of warning messages per prescription, whereas the registrars (who used the system the most) generated the fewest warning messages per prescription (results not shown).
In a multivariate analysis (stepwise logistic regression) the percentage of warning messages disregarded was significantly related to both the grade of the prescribers and their familiarity with the system.
Activity
The activity of the unit varied only slightly over the 2 months studied. The number of patients who received medicines each day varied from 49 to 65. The mean number of prescriptions per day was 79 (range 20–129). The number of warning messages generated per day varied from 40 (on two occasions) to 206. If we consider the data for July, removing the confounding effect of the number of new doctors who started on the unit in August, there is no significant correlation between the number of patients on the unit and the number of warning messages per prescription (r = 0.08, p = 0.67).
DISCUSSION
Doctors who were experienced in the use of the system and who finished working on the unit in July were much less likely to generate a warning message than the six doctors who were new to the system in August. However, the new doctors rapidly improved their prescribing, as judged by the number of warning messages per prescription. This may have been in part because the pre-registration house officers who started in August were new to prescribing and their prescribing behaviour improved generally over the 3 week period, as well as because they were learning to use the system.
There was an association between the grade of doctor and the number of warning messages generated. Junior doctors were less likely to disregard these warning messages (and proceed with the prescription) than more senior doctors. For example, warnings of “doxazosin: interactions with calcium channel blockers” and “warfarin interacts with omeprazole” were often ignored by consultants. Since doxazosin was prescribed for refractory hypertension, and since blood clotting was regularly monitored in patients on warfarin, the theoretical dangers were largely removed in the context of the renal unit.
Bates et al have previously described how a system of computerised physician order entry reduced the number of serious medication errors in medical and surgical wards.10 The system used in that study had some “expertise”—for example, in displaying the results of recent relevant investigations such as blood urea concentration, or in warning of some drug interactions. They found that the number of “non-intercepted preventable adverse drug events” that actually occurred fell by 17% when physician order entry was computerised, but the number of potential adverse drug events fell by 84% over 9 months. Bizovi et al found that errors in prescriptions written in an emergency department were reduced by two thirds following the introduction of computerised prescribing.6 Both studies suggest that a major factor was the reduction in errors due to poor handwriting.
Clearly, expert systems mitigate or eliminate errors due to handwriting in the same way as less sophisticated systems, but they also have additional benefits. We were surprised how rapidly prescribing changed. After 3 weeks of using the expert system the number of warning messages fell by half. We did not examine the number of non-intercepted medication errors that continued to occur, but we looked at the way in which the prescribing behaviour of doctors was modified by interaction with an expert computerised prescribing system. Since the computer generates warning messages when errors are made in prescribing, a reduction in the number of warning messages equates to improved—that is, safer—prescribing.
Key messages
-
Computerised prescribing systems can help doctors to modify their prescribing to reduce errors.
-
The number of warning messages generated by a prescriber falls as experience with the system increases.
-
Changes in prescribing behaviour occur within weeks.
We do not know whether improved prescribing behaviour persists when doctors move to areas where there is no computerised prescribing. This is a potentially important question to answer. If behaviour is modified in the medium or long term, then there is hope that expert systems will be able to “train” prescribers to adopt safer patterns of work. Thus, expert prescribing systems may have beneficial effects far beyond the areas in which they are used. By contrast, systems which replace a handwritten prescription with a computer printed one are unlikely to have any educational benefit.
We conclude that clinical staff adapt rapidly to computer prescribing and that their prescribing behaviour is modified to reduce the number of warning messages of serious danger displayed by the system. Provided the rules governing warning messages are carefully constructed, the alignment of doctors’ prescribing practice with the rules should improve patient safety. It remains to be seen whether the benefits of exposure to such computerised systems are maintained after prescribers cease to use them.