The Institute of Medicine Committee on Identifying and Preventing Medication Errors observed that adverse drug events (ADEs) are common, especially among older adults, but acknowledged that rates in ambulatory settings are largely unknown.1 Various methods of estimating rates have been proposed (including chart review, observation, incidents reports, automated data mining tools, and combinations of these) but are in need of further study before they can be relied upon to accurately determine ADE rates.2^–7

In the mean time, practising physicians in ambulatory settings face a pressing need to improve medication-related quality of care. In the absence of reliable rate information as a driver for quality improvement, a practical approach is to identify and learn from individual incidents. Resar et al have suggested that identifying quality problems through manual chart review conducted by members of the practice team “brings the participant to the ‘front line’ of data collection” and thereby may help to more actively engage them in process improvement.4 In primary care offices, where infrastructure for quality improvement is typically lacking, actively engaging staff in this way is an approach that appears to have potential.

Chart review can, of course, be problematic due to labour-intensity, variability of interpretation (particularly with regard to definition and classification of error and harm) and difficulties related to the quality of recorded data in medical charts. In an effort to bring some standardisation and efficiency to the chart-review process, a two-step method described in 1974 uses, as the first step, a set of screening criteria identifying the presence of certain sentinel words or results in the chart that potentially indicate an adverse event, and therefore warrants further scrutiny.8 This approach, now known by the term “trigger tool,” using either manual or computer-automated methods, has been adapted for a variety of healthcare settings to identify potential adverse events4–79^–12 and has been described in the literature. In the second step the “triggered” charts are reviewed further to determine whether an adverse event occurred and, if so, to obtain additional details such as preventability and severity. A physician, nurse and/or pharmacist typically carry out this review step, which relies on their clinical judgement.

Manual trigger tool-based approaches have been shown in hospital settings to be effective and efficient when used with appropriately trained reviewers.51012 We found only one prior report in the literature of the use of ADE trigger tools in the ambulatory setting (described below).3 We did not find any reported work specific to primary care.

Purpose of this study

The overall goal of this work is to aid in the development of practical approaches for primary care practices to engage staff around quality improvement for medication safety. This paper reports our experience at six ambulatory primary care sites using a 39-item trigger tool with manual chart review. Our objectives were to evaluate: (1) the positive predictive values (PPVs) of the various triggers as signals of ADEs in primary care settings and (2) the possibility of devising a briefer and potentially more practical and efficient trigger tool that might have a higher likelihood of adoption.

METHODS

Development of the trigger tool

We began by reviewing the literature to identify an appropriate ADE trigger tool for use in the ambulatory setting. Literature search revealed only one report of the use of an ADE trigger tool in the ambulatory setting.3 In this report, Gurwitz and colleagues describe their use of computer-generated signals as part of a multimethod approach to measuring rates of ADEs among ambulatory older adults in a large multispecialty group practice. Their computerised “trigger tool” included 58 items under the headings: Serum Drug Levels, Diagnoses (ICD-9-CM codes, including codes for drug poisonings and other possible drug effects), Laboratory Results, and Antidotes/Treatments. In developing our trigger tool, we started with Gurwitz’s 58 triggers and, borrowing the concept of “Life Events” from the work of Resar et al,4 we added triggers for “Emergency room visit,” “Unplanned hospitalisation,” and “Death.” On reviewing the inpatient trigger tool reported by Rozich et al,5 we identified an additional trigger, “Abrupt medication stop,” that can also be a signal of an ADE. In early feasibility testing, we found that the ICD-9-CM triggers were very rarely found in written charts, and the ability to extract this information using billing data was variable. Therefore, we eliminated these 23 items. Our final trigger tool therefore had the 39 items shown in fig 1.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Trigger tool.

Trigger screening

The goal was to target the highest-risk patients in the practice so as to maximise the yield of ADEs from which practices could learn. Older adults are known to have a higher risk of ADEs. Prior work showed that among older adults, the drugs most commonly associated with ADEs are cardiovascular drugs.3 Therefore, our chart eligibility criteria were as follows: (1) age 65 or above; (2) at least one visit for cardiovascular disease (ICD-9-CM codes 390–459) during the 12-month period under study; and (3) established care at the office prior to the 12-month period. Eligibility was determined by billing data, where available, or by manual chart review. During the chart eligibility screening process, we found that more than 90% of charts of patients aged 65 or greater had documentation of at least one cardiovascular disease. Therefore, this criterion served little purpose and likely has little implication for the findings presented here. Where there were 150 or fewer eligible charts, all were screened; otherwise a random sample of 150 was taken.

The research assistants, who were both experienced in chart review, each underwent training comprising of detailed exposition of all the sentinel words in the trigger tool followed by half a day of hands-on chart review at one site supervised by one of the physician investigators (RS). This training provided an opportunity to identify and clarify ambiguities in the protocol. In the final protocol, every part of the outpatient chart was screened for potential triggers including (but not limited to) medication list, progress notes, telephone communications, consultation reports, laboratory results, emergency department notes and hospital discharge summaries. Positive triggers were recorded on the scannable form (fig 1). Other potential problems could be recorded as “Other” triggers.

Trigger review

Charts that contained triggers were subsequently reviewed by a Physician and Pharmacist team to determine, for each trigger: (1) whether an ADE did occur, and if so, (2) whether the event was preventable, (3) the stage of the medication use process where the ADE originated (prescribing, dispensing, administration or monitoring) and (4) the effect on the patient (none/minimal, mild, moderate or severe). An ADE is deemed preventable if it is due to an error or equipment failure.313 Examples of errors include missed allergy, wrong dosage, errors in dispensing, administration errors and failure to order or complete laboratory monitoring. The term error-associated ADE is sometimes used to avoid potential ambiguity associated with the term “preventable;” in this paper we use the term preventable ADE as synonymous with error-associated ADE.

The review only examined triggers that were identified by the initial screening process. The physician and pharmacist worked together, reviewing, discussing and recording their consensus opinion for each trigger. If they were unable to reach consensus, this was also recorded. If the reviewers identified any potential or actual harm that had not been previously recognised and addressed, they notified the primary physician or site medical director.

To improve efficiency and eliminate the scannable paper forms, we have since implemented a system of direct data entry using laptop computers in the field during both the trigger ascertainment and review steps.

The study protocol was approved by the Social and Behavioral Sciences Institutional Review Board of the State University of New York at Buffalo.

RESULTS

Screening took approximately 20 min per chart. A total of 1289 charts were screened, of which 645 (50.0%) had triggers; there were 1733 triggers in total. The physician and pharmacist spent one-half to one day per site reviewing a random sample of those charts that contained triggers. They reviewed a total of 383 charts (59% of those that had triggers), for an average of 64 charts per site (range 56 to 69), containing a total of 908 triggers (52% of the total). They were able to reach agreement on all triggers.

Of the 908 triggers reviewed, 118 (13.0%) were found to be either duplicates of previously reviewed triggers that did not require further review (eg, patient with repeatedly elevated alanine aminotransferase level due to chronic viral hepatitis) or false positives (the reviewers determined that the trigger did not occur). Among the remaining 790 “qualifying triggers,” 232 were deemed to represent ADEs, of which 92 were judged to be preventable. Preventable ADEs most commonly originated during the prescribing or administration of medications; 38% were judged to have “Minimal” effect on the patient (eg, abnormal labs with no symptoms), while 30% were classified as “Severe” (resulting in either hospitalisation, permanent disability or death).

Table 2 summarises the performance of the nine triggers with the highest PPVs. All other triggers had PPVs less than 5%. Medication discontinuation, unplanned hospitalisation and emergency department visits were by far the most common triggers, and were the largest contributors to ADEs and preventable ADEs.

Table 3 shows, for each site, the rates and PPVs of the top nine triggers, and compares the numbers of triggers and ADEs detected by these top nine triggers with those detected by the whole trigger tool. These nine triggers accounted for 94.8% (861/908) of all the triggers and detected 94.4% (219/232) of the ADEs that were identified using the full Trigger Tool.

DISCUSSION

The overall goal of this work was to explore an approach to identifying ADEs among older adults in primary care. From an internal quality-improvement perspective, individual primary care practices may find this approach helpful to the extent that it creates opportunities for practices to learn from their own experiences. The Institute for Healthcare Improvement’s trigger tool program, which currently includes various trigger tools for hospitals and a general ambulatory adverse event trigger tool, can serve as a model for this process.1114 In the spirit of Resar’s call to bring staff to the “front line” of data collection,4 trigger screening might be performed by administrative and junior nursing staff, while senior nursing and medical staff could perform the review step. Further investigation is needed to establish the usability and acceptability of such tools in busy primary care offices and to explore whether and to what extent this kind of approach can engage staff in a productive way to improve quality.

As table 3 shows, there was considerable variation in the number of triggers and ADEs identified between the six sites in this study. This could be due to differences in our ability to identify triggers (due to, for example, the chart format and content) as well as true differences in rates, which in turn may be due to multiple systemic factors. This observation attests to the uniqueness of each primary care setting and to the need for each site to examine and learn from its own ADEs.

Triggers that have higher PPVs are desirable because they result in fewer unnecessary reviews. However, this is sometimes at the expense of sensitivity. For example, the trigger that requires the use of both an antihistamine and a steroid as a signal of a possible allergic drug reaction has a higher PPV than one which requires only an antihistamine (because of false positives due to antihistamines used for sleep and other indications) but will not be as sensitive because many allergic reactions are managed without steroids. Therefore, in designing triggers, a balance needs to be struck. The best choice of trigger may depend on the type of setting under study, the goal of the effort and the resources available.

It should be emphasised that the approach and triggers presented here should neither be used to estimate rates of ADEs nor be used for benchmarking. In the absence of a gold standard for identifying ADEs, the sensitivity of the trigger tool methodology for detecting ADEs is unknown. Furthermore, the reliability and sensitivity will vary between sites, and even between providers, depending on the format and completeness of the charts (especially because we limited our review to the office chart). These are major issues that pose significant challenges warranting future study.

Among the items in our trigger tool, most occurred very infrequently and had PPVs of less than 5%. The remaining nine items accounted for 94.4% of all ADEs detected. This finding was fairly consistent across the sites in our study. This suggests that, in most primary care settings, a trigger tool consisting of these nine items would have acceptable performance as a screening tool, when compared with the full trigger tool that was employed in this study. Further work is needed to determine whether such a shortened tool has any significant advantages in terms of efficiency and its ability to drive quality-improvement efforts.