Article Text
PDF
Abstract
Objectives To evaluate how often and why providers overrode drug allergy alerts in both the inpatient and outpatient settings.
Design A cross-sectional observational study of drug allergy alerts generated over a 3-year period between 1 January 2009 and 31 December 2011.
Setting A 793-bed tertiary care teaching affiliate of Harvard Medical School and 36 primary care practices.
Participants Drug allergy alerts were displayed for a total of 29 420 patients across both settings.
Main outcome measures Proportion of drug allergy alerts displayed and overridden, proportion of appropriate overrides, proportion of overrides in each medication class, different reasons for overriding and types of reactions overridden.
Results A total of 158 023 drug allergy alerts were displayed, 131 615 (83%) in the inpatient setting and 26 408 (17%) in the outpatient setting; 128 157 (81%) of which were overridden. A random sample of inpatient (n=200, 0.19%) and outpatient (n=50, 0.25%) alert overrides were screened for appropriateness, with >96% considered appropriate. Alerts for some drug classes, such as ‘non-antibiotic sulfonamides’, were overridden for >81% of prescriptions in both settings. The most common override reason was patient has taken previously without allergic reaction. In the inpatient setting alone, 70.9% of alerts that warned against the risk of anaphylaxis were overridden.
Conclusions The information contained in patients’ drug allergy lists needs to be regularly updated. Most of the drug allergy alerts were overridden, with the majority of alert overrides in the subsample considered appropriate. Some of the rules for these alerts should be carefully reviewed and modified, or removed. Further research is needed to understand providers’ overriding of alerts that warned against the risk of ‘anaphylaxis’, which are more concerning with respect to patient safety.
- Decision support, computerized
- Health services research
- Medication safety
Statistics from Altmetric.com
Introduction
The prevalence of allergies is increasing not only in the UK but across the world, with as many as 30% of adults and 40% of children affected by allergies in the USA.1 Drug allergies, in particular, are becoming increasingly common. It has been estimated that the drug penicillin is responsible for approximately 75% of fatal anaphylactic cases in the USA, leading to 500–1000 deaths per year.2
Computerised physician order entry (CPOE) with clinical decision support (CDS) allows physicians to enter medication orders electronically and provides real-time guidance and support to assist them with their prescribing.3–6 This guidance can take many different forms, including visual alerts or reminders about potential hazardous drug–drug or drug allergy interactions.7 ,8 However, the problem of alert fatigue has been well documented; prescribers are often exposed to too many alerts, which can result in them ignoring or failing to respond appropriately to even important alerts that can result in serious harm.9 ,10
Drug allergy alerts differ from other types of medication-related alerts in that they are heavily dependent on patient recall rather than those that are dependent on discrete lab values, diagnoses or known interactions with other drugs.11 In CPOE systems, prescribed drugs can be checked against the patient's allergy list, and a drug allergy alert can warn the prescriber about a possible ‘drug allergy’. A ‘drug allergy’ is a type of unpredictable drug hypersensitivity reaction, which can be either immediate (immunoglobulin E (IgE)-mediated) or delayed (non–IgE-mediated).12 These allergic reactions may often present with a broad range of symptoms, including prominent physical signs such as an itchy red skin rash (eg, urticarial), nausea, vomiting or mental status changes. Drug-induced anaphylaxis is potentially fatal and usually characterised by rapid-onset cardiovascular collapse. A number of studies have reported very high override rates of drug allergy alerts in primary care.9 ,13 However, it is currently unclear which drug classes and types of reactions were highly overridden. Another study recently evaluated providers’ drug allergy alert overrides in two US hospitals.14 However, this study does not consider the appropriateness of providers’ drug allergy alert overrides or how often specific drug classes were overridden in other clinical settings such as primary care.
Another factor complicating safety is the existence of cross-sensitisation. Once an individual has become sensitised to a drug, the possibility exists that the same individual may also react to drugs with a close structural chemical relationship or to immunochemically similar metabolites.12 Drug allergy alerts may be generated for possible cross-reactivity associations between drugs, such as penicillins and other antibiotics containing a beta-lactam ring, for example, cephalosporins, carbapenems and monobactams.15 According to the World Allergic Organization, the range of cross-sensitisation varies greatly among individuals and is often difficult to predict with any certainty.12 In this study, we evaluated how often and, more importantly, why providers overrode drug allergy alerts in both the inpatient and outpatient settings. Specific outcomes were (1) the appropriateness of providers’ drug allergy alert overrides, (2) proportion of overrides in each medication class, (3) the reasons why providers chose to override these alerts and (4) the different types of reactions overridden.
Methods
Research study site
This study included data from the Brigham and Women's Hospital (Boston, Massachusetts, USA), a 793-bed tertiary care teaching affiliate of Harvard Medical School, and 36 primary care practices. All these health organisations are part of a regional integrated healthcare delivery system, Partners HealthCare.
Electronic health record and CDS
At the time of this study, all prescriber orders were entered through the Brigham Integrated Clinical Information System (BICS) at the Brigham and Women's hospital, and the Longitudinal Medical Record (LMR) in the ambulatory care setting. The BICS and LMR provide clinical, administrative and financial computing functions, which included patient-specific decision support usually presented to providers in the form of medication alerts at the time of prescribing. These included drug allergy interaction alerts, which are triggered if the patient has a documented allergy or intolerance in their electronic health record to the prescribed product or drug agent. Drug allergy alert logic was initially sourced from the commercial knowledge base, First DataBank (First DataBank, South San Francisco, California, USA) and reviewed by an expert committee. In contrast to the Partners drug–drug interaction knowledge base, the drug allergy alert knowledge base has not been modified or iteratively improved over time to reduce the number of false positive alerts.
When a drug allergy interaction alert is generated in either the inpatient or outpatient setting, a specific recommendation is presented to the physician, which is linked to a monograph. The alert message will indicate whether the patient has a definite (the drug being ordered is an exact match to the allergen, eg, penicillin documented in allergy list and penicillin ordered), probable (the drug being ordered is in the same allergen group, eg, penicillin (penicillins) documented in allergy list and amoxicillin (penicillins) ordered) or possible reaction (cross-sensitivity is considered likely, eg, penicillin (penicillins) documented in allergy list and cephalexin (cephalosporins) ordered), as well as the specific type of reaction that the patient may be at risk of developing, for example, hives, anaphylaxis (figures 1 and 2).
Screenshot of a drug allergy alert in the inpatient setting.
Screenshot of a drug allergy alert in the outpatient setting.
In the outpatient setting, the alert warning gives the provider the choice of selecting patient does not have this allergy, will discontinue pre-existing allergy or overriding the alert. Should the physician choose to override the alert, they are required to select one of the following coded ‘override’ reasons in order to proceed: patient has taken previously without allergic reaction, low-risk cross-sensitivity, will monitor, no reasonable alternatives, or other (figure 2). If other is selected, further details should be entered in the free-text field. In the inpatient setting, the alert warning gives the provider the choice of cancelling the current order or keeping it, in which case they are required to enter a reason justifying their decision to override the alert in the free-text field provided (figure 1).
Study design and sample selection
This was a cross-sectional observational study of drug allergy alerts generated over a 3-year period between 1 January 2009 and 31 December 2011. All drug allergy alerts triggered in the inpatient and outpatient settings during the study period were downloaded. This included patients’ names and medical record identification numbers; names of both the allergic reaction and medicine that triggered the drug allergy alert; date of alert; practice location (outpatients)/specialty (inpatients); and the reasons given by providers at the time of overriding the alert.
Appropriateness check
Alert overrides of drug allergy alerts were considered appropriate for new orders in the inpatient and outpatient settings if the reason indicated by the provider, for example, patient does not have this allergy, will discontinue pre-existing allergy, low-risk cross-sensitivity, will monitor, no reasonable alternatives, patient has taken previously without allergic reaction or other could be verified on chart review. For renewal orders, the patient's chart was reviewed to confirm that there was no history that the patient had an allergic reaction to the drug (eg, rash, hives, anaphylaxis, mental status change, nausea and vomiting) and the patient did not experience an adverse event after the medication was ordered for the patient.
A pharmacist (DLS) screened a random sample of inpatient (n=200, 0.19%) and outpatient (n=50, 0.25%) alert overrides for appropriateness. A second academic pharmacist (MGA) independently reviewed the electronic inpatient medical charts (n=200), and inter-rater agreement comparing the appropriateness versus inappropriateness of each alert override was calculated and found to be excellent (κ=0.76). An attending physician (KCN) and academic nurse (IC) independently reviewed the electronic outpatient medical charts (n=50), and inter-rater agreement was also found to be excellent (k=0.86). Any disagreements were resolved by discussion with another reviewer (DWB).
Data analysis
Data were processed using structured query language statements. Natural language processing algorithms were developed to complete the categorisations of free-text override reasons in the outpatient setting and uncoded allergic reactions in both settings. For alerts that displayed more than one different type of allergic reactions, only the most severe reactions were considered. All drugs triggering allergy alerts were mapped to ATC codes (Anatomical Therapeutic Chemical Classification System, WHO, Geneva, Switzerland) in order to enable grouping of drugs into drug classes. In the inpatient setting, a research assistant (NM) mapped each of the free-text entries given by providers to justify their decision to override the alert (n=20 699) to one of 30 individual categories. Each category contained a group of words or phrases with similar meaning or connotations, and included the coded reasons in the outpatient setting for comparison (eg, patient has taken previously without allergic reaction (category 2), low-risk cross-sensitivity, will monitor (category 3), no reasonable alternatives (category 5)); further details can be found in table 3. Words and phrases that could not be grouped were classified under ‘other’ (category 30). DLS provided a second check of this classification process. Descriptive statistics were computed using the software R, V.3.0.2 (R Foundation for Statistical Computing, Vienna, Austria).
Results
Summary
A total of 158 023 drug allergy alerts were displayed, 131 615 (83%) in the inpatient setting and 26 408 (17%) in the outpatient setting, over the 3-year time period; 128 157 (81%) of which were overridden. These alerts were displayed for a total of 29 420 patients across both settings. The drug classes ‘opioids’, ‘non-antibiotic sulfonamides (without coxibs)’, which included drugs like furosemide (Lasix) and indapamide (Lozol), and ‘non-antibiotic sulfonamides (coxibs)’, which included drugs like celecoxib (Celebrex), were overridden over 81% of the time in both settings. The most common reason for overriding drug allergy alerts in both settings was patient has taken previously without allergic reaction. However, in a random sample of inpatient (n=200, 0.19%) and outpatient (n=50, 0.25%) alert overrides, >96% (n=240) of alert overrides in both settings were considered appropriate. In the inpatient setting, 70.9% (n=1682) of alerts that displayed anaphylaxis were overridden compared with 56% (n=130) in the outpatient setting.
Number, type and overrides of drug allergy alerts
A total of 131 615 drug allergy alerts were displayed in the inpatient setting over the study time period. The provider indicated that the patient did not have this allergy for 2% (n=2602) of these alerts. The overall alert override rate was 81.9% (n=107 812) in the inpatient setting. Table 1 describes the top drug classes that were overridden. The override rate for monoclonal antibodies was 98.1% (n=51) and included for comparison with the top drug classes in table 1. The number of alerts overridden in the drug class ‘opioids’ was 87.2% (n=58 348), followed by the class ‘non-antibiotic sulfonamides (without coxibs)’ at 84.4% (n=514) and ‘non-antibiotic sulfonamides (coxibs)’ at 81.6% (n=129). The override rate for ‘sulfonamide antibiotics’ was far lower than other sulfonamides (59.6%, n=627). The ‘other drugs’ category contained individual drugs that were collectively overridden 74.5% (n=24 463); drug allergies triggered for the top five ‘other drugs’ included paracetamol (4.4%, n=4375), metoprolol (1.6%, n=2168), prochlorperazine (1.1%, n=1444), simvastatin (0.9%, n=1154) and rosuvastatin (0.6%, n=826). The drug class ‘opioids’ generated the highest number of total orders in the study period and were the second highest class to trigger an alert; orders for non-antibiotic sulfonamides triggered the highest percentage of alerts.
Top drug classes that were overridden in the inpatient setting
In the outpatient setting, the number of drug allergy alerts displayed (n=26 408) was only a fifth of those displayed over the same time period in the inpatient setting (n=131 615). When compared with the total number of orders placed in both settings, drug allergy alerts were triggered less frequently in the outpatient setting (1.3%) than the inpatient setting (2.45%). The providers indicated that the patient did not have this allergy, will discontinue pre-existing allergy in >5% (n=1418) of cases. Overall alert override rate was statistically lower in the outpatient setting (77%, n=20 345), p<0.0001. Table 2 describes the top drug classes that were overridden in the outpatient setting. There was only one alert for monoclonal antibodies, which was not overridden. The alert override rate for ‘non-antibiotic sulfonamides (coxibs)’ (87.2%, n=157) was higher than for ‘non-antibiotic sulfonamides (without coxibs)’ (81.9%, n=521), which differed from the inpatient setting. The top five drugs triggered the most frequently in the ‘other drugs’ category were pravastatin (4%, n=1046), guaifenesin (3.8%, n=998), rosuvastatin (3.7%, n=983), azithromycin (3.1%, n=827) and simvastatin (3.1%, n=821); all drugs in this category were collectively overridden 79.3% (n=11 092). Similar to the inpatient setting, the drug class ‘opioids’ also generated the highest number of total orders in the study period. Drug allergy alerts for this drug class were triggered less frequently in the outpatient setting (20.6%) than the inpatient setting (50.9%). The drug class ‘non-antibiotic sulfonamides’ was also found to trigger the highest number of alerts per percentage of orders (10.67%) in the outpatient setting. The percentage of orders for the drug class ‘cephalosporins and other beta-lactams’ triggered a much higher number of alerts in the inpatient setting (22.3%) than the outpatient setting (8.8%).
Top drug classes that were overridden in the outpatient setting
Reasons for alert overrides
In the inpatient setting, the most common free-text reason for overriding drug allergy alerts was patient has taken previously without allergic reaction (category 2, 57.4%, n=61 858), followed by physician aware (category 7, 17.2%, n=18 583) and low-risk cross-sensitivity, will monitor (category 3, 12.3%, n=13 202) (table 3). In 1386 (1.3%) of cases, the provider commented that they were administering the drug as per the desensitisation protocol (category 17, 0.9%, n=927) or that they were testing or trialling the drug on the patient (category 19, 0.4%, n=459). In 224 (0.2%) of cases, the provider wrote that they would administer slowly (category 11, 0.1%, n=112), discontinue the drug (category 12, 0.05%, n=57), will discontinue if the patient experiences an adverse reaction (category 13, 0.03%, n=37) or adjust dose (category 9, 0.02%, n=18). According to providers, the patient either explicitly requested the drug or agreed to the drug being prescribed in 46 cases (category 28, 0.04%).
Reasons given by providers for overriding drug allergy alerts
Similarly in the outpatient setting, the most common coded reason given was patient has taken previously without allergic reaction (56.9%, n=11 594), followed by low-risk cross-sensitivity, will monitor (27.9%, n=5685) and no reasonable alternatives (4.9%, n=986). Providers chose the coded reason other in 10% (n=2037) of cases (table 3) and typed the free-text reason ‘OK’ (indicating that the prescriber understood the alert) in 0.1% (n=20) of cases. In the outpatient setting, a far lower number of providers commented physician aware (0.11%, n=23), that they were administering the drug as per the desensitisation protocol (0.08%, n=17) or that they were testing or trialling the drug on the patient (0.07%, n=14). Almost the same percentage of providers in the outpatients setting as in the inpatient setting stated that they would discontinue the drug (0.04%, n=8) or discontinue if the patient experiences an adverse reaction (0.03%, n=7).
Appropriateness of alert overrides
In the inpatient setting, 96.5% (193/200) of the chart-reviewed alert overrides were judged to be appropriate on first review. Of the 193 alerts that were overridden appropriately, 3 displayed the reaction ‘anaphylaxis’. Of the seven specific alert overrides that were considered by the two independent reviewers to have been possibly inappropriately overridden, three were discussed with a third reviewer (DWB) before reaching final agreement; an inadequate explanation was provided in the free-text field for one of these overrides, and reviewers were unable to verify on chart review whether the patient received the drug and had an allergic reaction for a second of these overrides. Consensus was reached that a total of seven alert overrides were judged to be inappropriate in the inpatient setting.
In the outpatient setting, 94% (47/50) of alert overrides were considered appropriate. Three specific alert overrides were discussed with a third reviewer (DLS) before reaching final agreement; in each of these cases, the provider chose the reason ‘other’ and provided no free-text explanation; the third reviewer was unable to verify on chart review whether the patient had received the drug already without having an allergic reaction. Consensus was reached that a total of three alert overrides were judged to be inappropriate in the outpatient setting.
Types of reactions
The most common type of reaction displayed in the alert warnings in the inpatient setting related to the skin (27.4% of the total), which included rash (14.6%, n=19 172), hives (7.1%, n=9353) and itching (5.7%, n=7547). Over 21% of alerts (n=27 650) indicated some kind of gastrointestinal upset and were overridden in 85.3% (n=23 599) of cases. Table 4 describes the top 10 reactions that were displayed to providers and the override rates of these specific types of potential reactions. The percentages shown in columns 2 and 4 show the fraction of how many alerts warned against the risk of the respective reaction in the inpatient and outpatient settings, respectively. The percentages shown in columns 3 and 5 present the fraction of how many of the respective alerts were overridden by the providers in the inpatient and outpatient settings, respectively. The number of alerts that displayed shortness of breath and anaphylaxis in the inpatient setting was 2541 (1.9%) and 2373 (1.8%), respectively, and these were overridden in 81.2% (n=2063) and 70.9% (n=1682) of cases.
Top 10 reactions displayed to providers and the override rates of these specific types of potential reactions
In the outpatient setting, >22.8% of alerts (n=6009) displayed gastrointestinal upset and 81.9% (n=4902) of these were overridden. Skin reactions were also commonly displayed (19.6%, n=5167); these included rash (11.2%, n=2956), hives (5.1%, n=1353) and itching (3.2%, n=858), and were overridden on average 66% (n=3413). The number of alerts that displayed shortness of breath and anaphylaxis was far lower in the outpatient setting and overridden much less 70.8% (n=289) and 56% (n=130), respectively.
Discussion
Over the last 20 years, there has been an increase in the drug allergy alert override rate at this hospital, from about 50% in 199516 to 81.9% (inpatients) and 77% (outpatients) as found in this study. These high override rates are similar to those of previous studies that analysed orders from the same and different hospitals,16–18 although a little lower than the 91% override rate found by Weingart et al13 in the outpatient setting. Over 96% of the chart-reviewed alert overrides in this study were judged to be appropriate. Overall, these data suggest that better documentation and updating of allergy records are needed in patient's health records to ensure that these are correct and up to date, and further studies are needed to improve the sensitivity and specificity of allergy alerts, thus helping to specify which alerts should be turned on and which should be turned off.
A few classes of drug accounted for a large proportion of this high drug allergy override rate, suggesting it could be readily addressed. Opiates in particular were one of the most frequently ordered and overridden drug classes, similar to other studies.11 ,17 A random sample of alert overrides in both settings was reviewed, and >94% were considered appropriate. Cross-reactivity in particular is an issue; many patients have a reaction to one opiate, but it is not clear whether providers need additional warnings when others are given. Hsieh et al also reviewed the charts of a random subset of patients and found that none of the adverse drug events attributed to the overridden drug alert was preventable.17 The authors concluded that clinical need for the drug outweighed the risk of a serious allergic reaction, thus the drug allergy alert overrides were deemed clinically justifiable.
Providers must ensure that patient self-reported allergies or intolerances are documented correctly in the electronic patient record and frequently updated. Many different types of providers can usually enter this information into the system at the time of the patient's visit.13 Hsieh et al17 found that the degree of completeness of patients’ allergy lists was generally poor in the CPOE system. The accuracy of the information included in these lists was also questionable, and it was often unclear whether the specific reactions were IgE or non-IgE-mediated. Often, opioid side effects are recorded as allergies.11 This is because the reporting of allergies and intolerance and adverse effects is handled through the same mechanism. This may be acceptable, but applications should ask providers to distinguish between them when they are being reported. Hsieh et al17 also found that nausea and vomiting, which are not true IgE-mediated reactions, were the most common adverse drug reactions documented to narcotics in the patient's allergy list. Further, in our study, the provider indicated that the patient (for whom a drug allergy alert was triggered) did not actually have the allergy in 4020 (2.5%) of cases. Providers overrode more than half of all drug allergy alerts giving the reason that the patient has taken previously without allergic reaction, with the scenario most often being that the patient had an allergy recorded to a drug but the patient was taking it without difficulty. Similar findings were reported in another study conducted in the inpatient setting with providers choosing the same reason (49%) when overriding drug allergy alerts.19 On selecting this override reason, it should be possible to encourage providers to deactivate allergies by providing an automatic link for providers to update the drug allergy list. Additional studies are also needed to look at the more efficient knowledge management of the drug allergy databases.
One of the most common override reasons given in both settings was low-risk cross-sensitivity, will monitor. Previous studies have revealed how the majority of drug allergy overrides (90%) occurred when the allergen and drug belong to the same family but were not identical (eg, codeine and hydromorphone).17 For some of the non-exact drug allergy matches presented in the study by Hsieh et al, the likelihood of an adverse reaction occurring to the patient on administration of the drug was felt to be extremely low. For example, the drug classes ‘non-antibiotic sulfonamides’ were overridden >81% of the time in both inpatient and outpatient settings. Strom et al20 conducted a retrospective cohort study examining patients who previously had an allergic reaction to a sulfonamide antibiotic and found that <10% of these patients had an allergic reaction to a sulfonamide non-antibiotic (eg, celecoxib). The authors concluded that this association appeared to be due to a predisposition to allergic reactions in general rather than to cross-reactivity with sulfonamide-based drugs. In other words, these patients were no more likely to have a subsequent hypersensitivity reaction to a drug containing a sulfonamide functional group (eg, furosemide) than to a drug that did not (eg, penicillin).20 This highlights the importance of incorporating clinical evidence on drug cross-reactivity into the decision support system rather than relying on pharmacological or structural similarity.21
Over 70% of the drug allergy alerts that displayed the reaction ‘anaphylaxis’ in this study were overridden. This is a worrying finding as a true allergic reaction severe enough to cause anaphylaxis will most often occur within minutes or hours of exposure. It is unclear from our study whether any of these patients (for whom an alert was triggered) had previously experienced such a reaction or went on to experience any adverse events as a result of taking the prescribed drug (beyond the scope of this study). Identification of patients with a history of anaphylaxis is important as over one-third of these patients could suffer a serious recurrence.22 However, with recent figures suggesting that the rate of anaphylaxis hospitalisations is on the rise in both the UK and the USA,23 ,24 further research is needed to explore why providers chose to override these alerts. In the inpatient setting, Topaz et al14 reported how only about one-tenth of the alerts for potentially life-threatening reactions (eg, anaphylaxis) were based on an exact match between the allergy and the prescribed drug, while the rest were based on either the cross-sensitivity or allergy group. These findings further suggest that patients’ medical records may contain inaccurate allergy information or the clinical evidence on drug cross-reactivity may need to be updated. This may be of particular importance to paediatric patient populations whose drug allergy prevalence varies substantially from adults.25
This study was undertaken within a single healthcare delivery system using two different homegrown prescribing systems (BICS and LMR) and, as such, is difficult to assess how generalisable the results are to other sites. However, our proportions are comparable with the results from different sites published by other research groups. The random sample of alert overrides reviewed for appropriateness was small compared with the total number of alerts triggered, and the accuracy of this review was reliant on the completeness of information contained in the patients’ charts. We did not control for the possibility that some alerts may have been repeated for the same patient. This paper presents the reasons why providers chose to override drug allergy alerts. Although providers in the inpatient setting enter this reason in the free-text field provided, providers in the outpatient setting selected one of the coded reasons provided. Notwithstanding these limitations, we believe that our findings have important implications for allergy alerting in other systems.
Conclusion
Drug allergy alerts are one of the most frequently displayed alerts in CPOE systems. There is little evidence, however, regarding the efficacy of these alerts. Most of the drug allergy alerts in this study were overridden, with >94% of a subsample of alert overrides considered appropriate. Some of the rules for these alerts should be carefully reviewed and modified, or removed. It is very important that the information contained in patients’ drug allergy lists is correct and up to date, and clinicians are encouraged to deactivate allergies by providing them with an automatic link to update their patients’ drug allergy lists. The clinical evidence on drug cross-reactivity in the decision support systems should be updated. Finally, we found that providers overrode the majority of drug allergy alerts, including those that warned against the risk of ‘anaphylaxis’. More needs to be done to understand providers’ reasoning in overriding such alerts and reduce the possible threat to patient safety.
References
Footnotes
↵* SPS and PEB contributed equally and are joint first authors.
Twitter Follow David Bates at @dbatessafety
Contributors DWB conceived this study. DLS, MA, IC and KCN reviewed the patient charts, JMF extracted the data, and PEB processed these data and conducted the analyses. SPS led the writing of this manuscript with all coauthors commenting on subsequent drafts. All authors gave their approval for the final version to be published. SPS and DWB act as guarantors.
Funding Agency for Healthcare Research and Quality (AHRQ) (#U19HS021094). Dr Beeler was supported by the Swiss National Science Foundation.
Competing interests None declared.
Ethics approval The necessary approvals were obtained to access these data from the Partners Human Research Committee (PHRC), which is the Institutional Review Board (IRB) of Partners Research Management at Partners HealthCare.
Provenance and peer review Not commissioned; externally peer reviewed.