Article Text


Intended and unintended effects of large-scale adverse event disclosure: a controlled before-after analysis of five large-scale notifications
  1. Todd H Wagner1,2,
  2. Thomas Taylor2,
  3. Elizabeth Cowgill2,
  4. Steven M Asch2,3,
  5. Pon Su2,
  6. Barbara Bokhour4,5,
  7. Janet Durfee6,
  8. Richard A Martinello6,7,
  9. Elizabeth Maguire4,
  10. A Rani Elwy4,5
  1. 1Center for Innovation to Implementation, VA Palo Alto Health Care System, Menlo Park, California, USA
  2. 2Department of Health Research and Policy, Stanford University, Stanford, California, USA
  3. 3Division of General Internal Medicine, Stanford University, Stanford California, USA
  4. 4Center for Healthcare Organization and Implementation Research, Edith Nourse Rogers Memorial Veterans Hospital, Bedford, Massachusetts, USA
  5. 5Department of Health Policy and Management, Boston University School of Public Health, Boston, Massachusetts, USA
  6. 6Veterans Health Administration, Office of Public Health, Washington DC, USA
  7. 7Yale School of Medicine, Departments of Internal Medicine and Pediatrics, New Haven, Conneticut, USA
  1. Correspondence to Dr Todd H Wagner, Center for Innovation to Implementation, VA Palo Alto Health Care System, 795 Willow Rd, Menlo Park, CA 94025, USA; todd.wagner{at}


Background and objective How patients respond to being notified of a large-scale adverse event (LSAE), such as improper sterilisation of medical equipment that exposes them to bloodborne pathogens, is not well known. The objective of this study was to determine, using administrative data, the intended and unintended consequences of patient notification following a LSAE.

Methods We examined five LSAEs where patients may have been inadvertently exposed to hepatitis C virus (HCV), HIV, and hepatitis B virus (HBV). A total of 9638 cases were identified at five Department of Veteran Affairs (VA) medical facilities between 2009 and 2012. We identified controls at the same facility prior to the exposure period and at neighbouring facilities (n=45 274). Difference-in-differences models were used with Veterans Health Administration (VHA) and Medicare data to examine infectious disease testing rates and subsequent utilisation patterns.

Results Receipt of a LSAE notification was associated with a 73.2, 76.8 and 77.1 adjusted percentage point increase for HCV, HIV and HBV testing, respectively (all p<0.001). Compared with white patients, African–American patients were significantly less likely to return to VHA for follow-up testing. Patients exposed to a dental LSAE reduced their use of preventive and restorative dental care over the subsequent year, but they eventually came back to VHA for dental services 18-months post exposure.

Conclusions The majority of patients notified of a LSAE responded by getting tested for HCV, HIV and HBV, although there remains room for improvement. Potential exposure to a LSAE was associated with increased odds of subsequently using non-VA facilities, but the size and timing of the shift depended on the type of care.

  • Communication
  • Decision making
  • Healthcare quality improvement
  • Health services research
  • Patient safety

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See:

Statistics from


Communicating medical or healthcare system errors to patients in a way that guides them to appropriate action is one of the more difficult challenges in medicine. Such discussions are particularly challenging in the case of large-scale adverse events (LSAEs), in which patients are exposed to an adverse event that raises the risk of infection. Frequently clinicians must provide accurate and timely information to many patients, with whom most have no prior relationship.1 Often the increased marginal risk from a LSAE is extremely low; this can complicate the discussion as patients (and clinicians) often struggle to present and understand risks, particularly in the face of uncertain information.2

Developing a rapid response system for communicating LSAEs is increasingly important for healthcare organisations because patients value notification, even when the risk is small.3 Failure to do so raises the risk that the information will first reach individuals through other people in their social network and the news media.4 Given the proliferation of social media, and the ability for news to ‘go viral’, the US Centers for Disease Control and Prevention has built a toolkit to help healthcare providers manage the notification process, including best practices to keep patients apprised of new information and managing media communications.5

As organisations develop best practices, there are often questions about how to measure the damage created by a LSAE, or conversely the benefit that better communication about LSAEs might engender. Analysing patient behaviour through administrative claims databases is common in healthcare, including studies on health insurance design,6 but this method has never been used to examine how patients respond to a LSAE notification.

We chose to analyse patient behaviour after LSAE notification in the Veterans Health Administration (VHA), because the VHA systematically looks for LSAEs, tracks potentially exposed patients and communicates with them after LSAE notification.7 VHA policy instructs facility directors and clinicians to “provide factual information to the extent it is known, express concern for the patient's welfare, and reassure the patient or representative that steps are being taken to investigate the situation, remedy any injury and prevent further harm”.7 Many of these steps are part of the essential aspects of medical apologies8 and represent patient-centred care.9

Over the past 5 years, the VHA has notified patients in multiple LSAEs involving potential inadvertent exposure to infectious agents. In the events studied, all notification letters were signed by the local VA Medical Center Director and sent by mail to the potentially affected patients. The letters explained the general situation, that improperly cleaned equipment was found during a quality inspection and that testing for hepatitis C virus (HCV), HIV, and hepatitis B virus (HBV) was available. The letters included reassurances that the bloodborne pathogen testing and results would be timely. The letters also provided reassurance that the VA is working to address the current issue and prevent future issues. Letters varied in their use of apologies, ranging from an indirect apology (ie, “regret that this situation occurred”) to one in which a Director stated “let me sincerely apologise to you and let you know that I understand this letter may cause you concern”. A dedicated, toll-free number, which was available 24 h a day, 7 days a week, was provided in the letters, should patients or families have more questions about the events or wish to schedule a testing appointment.

We used administrative data to examine several of these events to address four related questions: First, when Veterans receive a notification about personal exposure from a LSAE, including instructions to get tested for a possible infection, do they obtain follow-up testing? Second, when Veterans are notified about an exposure at VHA, are they less likely to return to the same facility to receive the same type of care in the future? Third, when Veterans are notified about an exposure, does this affect their future decisions to return to VHA for any care? Finally, are Veterans over age 65 years and covered by Medicare more likely to switch to a non-VA provider after notification?


Potentially exposed patients

We identified six LSAEs disclosed to patients between 2009 and 2012. Each of these involved potential exposure to bloodborne pathogens. Because of the sensitivity of these events, facility names were removed to protect employees, patients and family members impacted by the events from further public attention. Exposures at Sites 1 and 2 were a result of improperly reprocessed ear, nose and throat endoscopes. Exposures at Sites 3 and 4 were a result of improper disinfection of auxiliary water tubing system used during reprocessing of the colonoscope. Finally, exposures at Sites 5 and 6 were related to two dental care exposures involving improper infection control practices and techniques performed by a particular dentist. All six of these exposures were formally investigated by the VHA Office of the Inspector General (OIG) with detailed chart reviews to identify those at risk, followed by patient notification.

We did not have access to the list of patients who were notified; we identified cases and controls based on exposure dates, clinic and Current Procedural Terminology (CPT) procedure codes using VHA administrative data. There are limitations with this method as the official look-back investigations used the full electronic medical records. In some cases, the official investigations deemed patients not at risk based on details in the electronic medical records that were not in the administrative records. Therefore, we compared the identified cases with those reported by the VHA OIG reports. In three sites, our cohorts were very similar in size to those reported by OIG (8603 in our cohort vs 8734), while in the smallest exposure site, we identified 378 patients and found no corresponding OIG number. Notable discrepancies between our cohort and the OIG report were found in two sites: at Site 2, our sample was smaller than the OIG report (675 vs 1069), while at Site 4 our sample was larger than the OIG report (6189 vs 3260; table 1). The misclassification in Site 4 adds noise to the analysis, therefore, we excluded Site 4 from the main analysis; sensitivity results showed that the findings held when Site 4 was included albeit with attenuated effect sizes for infectious disease testing. The undercount of cases in Site 2 does not create the same problems as Site 4. Therefore we included Site 2 in the study; we excluded it in a sensitivity analysis and the results were robust and remained qualitatively similar.

Table 1

Exposure and analytical sample size

Two sets of control patients were identified. First, we identified patients who received the same services at the exposure sites prior to the exposure (ie, a pre-exposure control group). We also identified controls at other VA medical centres, during the exposure period and prior to the exposure period. These control sites were chosen based on having a similar clinic volume and geographic proximity to the exposure site while being in a separate media market. The exposure sites, control sites and sample sizes are shown in table 1; the 54 912 observations represent 51 648 individuals because 3264 persons were present in the pre and post periods.


To determine whether notified patients returned for infectious disease testing, we identified HCV, HIV and HBV tests based on CPT codes within 1 year following the LSAE notification date. We identified receipt of testing at VA facilities, at non-VA facilities paid by VA and at non-VA facilities paid by Medicare. We also examined timing of testing, measured as the number of days between notification and testing. Use of Medicare services was identified by merging VA and Medicare records (MedPAR, Carrier and Outpatient files) from the Centers for Medicare and Medicaid Services. VA only provides confidential HCV, HIV, HBV testing, and there is no option for anonymous testing that might lead to missing data.

Being notified of an exposure may result in diverse patient reactions ranging from a reluctance to seek the same type of care again, to switching healthcare providers, when possible. To determine if patients were less likely to return for the same care at the same facility, we focused on patients who were exposed while receiving dental care at Site 5. We examined return rates to the dental clinic in the 18 months after notification, separated into months 0–6, 7–12 and 13–18. We used dental CPT codes to differentiate preventative, restorative and any other dental care. We did not examine return visits for routine colonoscopies as these are generally recommended every 10 years.

To examine whether patients responded to the notification by using less VA care and more Medicare, we followed patients for 1 year. We identified, by clinic, six types of VA outpatient services. We examined 10 types of Medicare services (7 outpatient, 3 inpatient) based on the place of service from the MedPAR, Carrier and Outpatient files. In VA and Medicare files, multiple visits to the same clinic on the same day were considered one visit. All VA and Medicare records that had a CPT code for HCV, HIV or HBV testing were excluded to isolate services not in direct response to exposure notification. We segmented the population into younger (<age 65 years) and older patients (≥age 65 years) to account for age-related Medicare eligibility. Of particular interest is the role of ambulatory surgery because dually eligible patients can play an active role in choosing where to get care and the timing of care.


We used a difference-in-differences analysis to compare outcomes between cases and controls. Statistical significance was tested in multivariate models, while controlling for age, gender, race/ethnicity and marital status. We also included a dummy variable for each LSAE site to control for any idiosyncratic difference across sites. All of these covariates were obtained from the VA Vital Status File, and 22% of the race/ethnicity data were missing or coded as unknown (table 2).

Table 2

Patient characteristics*

Receipt of infectious disease testing was estimated using logistic regression. Use of health services was estimated using two-part models: logistic regression for the probability of using care and then linear regression models for the volume of care, conditional on any use. Given some people were in the pre and post periods, the SEs were adjusted for patient-level clustering. Given the large literature on racial disparities in healthcare, we examined the results for a possible racial/ethnic interaction. The race/ethnicity data were missing or unknown in 22% of the cases, so we focused on the two largest groups (African-American and white Veterans). All statistical analyses were performed with Stata V.13 (StataCorp), and the multivariate models adjusted for patient-level clustering. The research protocol was approved by the Stanford University Institutional Review Board.



A majority of the sample was male (>90%), non-Hispanic white between 45 years and 64 years of age (table 2). VA enrolment grew over the decade, as was evident in the number of Veterans receiving services at all VA sites. All of the differences between the exposure and control sites in the exposure period were statistically significant (p<0.05).

Rate of infectious disease testing

Patients who received an exposure notification were much more likely to obtain HCV, HIV and HBV testing than controls. In the exposure sites, HCV, HIV and HBV testing increased by 72.6, 76.3 and 76.3 unadjusted percentage points, respectively (table 3). These effects were highly significant and translated to large adjusted ORs (AOR) of 49.7 (95% CI 41.2 to 60.0), 103.8 (95% CI 78.1 to 137.9) and 88.4 (95% CI 70.4 to 110.0), for HCV, HIV and HBV testing, respectively. This translated into adjusted percentage point increases of 73.2, 76.8 and 77.1 for HCV, HIV and HBV testing, respectively.

Table 3

Rates of infectious disease testing before and after exposure

Most Veterans (>98%) received follow-up testing at a VA facility; little testing occurred at non-VA facilities, irrespective of whether it was paid by VA or Medicare (data not shown). Among patients who received an exposure notification and sought testing, 51% were tested within 30 days following the notification and 74% were tested within 60 days.

Compared with white Veterans, African-American Veterans were significantly less likely to return to VHA for follow-up testing. In the logistic regression models, African–American Veterans had an adjusted odds of receiving HCV, HIV and HBV tests of 0.74 (95% CI 0.61 to 0.89), 0.46 (95% CI 0.37 to 0.56) and 0.66 (95% CI 0.54 to 0.81), respectively, compared with white Veterans (table 3).

Returning for the same type of care

In the 12 months following notification, there was a decrease in return visits for dental services at Site 5, but the use of dental services rebounded to pre-exposure levels between 13 months and 18 months (table 4). Patients exhibited changes in their use of preventative and restorative dental care services, but not other dental care. There were no statistically significant differences in dental care rates between African–American and white Veterans.

Table 4

Adjusted odds of returning for dental care following a large-scale dental notification (Site 5)

Switching providers

Among older Veterans, notification was associated with higher odds of many outpatient VA services in the 3 months after the notification and then decreased odds in VA utilisation in the subsequent 9 months (table 5). The increase in services in the first quarter after notification cannot reflect infectious disease testing because we excluded any care where an HCV, HIV or HBV test was ordered. However, it is possible that this spike in care was contaminated by care addressing LSAE-related issues that did not result in a laboratory test, for example, counselling or discussing test results.

Table 5

Use of health services associated with a large-scale notification

None of the LSAEs were related to ambulatory surgery and it is unlikely that patients would use ambulatory surgery for infectious disease counselling. Therefore, this is plausibly an uncontaminated measure in which to look at switching behaviours. Here we see strong and consistent results suggesting that Veterans who received a notification were more likely to hold off from getting ambulatory surgery at VA, as notification was associated with significantly lower odds of obtaining ambulatory surgery in VA. For patients over 65 years, the data suggest that many patients switched providers as notification was associated with higher odds of using a Medicare provider for ambulatory surgery (AOR=2.05, 95% CI 1.229 to 3.418) and a decreased odds of obtaining surgery at VA (AOR=0.75, 95% CI 0.574 to 0.983). Table 5 shows the ORs for ambulatory surgery along with the quarterly trends post notification.

Among Veterans over age 65 years, notification was associated with increased odds of using a Medicare provider for outpatient care. The adjusted odds were often largest in the first 3 months after the notification. There was little evidence that notifications were associated with increased odds of using inpatient services, either VA or Medicare. Few differences between African-American and white Veterans were observed in patterns of healthcare utilisation associated with the notification.


More than two-thirds of the potentially exposed patients returned for HCV, HIV and HBV testing following receipt of a notification letter. Receipt of notification was associated with a 72–76% point increase in HCV, HIV and HBV testing. Among those who sought testing, 56.8% were tested in the 30 days following the notification and 74% were tested within 60 days. The vast majority (>98%) of the testing was completed at VHA facilities. The results suggest that existing communication strategies are successful in guiding many patients to remedial action after an LSAE. However, additional follow-up communication may be needed for 60-day non-responders.

Healthcare organisations are very sensitive to the possibility that an LSAE can damage their reputation. One important measure of this harm is whether patients ‘vote with their feet’ and seek subsequent care elsewhere. We tested whether LSAEs were associated with reduced odds that patients return for the same type of service. In our analysis of dental care following a dental LSAE, we found decreased odds of using preventative and restorative care in the first 12 months, but by 18 months the odds of using dental care reverted to baseline levels.

Receiving a LSAE notification was also associated with other types of utilisation unrelated to the exposure, but these effects varied by time and whether the patient had another source of insurance. Notification was associated with a decrease in VA services for Veterans over age 65 years and an increase in Medicare services. The odds of using Medicare services were highest in the 6 months following notification and then the odds generally reverted to baseline levels between 9 months and 12 months following notification. This shift in providers could create potential challenges in caring for chronically ill patients as discontinuity and fragmented care patterns might impact the quality of care received during the transition.10

African–American patients had lower rates of testing than white patients, raising the question, could foundational levels of trust and/or experiences of discrimination moderate successful remediation of these adverse events? African-American patients have been shown to have lower levels of trust in the medical care system.11 A recent study found that even after controlling for race/ethnicity, language and other sociodemographic variables, trust in primary care providers was the only significant predictor of colorectal cancer screening completion in low-income patients.12 Some factors, such as those related to racial differences in treatment preferences are considered fixed, and others, such as patients’ trust in their providers, are viewed as modifiable factors that can alleviate differences in patients’ health behaviours.13 Hence, it seems that interventions to increase African-American patients’ trust in the VA and VA providers should be considered when trying to achieve higher screening and testing rates. The validity of our finding is threatened by high rates of missing race data in our sample,13 however this issue is not unique to VHA and remains an issue in other administrative data sets.14 ,15 Validation studies have demonstrated that non-Hispanic white and African-American race data are reasonably representative of patient self-perceptions.16 These caveats aside, these data suggest that focused outreach efforts for minorities may be warranted after an LSAE.

This racial difference highlights the fact that we know too little about how to communicate to patients about an LSAE,17 especially if prior levels of trust and/or experiences of discrimination affect how a patient responds to a notification. African-American patients are more likely to trust health messages when delivered via the media compared with white patients,18 suggesting that how healthcare systems communicate information on LSAEs needs to be tailored to different racial and ethnic populations. Given that African-American patients trust providers less, perhaps media, including social media, messages can be used for communicating LSAEs to vulnerable populations.

Few studies examine patient trust following LSAE notification, but the limited evidence that does exist suggests that patients want to be informed of possible infection even though this will increase their anxiety. In a recent survey of patients affected by LSAE notification at an academic hospital (n=119), 75% felt that the notification letter provided information that patients needed to understand the endoscope sterilisation event, 64% were somewhat to very concerned about potential health problems that might arise from this event, and yet 81% still thought the medical centre was right to inform patients of the potential risks following this event.3 ,19 Assuming Veterans share these preferences, VHA's policy on LSAE notification is appropriate and the next step is to determine the optimal patient-centred communication strategies for disclosing the event.

This study provides novel insights on the intended and unintended effects of LSAE notifications. However, there are some limitations that warrant discussion. First, our cohort might exclude notified patients or include unaffected patients, because we did not have the list of specific patients who were notified. Patients are notified after an extensive chart review,20 and the OIG reports summarised the timing and procedures linked to the LSAE, along with the number of patients sent notification letters. Using this information, we constructed our cohorts across six LSAEs. We excluded one LSAE from the analysis because our cohort was much larger than the number of patients reported by the OIG. Sensitivity analysis showed similar results when we included this sixth site, although there were lower rates of HCV, HIV and HBV testing, as expected. We included a fifth site where the numbers from OIG were higher than was observed in the data. A sensitivity analysis with this fifth site found that the results were robust. A second limitation is the lack of non-VA data for patients under age 65 years. Although many of these Veterans only have VA coverage, some have alternative coverage through employers, unions and Medicaid, which we cannot identify. Given these limitations, we think the results generalise to older adults in the USA, and possibly other countries, who are notified of an adverse event. Finally, we tracked patient HCV, HIV and HBV testing for 1 year after notification. It is possible that we missed people who obtained testing outside this window, although only 0.09% (n=9 of 9638) were tested after 330 days so the probability of missing tests after 1 year seems unlikely.

In conclusion, while more than two-thirds of the potentially exposed patients returned for HCV, HIV and HBV testing following receipt of a notification letter, these data suggest that patients were less likely to return to VA for the same type of care that led to the exposure. Lower rates of returning for dental care tended to be greatest immediately following the notification and then utilisation reverted to baseline rates over a period of 12–18 months. The data also show that reductions in use of VA care unassociated with the exposure tended to be greatest between 3 months and 12 months, suggesting that some patients vote with their feet and shift providers.


View Abstract


  • The data in this manuscript were presented at the 2014 Academy Health Conference, San Diego, CA, USA.

  • Contributors All authors contributed to this work. All authors contributed to and approved the final manuscript.

  • Funding This study was supported by the Department of Veterans Affairs, Health Services Research and Development Service grant (grant number SDR 11-440).

  • Competing interests None.

  • Ethics approval This study was approved by the Stanford University IRB and the Palo Alto VA Health Care System Research and Development Committee.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Disclaimer The views, opinions, and content of this publication are those of the authors and do not necessarily reflect the views, opinions, or policies of the Department of Veterans Affairs of the United States Government.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles