Article Text
Abstract
Background The Accreditation Council for Graduate Medical Education (ACGME) enacted a policy in 2011 that restricted first-year resident physicians in the USA to work no more than 16 consecutive hours. This was rescinded in 2017.
Methods We conducted a nationwide prospective cohort study of resident physicians for 5 academic years (2002–2007) before and for 3 academic years (2014–2017) after implementation of the 16 hours 2011 ACGME work-hour limit. Our analyses compare trends in resident physician-reported medical errors between the two cohorts to evaluate the impact of this policy change.
Results 14 796 residents provided data describing 78 101 months of direct patient care. After adjustment for potential confounders, the work-hour policy was associated with a 32% reduced risk of resident physician-reported significant medical errors (rate ratio (RR) 0.68; 95% CI 0.64 to 0.72), a 34% reduced risk of reported preventable adverse events (RR 0.66; 95% CI 0.59 to 0.74) and a 63% reduced risk of reported medical errors resulting in patient death (RR 0.37; 95% CI 0.28 to 0.49).
Conclusions These findings have broad relevance for those who work in and receive care from academic hospitals in the USA. The decision to lift this work hour policy in 2017 may expose patients to preventable harm.
- patient safety
- duty hours/work hours
- medical error, measurement/epidemiology
Data availability statement
Data are available on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Key messages
What is already known on this topic
Work-hour limitations for first-year resident physicians have been enacted and then rescinded in the USA, with studies showing mixed benefit depending on the outcome of interest.
No study has examined the impact of the 16 hours 2011 Accreditation Council for Graduate Medical Education work-hour limitation on resident physician-reported patient safety outcomes for patients under their direct care.
What this study adds
The national policy limiting first-year resident physicians to work no more than 16 consecutive hours was associated with substantial reductions in harmful and fatal resident-reported medical errors.
How this study might affect research, practice and/or policy
Policy-makers in the USA should consider limiting the work hours of resident physicians to protect patient safety.
Introduction
Resident physicians’ work hours have been a subject of controversy in the USA for more than two decades.1 From 2003 to 2011, the Accreditation Council for Graduate Medical Education (ACGME) limited residents in their first postgraduate year to a maximum of 30 consecutive work hours, including 6 hours for continuity of care and educational activities (30 hours 2003 ACGME work-hour limit).2 Subsequent evaluations found that shifts of 24 or more hours were associated with increased odds of fatigue-related medical errors and preventable adverse events (PAEs),3 percutaneous injuries4 and motor vehicle crashes.5 A randomised controlled trial found that limiting first-year resident physicians to 16 consecutive work hours significantly improved resident alertness and patient safety.6 7 Altogether, a body of evidence accumulated suggesting that reducing or eliminating shifts longer than 16 hours did not negatively impact resident education and likely improved patient safety and resident quality of life.8 Subsequently, the Institute of Medicine of the National Academies (IOM) reviewed the available evidence and concluded that it was unsafe for any resident physician to provide clinical care for >16 consecutive hours without sleep.9 10 In response, the ACGME issued new work-hour regulations on 1 July 2011, limiting first-year resident physicians to a maximum of 16 consecutive work hours and emphasising a commitment to patient safety and mitigation of fatigue-related risks (16 hours 2011 ACGME work-hour limit).11
The response within the medical community to the 16 hours 2011 ACGME work-hour limit was mixed.12 Many stakeholders expected the changes to diminish the educational experience.13 The increased frequency of patient handoffs raised concerns, as physician-to-physician handoffs have historically been non-standardised and prone to error.14 In addition, the work-hour limitations were not accompanied by an increased number of residency slots, leading to work compression and a shift in some responsibilities to other clinical providers,15 as well as concerns about resident physician understaffing. Several studies of the 16 hours 2011 ACGME work-hour limit found that it had no impact on hospital-level mortality or mortality following surgical procedures.16–18 In light of these studies and opposition to the work-hour limit from within the medical community, the ACGME lifted the 16-hour limit as of 1 July 2017, again allowing first-year resident physicians to be scheduled for 24 hours of continuous work, plus up to 4 hours for care transitions (28 hours 2017 ACGME work-hour limit).
A limitation of several studies that informed this latest policy change is that the primary end points lacked sensitivity (eg, hospital-wide mortality) or were only tangentially related to the care provided by first-year resident physicians (eg, evaluating the impact of the 16 hours work-hour limit for first-year surgical residents using the outcome of fatal or serious complications after surgical procedures within institutions where first-year surgical residents have a minimal role in surgical procedures).18–20 The impact of the 16 hours 2011 ACGME work-hour limit on patient safety outcomes directly reported by resident physicians has not been studied. Therefore, we undertook an evaluation to determine whether resident physician-reported patient safety outcomes changed following implementation of the 16 hours 2011 ACGME work-hour limit.
Methods
We carried out nationwide prospective cohort studies of resident physicians for 5 academic years (2002–2007) before and for 3 academic years (2014–2017) after implementation of the 16 hours 2011 ACGME work-hour limit.21 Data were not available for transition years (2008–2013). During the transition period resident physician work-hour changes were being made nationwide in response to the IOM report (findings first made public in 2008),10 and a 2010 ACGME report reaffirmed many of the IOM conclusions and established a target date of 1 July 2011 for implementation of the new policies, with exceptions for programmes that needed one or more additional years to make the transition.11 Our analyses compare trends in resident physician-reported medical errors between the two cohorts. A Certificate of Confidentiality was issued by the Centers for Disease Control and Prevention. All variables used in this analysis were collected using the same wording and frequency in both cohorts. The methods have been described in detail in prior publications.3–5 21
Recruitment and data collection
In the initial cohort, all US medical school graduates and all additional individuals who matched to a US residency programme received an invitation to participate in the study. In the second cohort, all medical school graduates who completed an application through the Electronic Residency Application Service were emailed the same invitation to participate.
Respondents were asked to provide electronic informed consent. In the first month of residency, individualised links were sent via email to complete the baseline survey. The baseline survey collected demographic information. Subsequent monthly reports collected work-hour information, including the frequency of extended duration shifts, hours engaged in patient care and total hours of work. Participants also self-reported the primary outcome: significant medical errors in the past month (“Do you believe that you made a significant medical error, whether or not an adverse patient outcome occurred”), whether or not they believed fatigue contributed to the error, the number of errors and the patient outcome resulting from the errors. Resident physician-reported errors resulting in patient harm were considered PAEs. We also queried how many times participants nodded off or fell asleep (attentional failures) during specific patient-care activities (during surgery or while talking to, or examining, patients).
Statistical analysis
The primary analysis tested the association between the 16 hours 2011 ACGME work-hour limit and the resident physician-reported patient safety outcomes of interest. Secondary analyses tested the association between weekly work hours, extended duration shifts and resident physician-reported patient safety outcomes independent of cohort. We excluded months where participants reported ≥14 work-free days, or where work-hour information was missing or exceeded 168 hours of work per week. We further excluded months where no direct patient care was reported (eg, research rotation).
Demographics were compared between groups using Wilcoxon rank-sum test and Pearson’s χ2 test. The sum of hours spent physically awake in the hospital, classes or workplace (excluding occasional employment outside the residency (moonlighting)), plus the number of hours asleep in the hospital were considered weekly work hours. Monthly reports of work, sleep and stress were compared using generalised linear mixed models which accounted for clustering of respondents and the dependence between repeated measures.
We calculated the incidence of each resident-reported patient safety outcome before and after the 16 hours 2011 ACGME work-hour limit. Incidence rate ratios (IRR) were constructed, and the significance of the IRR was tested using likelihood ratio tests in log-linear models. Similar results were obtained using Pearson’s and deviance-based scaled Poisson models. Rare outcomes were dichotomised. We used generalised linear mixed models with a binomial distribution and log-link function to estimate the risk of each outcome. All models adjusted for hours spent in patient care that month. We identified additional confounders a priori based on relevance to the research question and biological plausibility. We imputed age with the median when missing using the missing indicator method.22 Fully adjusted models controlled for age, gender, specialty programme, university-based versus community hospital, an imputation indicator variable and the hours of patient care for that month. Secondary analyses used mixed effects models with data from both cohorts to examine extended duration shifts or weekly work hours as independent variables of interest. These pooled analyses controlled for cohort using a conditional likelihood approach. Sensitivity analyses stratified by cohort and surgical/non-surgical specialty. SAS (V.9.4, Cary, North Carolina, USA) was used for statistical analysis. All tests were two-sided and p<0.05 was considered statistically significant.
Results
A total of 21 862 residents who matched to a US residency programme during the study interval consented to participate (including 9% of all US medical residents from 2002 to 2007 and 18% of US medical residents from 2014 to 2017). Approximately 75% of those who consented provided data and were included in the study cohort (cooperation rate). Six thousand two hundred eleven first-year resident physicians completed 45 261 monthly reports between 2002 and 2007 and 9778 first-year resident physicians completed 61 205 monthly reports between 2014 and 2017 (106 466 total monthly reports). After applying exclusion criteria, 80 266 monthly reports from 15 276 first-year resident physicians were available21 and 14 796 individual residents reporting on 78 101 months with direct patient care were included in the current analyses (online supplemental figure 1).
Supplemental material
Characteristics of the study sample
Resident physicians who contributed data after 2011 were less likely to report female gender and affiliation with a university-based programme (table 1).21 The composition of specialties was largely similar. Weekly hours of work decreased following the 16 hours 2011 ACGME work-hour limit from 71 hours (SD 18) to 62 hours (SD 17), p<0.001, while hours engaged in patient care remained the same (50 hours (SD 20) vs 49 hours (SD 19), p=0.40). Participants reported fewer extended duration shifts per month (4.0 (SD 3.4) vs 0.2 (SD 1.2), p<0.001).
Incidence and adjusted risk of resident physician-reported adverse safety outcomes following the 16 hours 2011 ACGME work-hour limit
We observed an absolute excess of 0.611 (95% CI 0.305 to 0.913) resident physician-reported serious medical errors per-person-year, 0.086 (95% CI 0.043 to 0.129) PAEs per-person-year, and 0.042 (95% CI 0.020 to 0.064) errors resulting in patient death per-person-year prior to the 16 hours 2011 ACGME work-hour limit (relative to after the work-hour limit). The incidence for each adverse resident physician-reported patient safety outcome studied was significantly reduced following the 16 hours 2011 ACGME work-hour limit, both in unadjusted analyses (table 2) and after adjustment for potential confounders (figure 1). The risk of at least one significant medical error reported in a month was reduced 32% (rate ratio (RR) 0.68; 95% CI 0.64 to 0.72, fully adjusted model). A similar decrease was noted in the rate of reported PAEs (RR 0.66; 95% CI 0.59 to 0.74), while the risk of first-year resident physicians reporting a fatal PAE was reduced >60% (RR 0.37; 95% CI 0.28 to 0.49). The rate of patient-care-related attentional failures was reduced >40% (IRR 0.58; 95% CI 0.54 to 0.62).
Work hours, extended duration shifts and resident physician-reported patient safety outcomes
Participants in both cohorts reported exceeding 80 hours of work per week (averaged across 4 weeks) and working extended duration shifts (EDS). Independent of cohort, working >80 hours per week and working EDS were each independently associated with an increased risk of resident physician-reported significant medical error (table 3). Compared with working no EDS, working even one EDS in a month increased the risk of resident physician-reported adverse outcomes. Similarly, compared with working up to 60 hours per week, increased risks of resident physician-reported adverse outcomes were seen for those residents working >60–70 hours per week, those working >70–80 hours per week and those working >80 hours per week in a dose-dependent fashion (table 4). Stratified analyses are presented in online supplemental tables 1 and 2.
Discussion
Following implementation of the 2011 ACGME work-hour restrictions, we found that first-year resident physicians reported making fewer significant medical errors, errors resulting in patient harm and errors resulting in patient death. Despite fears that the policy would limit hands-on training, reported hours spent in patient care remained unchanged. Extrapolating these findings to the first-year resident physician population in the USA would suggest that prior to the implementation of the 2011 reforms, first-year resident physicians perceived that they committed an additional 22 000 errors, 3100 of which led to PAE, and 1500 of which resulted in death, on a yearly basis compared to after the reform.
Our results are in contrast with other recent evaluations of the ACGME’s implementation of the 16-hour shift limit in 2011. In the FIRST (Flexibility in duty hour Requirements for Surgical Trainees) trial, first-year resident physicians at surgical programmes assigned to the flexible work-hour group were allowed to exceed 16 consecutive hours of work and required time off between shifts was reduced.17 The mortality rate following general surgery was not inferior at teaching hospitals allowed to exceed the 16 hours 2011 ACGME work-hour limit compared with hospitals assigned to follow those limits.17 However, in those hospitals, the first-year surgical residents have minimal roles in the surgical outcomes that were evaluated (surgical mortality and major complications, the study’s primary end point), as first-year surgical residents spend only a small fraction of their time in the operating room (estimates suggest 1 hour per day),23 and their roles there are constrained by design (with involvement in 1% of complex cases).24 Moreover, no information was collected on the actual hours worked by the surgical residents in the FIRST trial, or whether the institutions that were allowed to schedule their surgical residents for >16 hours per shift actually did so, given the potential exposure to malpractice liability due to the IOM report9 and concerns regarding the ethics of the trial.25
Another recent effort, the iCOMPARE (Individualized Comparative Effectiveness of Models Optimizing Patient Safety and Resident Education) trial,20 compared programmes randomised to flexible work hours (where extended duration shifts were permitted) with programmes that maintained the 2011 ACGME work-hour restrictions, and similarly found that flexible programmes were not inferior to standard programmes with respect to 30-day hospital-level mortality (primary outcome) or patient safety indicators. Like the FIRST trial, the iCOMPARE trial had a primary end point that lacked sensitivity and a direct connection to the care provided by first-year resident physicians. In addition, the prespecified non-inferiority threshold allowed for non-inferiority to be declared as long as hospital-level 30-day mortality for any cause did not increase in the flexible group by >1%. Whether this prespecified threshold is consistent with an acceptably small degree of harm26 is questionable, and clinically significant decrements in patient safety may have gone unrecognised as a result.27 In any case, neither of these studies suggested limiting work hours is harmful to patient safety. Therefore, given the associated improvements in physician safety that we have previously reported,21 and the improvement in resident physician-reported patient safety that we report here, taken together, these studies indicate that the 2011 ACGME policy limiting first-year resident physician work hours was associated with an improvement in both physician and patient safety.
In addition to these two trials, prior retrospective studies using administrative data similarly concluded that the 16 hours 2011 ACGME work-hour limit did not appreciably impact hospital-level mortality or process of care measures,16 18 28 but like FIRST and iCOMPARE, these studies relied on process and outcome measures that were relatively insensitive and lacked a direct connection to the work of first-year resident physicians. In contrast to previous efforts, our study design enabled collection of patient-safety and work-hour data directly from the residents who provided care. We found that resident physician-reported significant medical errors, PAEs and errors resulting in fatal patient outcomes were reported at substantially increased rates in months when extended duration shifts were worked. The findings from this study, which focused on the impact of resident physician work-hour limits on patient safety using metrics that were directly related to care provided by the resident physicians, are consistent with extensive evidence compiled by the ACGME indicating that sleep deficiency is associated with degradation of performance among physicians performing clinical tasks.29
We also found that when resident physicians averaged 70–80 hours of work per week over a month, which is within the current standard of an average of 80 work hours per week averaged over a month, they had >2.5 times the odds of reporting a significant medical error, a medical error that injured a patient and a medical error that resulted in the death of a patient, as compared with weeks in which the resident physicians worked no more than 60 hours per week on average. In the USA, resident physicians are often scheduled to work more weekly work hours than equivalents in other countries.30–32 We found that even after controlling for the increased hours in patient care that could be expected to result from increased total work hours, increasing total work hours was associated with increased risk of reported significant medical errors, PAEs and errors resulting in patient death. Our data suggest that reducing weekly work hours would improve patient safety. The increased risk of long work weeks and extended duration shifts should be made more transparent to patients and providers.33 34
Our study has several limitations. Our study population included 9% of all first-year US medical residents from 2002 to 2007 and 18% of all first-year US medical residents from 2014 to 2017. We examined demographic characteristics for evidence of non-response bias. While the demographic characteristics of participants in this sample are similar to the demographic characteristics of resident physicians nationally (online supplemental table 3),21 those who chose to participate in the study may differ from non-respondents in ways that are not captured by our analysis. As it was observational, we are unable to directly attribute the findings to the 2011 ACGME work-hour restrictions. The estimated associations could be affected by residual confounding or other forms of bias. We controlled for multiple potential confounders, but residual confounding may persist as a result of factors we did not collect or imprecise measurement of the confounders that were collected.
Significant medical errors were collected by self-report. This approach to the collection of medical errors could be subject to social desirability bias, recall bias or erroneous self-observation. Self-report is an established methodology that is commonly used. Prior work has shown that self-reported errors are corroborated in the medical record >80% of the time35 and produce similar or lower estimates of medical error compared with retrospective chart review.36 37Alternative detection methods, such as trigger tools and direct observation have been shown to detect higher rates of medical errors compared with self-report.6 38 By using self-report, our estimates for medical errors are likely to be conservative, which would bias our results towards the null. Although we expect that resident physician-reported errors represent only a fraction of all errors that occur throughout the healthcare system, we do not have reason to believe that the propensity to report errors would have changed over time in a manner that introduced bias. Clinician perceptions of medical errors can also have a pervasive impact on physician well-being, and have been associated with increased odds of physician burnout and depression.39
While some reductions in the rate of medical errors may be attributable to secular trends, most studies have not found temporal reductions in the overall rate of harm attributable to medical care over the time period under study that could explain our results.40–43 Evaluations prior to the reform showed no reduction in the rate of medical error over time, although one recently published effort reported a declining trend in patient safety indicators from 2000 to 2013.44 Most estimates following the reform suggest the problem may be larger than previously reported, and remains highly prevalent.41 43 45 Our findings have important policy implications for >100 000 resident physicians in the USA. The very substantial reductions in harmful resident physician-reported medical errors over a relatively short period of time are striking, and together with the previously reported improvements in occupational safety among the resident physicians,21 suggest a causal link to the policy implemented at that point in time for this subset of clinicians and the improvements in both patient and physician safety.
Conclusions
In a large nationwide prospective cohort study, we found that implementation of a policy that eliminated shifts exceeding 16 consecutive hours for first-year resident physicians was associated with a substantial reduction in harmful and fatal medical errors reported by the resident physicians themselves. Extended duration shifts were associated with significant and harmful medical errors even after the implementation of the policy. Surveillance efforts are needed now that national regulations allow shifts of up to 28 consecutive hours for first-year resident physicians.
Data availability statement
Data are available on reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
This study was approved by Partners Human Research Committee (Protocol #: 2002P000202). Participants gave informed consent to participate in the study before taking part.
Acknowledgments
We thank the resident physicians who took time from their busy work schedules to participate in this study; to the Association of American Medical Colleges, especially Marie Caulfield and Brianna Gunter, for their assistance with recruitment; to Steven W. Lockley for his assistance with the study design and data interpretation; to research assistants Michael Shreeve and Justin Buie, and students Kelly Racicot, Lars Johnsen, Erita Rothschild, Autumn Gertz, Niamh Mulrooney, Matthew Stewart, Jacob Stroud, Gregory Pavlov, Krystal Lipomi, Arlene Perez, and Julian Smith-Sparks.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors CPL, CAC and LKB designed the parent study and developed the data collection tools. SQ managed data collection and data security. JPS, CSO and NV provided daily study oversight. MDW conducted the analysis and wrote the initial draft. All authors provided substantive contributions and critical revisions to the final product. MDW is the guarantor, accepting full responsibility for the work and the conduct of the study, had access to the data, and controlled the decision to publish.
Funding This study was funded by National Institute for Occupational Safety and Health (R01OH010300, R01OH011773, R01OH07567) and the National Heart, Lung, and Blood Institute (F32HL134249, R56HL151637, T32HL007901, U01HL111478).
Competing interests MDW reports grant support from the CDC, NIOSH, NHLBI and the Brigham Research Institute. He also reports institutional support and/or consulting fees from Delta Airlines, the National Sleep Foundation, Puget Sound Pilots, and the University of Pittsburgh. CPL has served as a paid consultant to the Midwest Lighting Institute to help study the effect of blue light on health care provider performance and safety. CPL has consulted with and holds equity in the I-PASS Institute, which seeks to train institutions in best handoff practices and aid in their implementation. He has consulted with the Missouri Hospital Association / Executive Speaker's Bureau regarding I-PASS. In addition, CPL has received monetary awards, honoraria, and travel reimbursement from multiple academic and professional organizations for teaching and consulting on sleep deprivation, physician performance, handoffs, and safety, and has served as an expert witness in cases regarding patient safety and sleep deprivation. CAC reported receiving grants to BWH from CDC, NHLBI, NIA, NIOSH and DOD during the conduct of the study; being a paid consultant to or speaker for Ganésco, Institute of Digital Media and Child Development, Klarman Family Foundation, M. Davis and Co, Physician's Seal, Puget Sound Pilots, Samsung Group, State of Washington Board of Pilotage Commissioners, Tencent Holdings, Teva Pharma Australia, and Vanda Pharmaceuticals, in which Dr Czeisler holds an equity interest; receiving travel support from Aspen Brain Institute, Bloomage International Investment Group, UK Biotechnology and Biological Sciences Research Council, Bouley Botanical, Dr Stanley Ho Medical Development Foundation, Illuminating Engineering Society, National Safety Council, Tencent Holdings, and The Wonderful Co; receiving institutional research and/or education support from Cephalon, Mary Ann and Stanley Snider via Combined Jewish Philanthropies, Harmony Biosciences, Jazz Pharmaceuticals PLC, Johnson and Johnson, Neurocare, Peter Brown and Margaret Hamburg, Philips Respironics, Regeneron Pharmaceuticals, Regional Home Care, Teva Pharmaceuticals Industries, Sanofi S.A., Optum, ResMed, San Francisco Bar Pilots, Schneider National, Serta, Simmons Betting, Sysco, Vanda Pharmaceuticals; being or having been an expert witness in legal cases, including those involving Advanced Power Technologies; Aegis Chemical Solutions; Amtrak; Casper Sleep; C and J Energy Services; Complete General Construction; Dallas Police Association; Enterprise Rent-A-Car; Steel Warehouse Co; FedEx; Greyhound Lines; Palomar Health District; PAR Electrical, Product, and Logistics Services; Puckett Emergency Medical Services; South Carolina Central Railroad Co; Union Pacific Railroad; UPS; and Vanda Pharmaceuticals; serving as the incumbent of an endowed professorship provided to Harvard University by Cephalon; and receiving royalties from McGraw Hill and Philips Respironics for the Actiwatch-2 and Actiwatch Spectrum devices. Dr Czeisler's interests were reviewed and are managed by the Brigham and Women's Hospital and Mass General Brigham in accordance with their conflict-of-interest policies. LKB was supported by grants from NIOSH, NIA and NHLBI, and has consulted with and/or received institutional support from Boston Children’s Hospital, the AAA Foundation, Delta Airlines, Puget Sound Pilots, the University of Pittsburgh, and the CDC.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.