The Institute of Medicine (IOM) released a report in 1999 entitled “To Err is Human: Building a Safer Health System”.¹ The report stated that errors cause between 44 000 and 98 000 deaths every year in American hospitals, and over one million injuries.¹ Health care appeared to be far behind other high risk industries in ensuring basic safety. The IOM report called for a 50% reduction in medical errors over 5 years.¹ Its goal was to break the cycle of inaction regarding medical errors by advocating a comprehensive approach to improving patient safety.

This IOM report received tremendous attention from both the public and the healthcare industry.² There was extensive media coverage that was closely followed by the American public.^2,3 The healthcare industry responded almost immediately with a wide range of patient safety efforts.^4,5 The federal government appropriated $50 million annually for patient safety research.⁶ Non-governmental organizations issued briefs indicating that patient safety was now a priority.⁵ Healthcare purchasers such as The Leapfrog Group encouraged hospitals to adopt safer practices and emphasized that safety was also now a priority for them.⁷

The 5 year anniversary of the IOM report has sparked debate regarding its impact on patient safety and quality of health care.⁸ Critics of the report have suggested that, although safety is a vital component of healthcare quality, the report may have done more harm than good.^8,9 They contend that, by focusing undue attention on accidental deaths which are difficult to study and prevent, limited resources are being drawn away from other important quality improvement initiatives.^8,10 Conversely, patient safety advocates argue that the IOM report has galvanized the public and the healthcare industry into making necessary changes and we are beginning to see the first signs of progress.^4,5,11,12 However, objective assessment of the impact of the IOM report has been difficult as no comprehensive nationwide monitoring system exists for patient safety.

One objective and readily available measure relating to patient safety is the health sciences literature. Although research and academic publications will by themselves not improve patient safety, they are a means of knowledge development and transfer and will be an integral component of any efforts to improve patient safety. The health sciences literature and its funding also provide a gauge of the relative importance and cultural attitudes towards healthcare issues. We therefore sought to investigate the effects of the IOM report “To Err is Human” on the publication of patient safety articles and granting of federally funded patient safety research awards.

METHODS

Study design

Using data from a period of 10 years, we evaluated changes in patient safety publications in MEDLINE indexed journals and federal research funding associated with the release of the IOM report “To Err is Human”. Changes in publications and research awards were estimated by interrupted time series analysis in which rates during the 5 year periods before and after the IOM report were compared.

Data sources

Data on patient safety publications were searched using MEDLINE. The search was conducted by identifying all English language articles on patient safety, limited to humans, published between 1 January 1994 and 1 January 2005 by using both medical subject headings (“medical errors”, “medication errors”, “iatrogenic disease”, “safety management”, “risk management”, “quality assurance, health care”, “patients”, “safety”) and keywords found in titles and abstracts (“safe”, “safety”, “error”, “patient”, “medical”, “medication”, “non-medical”, “nonmedical”). We combined the following search terms: (1) MeSH terms “patients” and “safety”; (2) MeSH term “risk management” and keyword “safe”; (3) MeSH term “quality assurance, health care” and keyword “safe”; (4) keywords “patient” and “safety”; and (5) keywords “medical” or “medications” or “non-medical” or “nonmedical” and “error”. Finally, we compiled articles identified by the MeSH terms “medical errors”, “medication errors”, “iatrogenic disease” and “safety management” with articles identified using the five combination search terms.

Data on patient safety research projects funded by the federal government of the USA were searched using the Computer Retrieval of Information on Scientific Projects (CRISP) database.¹³ The database is maintained by the Office of Extramural Research at the National Institutes of Health and includes projects funded by the National Institutes of Health, Substance Abuse and Mental Health Services, Health Resources and Services Administration, Food and Drug Administration, Centers for Disease Control and Prevention, Agency for Health Care Research and Quality, and the Office of the Assistant Secretary of Health. The CRISP search was conducted by identifying all research awards for the fiscal years 1995–2004 using the CRISP thesaurus search terms “patient safety”, “medical error”, and “iatrogenic disease”.

A team of four reviewers (RG, JM, SP and CS), blind to the study hypotheses, independently reviewed in random order the titles and abstracts of both the publications and research awards identified in our two database searches. Each publication and research award was evaluated to determine whether its principal focus was patient safety or medical errors. Selected publications were classified according to publication type (reports of original research, editorial, letter to the editor, review, guideline, news item or other) and principal subject (single most relevant MeSH term not employed in the search strategy). Reports of original research were further classified according to their methodology (qualitative studies, case reports/case series, correlational studies, cross-sectional surveys, case-control studies, cohort studies, intervention studies, systematic reviews or decision analyses). A fifth reviewer (HTS), blinded to the initial reviews, classified a 10% random sample of publications and research awards to calculate inter-rater reliabilities.

Statistical analysis

Publications were aggregated into 3 month intervals and data analysis was limited to the 5 year periods before (1 November 1994 to 1 November 1999) and after (1 November 1999 to 1 November 2004) the 1 November 1999 release of the IOM report. Patient safety research awards were analyzed in yearly intervals to coincide with funding decisions for each fiscal year (1 October to 30 September). Data analysis was limited to the five fiscal year periods before (1995–1999) and after (2000–2004) the release of the IOM report.

Analyses were performed assuming a Poisson distribution. We used a two step procedure to examine the data. We first compared publication and research award rates before and after the release of the IOM report. Interrupted time series regression models were then developed to estimate changes in the rates of patient safety publications and research awards that occurred after the release of the report. Our models included a constant, an offset, a baseline trend over time, and terms estimating changes in the level and trend of patient safety publications after the release of the IOM report.¹⁴ The offset for models of publications was the logarithm of the number of MEDLINE publications per 3 month interval while, for models of research awards, it was the logarithm of the total number of federally funded awards each fiscal year.

We repeated all analyses for a subgroup of articles published in the six general medicine journals with the highest impact factors in 2004 that published original research (New England Journal of Medicine, Journal of the American Medical Association, Lancet, Annals of Internal Medicine, Archives of Internal Medicine, and the British Medical Journal).¹⁵

Agreement on the classification of publications and research awards was assessed with Cohen kappa (κ) reliability coefficients.¹⁶ Statistical analyses were performed using Stata Version 8.0 (Stata Corp, College Station, TX, USA) with two tailed significance levels of 0.05. We reported results as rates, percentages, absolute percentage changes, and odds ratios.

RESULTS

Identification of publications and research awards

The literature search identified 12 429 articles from among 5 207 194 MEDLINE publications between 1 January 1994 and 1 January 2005. Thirteen duplicates were identified leaving 12 416 publications for review. Patient safety or medical errors were identified as the principal focus for 5905 publications (48%). Six articles were excluded because the date of publication could not be identified. Among the remaining articles, 5514 were published between 1 November 1994 and 1 November 2004 in 1095 journals from 40 countries and were included in the principal analyses. The search of the CRISP database identified 1745 awards out of 732 826 federally funded research awards granted for the fiscal years 1995–2004. Patient safety or medical errors were identified as the principal focus for 567 (32%) of the research awards. Agreement on the classification of publications and research awards was good: principal publication focus on patient safety or medical errors (agreement 86%, κ = 0.71), publication type (agreement 74%, κ = 0.67), publication subject (agreement 60%, κ = 0.57), methodology of reports of original research (agreement 68%, κ = 0.58), and principal research award focus on patient safety or medical errors (agreement 90%, κ = 0.77).

Changes in patient safety publications

A large shift in the number of patient safety publications followed the release of the IOM report (fig 1). An average of 59 patient safety articles were published per 100 000 MEDLINE publications in the 5 years before the IOM report; this increased to 164 articles per 100 000 MEDLINE publications in the 5 years after publication of the report (p<0.001). Even after controlling for an existing 3% per quarter upward trend (p<0.001), the rate of patient safety publications increased immediately after the release of the IOM report by 64% (p<0.001). Significantly increased rates of publication were observed for all types of patient safety articles (table 1). Rates of patient safety publications in the top general medical journals mirrored those in MEDLINE indexed journals, averaging four articles per 100 000 MEDLINE publications before the IOM report and 13 articles per 100 000 MEDLINE publications after the IOM report (p<0.001).

Changes in patient safety research

A large increase in patient safety research followed the release of the IOM report (fig 2). Before the IOM report an average of 24 reports of original research were published per 100 000 MEDLINE publications; this increased to 41 reports of original research per 100 000 MEDLINE publications after the release of the report (p<0.001). Before publication of the IOM report there was a 3% per quarter upward trend (p<0.001) in the rate at which reports of original research were being published. The release of the IOM report coincided with a fall of 21% in the rate of publication of reports of original research (p = 0.036). However, in the 5 year period following the IOM report the upward trend increased by 2% (p = 0.05) from 3% to 5% per quarter, leading to an overall increase in research publications in the 5 year period after the IOM report. Comparing the 5 year period before and after the IOM report, there were significant increases in the rates of qualitative studies, cross sectional surveys, case-control studies, intervention studies, systematic reviews, and decision analyses (table 2). No differences were observed for case reports or case series, correlational studies, or cohort studies.

The number of federally funded patient safety research awards increased after the release of the IOM report. There was an average of five research awards per 100 000 federally funded biomedical research awards before the IOM report and 141 after publication of the report (p<0.001). Before the IOM report there was an existing upward trend of 62% per fiscal year (p<0.001) in the rate of patient safety related research awards. After controlling for this baseline trend, the rate of patient safety research awards did not change significantly until the 2001 fiscal year when it increased by 569% (p<0.001).

Changes in subject matter of patient safety publications

Review of the patient safety articles identified 1156 unique MeSH terms. After combining similar terms, 918 MeSH terms remained. Examination of the 25 most common MeSH terms, which represented 2276 (41%) articles, suggested that the principal subject matter of patient safety articles was different before and after the publication of the IOM report (fig 3). The most frequent subject of patient safety publications before the IOM report was malpractice (6% v 2%, p<0.001), while after publication of the report the most frequent subject was organizational culture (1% v 5%, p<0.001).

DISCUSSION

We have examined the impact of the IOM report “To Err Is Human” on the health sciences literature and found a substantial increase in the number of patient safety publications and research awards following the release of the report. Increased rates of publication were observed for all types of patient safety articles. Publications of original research and research awards were more common following the IOM report. The subject matter of patient safety publications also changed. Before publication of the report the most frequent subject of patient safety publications was malpractice; after its release the most frequent subject was organizational culture.

Improving patient safety

Our study provides some of the strongest evidence to date of the impact of the report on efforts to promote patient safety. Firstly, publication of the report has clearly triggered a patient safety conversation in the health sciences literature. Patient safety has progressed from being the subject of occasional publications to being the focus of dedicated issues¹⁷ and series^18,19 in prominent medical journals. Secondly, the IOM report has changed the very nature of the patient safety conversation from focusing on dispensing blame to improving systems. Efforts to promote patient safety originated from studies in the 1990s designed to understand medical malpractice rather than improve health care. The IOM report introduced the concept of preventable injury secondary to systems issues. A paradigm shift is underway. Thirdly, patient safety is a new field and both time and stable funding are needed for meaningful research to develop. Many of the largest patient safety studies were published before the IOM report.^20–22 There has been a limited increase in the number of research publications. However, a distinct change in the methodology of these publications has already emerged with a new emphasis on interventions to improve patient safety. In addition, health sciences researchers are increasingly collaborating with scientists from fields of human factors engineering, psychology, and informatics creating prospects for innovative approaches to longstanding safety challenges. However, for these gains to be sustained, ongoing federal funding at present or higher levels will be needed. The level of patient safety funding in future AHRQ budgets is uncertain.

Our study also underscores how a policy report can transform a healthcare issue into a national priority. The medical community discovered patient safety with the publication of “To Err is Human”. Before the report was published there was sporadic interest in patient safety that accompanied high profile medical journal articles or media coverage of sensational medical errors.²³ The Harvard Medical Practice study was published in 1991, yet it was the IOM report that widely publicized the fact that between 44 000 and 98 000 people die in hospitals each year because of preventable medical errors.²⁰ The IOM report also personalized the discussion of patient safety by recalling previous celebrity patients such as Libby Zion and Betsy Lehman who had died from medical errors.²³ Finally, the report quantified the impact of medical errors on patient safety using the simple yet stunning analogy of one jumbo jet crashing per day.

“To Err is Human” illustrates the impact that a simple call to action can have. However, it is now more important than ever for the medical community to evaluate objectively the progress in efforts to promote patient safety. As time passes the paucity of evidence that patients are safer today than they were before the report was published is allowing critics increasingly to question the role of patient safety within healthcare quality. Brennan et al⁸ have argued that patient safety is something of a fad and not as important a priority as quality, so that investment would be better directed at quality than safety. We believe that separating patient safety from healthcare quality represents a false dichotomy because patient safety is a first step in providing quality care, and that both are valuable. The problem is that, historically, efforts to promote patient safety as well as broader efforts to promote healthcare quality have received limited attention and funding. “To Err is Human” has provided a window of opportunity for improving patient safety in health care. The current focus on patient safety should not discourage healthcare quality advocates. Rather, there is a need for continued patient safety research support and increased healthcare quality research support which has recently stalled. Otherwise, there is a risk that patient safety will be dropped as a priority due to a perceived lack of progress, and the impact of “To Err is Human” will be short lived.

The results of this study need to be interpreted within the context of its limitations. Firstly, although we employed both extensive MEDLINE and CRISP search strategies, we may have missed patient safety publications and research awards during the study period. Nevertheless, the same search strategies were used before and after the release of the IOM report and therefore should, at a minimum, provide similar sampling frames. Secondly, our analyses compared patient safety publications before and after the release of the IOM report with no allowances made for the time required to generate different types of publications. For example, it is likely to take longer to generate reports of original research than editorials, letters to the editor, and reviews. Thirdly, our results do not establish a causal relationship between release of the IOM report and changes in patient safety publications and research awards. During the period of our study there were other important patient safety events—for example, the publication in June 2000 of “An Organization with a Memory”. Although our quasi-experimental design allowed us to avoid many of the selection biases that plague non-randomized policy studies, it only permitted us to determine that there was an association between the release of the IOM report and subsequent changes in patient safety publications and research awards.

The increasing number of patient safety publications and research awards shows that the IOM report had a major impact on patient safety research, which is probably correlated with increased efforts by organizations to promote patient safety. However, the extent to which “To Err is Human” has improved the safety of patients cannot be determined by the results of our study. A window of opportunity remains for health care to follow other high risk industries in establishing basic safety.