Medical harm and hospital mortality: some history

The concept of medical harm has existed since antiquity, famously discussed by Hippocrates and passed on in the term ‘iatrogenesis’, from the Greek for ‘originating from a physician’. The influential 9th century Arab physician, al-Ruhāwī, wrote on this topic, and many others acknowledged the problem of medical harm over the centuries. In the 1860s, Oliver Wendell Holmes, Sr, a prominent American physician and popular writer, stated in an address to the Massachusetts Medical Society: “If the whole materia medica, as now used, could be sunk to the bottom of the sea, it would be all the better for mankind—and all the worse for the fishes”. This same decade also saw the first appearance of Florence Nightingale's work with William Farr on hospital mortality reports, drawing attention to the much higher mortality in London hospitals compared with rural ones.1

In the early 20th century, Ernest Codman introduced morbidity and mortality conferences, promoting the idea of systematically monitoring patient outcomes in order to improve quality.2 Formal studies of iatrogenic injury appeared in the 1960s,3–6 and Ivan Illich's indictment of modern medicine in the 1970s7 briefly brought the problem of iatrogenesis to popular attention.

By the 1980s, many individuals had begun to labour in the vineyard of improving healthcare quality, but their efforts produced only sporadic general interest. Even the Harvard Medical Practice Study8 did not generate sustained attention for the problem of medical harm or healthcare quality. Large scale studies began to appear from other countries,9 and the Harvard Medical Practice Study was replicated in Utah and Colorado.10 However, these studies would not have gained much traction were it not for the US Institute of Medicine publication, To Err is Human, in late 1999.11 The committee that produced this report included many individuals who had spent their careers trying to advance the quality agenda in healthcare. Perhaps they reasoned that medical errors would capture the popular imagination in a way that unexplained variations in rates of discretionary procedures, failures to implement established processes of care, and other such drier topics had failed to do.

Whatever the reasoning, it worked. The widely quoted estimates of 44 000–98 000 annual deaths due to medical error in US hospitals11 and the associated analogy of a jumbo jet crashing every day was a tipping point for quality improvement. Further studies of medical injuries (‘adverse events’) modelled after the Harvard Medical Practice Study were conducted in other countries,12–15 agencies focused on patient safety were set up in various countries, and new funding opportunities materialised.

The attention grabbing focus on medical error initially appeared as if it might work too well, with little interest left over for quality problems that could not be cast as medical errors. However, recent years have seen substantial interest in the optimal management of chronic illnesses, equitable access to healthcare, patient-centeredness, and the pressing need to eliminate wasteful practices, among other important quality problems.

Hospital mortality as a quality and safety measure

Even with the pendulum appropriately swinging away from the early fixation on ‘deaths due to medical error’, hospital mortality has remained a focus of performance measurement, most obviously in the form of the widely promulgated hospital standardized mortality ratio (HSMR).16 ,17 The dissemination of HSMR as a performance metric remains somewhat mysterious, as the choice to measure quality using hospital death rates, especially when derived solely from administrative data, runs counter to the results of decades of research.1 ,18–20 In fact, two prominent researchers have characterised HSMR as a “bad idea that just won't go away”.21

The flaws associated with HSMR include case-mix adjustment driven exclusively by comorbid conditions with no measure of illness acuity,20 technical problems with the methods of case-mix adjustment,22 the influence of coding practices, and geographical variations in the availability of alternatives to hospitalisation for dying patients.23 Moreover, this metric simply does not perform well as a ‘diagnostic test’ for poor quality care.24

Even with all these flaws, however, reporting standardised hospital mortality might still serve a useful purpose. ‘Your hospital has a higher than expected mortality’ sounds compelling in a way that ‘Experts recommend that all hospitals periodically review charts to look for evidence of quality problems’ does not. The former will far more likely achieve the desired of goal of galvanising hospital staff to review case records to identify problematic patterns in the care they deliver. The key question, then, is the degree to which quality problems can be identified by reviewing the charts of deceased patients.

Are hospital deaths the best place to look for quality problems?

One previous study25 directly measured the proportion of hospital deaths due to quality of care problems. Among 111 in-hospital deaths from seven US Veterans Affairs medical centres, physician chart reviewers rated 6.0% as probably or definitely preventable. Despite appearing in a prominent medical journal, this study did little to alter the impression that hospital mortality and the charts of deceased patients should attract careful study in efforts to improve patient safety.

The study by Hogan et al26 in this issue of the journal thus comes as a welcome addition to the literature on the topic. Investigators reviewed the records of 100 randomly selected patients from 10 acute hospital trusts (randomly sampled with stratification by geographic location, size and academic status) who had died at each hospital in the year 2009, for a total of 1000 death reviews. Patients admitted to paediatric, obstetric, and psychiatric services were excluded, since mortality fortunately occurs infrequently in these settings, as were patients admitted explicitly for palliative care.

Physician reviewers with experience in general acute hospital care (15 from internal medicine and two from general surgery) conducted the chart reviews, obtaining input from specialists as necessary. Reviewers were asked to consider the entire record for the index admission, including nurses’ and allied health professionals’ notes, drug charts and diagnostic test results, to identify problems in care, rate the overall quality of care delivered, and judge the degree to which death could have been prevented. Preventability was captured using a six-point probabilistic scale (similar to other major studies), with scores of 4–6 indicating a greater than 50% chance that death could have been prevented.

The single most important finding of the study is that, among these 1000 cases representative of deaths in English acute care hospitals, reviewers judged 5.2% (95% CI 3.8 to 6.6) as preventable (ie, having a greater than 50% probability that better care would have prevented death). The second most important finding, at least in the context of the oft-asserted equivalence of the death toll from medical error to a jumbo jet crashing every day, concerns the prognosis of these patients. Reviewers judged patients experiencing a preventable death in hospital as having a median life expectancy of 6 months (IQR 4 months–2 years). Thus, in contrast to the tremendous toll in terms of years of life lost for passengers on a crashed jumbo jet, most hospitalised patients who die from problems in medical care are already near the end of their lives.

The previous US study of preventable deaths25 made this point even more directly. In that report, reviewers were specifically asked if, in the absence of any problems in care, patients would have lived at least 3 months in good cognitive health. Clinicians estimated that only 0.5% (95% CI 0.3% to 0.7%) of the patients who died would have achieved even this limited goal, representing roughly one patient per 10 000 admissions to the study hospitals.25

What do these results imply for the future?

First, the low prevalence of preventable hospital deaths in the study by Hogan et al26 does not in any way undermine the importance of improving patient safety. Mortality by itself fails to capture the burden of many major illnesses. It may serve as a rallying cry, but, for many acute and chronic conditions, the real toll comes in the form of distressing symptoms, loss of function, loss of income, and resources expended by society. Similarly, most patient safety problems do not cause death. Many cause no harm at all. However, a substantial minority cause pain, anxiety, temporary or permanent disability, and increased utilisation of resources. These remain important goals for improvement. In most areas of medicine, the goal of care is not the avoidance of death, and patient safety is no different.

The results of this study26 do, however, undermine the use of hospital mortality as a performance measure. Even putting aside the many well documented problems with this metric,18 ,20 ,21 only about one in 20 hospital deaths are preventable.25 ,26 To say that looking for quality differences between hospitals on the basis of variations in hospital mortality amounts to looking for a needle in a haystack would be an overstatement, but not by much. The small proportion of preventable deaths presents a major signal to noise problem for hospital death rates. An increased HSMR will falsely signal poor quality far more often than it identifies true increases in the proportion of preventable deaths.18 ,20–22 ,24

Nevertheless, maybe these false positive signals will achieve important benefits by motivating hospitals to undergo a process they would not otherwise feel inclined to pursue, namely periodic chart reviews to identify problems in the care they deliver. Hospitals could commit to record reviews every few years (some already do so using the Global Trigger Tool27), but most need a nudge, and monitoring HSMR may provide just that.

Putting aside the potential harm to a hospital's reputation and staff morale from false positive signals, the key question is whether or not reviewing hospital deaths provides an efficient means of identifying patterns of errors (or other quality problems). I believe the answer is ‘no’. In practice, many hospitals identify ways in which they have been under-coding comorbid conditions and so ‘fix’ the problem of an elevated HSMR simply through more assiduous coding. Even if hospitals review the charts of deceased patients more carefully, the focus on identifying possibly preventable deaths may distract hospital personnel from identifying pervasive patterns of problems (eg, poor communication, teamwork problems, equipment and design issues) because chart review will generally not suggest (or even hint at) their presence or their impact.

More fundamentally, mortality reviews probably do not capture the full range of patient safety problems worth addressing. First, clinical services that have low death rates (such as those excluded from the study by Hogan et al,26 but also many surgical subspecialties) may well have different major categories of safety problems. One can hope that broad categories, such as those identified by Hogan et al26—mainly problems with clinical monitoring, delayed diagnosis and medication safety—obtain in these settings as well. However, the items in these categories will likely differ and the strategies to address them may differ as well.

One can also hope that, while harms caused by medical error on, say, the ophthalmology service will not include death, the latent causes of these less morbid errors are the same as the underlying problems encountered elsewhere—communication problems, poor teamwork, human factors issues and so on. However, hospitals rarely tackle these deep problems after reviewing the charts of inpatient deaths. Moreover, some important problems will rarely, if ever, appear in mortality reviews, for instance patient identification errors and problems with care coordination at the time of hospital discharge, to name just two.

Moving forward

Many strategies exist for identifying patient safety problems—from incident reporting and chart reviews to patient safety walk rounds, malpractice claims, patient complaints and the use of electronic trigger tools. These strategies yield substantially different views of the dominant patient safety issues.28 ,29 Hospitals can mitigate this difficulty by using several methods to characterise their patient safety problems. For some hospitals, mortality reviews might make sense as one of these strategies. At this stage in the field, however, we know enough about many categories of problems that it would seem reasonable to move beyond broad forms of measurement and start delving deeper into the categories of problems that have been found so often in research studies and by the monitoring strategies routinely used by hospitals for operational purposes—medication safety, problems with clinical monitoring and diagnostic errors, as well as more systems oriented categories, such as teamwork, communication and human factors problems.

Importantly, these categories are not monolithic. ‘Communication problems’, for instance, include many important subcategories and they will not all be amenable to the same interventions. Diagnostic errors and medication safety likewise include many different subcategories that will differ in their causes and the interventions likely to reduce them. Further characterising these major categories of patient safety problems in terms of the major contributing factors and their associated burden of harm would seem the more appropriate course of action at this point than ongoing mortality reviews.

The image of jumbo jets crashing stimulated widespread interest in patient safety. That the number of deaths due to medical care is more accurately analogous to less dramatic transportation disasters is not the point. The point is that we know that mortality represents only a fraction of the total toll of injuries from medical error, we know many of the broad causes of these injuries and we have some evidence about strategies for reducing a handful of specific patient safety problems. The study by Hogan et al26 will hopefully succeed in redirecting some of the excessive interest in hospital mortality towards the development of better measures of patient safety and effective strategies for improving it.