Article Text
PDF
Abstract
Introduction Monitoring hospital mortality rates is widely recommended. However, the number of preventable deaths remains uncertain with estimates in England ranging from 840 to 40 000 per year, these being derived from studies that identified adverse events but not whether events contributed to death or shortened life expectancy of those affected.
Methods Retrospective case record reviews of 1000 adults who died in 2009 in 10 acute hospitals in England were undertaken. Trained physician reviewers estimated life expectancy on admission, to identified problems in care contributing to death and judged if deaths were preventable taking into account patients' overall condition at that time.
Results Reviewers judged 5.2% (95% CI 3.8% to 6.6%) of deaths as having a 50% or greater chance of being preventable. The principal problems associated with preventable deaths were poor clinical monitoring (31.3%; 95% CI 23.9 to 39.7), diagnostic errors (29.7%; 95% CI 22.5% to 38.1%), and inadequate drug or fluid management (21.1%; 95% CI 14.9 to 29.0). Extrapolating from these figures suggests there would have been 11 859 (95% CI 8712 to 14 983) adult preventable deaths in hospitals in England. Most preventable deaths (60%) occurred in elderly, frail patients with multiple comorbidities judged to have had less than 1 year of life left to live.
Conclusions The incidence of preventable hospital deaths is much lower than previous estimates. The burden of harm from preventable problems in care is still substantial. A focus on deaths may not be the most efficient approach to identify opportunities for improvement given the low proportion of deaths due to problems with healthcare.
- Hospital mortality
- patient safety
- medical errors
- adverse events
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/3.0/ and http://creativecommons.org/licenses/by-nc/3.0/legalcode.
Statistics from Altmetric.com
Background
Following the US Institute of Medicine's report To err is human,1 the Chief Medical Officer for England estimated that 60 000 to 255 000 NHS patients each year suffer serious disability or death as a result of healthcare interventions.2 This estimate was derived from retrospective case record review (RCRR) studies conducted in USA in the 1980s and 90s.3 ,4 These and other national studies using comparable methods were not designed to establish the proportion of deaths that were preventable.5–8
Two smaller studies have specifically assessed the degree to which problems in care contributed to death. In one study of 111 deaths in US hospitals, reviewers judged 6% as either probably or definitely preventable.9 A study from New Zealand concluded that 3.4% of 118 deaths were related to preventable errors in healthcare.10 More recently, a large RCRR study in the Netherlands reported a figure of 4.1%,11 which would be consistent with a more modest estimate of 9000 such deaths annually in England. These findings suggest that existing estimates in England based on extrapolations from studies with small numbers of deaths have overestimated preventable deaths.10–12
Given the considerable attention paid to hospital mortality as an indicator of quality of care,13 ,14 we aimed to estimate more accurately the number of preventable deaths among hospitalised patients in England, to describe the problems in care that are responsible (type, phase of care) and to estimate the life expectancy of those affected.
Methods
Design
RCRR is a method based on experts' retrospective reviews of healthcare records, assessing the quality and safety of care provided during an index admission. It is the most sensitive approach in determining the proportion of hospital deaths that are preventable.15–17 Our study design was adapted from previous RCRRs in the UK and the Netherlands,7 ,11 which in turn, were based on the Harvard Medical Practice Study.3 It also drew on a study of deaths by Hayward and Hofer.9
Sampling strategy
Deceased patients were identified at 10 randomly selected English acute hospital Trusts. To increase generalisability, we stratified our sampling on the basis of region (London, South, Midlands and North); teaching status; and bed size (<500, 500–700, >700) before random selection of the 10 sites from across these strata.
We estimated that 6% of deaths would be judged preventable.9 A simple random sample would require 347 deaths to yield a 95% CI with a width of 2.5% on each side. Taking into account the two-stage sampling strategy and clustering effects at the hospital level increased the required sample size to 1000 cases. (This estimation used an intracluster correlation of 0.037 derived from the Dutch Adverse Event Study11).
One hundred case records of patients who had died in hospital during 2009 were randomly selected using the hospital administration system in each Trust. As in previous studies, obstetric, psychiatric and paediatric patients (who in total accounted for less than 5% of all hospital deaths in England and Wales in 200918) were excluded. Of the 1000 randomly selected patients, 13 patients were admitted explicitly for planned palliative care and, therefore, were excluded and replaced.
Judgements of preventable deaths
The judgement of preventable deaths was undertaken in two stages. First reviewers were asked to judge whether there had been any problem in care that had contributed to the patient's death. Problems in care were defined as patient harm resulting from acts of omission (inactions), such as failure to diagnose and treat, or from acts of commission (affirmative actions) such as incorrect treatment or management, or harm as a result of unintended complications of healthcare. This definition was seen as more helpful than adverse event, patient safety incident, or error (box 1) because it extends beyond single discrete incidents to take a wider view of the overall quality of care provided and its contribution to a patient's death. The definition was also more likely to ensure that deaths related to failure to act (omissions) were recognised, particularly if these occurred over days or weeks.
Definitions previously used to describe harm due to care
Adverse event
An injury related to medical management, in contrast to complications of disease. Medical management includes all aspects of care, including diagnosis and treatment, failure to diagnose or treat, and the systems and equipment used to deliver care. Adverse events may be preventable or non-preventable.19
Patient safety incident
Any unintended or unexpected incident that could have or did lead to harm for one or more persons receiving healthcare.20
Error
The failure of a planned action to be completed as intended (ie, error of execution) or the use of a wrong plan to achieve an aim (ie, error of planning). Errors may be errors of commission or omission, and usually reflect deficiencies in the systems of care.19
Then, for each case where a problem in care that had contributed to death had been identified, reviewers judged the preventability of death. This two-stage approach was adopted because some problems in care contributing to death are not the result of poor practice (eg, a patient experiencing an intracerebral bleed after appropriate administration of a thrombolytic drug following myocardial infarction). Neither the problem nor the death would be regarded as preventable. In other cases where a problem in care had contributed to death, the problem may have been preventable but the patient's concurrent illness was so complex or severe that, the death itself was not judged preventable during that admission. Reviews focused on the admissions during which death occurred, but reviewers identified problems that occurred prior to that admission if these appeared to have contributed to a patient's death.
In line with previous RCRRs, reviewers assessed preventability on a 6-point Likert scale (box 2) which reflects the probabilistic nature of reviewers' decision making more closely than requiring a simple ‘yes’ or ‘no’ response.4 The validity of this approach was demonstrated in the Harvard Medical Practice Study. Deaths were judged preventable if reviewers felt that there was more than a 50% chance the death was preventable (4–6 on the scale). This included all deaths in which reviewers judged the death was ‘definitely preventable’, ‘strong evidence it was preventable’ and ‘probably preventable’. It excluded those deemed ‘definitely not preventable’, ‘slight evidence of preventability’ and ‘possibly preventable but not very likely’.
Scale used to judge preventability of death
Definitely not preventable.
Slight evidence for preventability.
Possibly preventable but not very likely, less than 50–50 but close call.
Probably preventable, more than 50-50 but close call.
Strong evidence for preventability.
Definitely preventable.
The review process
The reliability of the reviews was maximised by: the use of experienced medical reviewers; providing reviewer training and written guidance; ongoing support from the research team with the opportunity to raise and discuss questions within the reviewer group; and the use of a structured data collection form.4 ,11 ,21 ,22
Given that previous studies have found that many problems in care are related to general clinical care processes (including omissions of care) rather than specialist technical processes and that our sample would include many patients with multiple pathologies, we required reviewers with a generalist orientation. To achieve this, 17 recently retired doctors all of whom had extensive experience as generalists (15 internal medicine and 2 general surgeons) were recruited through the Royal College of Physicians and other contacts. When necessary, specialist medical advice was available either from other reviewers within the group or from outside. This was most often used to obtain a surgical opinion.
Reviewers underwent one day of training in the review technique and could contact the principal investigator (HH) with any queries during the reviewing period. In addition, each case that was considered to be a preventable death was discussed with the principal investigator and an expert reviewer (GN). Two reviewers were allocated to each site and each reviewed 50 records to make a total of 100 for that site. Reviewers had previously had no connection with their allocated site. As reviews took place on site, they were able to request additional materials such as laboratory reports stored on computer, if these were missing from the clinical record. To determine inter-rater reliability, 25% of the records were re-reviewed by another reviewer.
Medical review form
Reviewers were asked to consider all aspects of patient care and review the entire record for the index admission, including nurses' and allied health professionals' notes, drug charts and diagnostic test results. Information was recorded by hand on a structured Medical Review Form. Demographic and clinical information on each patient included age, sex, admitting specialty (medical; surgical), type of admission (elective; emergency), comorbidity (number of conditions), and functional impairment based on the Karnofsky Performance Status Scale (none; mild; moderate; severe).23 In all cases where a problem in care was judged to have contributed to death, reviewers reported on the type of problem, its timing and any associated causative or contributory factors before making a judgement as to whether the death was preventable.
Reviewers estimated life expectancy on admission taking into account admitting diagnosis, functional state and degree of urgency of the admission. The use of a prognostic epidemiological tool based on survival analysis was rejected as it requires information that may not be present in case records. A similar approach to the one we adopted was used both in empirical studies of adverse events,9 ,11 and in the development of tools to assess quality of care.24
Reviewers also rated overall quality of care by first rating each phase of care (initial assessment, treatment plan, ongoing monitoring and preparation for discharge) and then the overall quality of care on a scale from very poor to excellent, using a validated method.25
Analyses
Anonymised data were entered onto EpiData 3.1 and Microsoft Access databases and analysed using STATA (version 11.2) software. Demographic and health service utilisation data for the 10 hospital Trusts and for England were obtained from Hospital Episode Statistics. Summary statistics included proportions, means and medians. For all comparisons of rates, descriptive statistics and frequency tables were used, and tests for comparison of proportions in two independent groups corrected for binomial distribution.
Results
Study sample characteristics
The study sample was representative of patients who die in hospital in England as regards age, admitting specialty and type of admission (table 1). Reviewers made the ‘determination of a problem in care’ (κ 0.54; 95% CI 0.37 to 0.71) and ‘preventable death’ (κ 0.49; 95% CI 0.2 to 0.8) with moderate inter-rater reliability. The wider CIs for preventable deaths reflect the fact that there were 17 preventable deaths among the 250 charts randomly selected for double review. There was substantial intrarater agreement in assessing ‘life expectancy’ (weighted κ 0.66; 95% CI 0.53 to 0.79).
Comparison of study sample and all National Health Service hospital deaths in England (2009)
Patients experiencing a problem in care
In the first stage of review 131 (13.1%; 95% CI 10.9 to 15.1) patients were identified as having a problem in care that contributed to their death. There were no statistically significant differences (at p<0.05) in the characteristics of patients who experienced a problem in care and those that did not (n=869) as regards age, sex or comorbidity (table 2). However, on admission, patients who experienced a problem in care were more likely to be admitted under surgical specialties (23.6% vs 12.7%, p<0.005), as an elective admission (9.4% vs 4.5%, p<0.05) and be less severely impaired (46.3% vs 71.7%, p<0.001) than those in whom no problem was identified. Fifty-five (45.5%) of the former group were judged to have a life expectancy of more than 1-year compared with 86 (10.7%) of the latter (p<0.001).
Comparison of the characteristics of patients who died having experienced a problem in care that contributed to their death with those that did not
Reviewers rated the overall quality of care received by patients to be excellent or good for 726 (73.8%, 95% CI 70.9 to 76.4) patients. Inevitably, the proportion was lower for the 131 patients deemed to have experienced a problem in care, 47 (37.0%; 95% CI 29.1 to 45.7; p<0.001) (table 3).
Reviewers rating of the overall quality of care received by patients
Preventable hospital deaths
Fifty-two deaths (5.2%; 95% CI 3.8 to 6.6) were judged to be preventable (ie, received a rating of 4–6), that is 39.7% of the 131 cases found to have a problem in care contributing to death. There were no significant differences between the proportions of preventable deaths found at each hospital. Only four (8%; 95% CI 3.1 to 18.8) of these patients were judged to have received excellent or good quality care overall and in these cases the problem in care occurred before the final hospital admission. In 33 (66.0%; 95% CI 52.2 to 77.6) patients suffering a preventable hospital death the overall quality of care was rated poor or very poor compared to 25 (2.9%; 95% CI 1.78 to 4.0) (p<0.001) in those with no problem in care.
Patients with preventable deaths were more likely to be admitted under surgical specialities (30.8% vs 13.3%, p=0.0004). While problems occurred in all phases of care (table 4), 37 (44.0%; 95% CI 33.9 to 54.7) of the problems that contributed to a preventable death occurred during ward care. Of the rest, 13 (15.5%; 95% CI 9.3 to 24.7) patients experienced problems in care before admission (of whom five experienced no further problems in care after admission).
Phases of care during which problem in care that contributed to death occurred. (More than one option may apply for each patient)
A wide range of types of problems were identified in patients whose death was judged to be preventable (table 5). In 73.1% (95% CI 59.7% to 83.2%) of preventable deaths more than one problem in care was identified. The most frequent problems related to clinical monitoring (31.3%; 95% CI 23.9 to 39.7), diagnosis (29.7%; 95% CI 22.5 to 38.1) and drugs or fluid management (21.1%; 95% CI 14.9 to 29.0). Clinical monitoring problems included failure to act upon results of tests or clinical findings, to set up monitoring systems, to respond to such systems or to increase the intensity of care when required. Problems with diagnosis occurred at all steps in the diagnostic process from physical examination to seeking specialist help if necessary. Examples of cases are provided in box 3.
Types of problems in care that contribute to patient death (More than one option may apply for each patient).
Examples to illustrate nature of deaths judged to be preventable
A female patient in her early 80s presenting with watery diarrhoea where the diagnosis of inflammatory bowel disease took 18 days despite a past history of the disease. The patient had deteriorated significantly before appropriate treatment was commenced and failed to respond.
A middle aged male patient who developed infection at the site of a pharyngeal pouch excision. Antibiotic treatment was continued despite a failure to improve and subsequent open drainage proved too late.
A male patient in his 60s with previous history of ischaemic heart disease and treated carcinoma of the bladder (with no evidence of progression/ recurrence) underwent an unnecessary therapeutic ascitic tap when misdiagnosed as recurrent cancer when the actual diagnosis was congestive cardiac failure. He suffered a myocardial infarction after the procedure and went into multi-organ failure.
An obese woman in her 40s who presented with malaise, vomiting, anorexia, weight loss, early saity and night sweats. The diagnosis of ovarian malignancy took 21 days to confirm. On day 19 the patient's breathlessness and tachycardia were treated as a chest infection. Two days later she collapsed and subsequently died from pulmonary embolism. No risk assessment undertaken or thromboprophylaxis prescribed during stay.
A 30 year old man with a history of drug and alcohol use admitted with worsening shortness of breath and green sputum. Initially condition treated as a community acquired pneumonia until CT scan showed possible lung abscess or empyema. Patient developed clostridium difficile diarrhoea which delayed chest drainage and then went on to have a cardiac arrest when an attempt at drain insertion was subsequently made on the ward. Following transfer to the intensive care unit and drain insertion he continued to deteriorate and died.
A female patient in her 80s on warfarin for atrial fibrillation and admitted with an infected finger which had been treated with a combination of antibiotics by her general practitioner. Despite daily warfarin at a dose of 1mg being continued, the international normalised ratio (INR) was not checked until day 3, 1-day after blood was first noted in her stools. When the INR was found to be well above therapeutic levels at 10, vitamin K and fresh frozen plasma were administered with the clinical team commenting that a preferred treatment was not available at the time. Despite ongoing resuscitation she continued to deteriorate and died.
Impact of preventable hospital deaths
If 5.2% of deaths in hospital are preventable, there would be 11 859 (95% CI 8712 to 14 983) adult preventable hospital deaths in English National Health Service (NHS) acute hospitals each year (based on 228 065 adult deaths in acute hospitals in England in 2009).26 If a more demanding definition of preventable is employed (scores of 5 and 6 only on the Likert scales) our estimate of preventable deaths falls from 5.2% to 2.3% (5245 deaths), though this excludes deaths that reviewers thought were ‘probably preventable’. Using a more relaxed definition (scores of 3 to 6 on the Likert scale, thus including ‘possibly preventable but not very likely) the proportion rises from 5.2% to 8.5% (19 385 deaths).
The median estimated life expectancy of those suffering a preventable death in hospital was 6 months (IQR 4 months to 2 years) with 60% of cases having a life expectancy of <1 year.
Discussion
Main findings
Among 1000 adult patients dying in acute hospitals in England, death was considered preventable in 5.2% of cases (95% CI 3.8% to 6.6%). Preventable deaths were more common among surgical admissions. The problems associated with preventable deaths occurred in all phases of hospital care but were most likely in wards (44%) and involved poor clinical monitoring (31%), diagnostic errors (30%), or inadequate drug or fluid management (21%).
Our best estimate of preventable deaths is based on a midpoint threshold on the Likert scale (categories 4–6), which may over or underestimate the actual proportion. Adopting a stricter definition in which deaths that reviewers judged to be ‘probably preventable—more than 50–50 but close call’ were excluded resulted in 2.3% defined as preventable but this would have excluded some preventable deaths. In contrast, a more relaxed definition which includes ‘possibly preventable, but not very likely’ resulted in 8.5% preventable though this would include deaths which are unlikely to be preventable.
These findings suggest there would have been 11 859 preventable deaths among adults in acute hospitals in England in 2009. Many of these deaths occured in elderly, frail patients with multiple comorbidities, with 60% judged to have had less than 1-year of life left to live.
Strengths and limitations of the study
Our study has a number of strengths: the large, representative sample drawn from Trusts in different regions and of different size and teaching status; our use of ‘problem in care’ rather than the commonly used ‘adverse event’ to minimise the risk of overlooking errors of omission; and the various measures to standardise data collection and ensure high quality record review.
Nonetheless, several limitations need to be considered. First, medical records may not document all problems in care, though this limitation applies to all RCRR studies, including ones that have generated previous estimates of preventable hospital deaths. Second, the estimates of life expectancy were dependent on reviewers' judgement, a notoriously difficult task. Third, RCRR studies are often criticised because of the poor reliability of the reviewers' judgements. We used a number of approaches to improve reliability and obtained a moderately strong inter-rater agreement that compared favourably with previous studies. Some researchers have advocated using two reviewers for each case but this has not been shown to significantly improve reliability compared to employing a single reviewer.27 Moreover, had we required agreement between two reviewers to count a case as a preventable death our estimate would have fallen to 2.8%. Thus, any problem with reliability is likely to have led to overestimating preventable deaths, not underestimating them. Another problem in RCRRs is hindsight bias, in which knowing the outcome and its severity influence the judgement of causation and preventability.28 However, this problem would also be expected to overestimate preventable deaths, not underestimate them.
We chose to use experienced generalist reviewers rather than specialist reviewers, the majority of whom were physicians rather than surgeons. We thus ran the risk of biasing the judgement of the technical aspects of surgical care. This might have led to an underestimation of the number of preventable deaths if errors in these processes were not spotted. In fact, we found a higher proportion of both problems in care and preventable deaths among surgical patients than medical patients, the majority of these being related to ward care rather than technical care. Our findings do resonate with previous reports that highlight that surgical patients do not always receive optimal management of their medical conditions.29–31 However, it is possible that the greater risk for preventable deaths among surgical patients in our study reflects the impact of prognosis on reviewers' judgements. Reviewers typically did not judge deaths as preventable in the setting of imminent death or short life expectancy due to comorbid conditions. Patients with very short life expectancies due to underlying conditions are probably less likely to be admitted to surgical services. Consequently, surgical services have a greater proportion of patients for whom reviewers might judge problems in care judged as directly contributing to death.
Comparison with existing evidence
Our estimate of 11 859 preventable hospital deaths is similar to an estimate from the Netherlands which was based on 3983 patients dying in 25 Dutch hospitals in 2005.11 However, our estimate is much lower than that suggested in 2000 by the Chief Medical Officer (60 000 to 255 000 serious disability or death),2 derived from studies in USA which not only included relatively small numbers of deaths but did not examine the relationship between problems in care and death.3 ,4 Our estimate is also inconsistent with suggestions of 25 000 deaths in England from venous thromboembolism,32 if most of those are considered preventable. The difference from previous estimates is all the more surprising for two reasons: our more inclusive definition would have identified more ‘problems in care’ and, therefore, more preventable deaths; and the methodological limitations of this study outlined above suggest we probably overestimated the number of preventable deaths. The difference from earlier estimates appears to have arisen because these estimates were based on unjustified extrapolations.
The observation that patients were more likely to experience a problem in care if they were less functionally impaired, were elective admissions and had a longer life expectancy on admission was inconsistent with studies in other countries and might reflect a bias among reviewers towards discounting problems in the most frail, sick patients. We tried to avoid this bias by requiring reviewers to examine the entire record to the same depth and in the same structured way for all patients. Instead, we feel this finding may reflect a greater willingness in England than in some other countries to limit the extent of interventions in frail patients which would put them at less risk of experiencing a problem in care. This is inevitably speculative and would be worthy of further investigation in an international comparative study.
Implications for practice, policy and research
Although the quality of care that three-quarters of patients received was judged to be good or excellent, there is clearly plenty of scope for improvement in clinical practice. The principal area of concern is clinical monitoring on the ward. This finding is consistent with previously voiced concerns and has already prompted various quality improvement initiatives. These include Early Warning Score Systems to avoid delay in identifying deteriorating patients,33 explicit handover procedures to ensure vital clinical information is passed between clinicians,34 and critical care outreach services.35
There are also implications for policy. While the spectre of preventable hospital deaths may prove helpful in raising interest in patient safety and a commitment to improvement, overestimating the size of the problem and the risk to patients may induce unjustified levels of anxiety and fear among the public. In addition, confirmation of the relatively small proportion of deaths that appear to be preventable provides further evidence that overall hospital mortality rates are a poor indicator of quality of care.14
This does not mean that preventable deaths should be ignored and no attempt made to improve our understanding of their causes. Indeed, this is one of the key areas for further research and we shall report on more detailed analyses of the type, place and timing of problems in care. Analyses will focus on clinical monitoring problems to ascertain if areas such as the early identification of deteriorating patients continue to threaten patient safety. If so, this will raise questions as to why the impact of existing initiatives has not been greater.
Mortality reviews have been adopted as a tool to identify serious harm arising from healthcare. Furthermore, given that many patients who die in hospital have been subjected to a complex series of medical interventions, studying deaths is likely to help identify a wide range of problems in care. However, it would be unwise to limit safety and quality monitoring to this relatively small proportion of patients, when the majority of problems in care may result in morbidity and disability rather than death. Further research needs to adopt a wider perspective of outcomes. There is also a need to consider other areas of secondary care, in particular preadmission care in ambulances and accident and emergency departments, and primary care where little is known about problems in care leading to serious morbidity and preventable deaths. And finally, research is required into the ways in which feedback of information on hospital mortality can be used effectively to reduce the occurrence of problems in care.
Acknowledgments
The authors thank the 10 English acute hospital Trusts for agreeing to take part in this study, allowing us access to their medical records and providing support for the record reviewers while on site. The authors thank all the medical record reviewers for their time and the Royal College of Physicians for help with reviewer recruitment. The authors also thank Amanda Cale and Jenny Neuburger for advice, Martin Gulliford, Cameron Willis, Richard Lilford and Michael Rigby for insightful comments on earlier drafts of this manuscript, and the National Institute of Health Research, Research for Patient Benefit Programme for funding (PB-PG-1207-15215).
References
Supplementary materials
Press release
Press release
Files in this Data Supplement:
- Press release - Press release
Footnotes
Funding The funders of the study, the National Institute of Health Research, Research for Patient Benefit Programme had no role in study design, data collection, data analysis, data interpretation, or composition of the report. The corresponding author had full access to all data in the study and had final responsibility for the decision to submit for publication. The views expressed in this publication are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.
Competing interests All authors have completed the unified competing interest form at http://www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare that neither authors nor their family relations have a financial or non-financial interest that might be relevant to the submitted work.
Patient consent Patients in the study were deceased. Section 251 of the National Health Service Act 2006 for the use of patient identifiable information without consent was gained.
Ethics approval Ethics approval was received from the National Hospital for Neurology and Neurosurgery and the Institute of Neurology joint multi-centre research ethics committee and research governance approval was granted by each participating Trust.
Provenance and peer review Not commissioned; internally peer reviewed.