Is “failure to rescue” derived from administrative data in England a nurse sensitive patient safety indicator for surgical care? Observational study
Introduction
‘Failure to rescue’ refers to the death of a hospital patient after a treatable complication (Silber et al., 1992). The rate of failure to rescue, derived from routine administrative data, is recognised and used as patient safety indicator by the United States (US) Agency for Healthcare Research and Quality (PSI 4, now renamed “Death among Surgical Inpatients with Serious Treatable Complications” (Agency for Healthcare Research and Quality, 2007). It holds the promise of being more sensitive to the quality of care in a hospital than either conventional mortality or complication rates (Silber et al., 2007). Failure to rescue has been identified as being particularly sensitive to the quality of nursing (Clarke and Aiken, 2003) and endorsed as a nurse sensitive quality measure (National Quality Forum, 2004) but it has not been widely used or reported outside North America. In this paper we assess the feasibility of deriving failure to rescue indicators for surgical patients from English hospital administrative data, which have previously been assessed as unsuitable for the purpose, primarily because secondary diagnoses are not sufficiently well recorded (McKee et al., 1999). We also assess the relationship between failure to rescue and a number of markers of hospital quality including staffing by both nurses and physicians.
Mortality rates are widely used to indicate the quality of care in hospitals, but variation in mortality is largely due to factors unrelated to hospital care (Mant, 2001). Rates must be adjusted to reflect differences in the underlying risk of the population that is treated if valid comparisons are to be made between hospitals (Iezzoni, 1997). However, different risk adjustment models give different estimates of individual risk of death and identify different hospitals as performing outside normal limits (Iezzoni, 1997). Failure to rescue is proposed as an alternative, or complementary, indicator. It is hypothesised that the ability of a hospital to successfully treat (rescue) a patient who suffers a complication is strongly related to the quality of care provided, whereas the occurrence of the complication is more closely related to the patient's underlying risk (Silber et al., 1995a, Silber et al., 1995b). Because failure to rescue indicators consider only patients who have developed a serious but treatable complication, they offer a partial solution to the problems of risk adjustment, because the population is more homogenous and the underlying risk of death is less variable, since all patients included in the denominator are severely ill (McDonald et al., 2007, Silber et al., 2007, Silber et al., 1995a, Silber et al., 1995b). There is empirical evidence that failure to rescue rates are more closely associated with hospital characteristics including nurse staffing levels and less influenced by patient characteristics than either complications or mortality (Silber et al., 2007, Silber et al., 1995a, Silber et al., 1995b).
The potential significance of this measure is reflected in recent reports and research into responses to deteriorating patients in acute care that emphasise the numerous potential points of failure prior to initiating appropriate intervention including:
- •
not taking observations;
- •
not recording observations;
- •
not recognising early signs of deterioration; and
- •
not communicating observations (Clarke, 2004, Luettel et al., 2007).
Because of the role of nurses in early identification of deterioration, failure to rescue has been widely advocated as a nursing sensitive outcome indicator in hospitals (Clarke and Aiken, 2003, Griffiths et al., 2008, Naylor, 2007), since observation may be compromised when staffing is not adequate. An association between low levels of nurse staffing and high levels of failure to rescue is supported by meta-analysis of observational studies. The increased odds of failure to rescue is estimated as 16% per additional patient per nurse (Kane et al., 2007). Other characteristics which have been associated with hospital quality, such as nursing skill mix (richer), hospital size (larger) and teaching status (Aiken et al., 2002, Hartz et al., 1989, Jarman et al., 1999) have also been associated with lower rates of failure to rescue (Aiken et al., 2002, Silber et al., 2007) but numbers of doctors, an important part of the hospital staff in many countries outside the USA, have not been widely studied, even though doctors too have a role in surveillance and the medical response to deterioration is also likely to be an important determinant of outcome. Higher medical staffing levels have also been associated with lower mortality rates (Bond et al., 1999, Jarman et al., 1999).
Because failure to rescue indicators need to identify a group of patients who experience particular complications, the validity of the indicators can be compromised if coding of secondary diagnoses in the administrative data set is poor. In the absence of codes to indicate diagnoses that are present on admission, the indicators must also rely on complex exclusion rules in order to eliminate pre-existing comorbidity. Because of the difficulty doing this for medical cases (Horwitz et al., 2007, Moriarty et al., 2010), use of the indicators has generally been recommended for surgical cases only.
The under recording of secondary diagnoses in administrative databases is a known issue. Previously, McKee et al. reported that English hospital data from 1996/1997 and 1997/1998 were unsuitable for deriving failure to rescue measures, primarily because of low rates of coding (McKee et al., 1999). Doubts about both the accuracy and completeness of coding have continued to be raised in the UK and other countries (Casez et al., 2010, Leibson et al., 2008, Williams and Mann, 2002). Although a recent systematic review suggested improvements and overall acceptable accuracy for coding in the UK, studies revealed substantial variation between hospitals and mainly focused on the primary diagnosis/procedure (Burns et al., 2012).
While classic studies of failure to rescue have looked at mortality in a sub group of patients presumed to have treatable suffered complications (e.g.Aiken et al., 2002, Silber et al., 2007) an alternative has been proposed. The alternative approach is predicated on the recognition that death is not the only possible result of a “failure” to rescue. If failure to rescue results in serious deterioration that in turn leads to extended hospital stay, then stays that fall well outside the norm can be used as a proxy indicator of failure to rescue. This approach has previously been used in a UK study exploring links between nurse staffing and failure to rescue (Rafferty et al., 2007).
Thus this study is an exploratory study that aims to assess the potential for deriving mortality based failure to rescue indicators and a proxy measure, based on exceptionally long length of stay, from English hospital administrative data by exploring change in coding practice over time and measuring associations between failure to rescue and factors that suggest how the indicator will perform as a quality measure. These factors include the association between failure to rescue and depth of coding (number of complications recorded) and staffing by doctors, nurses and support workers.
Section snippets
Methods
Our assessment is based on the approaches undertaken by McKee et al. (1999) and Silber et al. (2007). Specifically we consider:
- •
Whether coding of secondary diagnoses has increased since the previous assessment (McKee et al., 1999) – indicating improved potential for deriving mortality based failure to rescue indicators.
- •
Whether failure to rescue rates are associated with coding practices (rates of secondary diagnostic coding and rates of complications coded) in order to determine potential for
Data sources
To calculate failure to rescue rates, we used hospital discharge data from the National Health Service (NHS) Commissioning Data Sets (CDS) data from April 1997 to March 2009 to identify all admissions for surgical procedures to general acute National Health Service (NHS) hospitals in England (146 hospital trusts in 2008/2009 – a trust may comprise several hospital sites). The CDS provides a record of admission method, diagnoses, procedures, discharge dispositions and patient demographic details
Results
Between 1997/1998 and 2008/2009, there were 66,100,672 eligible surgical admissions (day case and inpatient) of whom 442,462 (0.7%) died and 4,993,863 (7.6%) experienced a long hospital stay, above the 25th percentile for their HRG. The median percentage of surgical admissions with at least one secondary diagnosis recorded increased from 26% in year 1997/1998 to 40% in 2008/2009. Overall 2,496,356 patients (3.8%) had an eligible complication for FTR-A of whom 226,237 died (9.1%). Overall
Discussion
Our results point to improved coding practice in English hospital data and a relatively stable failure to rescue rate derived from them. We have observed several associations between failure to rescue and presumed markers of quality, including clinical staffing levels, which have been previously associated with hospital mortality and failure to rescue. This suggests that the FTR-A indicator we derived from English data may well be a valid measure of quality. However, the claim that failure to
Conclusions
We conclude that there is potential to derive mortality based failure to rescue indicators for surgical patients from routine administrative data in England. Such indicators may offer some advantages over standardised mortality measures, such as HSMR, for surgical patients. Our FTR-A indicator, based on the AHRQ definition, is a potentially valid quality indicator which can complement HSMR, but like overall mortality it needs to be properly risk adjusted to facilitate benchmarking and
Authors’ contributions
PG conceived and designed the study jointly with SJ and AB. AB and SJ extracted data. AB and SJ mapped the AHRQ indicators to English coding and ICD 10 with advice from PG on clinical codes. SJ undertook statistical analysis and AB, PG and SJ interpreted the results. PG and SJ drafted the paper and AB, PG and SJ commented on drafts and approved the final version. SJ is guarantor for the extraction of data and analysis, PG for other aspects of the paper including the design, interpretation of
References (38)
- et al.
ICD-10 hospital discharge diagnosis codes were sensitive for identifying pulmonary embolism but not deep vein thrombosis
Journal of Clinical Epidemiology
(2010) - et al.
Economic evaluation of nurse staffing and nurse substitution in health care: a scoping review
International Journal of Nursing Studies
(2011) - et al.
Impact of organisation and management factors ion infection control in hospitals: a scoping review
Journal of Hospital Infection
(2009) - et al.
Nurse turnover: a literature review
International Journal of Nursing Studies
(2006) - et al.
Outcomes of variation in hospital nurse staffing in English hospitals: cross-sectional analysis of survey data and discharge records
International Journal of Nursing Studies
(2007) Patient Safety Indicators (PSI) Risk Adjustment Coefficients for the PSI Version 4.3
(2011)Patient Safety Indicators (PSI) Version 3.1 Comparative Data Agency for Healthcare Research and Quality
(2007)- et al.
Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction
Journal of the American Medical Association
(2002) - et al.
Health care professional staffing, hospital characteristics, and hospital mortality rates
Pharmacotherapy
(1999) - et al.
Intelligent information: a national system for monitoring clinical performance
Health Services Research
(2008)
Systematic review of discharge coding accuracy
Journal of Public Health
Regression Analysis of Count Data
Failure to rescue: lessons from missed opportunities in care
Nursing Inquiry
Failure to rescue: needless deaths are prime examples of the need for more nurses at the bedside
American Journal of Nursing
Multicollinearity in regression analysis: the problem revisited
Review of Economics and Statistics
Systematic review and evaluation of physiological track and trigger warning systems for identifying at-risk patients on the ward
Intensive Care Medicine
State of the Art Metrics for Nursing: A Rapid Appraisal
Hospital characteristics and mortality rates
New England Journal of Medicine
Failure to rescue: validation of an algorithm using administrative data
Medical Care
Cited by (37)
Perioperative patient outcomes in the African Surgical Outcomes Study: a 7-day prospective observational cohort study
2018, The LancetCitation Excerpt :Surgery is a cost-effective and core component of universal health coverage,5–7 but it needs to be safe.4 Known barriers to the provision of safe surgical treatment in Africa include low hospital procedural volumes,8 few hospital beds,9 and a scarce number of operating theatres,10 all of which are compounded by the geographical remoteness of many surgical hospitals and an absence of adequately trained staff.11,12 The Lancet Commission on Global Surgery13 was established to develop strategies for safe, accessible, and affordable surgical care, but implementation of this strategy requires robust epidemiological data describing patterns of surgical activity and subsequent patient outcomes.7,13
Is there a “weekend effect” in emergency general surgery?
2018, Journal of Surgical ResearchCitation Excerpt :FTR has been defined as “death after a treatable complication.”17 It has been widely adopted as a quality metric and is thought to reflect the ability of health-care providers to respond effectively to complications.17,18 In this study, we examined a national database of US hospital discharges for evidence of a weekend effect in EGS.
Nurse staffing and patient outcomes: Strengths and limitations of the evidence to inform policy and practice. A review and discussion paper based on evidence reviewed for the National Institute for Health and Care Excellence Safe Staffing guideline development
2016, International Journal of Nursing StudiesSurgical resident involvement differentially affects patient outcomes in laparoscopic and open colectomy for malignancy
2016, American Journal of SurgeryCitation Excerpt :To support this notion, Raval et al,25 when controlling for multiple factors including operative time, demonstrated that 6.1 additional patients undergoing general surgical/vascular procedures had morbidity events when a resident was involved; however, 1.4 more lives were saved. It is important to note that failure to rescue has previously been associated with staffing levels, both physician and nursing.43 As such, our data may be a proxy for the notion that more eyes on the surgical patient, whether resident physician, attending physician, nurses, or other ancillary staff, produce less failure to rescue.
Ten years into the job: A nursing journal for the future
2015, International Journal of Nursing StudiesThe association between multi-disciplinary staffing levels and mortality in acute hospitals: a systematic review
2023, Human Resources for Health