Is “failure to rescue” derived from administrative data in England a nurse sensitive patient safety indicator for surgical care? Observational study

https://doi.org/10.1016/j.ijnurstu.2012.10.016Get rights and content

Abstract

Background

Failure to rescue’ – death after a treatable complication – is used as a nursing sensitive quality indicator in the USA. It is associated with the size of the nursing workforce relative to patient load, for example patient to nurse ratio, although assessments of nurse sensitivity have not previously considered other staff groups. This study aims to assess the potential to derive failure to rescue and a proxy measure, based on long length of stay, from English hospital administrative data. By exploring change in coding practice over time and measuring associations between failure to rescue and factors including staffing, we assess whether two measures of failure to rescue are useful nurse sensitive indicators.

Design

Cross sectional observational study of routinely collected administrative data.

Participants

Discharge data from 66,100,672 surgical admissions to 146 general acute hospital trusts in England (1997–2009).

Results

Median percentage of surgical admissions with at least one secondary diagnosis recorded increased from 26% in 1997/1998 to 40% in 2008/2009. Regression analyses showed that mortality based failure to rescue rates were significantly associated (P < 0.05) with several hospital characteristics previously associated with quality, including staffing levels. Lower rates of failure to rescue were associated with a greater number of nurses per bed and doctors per bed in a bivariate analysis. Higher total clinically qualified staffing (doctors + nurses) per bed and a higher number of doctors relative to the number of nurses were both associated with lower mortality based failure to rescue in the fully adjusted analysis (P < 0.05); however, the extended stay based measure showed the opposite relationship.

Conclusion

Failure to rescue can be derived from English administrative data and may be a valid quality indicator. This is the first study to assess the association between failure to rescue and medical staffing. The suggestion that it is particularly sensitive to nursing is not clearly supported, nor is the suggestion that the number of patients with an extended hospital stay is a good proxy.

Introduction

Failure to rescue’ refers to the death of a hospital patient after a treatable complication (Silber et al., 1992). The rate of failure to rescue, derived from routine administrative data, is recognised and used as patient safety indicator by the United States (US) Agency for Healthcare Research and Quality (PSI 4, now renamed “Death among Surgical Inpatients with Serious Treatable Complications” (Agency for Healthcare Research and Quality, 2007). It holds the promise of being more sensitive to the quality of care in a hospital than either conventional mortality or complication rates (Silber et al., 2007). Failure to rescue has been identified as being particularly sensitive to the quality of nursing (Clarke and Aiken, 2003) and endorsed as a nurse sensitive quality measure (National Quality Forum, 2004) but it has not been widely used or reported outside North America. In this paper we assess the feasibility of deriving failure to rescue indicators for surgical patients from English hospital administrative data, which have previously been assessed as unsuitable for the purpose, primarily because secondary diagnoses are not sufficiently well recorded (McKee et al., 1999). We also assess the relationship between failure to rescue and a number of markers of hospital quality including staffing by both nurses and physicians.

Mortality rates are widely used to indicate the quality of care in hospitals, but variation in mortality is largely due to factors unrelated to hospital care (Mant, 2001). Rates must be adjusted to reflect differences in the underlying risk of the population that is treated if valid comparisons are to be made between hospitals (Iezzoni, 1997). However, different risk adjustment models give different estimates of individual risk of death and identify different hospitals as performing outside normal limits (Iezzoni, 1997). Failure to rescue is proposed as an alternative, or complementary, indicator. It is hypothesised that the ability of a hospital to successfully treat (rescue) a patient who suffers a complication is strongly related to the quality of care provided, whereas the occurrence of the complication is more closely related to the patient's underlying risk (Silber et al., 1995a, Silber et al., 1995b). Because failure to rescue indicators consider only patients who have developed a serious but treatable complication, they offer a partial solution to the problems of risk adjustment, because the population is more homogenous and the underlying risk of death is less variable, since all patients included in the denominator are severely ill (McDonald et al., 2007, Silber et al., 2007, Silber et al., 1995a, Silber et al., 1995b). There is empirical evidence that failure to rescue rates are more closely associated with hospital characteristics including nurse staffing levels and less influenced by patient characteristics than either complications or mortality (Silber et al., 2007, Silber et al., 1995a, Silber et al., 1995b).

The potential significance of this measure is reflected in recent reports and research into responses to deteriorating patients in acute care that emphasise the numerous potential points of failure prior to initiating appropriate intervention including:

  • not taking observations;

  • not recording observations;

  • not recognising early signs of deterioration; and

  • not communicating observations (Clarke, 2004, Luettel et al., 2007).

Because of the role of nurses in early identification of deterioration, failure to rescue has been widely advocated as a nursing sensitive outcome indicator in hospitals (Clarke and Aiken, 2003, Griffiths et al., 2008, Naylor, 2007), since observation may be compromised when staffing is not adequate. An association between low levels of nurse staffing and high levels of failure to rescue is supported by meta-analysis of observational studies. The increased odds of failure to rescue is estimated as 16% per additional patient per nurse (Kane et al., 2007). Other characteristics which have been associated with hospital quality, such as nursing skill mix (richer), hospital size (larger) and teaching status (Aiken et al., 2002, Hartz et al., 1989, Jarman et al., 1999) have also been associated with lower rates of failure to rescue (Aiken et al., 2002, Silber et al., 2007) but numbers of doctors, an important part of the hospital staff in many countries outside the USA, have not been widely studied, even though doctors too have a role in surveillance and the medical response to deterioration is also likely to be an important determinant of outcome. Higher medical staffing levels have also been associated with lower mortality rates (Bond et al., 1999, Jarman et al., 1999).

Because failure to rescue indicators need to identify a group of patients who experience particular complications, the validity of the indicators can be compromised if coding of secondary diagnoses in the administrative data set is poor. In the absence of codes to indicate diagnoses that are present on admission, the indicators must also rely on complex exclusion rules in order to eliminate pre-existing comorbidity. Because of the difficulty doing this for medical cases (Horwitz et al., 2007, Moriarty et al., 2010), use of the indicators has generally been recommended for surgical cases only.

The under recording of secondary diagnoses in administrative databases is a known issue. Previously, McKee et al. reported that English hospital data from 1996/1997 and 1997/1998 were unsuitable for deriving failure to rescue measures, primarily because of low rates of coding (McKee et al., 1999). Doubts about both the accuracy and completeness of coding have continued to be raised in the UK and other countries (Casez et al., 2010, Leibson et al., 2008, Williams and Mann, 2002). Although a recent systematic review suggested improvements and overall acceptable accuracy for coding in the UK, studies revealed substantial variation between hospitals and mainly focused on the primary diagnosis/procedure (Burns et al., 2012).

While classic studies of failure to rescue have looked at mortality in a sub group of patients presumed to have treatable suffered complications (e.g.Aiken et al., 2002, Silber et al., 2007) an alternative has been proposed. The alternative approach is predicated on the recognition that death is not the only possible result of a “failure” to rescue. If failure to rescue results in serious deterioration that in turn leads to extended hospital stay, then stays that fall well outside the norm can be used as a proxy indicator of failure to rescue. This approach has previously been used in a UK study exploring links between nurse staffing and failure to rescue (Rafferty et al., 2007).

Thus this study is an exploratory study that aims to assess the potential for deriving mortality based failure to rescue indicators and a proxy measure, based on exceptionally long length of stay, from English hospital administrative data by exploring change in coding practice over time and measuring associations between failure to rescue and factors that suggest how the indicator will perform as a quality measure. These factors include the association between failure to rescue and depth of coding (number of complications recorded) and staffing by doctors, nurses and support workers.

Section snippets

Methods

Our assessment is based on the approaches undertaken by McKee et al. (1999) and Silber et al. (2007). Specifically we consider:

  • Whether coding of secondary diagnoses has increased since the previous assessment (McKee et al., 1999) – indicating improved potential for deriving mortality based failure to rescue indicators.

  • Whether failure to rescue rates are associated with coding practices (rates of secondary diagnostic coding and rates of complications coded) in order to determine potential for

Data sources

To calculate failure to rescue rates, we used hospital discharge data from the National Health Service (NHS) Commissioning Data Sets (CDS) data from April 1997 to March 2009 to identify all admissions for surgical procedures to general acute National Health Service (NHS) hospitals in England (146 hospital trusts in 2008/2009 – a trust may comprise several hospital sites). The CDS provides a record of admission method, diagnoses, procedures, discharge dispositions and patient demographic details

Results

Between 1997/1998 and 2008/2009, there were 66,100,672 eligible surgical admissions (day case and inpatient) of whom 442,462 (0.7%) died and 4,993,863 (7.6%) experienced a long hospital stay, above the 25th percentile for their HRG. The median percentage of surgical admissions with at least one secondary diagnosis recorded increased from 26% in year 1997/1998 to 40% in 2008/2009. Overall 2,496,356 patients (3.8%) had an eligible complication for FTR-A of whom 226,237 died (9.1%). Overall

Discussion

Our results point to improved coding practice in English hospital data and a relatively stable failure to rescue rate derived from them. We have observed several associations between failure to rescue and presumed markers of quality, including clinical staffing levels, which have been previously associated with hospital mortality and failure to rescue. This suggests that the FTR-A indicator we derived from English data may well be a valid measure of quality. However, the claim that failure to

Conclusions

We conclude that there is potential to derive mortality based failure to rescue indicators for surgical patients from routine administrative data in England. Such indicators may offer some advantages over standardised mortality measures, such as HSMR, for surgical patients. Our FTR-A indicator, based on the AHRQ definition, is a potentially valid quality indicator which can complement HSMR, but like overall mortality it needs to be properly risk adjusted to facilitate benchmarking and

Authors’ contributions

PG conceived and designed the study jointly with SJ and AB. AB and SJ extracted data. AB and SJ mapped the AHRQ indicators to English coding and ICD 10 with advice from PG on clinical codes. SJ undertook statistical analysis and AB, PG and SJ interpreted the results. PG and SJ drafted the paper and AB, PG and SJ commented on drafts and approved the final version. SJ is guarantor for the extraction of data and analysis, PG for other aspects of the paper including the design, interpretation of

References (38)

  • E. Burns et al.

    Systematic review of discharge coding accuracy

    Journal of Public Health

    (2012)
  • A. Cameron et al.

    Regression Analysis of Count Data

    (1998)
  • S.P. Clarke

    Failure to rescue: lessons from missed opportunities in care

    Nursing Inquiry

    (2004)
  • S.P. Clarke et al.

    Failure to rescue: needless deaths are prime examples of the need for more nurses at the bedside

    American Journal of Nursing

    (2003)
  • D.E. Farrar et al.

    Multicollinearity in regression analysis: the problem revisited

    Review of Economics and Statistics

    (1967)
  • H. Gao et al.

    Systematic review and evaluation of physiological track and trigger warning systems for identifying at-risk patients on the ward

    Intensive Care Medicine

    (2007)
  • P. Griffiths et al.

    State of the Art Metrics for Nursing: A Rapid Appraisal

    (2008)
  • A.J. Hartz et al.

    Hospital characteristics and mortality rates

    New England Journal of Medicine

    (1989)
  • L.I. Horwitz et al.

    Failure to rescue: validation of an algorithm using administrative data

    Medical Care

    (2007)
  • Cited by (37)

    • Perioperative patient outcomes in the African Surgical Outcomes Study: a 7-day prospective observational cohort study

      2018, The Lancet
      Citation Excerpt :

      Surgery is a cost-effective and core component of universal health coverage,5–7 but it needs to be safe.4 Known barriers to the provision of safe surgical treatment in Africa include low hospital procedural volumes,8 few hospital beds,9 and a scarce number of operating theatres,10 all of which are compounded by the geographical remoteness of many surgical hospitals and an absence of adequately trained staff.11,12 The Lancet Commission on Global Surgery13 was established to develop strategies for safe, accessible, and affordable surgical care, but implementation of this strategy requires robust epidemiological data describing patterns of surgical activity and subsequent patient outcomes.7,13

    • Is there a “weekend effect” in emergency general surgery?

      2018, Journal of Surgical Research
      Citation Excerpt :

      If differences in staffing and hospital resources contributed to the weekend effect, national data might be expected to show higher rates of failure to rescue (FTR) for patients admitted at weekends. FTR has been defined as “death after a treatable complication.”17 It has been widely adopted as a quality metric and is thought to reflect the ability of health-care providers to respond effectively to complications.17,18

    • Surgical resident involvement differentially affects patient outcomes in laparoscopic and open colectomy for malignancy

      2016, American Journal of Surgery
      Citation Excerpt :

      To support this notion, Raval et al,25 when controlling for multiple factors including operative time, demonstrated that 6.1 additional patients undergoing general surgical/vascular procedures had morbidity events when a resident was involved; however, 1.4 more lives were saved. It is important to note that failure to rescue has previously been associated with staffing levels, both physician and nursing.43 As such, our data may be a proxy for the notion that more eyes on the surgical patient, whether resident physician, attending physician, nurses, or other ancillary staff, produce less failure to rescue.

    • Ten years into the job: A nursing journal for the future

      2015, International Journal of Nursing Studies
    View all citing articles on Scopus
    View full text