• critical care
• healthcare quality improvement
• quality measurement

Sepsis is a leading cause of death and suffering, afflicting 1.7 million adults annually in the USA and contributing to over 250 000 deaths.1 The high burden of sepsis has catalysed numerous performance improvement and policy initiatives, including mandatory sepsis protocols in a growing number of US states, the Centers for Medicare and Medicaid Services’ (CMS) ‘SEP-1’ measure, and WHO’s resolution declaring sepsis a global health priority.2 Hospitals around the world are dedicating considerable resources to improving sepsis recognition and compliance with treatment bundles.

However, accurately measuring the impact of sepsis quality improvement efforts is challenging. The core problem is that diagnosing sepsis involves considerable subjectivity.3 Sepsis is a heterogeneous syndrome without a pathological gold standard. It is defined as infection leading to organ dysfunction,4 but it is often unclear whether a patient is infected and whether organ dysfunction is due to infection or other factors such as dehydration, medications, cancer, or inflammatory diseases.

The challenge of sepsis measurement is compounded by the rapidly changing clinical and regulatory milieu. Clinicians are being encouraged to screen for sepsis more aggressively and both clinicians and administrators are being encouraged to code for sepsis and organ dysfunction more diligently to maximise reimbursement. The net effect is that many patients that previously were never labelled with sepsis are now being counted.3 5–8 These additional cases tend to have milder disease and lower mortality rates. If a hospital uses administrative codes to track sepsis, there is a high probability they will see higher sepsis case counts and lower sepsis mortality rates that are due at least in part to more ascertainment.9 10

Some hospitals prospectively track cases that trigger sepsis screens or perform retrospective audits of hospitalisations flagged by administrative data.11 12 New York state’s ‘Rory’s Regulations’ allow hospitals to use either method, while the CMS SEP-1 process measure requires retrospective case abstractions. These methods are more rigorous than using diagnosis codes alone but are still susceptible to suggesting misleading rises in sepsis rates and declines in mortality because of increasing sepsis awareness and recognition. Ascertainment bias is particularly problematic when all patients with suspected infection are counted, not just those in whom infection is confirmed or even probable. As an example, one healthcare system reported a reduction in sepsis mortality by more than 50% after implementing a quality improvement initiative, but tellingly twice as many sepsis cases were included in their case counts.13

These challenges in surveillance are a major impediment for hospital sepsis quality improvement initiatives. The uncertainty in knowing whether decreases in sepsis mortality are due to improvements in care versus simply expanding the pool of patients being classified as septic limits the ability of hospitals to know if their sepsis initiatives are bearing fruit. This in turn limits hospitals’ capacity to make informed decisions about how to optimise their sepsis programme.

Hospitals and policy-makers need a more objective measure to track progress and inform further improvements. Recognising this, the U.S. Centers for Disease Control and Prevention (CDC) created an ‘Adult Sepsis Event’ toolkit to help hospitals measure sepsis rates and outcomes using standardised clinical criteria that can be automated using routine electronic health record (EHR) data.14 This new tool offers hospitals a more reliable means to track sepsis since the criteria are clear and reproducible, have concrete rules for associating organ dysfunction with infection and focus on patients who receive sustained courses of antibiotics rather than simply suspected to have infection. Automated Adult Sepsis Event surveillance is more objective than tracking diagnosis codes and sepsis screens, easier to maintain than registries or retrospective audits, and able to track all hospitalised patients with sepsis rather than the small sample required by CMS.

The Adult Sepsis Event definition was initially developed as part of a multicentre study conducted to estimate the US national burden of sepsis using EHR data.1 In accordance with the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3),4 the Adult Sepsis Event definition identifies patients with presumed serious infection and concurrent organ dysfunction. Presumed serious infection is defined as a blood culture order and administration of at least 4 days of antibiotics (fewer if death or discharge occurs before 4 days). Acute organ dysfunction is defined as the initiation of vasopressors or mechanical ventilation, elevated lactate, or changes in creatinine, total bilirubin or platelets relative to clearly delineated baseline values. A comparison of Adult Sepsis Event, Sepsis-3 and CMS SEP-1 criteria is shown in table 1, while a comparison with other sepsis surveillance methods is shown in table 2.

In contrast to Sepsis-3 clinical criteria,15 the Adult Sepsis Event definition was designed for retrospective surveillance rather than early detection and real-time clinical decision-making. The definition requires ≥4 days of antibiotics to minimise false positives from patients who get just a few days of empiric antibiotics that are stopped once infection is no longer suspected. In addition, it simplifies and adapts the Sequential Organ Failure Assessment (SOFA) score by eliminating components that are inconsistently measured, documented and stored in EHRs, such as mental status, vasopressor doses, urine output, blood gas results (which are often unreliably labelled as arterial vs venous) and FiO₂ at the time blood gases were drawn. This makes it more objective and suitable for widespread automated use among hospitals with different EHR systems.

Medical record reviews suggest that Adult Sepsis Events has superior sensitivity and similar positive predictive value compared with sepsis diagnosis codes for identifying patients meeting Sepsis-3 criteria.1 It also generates more credible estimates of sepsis trends since the sensitivity of EHR-based clinical criteria is more stable over time than diagnosis codes.10 Applying this definition to EHR data from over 400 US hospitals showed that sepsis incidence and short-term mortality rates were stable from 2009 to 2014, in contrast to administrative codes from the same hospitals which gave the impression of steep increases in incidence and decreases in mortality.1

We believe that implementing Adult Sepsis Event surveillance can yield important benefits for hospitals, clinicians and patients. First, it will enable hospitals to better measure the impact of their sepsis prevention and treatment initiatives since it identifies patients in a consistent manner independent of diagnoses assigned by physicians, hospital coders or quality auditors. This will provide more confidence to hospitals that mortality improvements are due to better care rather than better detection and labelling of less ill cases. Conversely, outcomes that remain static can help identify the need for further improvements in processes and care.

Second, Adult Sepsis Events can provide hospitals with a more complete picture of sepsis incidence, trends and outcomes than the CMS ‘SEP-1’ measure since the latter is a process measure and only includes a maximum of 20 cases per month. Applying Adult Sepsis Events electronically allows hospitals to track all sepsis cases. In addition, it sidesteps some of the variability inherent in SEP-1’s requirement to abstract time zero.16 This is a time-consuming and subjective task that requires reviewers to pore over multiple notes, vital signs, laboratory data, medications and nursing flowsheets to find the first documentation of suspected infection and judge whether organ dysfunction is present, new and related to infection.16 Adult Sepsis Events avoids these challenges by focusing instead on the day of sepsis onset as marked by concurrent blood culture draws, antibiotic starts and organ dysfunction on the same or adjacent calendar days.

Third, Adult Sepsis Events give more insight into when sepsis occurred during hospitalisation compared with discharge diagnosis codes, which are limited in their granularity to present-on-admission flags. Hospitals can use this to facilitate retrospective audits of processes of care. It can also give hospitals a better window into community-onset versus hospital-onset sepsis since present-on-admission codes are often inaccurate and variably applied by hospitals.17 Monitoring community-onset sepsis incidence rates could inform pre-hospital or ambulatory prevention initiatives to identify and treat infections before they progress to full-blown sepsis. Tracking hospital-onset sepsis incidence may help inform hospitals’ efforts to prevent nosocomial infections.

Fourth, objective sepsis surveillance is a first step towards more meaningful comparisons between hospitals. Currently, it is difficult to discern whether observed variations in sepsis outcomes reflect differences in case mix, quality of care, or diagnosis and coding practices. In New York state, where sepsis criteria and case-finding methods are not standardised, early analyses have demonstrated considerable variability in the percentage of hospitals’ sepsis cases found in administrative data that are reported to the state.18 Hospital networks may find Adult Sepsis Event surveillance helpful for internal quality improvement efforts by identifying hospitals with unexpectedly low or high sepsis mortality in order to elucidate the organisational or care factors that separate those hospitals. Credible comparisons will also require rigorous risk adjustment to account for differences in patient populations. This is not currently included in the Adult Sepsis Events toolkit, but CDC has flagged this as a priority for future development. Risk adjustment for Adult Sepsis Events could use methodology similar to New York state’s model19 but focus on data routinely found in EHRs.

Fifth, tracking Adult Sepsis Events provides another avenue to monitor and improve antibiotic use given that it relies on ≥4 antibiotic days to identify presumed infection. Audits could assess the choice, timeliness and duration of antibiotics. Integrating antimicrobial stewardship efforts with hospitals’ sepsis campaigns may help balance the need for early and aggressive antibiotics to begin treatment with aggressive de-escalation once a pathogen is identified or infection no longer seems likely.

To be sure, the Adult Sepsis Event definition has important limitations. Like all sepsis surveillance methods, the Adult Sepsis Event definition incorporates clinical judgements such as the decision to draw a blood culture, prescribe antibiotics or start vasopressors. It thus remains susceptible to variation and changes in practice patterns between clinicians and over time. It may also flag some patients who were not truly infected (despite receiving ≥4 days of antibiotics) or who had organ dysfunction unrelated to infection. This is also true of all other sepsis surveillance methods, but Adult Sepsis Events at least associate presumed infection with organ dysfunction in a consistent and reproducible manner. Validations of the accuracy of Adult Sepsis Events have thus far have been encouraging,1 but further work is needed to assure consistency and generalisability across diverse settings. Adult Sepsis Events may miss some patients who meet Sepsis-3 criteria since the organ dysfunction criteria for the two definitions are similar but not identical. Patients missed by Adult Sepsis Events, however, tend to have milder cases of organ dysfunction that trigger SOFA organ dysfunction criteria but not Adult Sepsis Event criteria, such as hypoxaemia without need for mechanical ventilation or hypotension without need for vasopressors.1 The Adult Sepsis Event also risks sowing confusion by adding another definition to a crowded field. However, we believe different sepsis definitions are appropriate for different purposes, particularly real-time guidance for clinical care (eg, Sepsis-3) versus retrospective surveillance for rigorous case-counting and outcome monitoring.20 Additional research is needed to better understand the overlap and differences between Adult Sepsis Events and other sepsis criteria, and to adapt the definition for use in paediatric populations.

Lastly, from an operational standpoint, automating Adult Sepsis Event surveillance requires informatics expertise, resources and a reasonably sophisticated EHR. However, a common data specification and analytic code is available from CDC,1 14 and the effort to maintain surveillance should be low after the initial programming and validation stage. If electronic implementation cannot be done, hospitals could alternatively choose to manually abstract Adult Sepsis Event cases. This would be time-consuming but more feasible if performed on a random sample of patients, and might still be simpler and more objective than applying SEP-1 or Sepsis-3/SOFA criteria.

In conclusion, the Adult Sepsis Event definition is an important step towards making sepsis surveillance more objective and providing more reliable information on sepsis incidence and outcomes to clinicians, quality officers, policy-makers and public health officials. This in turn carries the promise of helping drive further innovations and improvements in the prevention, detection and management of this devastating illness.