Article Text
PDF
Abstract
Background and objective Incident, adverse event (AE) and complaint data are typically used separately, but may be related at the patient level with one event triggering a cascade of events, ultimately resulting in a complaint. This study examined relations between incidents, AEs and complaints that co-occurred in admissions.
Methods Independently and routinely collected incident, AE and complaint data were retrospectively linked for surgical admissions in an academic centre (2008–2014). Two investigators reviewed whether incidents/AEs in admissions were clinically related and in what sequence (incident preceding vs following AE). Likelihood of occurrence of AEs and AE cascades (ie, ≥3 AEs) was studied using logistic regression analyses.
Results Complaints were filed for 33 (0.1%) of 26 383 admissions. Complaints filed by patients with incidents and/or AEs (n=13) mostly addressed quality/safety problems, whereas other complaints mostly addressed relationship problems. Incidents and AEs co-occurred in 730 (2.8%) admissions, which seemed clinically related in 34% of these cases. Incidents with related AEs preceded as well as followed AEs (56.6%/44.4%). Patients with incidents were at greater risk of AEs than patients without incidents, even for seemingly unrelated AEs (OR 1.4; 95% CI 1.3 to 1.6). Risk of AE cascades was greater when patients with AEs also had incidents, regardless of whether these seemed related (unrelated: OR 2.0; 95% CI 1.6 to 2.5; related: OR 5.7; 95% CI 4.3 to 7.4) or whether incidents preceded or followed AEs in these admissions (53% vs 52%, P>0.05).
Conclusions Patient-level linkage of incident, AE and complaint data can reveal relations between events that otherwise remain obscured, such as incidents that trigger as well as follow AEs, introducing event cascades, regardless of whether clinical relations seem present.
- quality improvement
- quality measurement
- patient safety
- incident reporting
- adverse events, epidemiology and detection
Statistics from Altmetric.com
- quality improvement
- quality measurement
- patient safety
- incident reporting
- adverse events, epidemiology and detection
Introduction
Most hospitals have installed systems to collect quality and safety data, such as incidents, adverse events (AEs) and patient complaints. Previous studies have demonstrated that these systems each identify different types of issues.1–5 This would support using various approaches independently, and then synthesising the messages from each approach to inform improvement programmes.1 However, although each system captures different signals from the same patient journey, these may be related at the patient level. Some of these relations are obvious, but others can be less clear, such as when an AE makes a patient more vulnerable and increases case complexity, triggering a chain of events. Insight into (perhaps still unknown) relations between co-occurring events may be obtained by linking these data at the patient level. In contrast to previous studies that examined different systems for their accuracy to detect the same events,2 3 5 6 this study focuses on the relations between different events collected for the same admissions by independent systems.
The primary purpose of quality and safety data is to offer a ‘window’ onto the system, revealing underlying risks that need further investigation.7–9 However, in isolation, these data may not be used to their full potential. To illustrate, record review looks back to assess whether patient harm can be linked to preceding substandard care—if so, this is considered an AE. Incident reporting assesses the same relationship in the opposite direction by reporting suboptimal care processes and whether these cause harm. Thereby, these approaches identify one-on-one relations in one direction, that is, from process to subsequent harm. Real clinical practice is more complex, and events within a single admission can have many-to-many relations, which could also be in the opposite direction when initial harm triggers subsequent process problems (eg, delirium → incident with dislodged intravenous line → haemorrhage → anaemia → transfusion incident). Current methods are only able to capture patient harm with known relations with (problems in) care processes. After all, if the relation between a process problem and patient harm is yet unknown, the harm would not be considered an ‘AE’ by record reviewers because they are unaware of the relation with medical management, and these problems would not be considered ‘harmful incidents’ by reporters.
Linkage of the various information sources on incidents, AEs and complaints could potentially offer a more comprehensive view,10 allowing a more sophisticated analysis of (relations between) events occurring in the same admission. This would also connect different perspectives, as incidents are typically reported by nurses,4 11 whereas AE data are collected by physicians or from their notes in records, and complaints are filed by patients and their families. The Dutch healthcare system has three independent reporting systems to collect data on incidents, AEs and complaints, which each have a slightly different purpose and content (table 1). The aim of this study was to examine relations between incidents, AEs and complaints, separately reported for the same admissions, including how one event may trigger a cascade of events. This was done by retrospectively linking independent data systems for all patients hospitalised at an academic surgical department in a period of 7 years.
Methods
This retrospective cohort study linked all routinely and independently collected incident, AE and complaint data, for all 26 383 surgical inpatients discharged from a Dutch academic hospital between January 2008 and December 2014. As complaints may be lodged up to 2 years after hospitalisation,12 those received between January 2008 and June 2016 were included. The requirement for ethical approval was waived by the local Ethics Committee (#P15.352) based on the Dutch Medical Research Involving Human Subjects Act.
Definitions
An incident is defined as an event or circumstance which could have resulted, or did result in unnecessary harm to a patient, which follows the WHO definition.13 Incidents are process problems that can be harmful (ie, causing AEs) or non-harmful (ie, reportable circumstances, near misses or no-harm incidents).13 As in many other countries,3 9 14 Dutch hospitals actively encourage reporting of harmful as well as non-harmful incidents.15 It is common in the patient safety literature to refer to harmful incidents as merely ‘adverse events’,14 16 but this study distinguishes incidents (process problems regardless of patient harm) from AEs (harm regardless of process problems).
AEs represent undesired outcomes for patients, not all of which are necessarily preventable and caused by (observable) incidents. In the Netherlands, an AE is defined as any unintended or undesired event or state, occurring during or following medical care, that is so harmful to a patient’s health that adjustment of treatment is required or that permanent damage results. 17 This definition overlaps with the commonly used WHO definition (ie, injury caused by medical management rather than underlying disease 13), but additionally covers AEs related to underlying disease because the definition does not require judgement on the cause of the AE at time of reporting.
Patient complaints are defined as letters of complaint sent to the hospital by patients or on behalf of patients.
Reporting systems
Incidents and AEs are separately collected by independent reporting systems (table 1). These systems are not intended to capture the same events, but rather to offer insights into process problems regardless of outcomes (incident reporting) versus adverse patient outcomes regardless of the quality of processes (AE reporting). Incident reporting is similar to that in many other hospitals, with incidents reported by all clinical staff, but mostly by nurses, in a hospital-wide electronic system.3 14 Incident reports are short stories and usually also allow to report whether the specific incident was harmful or not. Therefore, this widespread method is only able to capture patient harm that has a one-on-one, well-known relation with the reported incident.
Features of the independent data collection systems for incidents, adverse events and complaints used in this study
AEs are reported by physicians in electronic medical records during patients’ stay.17–19 AE reports only include medical terms (eg, septic shock) and severity scores reflecting consequences for patients, that is, (1) recovery without (re)operation; (2) recovery after (re)operation; (3) (potential) irreversible harm; (4) death.18 AEs with severity ≥2 are considered ‘serious AEs’. For the present study, ‘AE cascades’ are defined as ≥3 AEs within the same admission.
Many other settings use record review to detect AEs rather than physician reporting. A prior study estimated that this type of physician-driven reporting underestimates annual AE rate by only 1.8% compared with retrospective record review.18 A benefit of this type of AE reporting combined with the Dutch definition is that all undesired outcomes are recorded without the need to identify causes in medical management,17–19 simplifying reporting and capturing a broad range of events. Because record review only captures AEs that can be related to (preceding) medical management, it will likely miss AEs related to process problems in ways that are yet unknown, or through a combination of events rather than a one-on-one relation.
Patient complaints are collected by the complaints handling and patient service offices, with copies sent to the departments involved (table 1).
Data linkage and methods
AEs were already linked to corresponding admissions, but incidents could only be linked using patient identifiers and reporting dates (available for 2708 of 3001 incidents (90.2%)). Incidents were matched if identifiers were equal and if the reporting date of the incident was on or between admission and discharge date, resulting in 2162 (79.8% of 2708) incidents matched to 1599 admissions. Most unmatched incidents had misspelled patient identifiers or concerned non-surgical patients (62.0%), and in other cases reporting dates seemed to be misspelled. Of the 104 complaints received by the surgical department (January 2008–June 2016), 43 were for inpatient admissions in the study period, of which 33 could be linked to admissions using patient identifiers and dates in letters.
Complaints were categorised using the Healthcare Complaints Analysis Tool (HCAT), scoring problems on clinical (ie, quality and safety of care), management and relational domains (ie, behaviour of staff towards patients and their family/friends).20 Incidents and AEs were classified using the WHO framework16 and a previously developed AE classification scheme.17
To assess whether incidents and AEs in the same admission were clinically related, an MD-researcher (MSdV) scored the likelihood of a clinical relation (ie, unlikely, potentially or likely) for all potential incident/AE pairs in admissions. For related incidents/AEs, the most likely sequence (incident preceding or following AE) was also scored. All potentially or likely related incident/AE pairs were additionally reviewed and scored by a second investigator (practising research nurse (JJCC-H)) and discussed until consensus was reached.
To assess whether data linkage of independent reporting systems could reveal well-known relations between incidents and AEs, two clinical themes were selected a priori: (1) delirium and patient accident incidents (using the WHO incident type, for example, falls or line removal); and (2) venous thromboembolism (VTE; ie, deep vein thrombosis or pulmonary embolism) and incidents with VTE prophylaxis (ie, low-molecular-weight heparin). Theme 1 represented an incident expected to frequently co-occur with the AE and to follow the AE, whereas theme 2 represented an incident expected to less frequently co-occur with the AE but to precede the AE.
Two AE types, wound infections and anastomotic leakage, were selected a priori to study whether incident co-occurrence would increase risk of AE cascades. For patients with wound infections, incident co-occurrence was expected to increase risk of AE cascades as it may further increase patient vulnerability, while anastomotic leakage was considered a more severe AE and hence expected to be associated with AE cascades regardless of incident co-occurrence.
Statistical analyses
Statistical analyses were conducted using SPSS Statistics (IBM, V.23) with a 0.05 α level. Complainants were compared with non-complainants on patient characteristics (age, gender, undergoing surgery or not, American Society of Anesthesiologists (ASA) physical status and emergency status at the first surgery, length of stay and presence of readmission within 30 days), overall and separately for cases with both incidents and AEs. χ2 tests were used for categorical variables (Fisher’s exact test if expected count was less than 5), with Kruskal-Wallis H tests for age and length of stay, and Mann-Whitney U test for number of incidents/AEs. Complaints for admissions with versus without incidents and/or AEs were compared on HCAT domains to study whether these addressed different issues.
To study co-occurrence, admissions with only incidents, only AEs, both incidents and AEs, and neither incidents nor AEs, were compared on the patient characteristics mentioned above. Complaints were assessed separately because of their low volume. The groups were compared on complaints filed, number of incidents/AEs, AE severity, and occurrence of serious AEs and AE cascades. Multivariable logistic regression was then performed to assess whether incident occurrence increased risk of AEs after adjustment for patient characteristics (age, gender, undergoing surgery or not and ASA status), both for all AEs and for seemingly unrelated AEs. Similarly, we assessed whether risk of long length of stay (ie, upper quintile) or readmission was increased for cases with both incidents and AEs rather than cases with only AEs.
To study risk of AE cascades, multivariable logistic regression was used among cases with AEs, comparing cases with (unrelated or related) incidents to cases without co-occurring incidents, adjusted for patient characteristics as above. The same analyses were performed conditional on having wound infections or anastomotic leakage. Consequences of AE cascades were assessed by studying risk of long length of stay and readmission for cases with AE cascades rather than only one or two AEs, using multivariable logistic regression adjusted as above.
Results
Patient complaints
Complaints were filed for 33 of the 26 383 admissions (0.1% or 1.3 per 1000) (figure 1). Most complaints were filed for cases without incidents/AEs (n=20, 60.6%) (table 2). Complainants were similar to non-complainants in all patient characteristics (data not shown), except for a longer length of stay (median: 6 days vs 3 days; P=0.015). Admissions with complaints seemed more likely than those without complaints to have both incidents and AEs, but group sizes varied greatly (4/33 (12.1%) vs 726/26 350 (2.8%); P=0.001) (figure 1). Complaints for admissions with incidents and/or AEs mostly addressed problems on the clinical domain (85% of 13), whereas other complaints mostly addressed the relational domain (75% of 20). In addition, in incident reports filed for 10 admissions, staff expressed complaints on behalf of patients or family (eg, ‘felt not taken seriously’ or ‘not informed about transfer to intensive care unit’), but none of these were filed as formal complaints.
Occurrence of complaints, incidents and adverse events in admissions.
Characteristics of admissions with or without adverse events and/or incidents
Co-occurrence of incidents and AEs
Incidents were reported for 1599 (6.1%) admissions, mostly by nursing staff (71.2%). Annual incident rates doubled following implementation of electronic reporting in 2011 from 3%–4% to 8%. AEs were reported for 4838 (18.3%) admissions, with annual AE rates ranging from 16.8% (2010) to 19.5% (2014). For 730 (2.8%) admissions, both incidents and AEs were reported (figure 1). More than half of all incidents (52.6% of 2162) were reported for patients with AEs, whereas 22.2% of all AEs (n=8870) were reported for cases with incidents. Most common incident type was medication (4.0% of all admissions) and most common AE type was infection (7.3%), which were also the most common types to cluster in admissions (data not shown).
Patient characteristics differed between groups with and without incidents/AEs (table 2). Looking at the data, patients with incidents and/or AEs seemed older, more often undergoing surgery and less often ASA 1–2 compared with other patients (table 2). In multivariable analysis, adjusted for patient characteristics, incident occurrence and AE occurrence were significantly associated (OR 3.0; 95% CI 2.7 to 3.3). Compared with patients with only AEs, patients with both AEs/incidents had more incidents and AEs, and more often serious AEs (table 2), and increased risk of long length of stay (OR 3.8; 95% CI 3.1 to 4.8) but not of readmission (OR 1.0; 95% CI 0.8 to 1.2).
Clinical relations
In 248 of the 730 (34.0%) admissions with co-occurring incidents and AEs, one or more clinical relations between incidents and AEs were identified (n=322 pairs). These included 36 unreported AEs mentioned in incident reports. In total, there were 4590 admissions with stand-alone or seemingly unrelated AEs, including 482 admissions with only unrelated incidents/AEs and 4108 admissions with only AEs. Multivariable analysis showed that incident occurrence also increased the risk of these stand-alone/seemingly unrelated AEs (OR 1.4; 95% CI 1.3 to 1.6), after adjustment for patient characteristics.
AE types that were commonly related to co-occurring incidents included ‘psychological disturbance’ (eg, delirium), ‘symptoms without diagnosis’ (eg, metabolic abnormality) and ‘rejection/allergy’, whereas ‘shock’ or ‘fistula’ were only rarely related (table 3). Incidents of the ‘patient accident’ type (eg, falls, unplanned removal of lines) were more often related than unrelated to co-occurring AEs (68.7%), whereas, for example, only 11%–13% of the incidents about documentation and administration had clinically related AEs (table 3).
Co-occurrence and presence of relations per adverse event and incident type*
Sequence
Among the 322 pairs of related incidents/AEs, incidents seemed to have preceded AEs in 55.6% and followed AEs in 44.4%. For example, one haemorrhage AE was preceded by a heparin overdose incident, while another was followed by a blood transfusion incident. Looking at the most common incident and AE types: medication incidents mostly seemed to have preceded related AEs (61.6% of 164), and incidents related to infections mostly seemed to have followed these AEs (65.1% of 63).
For the a priori selected themes, delirium was more common among cases with than without incidents of the ‘patient accident’ type (35.7% vs 1.6%; x2=594; P<0.001). This pattern of frequent co-occurrence was also visible over time (figure 2A). In general, incidents of the ‘patient accident’ type mostly seemed to have followed rather than preceded related AEs (65.2% of 46). A different pattern was observed for VTE and VTE prophylaxis. Only 2 of the 97 cases with VTE prophylaxis incidents also had VTE in the same admission (both preceding AE). These two cases occurred after a strong increase of VTE prophylaxis incidents (figure 2B), while overall VTE reporting rate remained stable (both before and after incident increase: mean 0.3% of cases per quartile).
Selected clinical themes: co-occurrence of specific incidents and adverse events in admissions over time. VTE, venous thromboembolism.
Cascades
Overall, AE cascades (≥3 AEs) were present for 845 admissions, of which 31.2% also had incidents and 0.2% had filed complaints. Admissions with incidents and AEs more commonly had AE cascades than admissions with only AEs (36.2% vs 14.1%; x2=208; P<0.001) (table 2). This difference remained in multivariable analysis, adjusted for patient characteristics, both for cases with related incidents (OR 5.7; 95% CI 4.3 to 7.4) and cases with only seemingly unrelated incidents (OR 2.0; 95% CI 1.6 to 2.5). AE cascades were just as common among cases with only incidents following related AEs as in cases with incidents preceding AEs (52.0% vs 53.0%; x2=0.023; P=0.880). Cases with AE cascades were more likely to have a long length of stay (OR 5.1; 95% CI 4.1 to 6.4), but no differences were observed for risk of readmission (OR 0.9; 95% CI 0.7 to 1.1). For cases with wound infections, having an incident strongly increased the risk of an AE cascade (55.4% vs 18.8%; P<0.001). This applied to patients with related incidents (OR 14.0; 95% CI 5.7 to 34.4) and those with only seemingly unrelated incidents (OR 3.1; 95% CI 1.7 to 5.6). Incident co-occurrence did not increase risk of AE cascades in cases with anastomotic leakage (OR 1.4; 95% CI 0.5 to 3.6), among which more than half of the patients had AE cascades, both those with incidents (65.5%) and without (56.9%).
Discussion
This study addressed patient-level relations between incidents, AEs and complaints by linking routinely collected data from independent systems. Most patients who filed complaints had no incidents or AEs and addressed relationship problems, whereas complaints for admissions with incidents and/or AEs mostly concerned quality and safety issues. Among admissions with co-occurring incidents and AEs, clinical relations between these events were identified in approximately one of three admissions. In terms of sequence, incidents seemed to have preceded related AEs in 55.6% and followed AEs in 44.4% of the clinically related incident/AE pairs. Overall, patients with incidents more commonly had AEs and AE cascades than patients without incidents, regardless of whether these AEs seemed clinically related or in what sequence. These findings demonstrate that although separate systems collect different signals from the same patient journey, these have relations at the patient level and should therefore be interpreted in relation to each other to obtain more comprehensive and detailed information for improvement efforts.
Breaking down the silos
Previous studies encouraged hospitals to use more than one method to collect data on quality and safety because each method provides complementary information, previously compared with the fable of the blind men and the elephant.1 3 21 Over the years, various systems to collect quality and safety data, such as incidents, AEs and complaints, have been implemented in different periods and isolated from each other.22 Consequently, co-occurrence cannot be evaluated and relations between events may remain obscured, such as cascades or clusters of seemingly unrelated events. While both incident and AE data (collected through reporting or record review) may be used to reveal suboptimal processes that cause harm, only linkage of these data allows an approach looking at co-occurrence and how initial harm may trigger further process problems or cascades of events. Integration of these systems would also connect perspectives of nurses, physicians and patients and may reveal unreported problems, as illustrated by our finding that incident reports revealed patient complaints not otherwise reported. Complaints are a particularly underused source of information for improvement because they mostly remain completely separated from quality and safety data.20 22 23 This study indicates that this may be a missed opportunity because complaints from patients with incidents and/or AEs specifically provided information on quality and safety from the patient perspective.
Ability to respond
Although incident reporting is known to be very poor at detecting AEs,2 3 incident reports are unique stories from the sharp end that may reveal local system hazards before harm is inflicted.24 Previous authors underlined that reports of near misses (ie, harmless incidents) can be used to study resilience of healthcare processes because these indicate successful responses to potentially harmful situations.14 25 26 Equally important is the capacity to respond to harm once inflicted, preventing that (responses to) initial AEs send patients ‘out of the frying pan into the fire’, which bears similarity to the concept of ‘failure to rescue’.27 28 Patient-level data linkage allows further study of critical elements of safety,29 namely the ability to anticipate, respond and adapt to difficulties, such as increased vulnerability and complexity of patients with AEs and incidents. This vulnerability and complexity was particularly illustrated in this study by the increased risk of AE cascades for patients with wound infections who also experienced incidents. The obtained insights can be used to enhance these abilities, which responds to calls for a more proactive and preventive approach to patient safety.30
Practical implications
Integration of quality and safety systems will require investments that may differ per institution, depending on whether information is available in a digital format and (can be) linked to corresponding admissions. For example, not all hospitals have digitalised patient complaints.22 Hospitals could start by providing a clear overview of a patient’s AEs and incidents in the medical record (integrating safety systems into electronic records), because this may support the ability of (rotating) staff to anticipate future problems for these patients. In addition, patients with both incidents and AEs could be sampled for higher priority in-depth analysis or discussion at team meetings, for example, morbidity and mortality conferences. These learning reviews should additionally address the team’s response to these events and any patient complaints, which means that complaints' data should be made accessible. This approach honours the principle that it is more valuable to thoroughly analyse a small number of events than to superficially study large volumes of data.8 31 Another practical implication would be to consider expanding the focus of record review to what happened after AEs (eg, whether the AE triggered incidents), and to encourage incident reporters to address what happened before incidents (eg, whether the incident was preceded by AEs), in order to identify chains of events.
Strengths and limitations
A strength of this study is that it is the first in its kind to study patient-level relations between different types of quality and safety data routinely collected over several years. A study limitation is that under-reporting may have affected incident rates. This may particularly apply to the years before electronic reporting after which incident rates doubled, as also observed elsewhere.1 32 AE rates were more stable and have been demonstrated to be similar to those obtained through record review.18 Moreover, incident, AE and complaint rates closely resembled those in other studies.12 32–35 We acknowledge that the AE cascade definition is arbitrary and that it remains unclear how these events are related, but this variable was used to reflect cases progressing from bad to worse, which likely resonates with clinicians. Accurate data on the exact timing of events is required to examine chains of events more closely. The finding of longer length of stay for patients with AEs/incidents should be interpreted with caution as this could also reflect greater complexity of these cases, which could have occurred regardless of incidents/AEs. Similarly, that cases with incidents were at increased risk of unrelated AEs may reflect their greater complexity for which we could not fully adjust. Although clinical relations were assessed by clinician researchers, this remains subjective, similar to record review studies.36 37 Even though generalisability is an important limitation of any single-centre study, this study presents more general messages potentially relevant for other institutions. The relations between events demonstrated in this study likely reflect a more universal underlying process of increased patient vulnerability and complexity. Therefore, these findings could encourage hospitals with other definitions or methods (eg, record review) to integrate available incident, AE and complaint data to obtain rich information that helps envision the bigger picture of patient safety.
Future directions
Future research is needed on clusters of seemingly unrelated incidents and AEs, and the impact of incidents after initial AEs. These studies could provide guidance for clinical practice by identifying what types of events warrant more vigilance in monitoring and management to prevent a negative cascade of events. Ideally, hospitals would use a linked registry to detect early warnings before (more) patients are harmed, but methods still need to be developed and validated. Another important extension could be to use integrated data from various sources to study particularly ‘safe’ teams or processes in order to increase understanding of why things go right 38 39 and to seek exemplary behaviour and solutions that are already present within the clinical community.40 41 With this study, we hope to inspire more research with patient-level linkage of currently available data in other settings and with other types of data, such as patient-reported outcome or experience data.
Conclusions
This study shows how patient-level linkage of incident, AE and complaint data can reveal relations that otherwise remain obscured, such as incidents emerging in the context of prior AEs or triggering AE cascades, even for seemingly unrelated events. As we have come to appreciate that the various data systems in hospitals offer different ‘windows onto the system’,8 we should start integrating these for a more ‘panoramic’ view on healthcare quality and safety.
Acknowledgments
The authors thank all clinicians involved in the data gathering over the years.
References
Footnotes
Contributors All authors contributed to conception and design of this study. MSdV and JJCC-H analysed the data, and MSdV, JFH and PJM-vdM interpreted the results. MSdV drafted the article and all other authors (JFH, JJCC-H, PJM-vdM) participated in revising it critically. All authors have seen and approved the final draft. All authors received access to all the data (including statistical reports and tables) in the study and take responsibility for the integrity and the accuracy of the data analysis. PJM-vdM is guarantor.
Funding MSdV was supported by a combined unrestricted grant from the board of directors of the Leiden University Medical Centre, The Vogelgezang Foundation (no. 1519-01), Leiden University Fund (no. 5265/12-11-15) and the Michäel-Van Vloten Surgery Fund. Funders had no role in the study design; data analysis and interpretation; in the writing of the report; or in the decision to submit the article for publication.
Competing interests None declared.
Patient consent Not required.
Ethics approval The local ethics committee waived the requirement for approval based on Dutch national law.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.