Article Text
PDF
Abstract
Background Adverse events (AEs) and no-harm incidents are common and of great concern in healthcare. A common method for identification of AEs is retrospective record review (RRR) using predefined triggers. This method has been used frequently in inpatient care, but AEs in home healthcare have not been explored to the same extent. The aim of this study was to develop a trigger tool (TT) for the identification of both AEs and no-harm incidents affecting adult patients admitted to home healthcare in Sweden, and to describe the methodology used for this development.
Methods The TT was developed and validated in a stepwise manner, in collaboration with experts with different skills, using (1) literature review and interviews, (2) a five-round modified Delphi process, and (3) two-stage RRRs. Ten trained teams from different sites in Sweden reviewed 600 randomly selected records.
Results In all, triggers were found 4031 times in 518 (86.3%) records, with a mean of 6.7 (median 4, range 1–54) triggers per record with triggers. The positive predictive values (PPVs) for AEs and no-harm incidents were 25.4% and 16.3%, respectively, resulting in a PPV of 41.7% (range 0.0%–96.1% per trigger) for the total TT when using 38 triggers. The most common triggers were unplanned contact with physician and/or registered nurse, moderate/severe pain, moderate/severe worry, anxiety, suffering, existential pain and/or psychological pain. AEs were identified in 37.7% of the patients and no-harm incidents in 29.5%.
Conclusion This study shows that adapted triggers with definitions and decision support, developed to identify AEs and no-harm incidents that affect patients admitted to home healthcare, may be a valid method for safety and quality improvement work in home healthcare.
- adverse events, epidemiology and detection
- patient safety
- trigger tools
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Background
Advanced medical care is moving from hospitals to homes for patients with severe or multiple diseases.1 2Patients in home healthcare often have multimorbidity and are high consumers of both healthcare and social care. Responsibility for their care is fragmented between different care levels and caregivers.3 This may lead to new challenges and patient safety risks that are important to identify. Patient safety has been investigated in hospital care in many studies,4–7 but more rarely in home healthcare settings.1 8 Studies of home healthcare in Canada showed an adverse event (AE) rate up to 13% and common AEs were falls with injury, adverse drug events, wound infections and pressure ulcers.1 8 9
Home care systems appear to differ both between and within countries.10 In Sweden, home healthcare can be provided by either county councils or municipalities, but the county councils always provide the physician resources. Home healthcare in Sweden is defined as healthcare that is administered in a patient’s home or the equivalent, and that is consistent over time.11 Home healthcare does not encompass home care organisations with unlicensed staff administering social care.3
Structured retrospective record review (RRR) of healthcare records is a valid and established method, proven to identify AEs that often go unnoticed when using, for example, incident reporting systems.12–15 One of the most commonly used RRR methods is the Global Trigger Tool, developed by the Institute for Healthcare Improvement, USA. This tool has been further developed for different settings and focus areas within healthcare.4 A trigger (or clue) is a word or an event in a record that could indicate that an AE has occurred, such as fall or transfusion. Some triggers are broad/implicit, such as readmission within 30 days, while some are narrow/explicit, such as pressure ulcer. A reviewer must investigate the record to determine the presence or absence of an AE.16 RRR focuses on AEs, while no-harm incidents are usually excluded. However, Schildmeijer et al 17 showed that no-harm incidents could also be efficiently and successfully identified through RRR. There is limited knowledge about both AEs and no-harm incidents in home healthcare.
Home healthcare in Sweden mainly targets the group of people who are most frail, that is, older people with multimorbidity, with multiple specialists involved in treatment and care. Provision of care, monitoring of symptoms and assessment of treatment is performed intermittently, in the patient’s private home, by healthcare professionals employed by different care providers, governed by different levels of government (municipal and county) and regulated by different laws (ie, the Social Services Act and the Health and Medical Services Act). There is an urgent need for methods to identify and learn from both AEs and no-harm incidents, regardless of caregiver, in home healthcare. The aim of this study was to develop a trigger tool (TT) for the identification of both AEs and no-harm incidents affecting adult patients admitted to home healthcare in Sweden, and to describe the methodology used for this development.
Methods
Design
The TT was developed in a stepwise manner, in close collaboration with experts with different skills, using (1) literature review and interviews, (2) a five-round modified Delphi process18 including face-to-face meetings with group discussions, and (3) two RRR tests of the TT before Delphi rounds 2 and 3, respectively. A flow chart of the development and validation process of the home healthcare TT is provided in figure 1. A thorough description of steps 1 and 2 in this process can be found in online supplementary appendix A.
Supplementary file 1
Flow chart of the development and validation process of the home healthcare trigger tool. RRR, retrospective record review.
Recruitment of review teams and Delphi panel
To test the triggers and collect clinical professionals’ feedback in a structured manner, review teams were invited through personal contacts or by email via the patient safety network of the Swedish Association of Local Authorities and Regions. All review teams interested in participation were included. In total, seven teams volunteered from municipal home healthcare and three from county councils’ specialised home healthcare. The teams represented geographically distinct parts of Sweden and consisted of one to three registered nurses and one or two physicians, all experienced in home healthcare, in total 28 clinicians. Some municipal teams had no physician actively involved in team discussions, but had a consultant available when needed.
The 28 clinicians in the review teams also participated as clinical experts in the Delphi panel, which consisted of 41 experts in home healthcare, patient safety, RRR methodology and/or TT design (online supplementary appendix B, table B.1).
Supplementary file 2
The RRR process
Definitions
In the RRR process, an AE was defined as suffering, physical or psychological harm, illness or death caused by healthcare or social care that was not an inevitable consequence of the patient’s condition or an expected effect of the treatment received by the patient because of her/his condition. A no-harm incident was an event caused by healthcare or social care that reached the patient and could have led to an AE, but resulted in no discernible harm.19
Based on the terminology in the Swedish Patient Safety Act,20 a preventable AE or no-harm incident was defined as an event that could have been prevented if adequate actions had been taken during the patient’s contact with healthcare or social care.
AEs and no-harm incidents related to acts of either omission or commission were included.
Study sample, inclusion and exclusion criteria
Two RRR tests were performed using different versions of the TT. For the second test, we estimated that at least one AE would occur in 17% of admissions and calculated that a sample of 600 randomly selected admissions would be sufficient to estimate the cumulative incidence of AEs with a 95% CI of ±3.01%. Admissions during 2015 of patients at least 18 years old were eligible for randomisation.
Each record was reviewed for a maximum of 90 days from admission to home healthcare (index admission). If a patient had been discharged from home healthcare and readmitted within the 90-day period, the review of the record continued.
To be included as an AE or no-harm incident in the study, one of the following criteria had to be met:
The AE or no-harm incident occurred during the index admission, that is, within 90 days after enrolment in home healthcare, regardless of caregiver.
The AE or no-harm incident derived from caregivers outside home healthcare (outpatient care, social care or inhospital care), occurred within 30 days prior to the index admission and was detected during the index admission.
AEs or no-harm incidents that occurred but were detected more than 90 days after the index admission or that occurred, were detected and for which action was completed before the index admission were excluded. Randomisation was performed by one of the authors (MU), using an online randomiser, to ensure it was carried out in the same way for all review teams. Oversampling was carried out with 10 records per team. If a patient in the random sample was receiving limited home healthcare once or twice a week, for example, blood pressure measurement or delivery of predispensed drugs, this patient was replaced by another randomly selected admission.
Team training
In addition to getting an overview of the TT methodology in connection with the first Delphi round, all review team members underwent a mandatory 1 day education in the TT methodology before the second test of the triggers. Before this education, all team members carried out independent reviews of six training records. Most team members were unfamiliar with the TT methodology before this study. During the education, a consensus process was carried out, including discussions of definitions, inclusion and exclusion criteria, interpretation and application of the triggers, and assessments of AEs, no-harm incidents and preventability. Strategy discussions were held concerning how to make the review process more reliable and efficient. An example of the description of a trigger is shown in online supplementary appendix B, table B.2. As a result of discussions during the education, three new triggers were added ahead of the second test: pressure ulcer, escape from home/special accommodation and absence of in-depth drug review.
RRRs in a two-stage procedure
In the first RRR test, the teams reviewed 60 non-random records using 35 triggers in four modules (Care, Laboratory, Medication, and Continuity and transition). The aim was to test the triggers and the study manual, as well as the feasibility of obtaining correct patient lists via the respective care providers’ patient administrative systems. The study manual included, for example, AE and no-harm incident descriptions, definitions, and inclusion and exclusion criteria.
In the second and main RRR test, the review teams reviewed 600 random records using 38 triggers.
The review teams had differing access to some parts of the records. Municipalities and county councils generally have different record systems. Some municipal review teams had to request physicians’ notes and laboratory values, for example, since these were stored in a county council’s record system.
RRR was carried out in a two-stage procedure. In most teams, the registered nurses carried out the primary and secondary reviews and then discussed their findings with the physicians until they reached consensus. In some teams, the physicians carried out some of the primary and secondary reviews. A trigger list and a study manual were used as decision support in both the primary and secondary review stages.
In the primary review, all records were reviewed, without any time restriction. The reviewers screened for the presence of one or more of the predefined triggers. For each trigger detected the reviewer judged whether it reflected the presence of a potential AE or no-harm incident. The reviewers documented how many times each respective trigger was present per record. Only records with triggers indicating at least one potential AE or no-harm incident went forward to secondary review. The reviewers also recorded demographic data.
In the secondary reviews, each potential AE/no-harm incident was reviewed separately. To qualify as an AE/no-harm incident, a score of 3 or higher on a 4-point Likert Scale was required (1=the AE/no-harm incident was not related to healthcare/social care, 2=the AE/no-harm incident was probably not related to healthcare/social care, 3=the AE/no-harm incident was probably related to healthcare/social care and 4=the AE/no-harm incident was related to healthcare/social care). A similar 4-point scale was used to judge the preventability of the AE/no-harm incident. The severity of the AE/no-harm incident was judged using two different severity scales.5 21 All secondary reviewers documented, for example, all triggers related to the AE/no-harm incident, how many times each respective trigger was present per event, as well as information on the source of the AE/no-harm incident (home healthcare, inpatient care, outpatient care or social care).
During the review process, support was available via email and telephone, primarily from one of the researchers (MU). The team could also pose questions to the physician (LN) in the research group.
At the end of the second RRR test, the review teams rated each respective trigger for clinical relevance, comprehensibility and utility, respectively, on a 4-point Likert Scale (1=low grade, 2=rather low grade, 3=rather high grade and 4=high grade).
Reliability and validity in the review process
The primary review process was evaluated concerning inter-rater reliability. Ten per cent of the records in the second test were at least double-reviewed to assess agreement between the reviewers’ judgements as to whether or not a record was to be included in secondary review. The judgements in the double-reviewed records were discussed between reviewers to reach consensus, as a basis for the secondary review stage.
An RRR expert (MU) monitored all primary and secondary reviews for completeness and adherence to the trigger definitions and study manual. No double review was performed in the secondary review stage. The secondary reviewers’ outcomes were compared with the primary reviews, trigger definitions and descriptions and the study manual, including methodology. All questions or discrepancies were referred back to the teams for resolution. If discrepancies were found, clarifying discussions were held with the respective teams.
Statistics
Categorical data from the RRR were summarised using frequency counts and percentages. Continuous variables are presented as means with SD and medians with range.
Cohen’s κ22 was calculated for inter-rater reliability between primary reviewers.
The positive predictive value (PPV) of each trigger was calculated as the number of times this trigger identified an AE or no-harm incident, divided by the total number of times that the trigger was found, multiplied by 100.
To compute an index for each trigger concerning clinical application, comprehensibility and utility, respectively, the rating of either 3 or 4 on a 4-point Likert Scale was divided by the number of respondents per trigger.
The statistical programme used in analyses was Statistica V.13 (StatSoft, Tulsa, Oklahoma, USA).
Results
Demographics
The study sample reflected 40 735 care days for index admissions and the mean number of reviewed days was 67.9 (SD 30.9, median 90 days, range 1–90). The mean age of patients was 78.2 years (SD 12.4, median 80.5 years, range 20–99), and 53.3% were female (n=320). Half of the patients (50.0%) were referred to home healthcare from hospital care. Malignancy and cardiovascular disease were the most common medical diagnoses (table 1).
Demographic data
Inter-rater reliability
The inter-rater reliability of the primary reviewers’ judgements concerning if a record was to be forwarded to secondary review was κ=0.801 (substantial).
Trigger outcome from the second RRR test
Triggers were identified 4031 times in total, in 518 (86.3%) records, resulting in a mean of 6.7 (SD 7.9) triggers per record with triggers (median 4, range 1–54). Patients who were affected by an AE and/or no-harm incident had a median of five triggers. Individual triggers varied in their detection of AEs (PPV range 0.0%–72.4%) and no-harm incidents (PPV range 0.0%–66.7%) after secondary review. The total PPVs for AEs and no-harm incidents were 25.4% and 16.3%, respectively, resulting in a PPV for the total TT of 41.7% (range 0.0%–96.1% per trigger) when using 38 triggers (table 2).
Outcome of each respective trigger in relation to adverse events (AEs) and no-harm incidents, sorted by positive predictive value (PPV) in total, % for the respective module
The Care module had the highest total PPV for AEs (28.5%) and no-harm incidents (17.2%). The Laboratory module was the least predictive for both AEs and no-harm incidents (table 2).
The triggers with the highest PPVs were fall, documentation related to insufficient coordination of care, communication and/or information and adverse drug event/adverse drug reaction. The most commonly found triggers were unplanned contact with physician and/or registered nurse and unplanned change in care-providing unit. Common explicit triggers were moderate/severe pain, blood vessel, skin and/or tissue harm, fall and moderate/severe gastrointestinal problem (table 2). The triggers detected events with a wide range of severity, from events that reached the patient without harm occurring, to events that contributed to patient death. The triggers most often involved in AEs that contributed to permanent harm or death were deviation from normal course after invasive procedure/surgical treatment, adverse drug event/adverse drug reaction, unplanned contact with physician and/or registered nurse, unplanned change in care-providing unit, and cardiac arrest and/or deterioration in vital signs.
AEs and no-harm incidents
After secondary review, AEs were identified for 37.7% of the patients and no-harm incidents for 29.5%. Of all AEs, 23.9% were derived from caregivers outside home healthcare and the corresponding for no-harm incident was 17.3%.
Trigger tool refinement
After the second RRR, the review teams rated each respective trigger for clinical relevance, comprehensibility and utility, respectively, on a 4-point Likert Scale. Each respective Likert Scale was computed to an index: for clinical relevance (range 0.5–1.0), for comprehensibility (range 0.67–1.0) and for utility (range 0.5–1.0) (table 2). The trigger other had the lowest index for clinical relevance and utility. Although this trigger had a low PPV in total (28.7%), it was retained, as a TT needs a trigger to categorise events not covered by other triggers.
In the fourth Delphi round, the review teams verbally evaluated the TT as useful, with relatively high ease of use, but reported some difficulties in applying a few of the trigger definitions, such as treatment. Several teams experienced that record documentation was in some cases fragmented and not easily accessible, as data were stored in several systems.
Triggers with low PPV (<50%) were seldom stand-alone or were infrequent. For example, treatment and deep vein thrombosis and/or pulmonary embolism were removed from the trigger list due to PPVs of 0%. Triggers such as treatment with drugs that increase the risk for haemorrhage and absence of and/or deviation from care plan were also removed, as corresponding AEs/no-harm incidents would be detected by triggers such as adverse drug event/adverse drug reaction or insufficient planning, coordination, communication and/or information in the final trigger list. Patients with severe or multiple diseases often have disease-related deteriorations in laboratory values. Furthermore, laboratory triggers were seldom detected and most of the linked events were detected by other triggers as well. Therefore, the module Laboratory value with its five triggers was removed. In total, 13 triggers were removed and three triggers were merged into one (table 2). Some of the remaining trigger definitions and descriptions were refined with the aim of achieving a more valid TT, thus reducing the false-positive trigger outcomes. For example, cardiac arrest and/or deterioration in vital signs was renamed unexpected cardiac arrest and/or deterioration in vital signs.
The outcome after the fifth Delphi round was a list containing 23 triggers in three modules: Care; Medication; and Continuity and Transition (box 1).
Final trigger list containing 23 triggers in three modules
Triggers
Care module
Unexpected cardiac arrest and/or deterioration in vital signs
Pressure ulcer
Blood vessel, skin and/or tissue harm
Neurological impairment and/or harm
Fall
Healthcare-associated infection
Moderate/severe pain
Moderate/severe worry, anxiety, suffering, existential pain and/or psychological pain
Moderate/severe change in psychological and/or behavioural status
Undernutrition
Insufficient oral health
Gastrointestinal malfunction
Distended urinary bladder
Deviation from normal course after invasive procedure/surgical treatment
Occurrence of any complication in connection with use of medical device
Documentation of mistake or dissatisfaction with care
Other
Medication module
Adverse drug event/adverse drug reaction
Drug that requires follow-up with blood sampling
Drug management
Continuity and transition module
Unplanned change in care-providing unit
Unplanned contact with physician and/or registered nurse
Insufficient planning, coordination, communication and/or information
Discussion
This study aimed to develop a TT for the identification of both AEs and no-harm incidents affecting adult patients admitted to home healthcare. The 38 empirical tested triggers were identified 4031 times in the records and constituted a rich material for trigger validation. This yielded an overall PPV of 41.7% for the TT. The final TT included 23 triggers in three modules which will form the basis for a national patient safety tool. AEs were identified in 37.7% of the patients and no-harm incidents in 29.5%.
There is no gold standard for validating RRR methods. Several studies have compared RRR with other methods for identifying AEs, and RRR has been found to be the best single method,12–15 although multiple methods are preferred.4 23 Another common way to validate a TT is to use trigger occurrence and PPVs,23 as in this study. We identified more triggers per record than several other RRR studies, reporting 2.2–4.7 triggers per record.24–27 The occurrence and PPV of triggers varied widely and is in line with the results of other studies regardless of setting and patient population.28–31 Our high overall PPV may in part be related to the fact that we included no-harm incidents as well. Furthermore, since PPV is highly influenced by prevalence, a high overall PPV value also may be reflected for high prevalence of AEs and no-harm incidents. Other TTs have a PPV ranging from 3.7% to 50%.25 26 31–36 However, it is difficult to find guidance in the literature concerning which PPV is acceptable for a trigger and RRR method.
Some of the triggers with low PPVs, such as moderate/severe pain and malnutrition, have been included in the final trigger list after clarification, as they represent important aspects of home healthcare quality. Most of the patients had multimorbidity and cancer diagnoses in a palliative stage, where laboratory triggers and triggers such as cardiac arrest and/or deterioration in vital signs were not applicable. The low PPV for some no-harm incidents was due to some triggers being AEs in themselves, such as healthcare-associated infection and pressure ulcer.
To our knowledge, there are few studies using the RRR method to assess AEs in home healthcare.1 8 In these studies unplanned hospital care was common, which indicates that patients in home healthcare have complex patient safety issues involving multiple caregivers. Our study found more triggers related to falls and drugs than those studies.
The final trigger list was constructed in accordance with the Swedish TT methodology,37 making it possible to include no-harm incidents (National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) Index categories C and D).21 We found 647 triggers related to no-harm incidents, with a PPV of 16.1%. This is in line with the conclusions of an earlier study17 showing that RRR methodology is suitable for identifying such safety information while searching records for AEs as a way to inform proactive patient safety management.
The Delphi process is considered a strong methodology when the aim is to get a panel of experts with different skills to achieve a degree of consensus on a specific topic. It has been applied in modified ways,38 39 giving rise to debates about the applicability of certain principles, for example, selection and definition of experts, anonymity and the number of ‘rounds’. Like other studies using the Delphi process,40–42 we found it suitable and valuable in the development of the TT. Although participant anonymity is often considered a key factor in a Delphi process,43 the literature is divided on this. Anonymity could be a facilitator which enables the participants to be open and truthful without being influenced by the other participants.44 45 On the other hand, discussions could increase knowledge and develop judgement.41 According to Keeney et al,38 complete anonymity cannot be guaranteed and it is difficult to predict how it affects findings. In our study, we used a combination of anonymity (written feedback) and non-anonymity (face-to-face meetings) based on earlier positive experiences.31 Our experience from the current study was that the experts were able to voice their opinions, contributed equally and that no member dominated the discussions. Furthermore, the face-to-face meetings gave a deeper and broader variety of clarifications of qualitative justifications for the TT development than the written feedback. We used controlled feedback at each stage of the process, which is considered to be a key strength in the Delphi process.18
Another challenge in Delphi is how many rounds are to be used before consensus is achieved. We had five rounds; while there are four rounds in the classic Delphi process.46 We adapted the number of rounds to fit the different stages of TT development, since we included RRR in this process. The criticism of having many rounds has been that participant interest and engagement will decrease.38 We experienced increased engagement over time, which could be related to increased knowledge of the method and the review teams’ empirical experiences of the triggers.
Interest and knowledge of home healthcare were important aspects when recruiting review teams. A majority of those performing the reviews were inexperienced in TT methodology, but were experts in home healthcare and skilled in assessing if an AE/no-harm incident was related to healthcare, social care or an underlying medical condition. In this context, the boundaries can be subtle. Severe pain related to a medical condition is an AE if it is not identified and treated in a timely and proper way.
Strengths and limitations
Our study was strengthened by teams representing a wide range of home healthcare services and different parts of the country. We have transparently described the review process, the Delphi process, as well as the heterogeneous Delphi panel’s composition, which was consistent with good characteristics for a Delphi panel.47 48 We also tested the triggers in a clinical context before the final revision as an integrated part of the Delphi process to increase implementability.48 However, we did not empirically test the final version of the TT as we did not add any new triggers, but only merged, slightly renamed or removed some triggers with low PPV or face and content validity. The final TT might improve the PPV but this remains to be tested in a large scale. One well known limitation is the RRR method, which requires high-quality record documentation. Events not reported or not accessible for the reviewers will have been missed, along with their triggers. Another limitation is that we have not compared the outcome of the TT with any other method. However, there is a consensus in the scientific literature that RRR is a valid method and identifies more AEs than most other methods.12–15 Further, we did not use anonymity throughout the Delphi process. While we experienced that all Delphi members were able to voice their opinions, there might have been occasions when someone felt they did not have that opportunity. Not all municipality review teams had a physician directly involved, but all teams had access to a physician when needed.
Conclusions
Caregivers and professionals need valid and reliable methods to identify quality and safety issues and to follow-up interventions. This study shows that use of adapted triggers with definitions and decision support, developed to identify AEs and no-harm incidents that affect patients admitted to home healthcare, may constitute a valid method for safety and quality improvement work in home healthcare.
Acknowledgments
The authors thank the members of the Delphi panel and the review teams, as well as the Swedish Association of Local Authorities and Regions (SALAR) and the National Board of Health and Welfare, which provided support during the study.
References
Footnotes
Contributors The authors jointly designed and conducted the study and contributed with their expertise in methodology and home healthcare to the development, analysis and revision process of the trigger tool. MU monitored the teams and record review outcomes. MU and LN performed analysis of the data from the RRR study, which was followed by discussions with all the authors. All authors drafted and approved the final manuscript.
Funding Financial support was provided through FORSS (No 470161) Medical Research Council of Southeast Sweden and the Region Östergötland (LiO537211). The funders have not been involved in any part of the study, in writing the manuscript or the decision to submit the manuscript for publication.
Competing interests None declared.
Ethics approval Regional Ethics Committee of Linköping (numbers 2014/150-31 and 2016/45-32).
Provenance and peer review Not commissioned; externally peer reviewed.