Article Text
PDF
Abstract
Objectives To quantify the prevalence and nature of adverse events in acute Irish hospitals in 2015 and to assess the impact of the National Clinical Programmes and the National Clinical Guidelines on the prevalence of adverse events by comparing these results with the previously published data from 2009.
Design and methods A retrospective chart review of 1605 admissions to eight Irish hospitals in 2015, using identical methods to those used in 2009.
Results The percentage of admissions associated with one or more adverse events was unchanged (p=0.48) at 14% (95% CI=10.4% to 18.4%) in 2015 compared with 12.2% (95% CI=9.5% to 15.5%) in 2009. Similarly, the prevalence of preventable adverse events was unchanged (p=0.3) at 7.4% (95% CI=5.3% to 10.5%) in 2015 compared with 9.1% (95% CI=6.9% to 11.9%) in 2009. The incidence densities of preventable adverse events were 5.6 adverse events per 100 admissions (95% CI=3.4 to 8.0) in 2015 and 7.7 adverse events per 100 admissions (95% CI=5.8 to 9.6) in 2009 (p=0.23). However, the percentage of preventable adverse events due to hospital-associated infections decreased to 22.2% (95% CI=15.2% to 31.1%) in 2015 from 33.1% (95% CI=25.6% to 41.6%) in 2009 (p=0.01).
Conclusion Adverse event rates remained stable between 2009 and 2015. The percentage of preventable adverse events related to hospital-associated infection decreased, which may represent a positive impact of the related national programmes and guidelines.
- adverse events
- epidemiology and detection
- healthcare quality improvement
- patient safety
- quality improvement
Data availability statement
Data are available upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- adverse events
- epidemiology and detection
- healthcare quality improvement
- patient safety
- quality improvement
Background
The publication of ‘To Err is Human’1 in 1999 drew attention to the concept of adverse events (AEs) in hospitalised patients and revealed the extent to which AEs affect the delivery of safe patient care. Since then, in order to assess and monitor AE rates both locally and nationally, many healthcare providers have adopted the Harvard Medical Practice Study (HMPS) methodology2–4 or Global Trigger Tool.5 In a recent systematic review, Panagioti et al6 concluded that approximately 6% of hospital inpatients experience a preventable AE.
To date, international healthcare providers and institutions have demonstrated little success in reducing AE rates with large-scale evidence-based patient safety initiatives.7–9 In Ireland, the first Irish National Adverse Event Study (INAES)10 reported the overall AE rate in Irish publicly funded hospitals to be 12.2 % (95% CI=9.5% to 15.5%) corresponding to 41 000 admissions associated with one or more AEs in 2009. The preventable AE rate for INAES was 9.1% (95% CI=6.9% to 11.9%) in 2009. Since this first study, 33 National Clinical Programmes (NCPs), each directed at specialty or disease-focused delivery of care, were launched to standardise models of care and develop guidelines, pathways and associated strategies for the delivery of publicly funded clinical care in Ireland. As well as better delivery of quality care, the programmes incorporated guidance on reducing perioperative mortality and morbidity, antibiotic resistance, morbidity and mortality after hip fracture, adverse drug events, mortality and morbidity from sepsis and hospital-associated infections (box 1).
Description of the National Clinical Programmes and Guidelines
Patient Safety in Ireland
The two main initiatives which may have influenced patient safety in Ireland between 2009 and 2015 were the National Clinical Programmes (NCPs) and the National Clinical Guidelines.
The National Clinical Programmes:
The NCPs were established by the Health Service Executive in 2010 to improve and standardise patient care by bringing clinical disciplines together and enabling them to share innovative solutions.57 The aims of the programmes are to provide access to safe and timely care as close to patient homes as possible. In total, 33 NCPs were introduced (https://www.hse.ie/eng/about/who/cspd/ncps/); however, it will not be possible to provide a complete overview of these programmes and their impact on patient safety in Ireland.
Many of the programmes aimed to address specific categories of adverse events (AEs). Examples of such programmes include:
Healthcare-Associated Infections Programme—aiming to reduce the incidence of healthcare-associated infections and related morbidity and mortality.58
Sepsis Programme—aiming for the early detection and prevention of sepsis and reduction in sepsis-related morbidity and mortality.59
Trauma and Orthopaedic Surgery Programme—aiming to reduce perioperative morbidity and mortality after fracture. Guidelines define delirium pathways, deep vein thrombosis protocols and the role of the ortho-geriatrician.60
Surgery Programme—aiming to prevent postoperative complications and wound infections.61
Emergency Medicine Programme—aiming for the early detection and management of delirium, prevention of healthcare-associated infections and sepsis management.62
Medicines Management Programme—aiming to reduce medication-related AEs.63
Anaesthesia Programme—aiming to reduce anaesthetic and perioperative morbidity and mortality.64
Other national clinical programmes aimed to improve patient safety by improving access to care, developing clinical pathways and improving staffing and resources. Such programmes include the NCP for acute coronary syndrome, acute medicine, asthma, chronic obstructive pulmonary disease, critical care, cystic fibrosis, dermatology, diabetes, epilepsy, gastroenterology and hepatology, heart failure, the older person, mental health, national transport medicine, neurology, obstetrics and gynaecology, ophthalmology, paediatrics and neonatology, palliative care, pathology, radiology, rare diseases, rehabilitation medicine, renal, rheumatology and stroke.57
The National Clinical Guidelines:
The National Clinical Effectiveness Committee was established by the Irish Department of Health in September 2010 and aimed to prioritise and quality assure guidelines, which would improve health outcomes, standards of care and decrease variation in care.65 The recommendations are graded based on the Appraisal of Guidelines for Research and Evaluation instrument that helps identify the level of evidence supporting the guidelines putting greater emphasis on the recommendations of a superior evidence base.66 The guidelines which were established between 2009 and 2015 were:
The National Early Warning Score—The recommendations aim to detect the clinical deterioration of a patient early by categorising the patient’s illness severity.14
Prevention and Control of Methicillin-Resistant Staphylococcus aureus.12
Surveillance, Diagnosis and Management of Clostridium difficile Infection in Ireland.13
Sepsis Management—The recommendations focus on detection, early management and appropriate escalation of care of patients with sepsis.15
In parallel with this, the National Clinical Effectiveness Committee sought to prioritise and quality assure guidelines that would improve standards of care and patient safety.11 By 2015, the National Clinical Effectiveness Committee had developed National Clinical Guidelines (NCGs)on the prevention and treatment of methicillin-resistant Staphylococcus aureus (MRSA)12 and Clostridium difficile infection,13 the early detection and management of the deteriorating patient14 and the management of sepsis15 (box 1).
This follow-up study, the Irish National Adverse Event Study-2 (INAES-2), aimed to quantify the prevalence, nature and costs of AEs in acute Irish hospitals in 2015. Additional objectives were to explore any impact of the NCPs and the NCGs by comparing these results with the 2009 study. As hospital incident reporting systems typically under-report AEs,16 we wished to compare our prevalence rates of AEs with rates of incidents reported to the National Incident Management System.
Methods
Study design and population
A retrospective two-stage review of charts examining admissions from 2015 was carried out and the findings were compared with those from the original INAES database, which examined admissions from 2009. We used the HMPS methodology of AE detection,17 as modified by the Canadian researchers who completed the Canadian Adverse Events Study.18 This method of chart review was identical to the original INAES study.10 In the 2009 study, eight Irish public hospitals had been randomly selected from the four healthcare regions in Ireland (one large and one small hospital from each region). The same eight hospitals agreed to participate in the second study (INAES-2). For the purpose of this study (and the original INAES study), large hospitals were defined as those which admitted over 100 000 patients via the emergency department or day care unit per year and/or had a National Cancer Centre. The remainder were classified as small hospitals.10 Half of the admissions reviewed were of patients who underwent surgical procedures, and half were medical patients as defined in the INAES sampling protocol (see online supplemental appendix 1). The procedure codes for general anaesthetic, regional and neuroaxial blocks acted as proxies to indicate that surgery was likely to have been performed during the admission. Eligible admissions had a hospital length of stay of at least 24 hours or had died within that time. Patients under the age of 18 years and admissions with an obstetric or psychiatric discharge principal diagnosis were excluded.10 Patients who had been transferred from another hospital were excluded as it was less likely that the full patient notes would be available. An oversampling of 400 randomly selected admissions per hospital for 2015 was generated locally at each site via the Hospital Inpatient Enquiry (HIPE) based on the above inclusion and exclusion criteria. HIPE is a national health information system that gathers clinical data on each hospital admission.19
Supplemental material
Power calculation
For the 2015 study, we determined a sample size of 1500 admissions, which was calculated on the basis of a 20% rate of AEs and ±2% precision. This allowed precision of ±5% on any subgroup constituting 20% or more of the total sample. Thus, at least 187 eligible admissions were required to be reviewed in each hospital.10
Patient chart review
Using an identical methodology to the original 2009 INAES study to determine an AE,10 a nurse reviewer initially screened the selected admissions for eligibility. The documentation in the medical records (which was largely paper based) of eligible admissions was reviewed for the presence of 18 triggers suggestive of the occurrence of AEs. The laboratory and radiology results were accessible electronically. Examples of these triggers include readmission following recent discharge from hospital, evidence of having suffered an adverse drug reaction and an unplanned admission to the intensive care unit (online supplemental appendix 2). This review involved assessing documentation up to 1 year before and 1 year after the index admission.
As in the original study, admissions that met any one of the trigger criteria were referred to a physician reviewer to determine if an AE had occurred, and if so to assess the degree of causation, resultant outcome and preventability of the AE.10 An AE was defined as an unintended injury or complication resulting in disability at the time of discharge, prolonged hospital stay or death that was caused by healthcare management rather than by the underlying disease process.10 An AE was classified as preventable if there was a greater than 50% chance of its prevention, assessed by the reviewer using a Likert scale (online supplemental appendix 3). The physician reviewer was required to determine the outcome of each AE. The possible outcomes for an AE were additional outpatient visits, the necessity for further intervention or treatment, prolonged hospital stay, subsequent hospitalisation, disability at the time of discharge or whether an AE may have contributed to death. It was possible that an AE could have one or more outcomes.
A physician reviewer assessed the clinical notes to categorise the clinical process causing the AE as surgery related, diagnosis related, medical procedure related, fracture related, anaesthetic related, therapeutic related and drug related. Each AE could be classified into one or more categories. For example, an AE due to a patient falling may have been caused by a sedative medication (ie, drug related) and a lack of resources, monitoring and supervision (ie, therapeutic related). This categorisation was based on the Canadian Adverse Event Study18 and was also used in the 2009 INAES study. In the event that physician reviewers were unable to determine the above criteria, consultant specialists including a surgical reviewer were available to review the case.
We included AEs that occurred during the patient’s index hospital admission, which were detected either during the admission or after discharge. We also included AEs that had occurred in the 12 months prior to the admission in the same hospital and was detected during the index admission.
Reviewer training
Ten nurses and three physicians, each with a minimum of 7 years clinical experience, performed the chart reviews. Six of the eight hospitals provided at least one nurse who carried out the role of nurse reviewer within their own hospital. Two hospitals were not able to provide a nurse to carry out this role and therefore a nurse reviewer was recruited externally. All three physician reviewers were recruited externally and had no affiliation to the hospital sites. One of the nurses and two of the physicians had participated in the previous INAES study. All reviewers underwent standard training prior to the study (regardless of whether they participated in the initial 2009 study). The training was delivered over 3 days by researchers who carried out the previous INAES study with the assistance of a Canadian Adverse Event Study researcher. After face-to-face training, reviewers independently reviewed 20 training charts, and inter-rater reliability was calculated using the Cohen’s kappa statistic (κ), which estimated the inter-rater agreement for the presence of triggers identified by nurse review and the presence of AEs by physician review yielding results for the nurses of κ=0.207 (95% CI=0.13 to 0.33) and the physicians of κ=0.66 (95% CI=0.42 to 0.93).20 Following a further period of training with 20 training charts, the kappa statistic for the nurse reviewers rose to κ=0.54 (95% CI=0.37 to 0.7).
During the data collection stage of the study, approximately 10% of records were reviewed by a second reviewer to determine the inter-rater reliability of the physician and nurse reviewers. The second reviewer was blinded to the outcome of the first review. The ‘in the field’ kappa statistic for the detection of the presence of triggers for the nurse reviewers was κ=0.80 (95% CI=0.70 to 0.91). The inter-rater kappa statistic for the physician reviewers’ detection of the presence of AEs was κ=0.50 (95% CI=0.29 to 0.72) and for the presence of preventable AEs was κ=0.37 (95% CI=0.21 to 0.53).
Statistical analysis
Statistical analysis was carried out using Stata Release 16 software.21 The Charlson Index was derived for each case to classify comorbidity.22 In Ireland, a medical card is issued by the Health Service Executive (HSE—responsible for the provision of publicly funded health and personal social care services), which grants the owner certain health services free of charge.23 To qualify for a medical card, the person’s weekly income must be below a certain figure for their family size and therefore can be used as a proxy for socioeconomic status.
Admissions were stratified to ensure that 50% had undergone a surgical procedure during the admission (using anaesthetic codes as proxies as described above), whereas the actual national percentage of those who underwent surgery in 2015 was 19% and in 2009 was 24%. The analysis was reweighted for our sampling frame (ie, the ratio of admissions with and without the anaesthetic procedure codes in each hospital’s eligible study population) in order that the results would be representative of the adult Irish hospital inpatient population.
The overall and preventable AE prevalence rates were defined as the percentage of admissions associated with one or more AE. The incidence density of overall and preventable AEs was calculated as a number of AEs occurring per 100 admissions. This statistic considers the occurrence of multiple AEs during the same admission. We only present weighted prevalence and incidence density rates in our results. The total number of AEs nationally was estimated by multiplying the weighted incidence density by the total number of adult patients (according to our eligibility criteria) admitted in 2015.
The data were further subcategorised to determine whether the national programmes and guidelines had an impact on the incidence and prevalence of particular AEs over the 6-year study time interval. This subcategorisation was carried out after the data was collected and was based on the clinical data collected (as described in table 2). The subcategories created reflect specific national programmes and guidelines. For example, it was possible to identify C. difficile infection-related AEs, allowing insight into the impact of the National Clinical Guidelines on C. difficile prevention. Of note, it was not possible to classify AEs due to sepsis, as the data required to make this clinical diagnosis was not captured in the data entry tool (similar to the original 2009 study).
Overall and preventable adverse event rates for 2009 and 2015, presented by category of adverse event
CIs for binary variables were modelled using logistic regression. Incidence rates were calculated using Poisson regression with robust variance estimation to account for overdispersion. Logistic regression was used to explore differences in prevalence rates, types of AEs as a percentage of total AEs and the outcome of AEs between the two periods of data collection. Therefore, p values were derived from logistic regression and significance was set at p<0.05. The cost of AEs was estimated by multiplying the average cost of a hospital bed per day (€839 in 201524) and the average additional bed days incurred by an AE and the estimated number of AEs in 2015.10 The Health Pricing Office of Ireland25 provided national admission demographic data for adult inpatients for the years 2009 and 2015 according to our HIPE search strategy (online supplemental appendix 1). This allowed the sample to be population-weighted for estimating national statistics. The HSE provided national incident data (reported to the National Incident Management System) for 2015 meeting our eligibility criteria. The national incidence density of incidents reported was calculated by dividing the number of incidents reported to the National Incident Management System by the total number of admissions meeting our eligibility criteria for 2015.
Results
Admissions reviewed
The collection of 2015 admission data commenced in November 2018 and finished in August 2019. Three thousand four hundred patient admissions were sampled (figure 1). Of the admissions sampled, 2025 admissions underwent initial screening by the nurse reviewers. The nurse reviewers excluded 420 admissions as they were ineligible (reasons for ineligibility are shown in figure 1). In total, 1605 admissions were reviewed by the nurse reviewers and 572 (35.6%) were triggered for physician review. Two more admissions were excluded by the physician reviewers as the patients were transferred from another hospital and therefore the full patient notes were not available, leaving 570 admissions to be reviewed by the physician reviewers.
Flowchart of admissions reviewed during the INAES-2 study.
Table 1 compares baseline demographics for the 2015 dataset with those of the earlier 2009 dataset. The median patient age increased from 55 years in 2009 to 60 years in 2015 (p<0.001). The Charlson Index was similar between both years (1.8 in 2009 (95% CI=1.8 to 1.9) vs 1.9 in 2015 (95% CI=1.7 to 1.9, p=0.5)). The percentage of patients entitled to free healthcare (ie, in possession of a medical card) increased from 51.3% (95% CI=48.8% to 53.7%) in 2009 to 58% (95% CI=55.5% to 60.7%, p<0.001) in 2015. The percentage of patients who died as an inpatient reduced from 4.8% (95% CI=3.8% to 5.9%) in 2009 to 2.4% (95% CI=1.6% to 3.1%) in 2015 (p<0.001). Other baseline characteristics were similar, including the length of stay and the percentage of women and emergency cases (table 1).
Comparison of patient baseline characteristics of both study samples (2009 and 2015) and characteristics of the total Irish inpatient population
Adverse events
Of the 3177 admissions reviewed (1574 in 2009 and 1603 in 2015), there were 247 AEs detected in 211 admissions for 2009 and 290 AEs detected in 238 admissions for 2015. There was no significant change (p=0.48) in AE prevalence from 12.2% (95% CI=9.5% to 15.5%) in 2009 to 14% (95% CI=10.4% to 18.4%) in 2015. The incidence density remained unchanged (p=0.97) and in 2009 was 10.3 AEs per 100 admissions (95% CI=7.2 to 13.4) and 10.4 AEs per 100 admissions (95% CI=6.5 to 14.3) in 2015.
There were approximately 390 000 admissions (meeting the eligibility criteria) to acute Irish public hospitals in 2015. Based on the findings of our study, this would extrapolate to 54 000 admissions being associated with one or more AEs for the year 2015. In 2015, patients who experienced an AE were judged to have required a median additional hospital stay of 5.6 days (95% CI=3.4 to 7.7). This is equivalent to an additional cost to the health service of approximately €4700 per AE for the hospital stay alone (this excludes litigation costs and societal costs), which when extrapolated nationally would equate to an annual cost of hospital-based AEs for adult inpatients of €190 million for 2015.
Preventable AEs
There were 179 preventable AEs in 159 admissions in 2009 compared with 161 preventable AEs identified in 129 admissions in 2015. The prevalence of preventable AEs was unchanged (p=0.33) from 9.1% (95% CI=6.9% to 11.9%) in 2009 to 7.4% (95% CI=5.3% to 10.5%) in 2015. The incidence density of preventable AEs was 7.7 (95% CI=5.8 to 9.6, p=0.23) AEs per 100 admissions in 2009 and 5.6 (95% CI=3.4 to 8.0, p=0.23) AEs per 100 admissions in 2015 (see case descriptions online supplemental appendix 4).
Impact of national programmes and guidelines
There was a statistically significant decrease in the percentage of preventable AEs classified as hospital-associated infection related from 33.1% (25.6% to 41.6%) in 2009 to 22.2% (15.2% to 31.1%) in 2015 (OR=0.6, 95% CI=0.4 to 0.9, p=0.01). The percentage of the other preventable AEs classified into categories thought to be targeted by the national programmes and guidelines (surgical-related, drug-related, fracture-related, MRSA-related and C. difficile infection-related AEs) were unchanged between 2009 and 2015 (table 2 and figure 2). When all AE categories thought to be targeted by the national programmes and guidelines were pooled, a statistically significant difference in the percentage of AEs targeted was not seen (p=0.1); 70.3% (95% CI=63.1% to 76.6%) in 2009 and 64.1% (95% CI=57.2% to 70.5%) in 2015.
Classification of preventable adverse events thought to be targeted by the national programmes and guidelines comparing 2009 to 2015. *It is possible for each adverse event to be classified into one or more categories.
AE outcomes
The percentage of AEs that resulted in disability increased significantly from 14.6% (95% CI=9% to 22.8%) in 2009 to 28.2% (95% CI=19.5% to 38.9%) in 2015 (p=0.003). Similarly, the percentage of AEs that resulted in additional treatment or intervention increased significantly from 2.6% (95% CI=0.7% to 9.1%) in 2009 to 9.2% (95% CI=4.9% to 16.8%) in 2015 (p=0.03). The percentage of AEs resulting in subsequent hospitalisation, prolonged hospital stay, additional outpatient visits and contributing to death remained similar (table 3).
Overall adverse events presented by outcome
National incident reporting
When the national incident data was examined (ie, incidents reported in adult inpatients, excluding maternity, paediatric and mental health services), it was determined that 4.4 incidents per 100 patient admissions were reported in 2015.
Discussion
This study is the first follow-up study of AE rates in the Irish healthcare system and compares AE rates before and after the implementation of the national programmes and guidelines. There was no statistically significant change in the prevalence of preventable AEs, which was 9.1% in 2009 and 7.4% in 2015 and in the incidence density of preventable AEs, which was 7.7 AEs (in 2009) per 100 admissions and 5.6 AEs per 100 admissions in 2015. However, the percentage of preventable AEs related to hospital-associated infections showed a statistically significant decrease from 33.1% in 2009 to 22.2% in 2015. There was no change in the prevalence of overall AE rates.
We determined that 4.4 incidents per 100 patient admissions were reported nationally in 2015 to the National Incident Management System. This figure falls short of our estimate of 10.4 AEs per 100 admissions and highlights the known limitation of under-reporting of incidents.16 This therefore emphasises the need for ongoing systematic and objective monitoring of AE data via studies like INAES-2. This figure of 4.4 incidents per 100 admissions is however an improvement since our previously reported national figure of 1.9 incidents reported per 100 admissions.10 This increase in reported incidents can be considered reflective of an improvement in patient safety culture within the Irish healthcare system.26
The NCPs and NCGs were introduced at a time of global economic recession. Ireland was the first European country to announce that it had entered recession in 2008.27 28 Therapeutic-related AEs were the second most prevalent classification of AEs chosen by the physician reviewer and often reflect a pervasive lack of timely care, resources and diagnostics.10 Compared with other ‘bailout countries’, Ireland suffered the most substantial decrease in government health expenditure as a percentage of its total government expenditure: a decrease of almost 22%.29 In 2009, the annual healthcare budget was €14.5 billion but was cut year on year to €13 billion in 2015.30 31 The healthcare budget was reduced by decreasing expenditure on staff pay, pharmaceutical expenditure, community home-care provision and by creating staff redundancies.32 33 By 2015,34 there were 6800 fewer staff in the HSE compared with 200935 (due to a combination of redundancies and a recruitment embargo36) while there was an 18% increase in inpatient admissions during this time.34 35Due to the lack of hospital bed space and staffing, many patients received inpatient care on trolleys in corridors and common areas.37 The Irish Nurses and Midwife Organisation reported a 46% increase in patients being treated on trolleys in 2015 (n=92 998) compared with 2009.38
The ageing population and older case-mix captured in this study may also have influenced our results and may have offset the positive effect of the national programmes and guidelines. This demographic change may also explain the increase in the percentage of AEs, which resulted in disability at discharge and requiring additional intervention, as severe AEs have shown to be more prevalent in older age groups.39 40 The median patient age had increased by 5 years between studies (from 55 years in 2009 to 60 years) although we did not see an increase in the Charlson Index score of comorbidity. However, a frailty index may provide a greater measure of patient complexity and physiological reserve in the older patient and allow better comparability, as AE rates and outcomes appear to be more related to complexity and frailty rather than solely age and comorbidity.41
It is not possible to determine whether our preventable AE prevalence rate of 7.4% is actually higher than the aforementioned meta-analysis rate of 6%.6 The variability among studies (heterogenicity) in the meta-analysis was high (I²=99%), as may be expected of a meta-analysis of epidemiological studies.42 Differences in patient demographics, eligibility criteria, settings, points in time, variation in timeframe over which AEs were detected and quality of documentation used for detecting AEs accounted for much of this heterogenicity.6 Therefore, a detailed comparison cannot be made.
Limitations
The limitations of the INAES-2 study are principally those of retrospective chart review. The patient charts reviewed during both studies were mainly paper based and 142 charts were deemed ineligible in the 2015 study as they were either unable to be located or incomplete. The kappa statistic (inter-rater reliability) for the detection of the presence of overall and preventable AEs was κ=0.50 and κ=0.37, respectively, and for the presence of a trigger by the nurse reviewers was κ=0.80, indicating fair to substantial agreement. The original INAES study reported similar inter-rater reliability kappa values for the presence of overall AEs by the physician reviewers (κ=0.59) and for the presence of a trigger by the nurse reviewers (κ=0.79).10 This is consistent with other studies using retrospective chart reviews for AE detection, which averaged κ=0.55 (range 0.24–0.8).43 This level of agreement between reviewers is likely due to the retrospective review process, which introduces the risk of hindsight and outcome bias. Errors made in evaluating the appropriateness of the decision when the outcome is already known (outcome bias) and the tendency of reviewers to overestimate their ability to have predicted an outcome that could not possibly have been predicted (hindsight bias) are prevalent in retrospective reviews.44 45 In particular, determining causality and preventability of AEs by the reviewer is based on reviewer judgement and limited to the information documented in the medical chart.
Over time different types of AEs have come to be seen as preventable and unacceptable; for example, hospital-associated infections are no longer seen as an unfortunate side-effect of good quality healthcare.46 In surgery, complications that were considered unavoidable a decade or so ago are now thought to be due to a complex interplay of causes (as outlined below).47,48 The scope of patient safety is expanding and potentially more AEs are being detected as a result.49 This may mean that the reviewers’ standard of review is becoming stricter, being influenced by cultural shifts and evolving evidence.
Patient outcomes are influenced by many factors and therefore adverse outcomes are not remediable by the implementation of any single intervention. Improvement in AE rates is determined by a combination of a positive patient safety culture, incorporation of patient safety education, implementation of patient safety initiatives, optimal staff workload, supervision and leadership, external policy content and training in patient safety operations.47 It is difficult to assess the various interactions between these factors and the resultant outcomes over a long period. Similarly, this study’s primary outcomes (overall AEs and preventable AEs) even when subcategorised are likely to be simplistic, heterogeneous and may not detect differences in patient safety interventions over time as smaller targeted improvements may be statistically neutralised when calculated using such composite measures.50 Traditionally, changes in national AE prevalence rates have been measured by retrospective chart review studies.9 51–53 Such uncontrolled before and after studies have been criticised for overestimating effect size and their difficulty in interpreting cause and effect.54 This approach may not capture nuanced changes, which more rigorously controlled designs (eg, a prospective time-series design) may achieve. Such an approach should aim to capture established measures for determining specific patient safety outcomes as suggested by Shojania et al.50 (eg, prospective laboratory-based surveillance of hospital-associated infections55 and a prospective registry for monitoring surgical outcomes56).
Conclusion
This study is the first longitudinal follow-up study of national AE data in Ireland. These results were obtained using an internationally standardised methodology and a sample representative of all healthcare regions in Ireland. These findings are, therefore, generalisable to our healthcare system. The increasing complexity of an ageing population and the financial and resource limitations seen during the economic recession would be predicted to result in increasing preventable AE rates. However, preventable AE rates have remained unchanged despite these changes with preventable hospital-associated infection-related AE rates seen to decrease. We conclude therefore that the NCPs and NCGs may have positively influenced patient safety in Ireland.
Data availability statement
Data are available upon reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
Ethical approval for INAES-2 was received from the ethics committees of the Royal College of Surgeons in Ireland (REC 1421) and the Royal College of Physicians in Ireland (RECSAF 04-2). A consent declaration was obtained from the Health Research Consent Declaration Committee in Ireland (AF3-19-001), which allowed data collection without obtaining explicit consent from each patient before we irrevocably anonymised data.
Acknowledgments
We want to thank and acknowledge the eight-hospital sites who facilitated the data collection. In particular, we would like to recognise the hospitals’ managerial and administrative staff who facilitated access for the study. We want to thank Ms Virginia Flintoff from the University of Toronto for her assistance with training and her advice. We would also like to thank our nurse and physician reviewers, our surgeon adviser and the INAES-2 advisory group. We also thank Dr Siobhan Young, Ms Loretta Jenkins, Ms Cora Mc Caughan, Dr Sarah Condell, Dr Paul O’Connor, Dr Philip Crowley, Prof Ross Baker, Ms Carol Hickey, Mr Gavin Maguire and Dr Colm Henry.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @warconnolly
Contributors WC: project management, data collection, analysis, interpretation of the data and drafting the manuscript. NR: staff training, study design, data collection, analysis and interpretation of the data and critical revision of the drafted manuscript. DJW and AH: design, review, analysis and interpretation of the data and critical revision of the drafted manuscript. RMC: design and data analysis and interpretation. CS: design, project management and critical revision of the drafted manuscript.
Funding Funding for the project was received via the Applied Partnership Awards (APA-2016-1877) from the Health Research Board (HRB) and the HSE.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.