The conception of ‘incident’ changes over time

Vincent and Amalberti argued that safety in healthcare is a moving target, because innovation and improving standards in healthcare alter the conceptions of both harm and preventability.5 This dynamic view of healthcare safety can be illustrated by the 1996 Dutch legal definition for a sentinel event (SE), the most serious class of incidents that healthcare organisations are mandated to report (see box 1). Since rising standards of care influence the way incidents are judged, an incident in 2005 can be judged differently in 2015 using the identical definition.6 Moreover, the standards of the healthcare quality community change faster, because they stay up to date on quality improvement innovations, than the standards of many frontline personnel. As a consequence, there is an endless discussion between hospitals and the healthcare inspectorate as to which incidents are classified as SE and should therefore be reported. Another consequence is that our national incident reporting system cannot provide the inspectorate with a representative view of clinical safety issues, or a means to measure safety improvement over time.

Standards for corrective actions change over time

Just as the standards for what is defined as incident shift, so do the standards for corrective actions. For example, Behr et al7 conducted research into three healthcare incidents that attracted significant public attention. These were a paediatric cardiology case in 2001, a cardiothoracic case in 2005 and a neurology case in 2009. Although the cases were similar in many respects, the authors found a shift in the way the cases were perceived. The 2001 case was seen as a professional problem, the 2005 case a managerial problem and the 2009 case a governance problem. Each perception led to corrective actions fitting that view: removal of the professionals, sacking of the board and problems for the regulator. What was deemed an adequate corrective action in 2001 was deemed inadequate 4 years later. If we are to keep up with advancements in safety-related insights then the goal of reporting systems should not be the corrective action itself, but the ability to determine appropriate corrective actions.

Healthcare providers need to be able to hit moving targets

Incidents are not something in themselves; they are symptoms of a larger problem. Instead of targeting the ‘symptoms’, it seems better in the long term to target the ‘causes of the disease’. That is to have the healthcare providers learn how to cope with the infinite variability of safety issues by learning how to analyse these and how to devise corrective actions that fit their local setting. Incident reporting systems should therefore lead to social and participative learning at the local level.3 Safety issues will keep sneaking up on healthcare providers from all directions. Hitting a moving target over and over again requires a different set of skills from hitting a fixed target once. Being engaged in analysing incidents can serve as a catalyst for changing the way healthcare providers think about risk and increase their vigilance.8 Healthcare providers will only learn to hit this continuous stream of moving targets once they properly analyse their incidents.

Focusing on the learning, not the outcome

Dutch hospitals are mandated by law to report all serious incidents, defined as SEs, to the Healthcare Inspectorate (IGZ) (see box 2). The IGZ receives about 800 SE reports annually from the 93 Dutch hospitals. In 2012, the IGZ decided to shift its focus from what hospitals learn from their SEs to how hospitals learn from their SEs. The inspectors felt that this would improve the effect of their work. In 2010, the IGZ had made a similar shift in their supervision of suicide reports, turning the focus on the organisational learning ability of mental healthcare institution.9 Organisational learning is defined as the process of creating and applying valid knowledge to enable an organisation to improve.10 The IGZ expected that the learning ability of hospitals would improve by addressing conditions for learning from SEs, which would contribute to safety.11 ,12

The quality of the learning process is now quantified using the 2012 WHO draft report ‘Concise Incident Analysis’, supplemented with extra items on patient engagement (see box 3).13 Each SE analysis report receives a score of between 0% and 100%, indicating the percentage of items addressed adequately. Since July 2013, the scores have been added to a database showing the quality of SE analysis reports over time. These figures are benchmarked to give the IGZ insight into the quality and rate of improvement of each individual hospital compared with other hospitals (see figure 1). These data are discussed with each hospital individually during an annual meeting between the hospital board and the IGZ as one of many agenda items. If the data are a cause for concern, then the IGZ plans a separate meeting with the board and the hospital's SE investigating committee. The data are not publicly accessible.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Example of one hospital. The columns represent the quality score (range 0% to 100%) of the 24 individual sentinel event (SE) analysis reports by Hospital X from July 2013 to October 2015. The reports are in chronological order from left to right. The dark line is the moving average, the mean score of Hospital X over the past five reports. The grey lines are the national highest, average and lowest scores (n=1675 SE reports). Hospital X scored below average and then showed a strong increase in the quality of its SE analysis reports, eventually dropping back to an average score. The national average is increasing.

Between July 2013 and August 2015, the overall average score for the quality of SE analysis reports moved from 64% to 78% (n=1675 SE reports). Shifting the focus from what hospitals learn to how hospitals learn has several advantages:

• It is independent of changes in the conception of ‘incident’.

• It gives healthcare professionals and boards the room to devise and apply tailor-made corrective actions.

• Changes in the learning curve can be tracked with each new analysis report.

• Data make it possible for the IGZ to focus on hospitals that lag behind.

• The effect of interventions by the IGZ on specific items (eg, patient engagement) can be tracked over time.

Many have stressed the value of multidisciplinary input, as well as physicians' and hospital boards' engagement, for optimising learning and reinforcing the importance of safety.11 ,14 Based on the SE analysis reports, the annual meetings with hospitals and the on-site inspections, we can conclude that all Dutch hospitals now have multidisciplinary investigating committees. All committees include physicians (often as chair) and all hospital boards provide support for their committees and reporting systems.

Several explanations can be given why this learning process has engaged physicians, although we cannot support these scientifically. One possible reason is that the IGZ has more or less mandated hospitals to engage physicians and checks this regularly. Other factors that might play a role are increased awareness of patient safety and greater engagement of physicians in other patient safety activities, creating more interest among physicians to contribute to SE analysis.

Limitations of focusing on learning

However hard the IGZ strives for a ‘just culture’, one that is defined as a focus on learning opposed to punishing, there could always remain a sense of fear among healthcare providers that could ultimately influence their openness and thus the validity of their SE analysis reports.15 Since 2013, the IGZ has been pushing hospitals to engage the patient or a patient representative in SE analysis. This provides the IGZ with an external check on the report's validity, but it might also increase the hospital's unease. This aspect deserves further research.

Another concern is that the focus on the process instead of the content makes the IGZ miss signals of recurrent safety issues. This is in part mitigated by the team that judges the analyses because they can recognise emerging themes. An example of such a theme is serious harm to infants after anaesthesia, which was independently reported by different hospitals and led to the IGZ informing the Dutch Society of Anaesthesia.

Relying on memory is clearly not a sustainable method for a regulator, and it is reasonable to assume that the IGZ misses relevant safety issues. We also recognise that SE analysis is just one step in the path to improvement. The next step is actually implementing the recommended corrective actions and demonstrating whether the improved learning ability of hospitals leads to safety improvement. Furthermore, learning from SEs is just one of the many activities needed to improve patient safety, and the score we assign to the quality of SE analysis reports is not a measure for the overall safety in that hospital.