Definition of adverse outcome

In The Netherlands, an “adverse outcome” is defined as:“… an unintended and unwanted event or state occurring during or following medical care, that is so harmful to a patient's health that (adjustment of) treatment is required or that permanent damage results. The adverse outcome may be noted during treatment or in a predefined period after discharge or transfer to another department. The intended results of treatment, the likelihood of the adverse outcome occurring and the presence or absence of a medical error causing it, are irrelevant in identifying an adverse outcome.”

The Dutch definition of an AO is more sensitive than the definitions used in retrospective studies in that those definitions excluded the consequences of the disease itself and/or co-morbidity.1–3 Using a definition with such exclusion rules in routine reporting would require a doctor to completely understand the cause of an AO and its preventability and/or blame. The Dutch definition obviates such judgement, with the aim of reducing subjectivity and inter-observer variability and thereby facilitating comparisons over time and/or across different settings.

Management of (contextual) information

A special software was designed to support the management of hospital information system and AO data. All the patients identified by the hospital information system search were automatically imported each week into an AO database (Access 2003) with a minimum dataset of contextual information, including name, gender, date of birth, operative procedures performed and the American Society of Anesthesiologists (ASA) classification. Operative procedures were automatically classified with respect to technical complexity, according to the standards of the Association of Surgeons of The Netherlands (on a scale ranging from 1 for simple to 5–7 for complex or difficult surgical procedures).

The occurrence or non-occurrence of all AO was recorded on paper for all patients. Selected AOs were checked and discussed at weekly meetings attended by surgeons, residents and interns and chaired by a staff surgeon. The selection of AOs was based on an algorithm that included patients (1) with three or more AOs, (2) with one or more AOs leading to reoperation, permanent damage and/or death (ie, grade 2 or higher; see below) or (3) with an AO report that seemed likely to contain a lesson for the future. The aim of the above selection was to ensure interesting meetings and thus maximise attendance. Each patient with an AO was presented by the intern/resident who was on the ward when the patient was discharged. The focus of the ensuing discussion was to interpret the underlying causes and thus learn how other such outcomes might be prevented. After the meeting, data forms of all patients, without and with AOs, discussed and non-discussed, are entered into the AO database using the software described above. Patients with one or more AOs are the denominator of the proportion of patients with an AO, the whole patient set being the numerator.

Master classification

AOs were classified according to a special classification system that was designed to match existing (inter)national classification systems as closely as possible. This “master classification” facilitated analysis by allowing AO to be automatically aggregated (by the software) into specific groups and subgroups, The master classification system defines the key characteristics of each AO in terms of the following three dimensions (see Online Appendix A):

1. Type of AO that is its pathology (eg, infection, bleeding)

2. Location, specified by

   • a. region (eg, thorax, abdomen) and/or by

   • b. organ or organ system (eg, lung, liver) and/or by

   • c. tissue

3. Contextual information and potential determinants (eg, medication).

The first dimension—type—is based on the International Classification of Diseases, 10th revision, but with adaptations to accommodate AOs (as opposed to spontaneous disease). The second dimension—location—is based on the Dutch WCC Standard Anatomical Location (http://www.rivm.nl/who-fic/Annuals/Paris26.pdf), developed by WHO. Of the various elements that fall under contextual information dimension, the coding of medication was derived from the Anatomical Therapeutic Chemical classification (http://www.whocc.no/atc_ddd_publications/guidelines/). Each dimension comprises two or more degrees of detail, to allow general and detailed analyses. AOs were graded according to four levels of severity: (1) temporary health disadvantage, recovery without (re)operation; (2) recovery after (re)operation; (3) (probably) permanent damage or function loss; and (4) death.

Discussion

The present study demonstrates that doctor-driven reporting of AO is feasible alongside routine clinical care over a long period. Even after adjusting for shifts in patient mix and operation mix, it was seen that the number of non-serious AO reported increased rather than decreased, whereas the number of deaths (severity 4 AO) decreased. This suggests that the reporting of AO has become increasingly sensitive over time and that surgical care has become safer with respect to mortality. AO reporting was used to support professionals in assessing and improving their own quality of care in two ways. First, through the weekly AO meetings, surgeons discuss and analyse AOs that their patients suffered case by case. The emphasis lies on ‘What can we learn from this?’ instead of on ‘What have they done wrong?’. The lessons learnt in this way averaged 55 per year in the third period, and applied to many areas ranging from thrombosis prophylaxis in subgroups of patients to technicalities of fracture management at the operating room, and from the appropriate use of antibiotics in trauma patients to timing and preoperative workup of liver transplant patients. Second, at a group level, the aggregation of AOs into master classification groups makes it possible to carry out a statistical analysis of AOs and possible determinants, independent of the weekly case-by-case analyses. As an example, the time-trend analysis presented here demonstrates a significant increase in functional disorders of the kidney, which is most likely related to new immune-modulating drugs used in transplantation and an increase in reported infections. A detailed analysis of the latter has been published in a separate paper.9 Infections identified by AO reporting differ from those identified by monitoring of the Centers for Disease Control and Prevention, as AO reporting includes all suspected or proven infections that do not resolve without treatment, that is, that require(d) treatment or have been treated. Apart from the fact that infection monitoring by the Centers for Disease Control and Prevention is not ubiquitous in all Dutch hospitals, it does not routinely provide detailed information on context or potential causes.

Outside the context of the registering department or hospital, doctor-driven AO reporting can also be used for “mutual monitoring”. When professionals from different departments or hospitals cooperate through a (confidential) national AO database, this can provide mirror information to allow doctors to compare their own AO data with anonymous AO data that have been collected by professionals working in similar hospitals or departments. For such a purpose, the system described here has been adopted as the national standard for AO reporting by the Association of Surgeons of The Netherlands. Obviously, additional important issues will need to be addressed, such as adjusting for under-reporting and for more detailed differences in patient and operation mix, to further improve the validity of mirror comparisons.

The Dutch Healthcare Inspectorate analyses the quality of care of all Dutch hospitals annually through national “external” benchmarking of a set of 20 quality performance indicators. One of these indicators is whether medical professionals and hospital departments practice AO reporting. The Inspectorate has, in line with recommendation 6.1 of the Institute of Medicine report “To err is human”i on peer-review protections, stated that AO data are and will at all times remain the sole property of the medical professionals and will never directly be used for benchmarking. However, if professionals are found to have out-of-range performance indicators, they themselves can use the detailed information provided by the AO reporting system to bring their performance up to standard.

Other research has been published on how to make healthcare safer.10–12 Chang et al13 proposed a standardised terminology and classification for adverse events. The approach presented here differs from Chang et al's in that it is not limited to outcomes that are related to errors in the medical (management) process. By eliminating causality, such as errors, as a criterion for reporting, the “Dutch approach” is more sensitive and has less observer-dependent reporting. In addition, it provides information about all AO and is thus a more relevant source of information because it does not exclude AOs that are not caused by errors. The fact that Dutch AO reporting is integrated into routine clinical practice has advantages over other initiatives that are not rooted in routine patient management, such as the National Confidential Enquiry into Patient Outcome and Death.14 ,15 Not the least of these advantages are the regular plenary discussions among clinicians and the lessons learnt directly each week.

Doctor-driven reporting inevitably has drawbacks as well, the most obvious being underreporting. However, we have shown in an earlier study that, with the overriding focus on learning, underreporting can be kept to a minimum.

The internal monitoring described here, and the mutual monitoring, which is presently part of an initiative in The Netherlands, demonstrate that doctor-driven AO reporting can be a powerful instrument with the potential to prevent AO in patients in the future.