• retrospective case record review

“Hospital medicine used to be simple, ineffective and relatively safe. Now it is complex, effective and potentially dangerous”.¹ The medical profession has been slow to address the dangers. Modern medical practice took off after the Second World War and within a decade the increased risks of iatrogenic disease were recognised. It was reported that admission to hospital carried a 20% chance of a “noxious” episode (excluding those caused by inadvertent errors and postoperative complications!). However, the attitude was “the dangers are generally warranted by the benefits” and “to seek absolute safety is to advocate therapeutic nihilism”.² It took claims of medical negligence by the public to rock the boat. Prompted by a tort system spiralling out of control, the insurance industry in California sponsored a study of the incidence and nature of adverse events based on retrospective case record review (RCRR).³ Despite the cumbersome and costly nature of RCRR, the method continues to be used, primarily as an epidemiological tool.

The basic review process has changed little. In phase I nurses or experienced hospital administrators are trained to identify case records that satisfy one or more of 18 well defined screening criteria that have been shown to be associated with an increased likelihood of an adverse event. In phase II trained doctors analyse positively screened records in detail to determine whether or not they contain evidence of an adverse event.³

The largest studies have been undertaken in the USA⁴ and Australia.⁵ In US studies 3–4% of case records were found to contain adverse events, of which about one quarter were found to be due to substandard (negligent) care. In Australia 16.6% of admissions were associated with an adverse event of which one half were deemed preventable. It seems unlikely that this large difference can be ascribed solely to the different aims of the two studies. About one half of the variance is probably due to methodological differences. Using American criteria the incidence of adverse events in Australian hospitals is reduced to 10.6%.⁶ This is similar to the incidence of adverse events in three smaller studies recently reported from London (10.8%), Copenhagen (9.0%), and Auckland (10.7%). About 40% of adverse events are regarded as preventable.

So how robust is the methodology and how useful are the data? Prompted incident reporting⁷ and checking risk management and litigation files⁸ show that RCRR misses up to 20% of adverse events. Those that are revealed depend on clinical judgements. Great efforts have been made to strengthen the accuracy and reproducibility of these judgements but reliability remains no better than moderate.⁹ Furthermore, hindsight bias—the tendency to impute causation when the outcome is known—is a potential inherent weakness. In published studies causation (whether or not an adverse event is due to healthcare management rather than the disease process) and preventability are judged to the legal standard of “more likely than not” which leaves considerable room for dispute. In the Harvard study¹⁰ paired expert opinions regarding the occurrence of an adverse event showed that extreme disagreement was more common than agreement. Agreement was highest for wound infections and lowest for failure to diagnose and incorrect treatment. Experienced reviewers are more likely to agree, but even in a most careful study from Australia there was only 80% agreement on the presence of an adverse event (κ=0.55) and 58% agreement on preventability (κ=0.33).⁵ Testing validity—that is, the extent to which the process approximates to the true value—without a “gold standard” is even more difficult. It has been concluded that the recognition of adverse events by RCRR has only moderate face, content and construct validity.¹¹

Whatever the true figure for adverse events in hospital practice, the results of RCRR need careful interpretation. In all reported studies about two thirds of adverse events either caused transient injuries (full recovery within 1 month) or occurred in very sick patients. The incidence of adverse events is higher among the elderly (probably because their conditions are more complex) and increases with the length of stay in hospital.^4,5,12 Regrettably, headline figures are dramatised by journalists who see adverse events lumped together irrespective of preventability and severity of outcome. Moreover, politicians, lawyers and healthcare managers often have difficulty in interpreting the data or use them selectively to satisfy particular agendas.

Despite these caveats, RCRR has led to some very significant initiatives. It has had an enormous and positive impact on efforts to improve hospital practice and has played an important part in the development of mission statements from the Institute of Medicine in the USA and the expert group advising the Chief Medical Officer in the UK regarding the building of safer health systems. In the USA, RCRR has triggered efforts to redesign systems in order to try to reduce the incidence of adverse events. For example, it was shown that 20% of adverse events were drug related. This has led to several studies indicating that information and decision support systems as well as improved methods for ordering, transcribing, dispensing and administering medications may reduce adverse events.¹³ In contrast, Australian researchers have focused on identifying the contributory causes of human error. In more than half the adverse events they found that cognitive failure was an important trigger. To improve care they called for better—or better implemented—policies and protocols, better quality monitoring, better education and training, and more consultation.¹⁴ The UK study¹² offers yet another exploratory route. Rather than trying to identify the underlying causes of specific errors by root cause analysis, stages in the hospital process at which care became suboptimal were defined and related to modes of clinical practice. In this way it may be possible to extend the value of RCRR.

However, there remains a need to devise more timely and more efficient methods of preliminary screening and more robust methods of contemporaneous record review to enable medical directors and heads of clinical units to assess quality of care. Reducing the background of low grade adverse events in hospital practice is not only vital for the improved care of patients, but also has important implications for health economics. Reported RCRR studies are remarkably consistent in finding that each adverse event equates with the use of 6–8 hospital bed-days. Furthermore, as pressure on hospital beds is a worldwide problem, there appears to be a pressing need for further and better studies.