RATIONALE FOR A REVISED REVIEW FORM

Earlier studies^3–5 focused on the incidence and type of adverse events with some attempt to identify causes and methods of prevention.¹² The original review form uses a mixture of taxonomies which are not always clearly distinguished. Reviewers in our study found that the structure of the review form meant that results did not always reflect the underlying clinical reality. For instance, with a standard RF2 a postoperative infection is classified as an operative event (because it occurred within 30 days of surgery) and non-technical (because it was not directly related to the operation itself). In a subsequent examination of the data we analysed in detail the narratives provided by the reviewers.¹³ This enabled us to define when in the process of care the adverse event occurred and the nature of the underlying problem (box 1, C1–C5). As a result, a postoperative chest infection would be classified as a ward based event and the problem as one related perhaps to drug administration (oversedation), to failure to monitor (such as over a weekend), or to failure to provide physiotherapy.

Adverse events frequently involve a chain of events and a series of care management or care delivery problems leading up to the incident itself.¹⁴ In previous versions of the form multiple problems were sometimes identified, but were not prioritised and were not sufficiently distinguished in the final analyses. However, we found that there is usually one critical period in which the primary problem occurred. We therefore introduced the concept of a “principal problem”, being the most important problem in the delivery of care. This enabled a precise identification of a particular period in the process of care, now incorporated in the review process, and a defined principal problem which greatly clarified the review process. To provide additional clarity we divided the review form into five sections each with a defined purpose, providing a modular structure to both the form and the review process.

A final important change has been a much stronger emphasis on the broader organisational, environmental, and other factors that contribute to adverse events. While earlier versions paid some attention to these factors, the approach was not sufficiently systematic. The present approach is based on a previously devised framework¹⁵ and method of individual case analysis.¹⁶ Identifying contributory factors is best done from observation and interview, although this is quite labour and resource intensive. We believe that expert reviewers or those familiar with the working environment can often comment on some of the major contributory factors. This is particularly important as the identification of such factors offers a route to devising methods of prevention.

Thus, we have attempted to address three areas in which the earlier review forms have limitations:

• We have divided hospital stay into distinct periods of care and then identified the clinical aspects associated with each period—to address the former mixed categories of the type of adverse event.

• We have introduced the term “principal problem” to help clarify the review process in order to identify the healthcare management or treatment, or delay or lack of treatment, that led to the adverse event.

• We have provided a comprehensive list of contributory factors to include organisational, environmental, and other contextual factors that influence the provision of health care.

THE MODULAR REVIEW FORM (MRF2)

The MRF2 comprises five stages or modules:

Once an adverse event has been identified following the first stage review (RF1), reviewers are required to complete stages A and B in full. When completing stage A they are required to identify one or more of five periods of care during which adverse event(s) occurred (C1–C5). Next they complete the relevant part(s) of stage C. In so doing, they are required to identify the nature of the underlying cause(s) of the adverse event ranging from errors in diagnosis to mismanagement at the time of discharge. This leads to the completion of one or more of subsections D1–D7. Finally, stage E provides the opportunity for identifying contributory factors and potential preventability. Details of each stage are shown in box 1 and fig 1. The modular structure allows reviewers to select which stages to complete, depending on the focus of the review. The MRF2 is available on the QSHC website (http://www.qshc.com/supplemental).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Flow chart of the Modular Review Form 2 (MRF2).

The objectives of this study were to pilot and evaluate the new review form (MRF2) using an evaluation questionnaire.

PILOTING AND PRELIMINARY EVALUATION OF THE MRF2

After publication of our case record review study,⁶ a considerable number of clinicians and researchers contacted us to ask for copies of the review forms with a view to conducting studies in their own hospitals. All those who had previously requested the materials were contacted and invited to take part in piloting the MRF2. Of 36 individuals contacted, 20 agreed to take part.

Procedure

Twelve teams (from the UK (2), Italy (2), France, Spain, Australia (2), New Zealand, Japan and USA (2)) took part in piloting the MRF2. Team members had a range of clinical expertise and specialist interests including epidemiology, paediatrics, nursing, infectious diseases, pharmacy, endocrinology, anaesthetics, nephrology, and emergency medicine. Eight teams sent 44 completed forms and the remaining four provided summaries of 20 adverse events. The number of reviewers per team ranged from 1 to 3 and the number of review forms completed per reviewer ranged from 1 to 6.

Participants were asked to identify 5–10 known adverse events and to review each one using the MRF2. The reviewers were warned that it takes time to understand the review process and, in particular, to interpret definitions correctly. The instruction document provided definitions of important concepts, an outline of the review process, and a list of conditions which reviewers have had difficulty with in previous review studies such as hospital acquired infection, pressure sores, and falls. Reviewers were given the opportunity to contact the research team with queries related to the review process.

After using the new forms, participants were asked to rate them on an evaluation questionnaire consisting of ratings of the statements shown in table 1 together with questions on the strengths and weaknesses of each stage of the review form and the form overall. In addition to this we posed some specific questions on issues of concern. For example: “Did you find the concept of the ‘principal problem’ in the process of care clear and meaningful? If not, do you have any alternative suggestions?” and “Did you find it possible to rate the contributory factors in a case when working from records alone?” Participants were required to either return fully completed review forms or to provide a summary of the adverse events they had reviewed on the evaluation questionnaire.

RESULTS

In most cases the MRF2 was completed adequately and, in some cases, the level of detail was exceptional. Most of the reviewers (7–10 teams) agreed with the 12 positively worded statements regarding the MRF2 (table 1). Only one reviewer gave consistent negative feedback (disagreeing with nine statements). Several useful comments were received on how the review form could be improved and, where possible, the final version has been modified accordingly. A summary of comments is included in box 2.

Three teams directly reported difficulty with the term “principal problem”. Of these, one was uncertain whether the question referred to the “principal problem for admission or with patient”, with two others finding that the principal problem was not sufficiently distinguished from the adverse event itself. The remaining teams reported that, on the whole, they found the term useful.

Five teams found “…it possible to rate the contributory factors in a case when working from records alone” whereas four teams did not. Of the three remaining teams, one lead reviewer left this question blank, another had “intimate knowledge of the adverse event”, and the third thought it was necessary to include someone from the department in the review team. Almost all said it was “…easier to rate contributory factors if you were familiar with the department in which the adverse event occurred”. Comments on the guidance notes, where given, were generally positive but additions and clarifications were requested, particularly for definitions of terms. One team also suggested moving more definitions into the review form itself. As a result of the study we have modified the MRF2 to take account of criticisms and suggestions (table 2).

DISCUSSION

The review form was generally thought to be comprehensive, well structured, and clear. The new format scored particularly well on its comprehensive nature and its division into modules, although some difficulties with terminology and definitions remained. A few teams reported difficulty with the term “principal problem”, although most found the term useful after some additional clarification had been provided. The definition of an adverse event needs particular emphasis, both on the review form and during the review process. We have since added further explanatory statements and have provided illustrative examples—such as was it a diagnostic error, technical mishap, or failure to monitor?

Identifying contributory factors is difficult but reviewers with detailed knowledge of the type of adverse event or local circumstances can provide useful indications on contributory factors which will form the basis of improvement and prevention strategies. Completion of this and other sections draws on the expertise and experience of the reviewer and, in that sense, their judgement is required. This can be addressed with training and reliability checks.

Devising a review form that is capable of encompassing almost all aspects of hospital care inevitably results in a long document. We have, where possible, tried to reduce the level of detail in the interest of making the whole process more manageable. We purposely kept the guidance notes short and inserted definitions within the review form itself, as some reviewers in our previous study⁶ found it irksome to consult the manual while completing the form. While the complete form may still look daunting, reviewing a single record only involves completing a relatively small number of specific subsections. The process could be made more efficient with an electronic version of the form. Reviewers may have to adapt the forms to their particular country and circumstances, and perhaps add additional guidance notes appropriate to the local setting. However, results from this pilot study suggest that the underlying structure should prove robust. Reviewers with a particular interest in a certain type of adverse event may wish to include much more detailed questions on some aspects of care.

Our experience of studying adverse events and piloting these forms indicates that training of reviewers remains of vital importance. Clinical ability and acumen, although essential, do not guarantee that the review process will be either understood or adhered to. Preparation by review of trial cases followed by discussion is essential. Without training in identifying the key issues quickly and efficiently, the process can become unduly time consuming. In our experience^6,13 most disagreements usually revolve around definitions and terminology rather than differences of opinion on clinical matters.

We acknowledge that the present study is only a preliminary test of the modified review form and that it needs to be compared with previous versions and tried out in larger studies in a variety of settings. As no formal comparison with the old form has been made, we are unable to comment on whether the MRF2 is an improvement on previous versions. Furthermore, we were unable to control for variation in the specific instructions and information that was given to the reviewers by team leaders or the variability of specialties and experience of each review team. However, it is worth acknowledging the practical setting in which these forms may be used. We consider that the positive nature of the present findings suggests that the modified form should be considered for use in future case record reviews.

In our view the full potential of case record review has yet to be explored. Most adverse event reviews conducted so far have been labour intensive, large scale studies. However, we believe that there is considerable potential for small scale studies which may be either local clinical review or formal research. Case record review could be especially valuable for routine review at the local level, particularly when specific types of cases are targeted and with a full exploration of causes and methods of prevention. The modular structure of the present form allows those with limited resources to cut down the review process to its essentials and adapt it to their particular purpose. There is also considerable scope for using record review prospectively in combination with staff interviews. Record review, whether retrospective or prospective, remains one of the primary methods for assessing the incidence of adverse events.^10,11