Objectives: To redesign the existing clinical review form (RF2) used in previous retrospective case record review studies in order to clarify the review process and provide a more powerful analysis of adverse events; and then to ask clinicians to pilot and evaluate the new modular review form (MRF2). The review form is divided into five sections, each with a defined purpose, providing a modular structure.
Design: Design and testing of the MRF2 on a sample of medical and nursing records, and evaluation of the reviewers’ responses regarding the new review form.
Setting: Hospital based teams from eight countries.
Results: The modular review form was reported to be comprehensive, well structured, and clear. Most of the reviewers agreed with the positive statements regarding the review form. Overall, the modular structure was thought to be helpful. Several modifications have been made to the final version to take account of criticisms and suggestions.
Conclusions: The full potential of case record review has yet to be explored. The benefits of this review form include a modular format which enables reviewers or project leaders to select the focus of their review based on resources and the purpose of the review, and to identify contributory factors which indicate areas for improvement and prevention. The training of reviewers is of vital importance for record review. Record review remains one of the primary methods for assessing the incidence of adverse events and the new format is suitable for both prospective and retrospective review.
Statistics from Altmetric.com
The use of case record review to identify adverse events has been important in the drive to improve patient safety.1,2 The Harvard Medical Practice study3 carried out in New York in the mid 1980s provided powerful evidence of the scale of harm to patients in hospital. These initial findings were substantiated in further studies in the United States,4 in Australia,5 the UK,6 Denmark,7 and New Zealand,8 and there are ongoing studies in other countries.
The basic methodology was developed in the early 1970s for the Californian Insurance Feasibility Study.9 Retrospective case record review studies are carried out in two stages. First, using Review Form 1 (RF1), medical records are screened according to 18 predefined criteria (for example, unexpected death, hospital acquired infection/sepsis, unplanned return to the operating theatre) to identify records of patients more likely to have suffered an adverse event. Records meeting one or more of the screening criteria are forwarded for clinical review using Review Form 2 (RF2). In this second stage trained clinicians examine each case record in detail to determine whether or not an adverse event has occurred and to extract information about the nature and causes of adverse events. Each research group4–8 made minor modifications to the RF2, adding or subtracting specific questions, but the basic format used in the Harvard study3 was maintained.
The methodology of case record review has considerable strengths. It has provided a more complete indication of the incidence of adverse events or critical incidents than reporting systems, even when they are backed by additional monitoring by a dedicated risk manager.10 Familiarity and other biases are reduced when external independent assessors are used to conduct the review. The review forms provide a standardised method of recording and data collection which is robust when used on a random sample of case records. The epidemiological data obtained are potentially useful for comparative studies, although any comparisons need to take account of variations in methodology, particularly with the definition and inclusion criteria.
Readers are invited to respond to the three questions listed at the end of the accompanying commentary by Ross Wilson on page 402 by using the rapid response function http://qhc.bmjjournals.com/cgi/eletter-submit/12/6/402.
Case record review is, however, wholly dependent on the accuracy, completeness and legibility of patient records. Some information, such as the effects of the adverse event on the patient, is not generally recorded and often the adverse event itself is not explicitly stated in the record and may not be recognised until the patient is readmitted. Low to moderate inter-rater reliability has been reported.5,11 Finally, retrospective case record review can be time consuming and expensive. Despite these limitations, however, case record review has yielded important epidemiological data that have had a major effect on governmental policies and action by healthcare providers.2
RATIONALE FOR A REVISED REVIEW FORM
Earlier studies3–5 focused on the incidence and type of adverse events with some attempt to identify causes and methods of prevention.12 The original review form uses a mixture of taxonomies which are not always clearly distinguished. Reviewers in our study found that the structure of the review form meant that results did not always reflect the underlying clinical reality. For instance, with a standard RF2 a postoperative infection is classified as an operative event (because it occurred within 30 days of surgery) and non-technical (because it was not directly related to the operation itself). In a subsequent examination of the data we analysed in detail the narratives provided by the reviewers.13 This enabled us to define when in the process of care the adverse event occurred and the nature of the underlying problem (box 1, C1–C5). As a result, a postoperative chest infection would be classified as a ward based event and the problem as one related perhaps to drug administration (oversedation), to failure to monitor (such as over a weekend), or to failure to provide physiotherapy.
Box 1 Structure of the modular review form (MRF2)
(A) Patient information and background to adverse event (AE)
Primary illness; prognosis
Main features of AE
Identification of principal problem
→ identify period(s) of care (C1–C5)
Adequacy of records
(B) The injury and its effects
Disability (including death) caused by AE
Effect of AE on hospital resources (e.g. additional bed days)
Additional treatment as a result of AE
(C) Period of hospitalisation during which AE occurred (as identified in stage A)
C1: Care on admission ward including preoperative care →
C2: During procedure including surgery, anaesthesia →
C3: Immediate postoperative and ITU/HDC care →
C4: General ward care →
C5: Care/assessment during discharge
→ principal problem (D1–D7)
(D) Principal problems in the process of care (as identified in relevant subsection(s) in C)
D1: Diagnostic/assessment error
D2: In relation to patient’s overall condition
D3: Medical management/monitoring including nursing care
D4: In relation to infection
D5: Procedure (including anaesthesia/surgery)
D6: In relation to drug/IV fluid/blood transfusion
D7: In relation to a resuscitation procedure
(E) Causative/contributory factors and preventability of AE
Causative factors: patient characteristics, task factors, individual factors, team factors, work environment, organisational/management factors
Preventability of AE
Expertise of reviewer
Adverse events frequently involve a chain of events and a series of care management or care delivery problems leading up to the incident itself.14 In previous versions of the form multiple problems were sometimes identified, but were not prioritised and were not sufficiently distinguished in the final analyses. However, we found that there is usually one critical period in which the primary problem occurred. We therefore introduced the concept of a “principal problem”, being the most important problem in the delivery of care. This enabled a precise identification of a particular period in the process of care, now incorporated in the review process, and a defined principal problem which greatly clarified the review process. To provide additional clarity we divided the review form into five sections each with a defined purpose, providing a modular structure to both the form and the review process.
A final important change has been a much stronger emphasis on the broader organisational, environmental, and other factors that contribute to adverse events. While earlier versions paid some attention to these factors, the approach was not sufficiently systematic. The present approach is based on a previously devised framework15 and method of individual case analysis.16 Identifying contributory factors is best done from observation and interview, although this is quite labour and resource intensive. We believe that expert reviewers or those familiar with the working environment can often comment on some of the major contributory factors. This is particularly important as the identification of such factors offers a route to devising methods of prevention.
Thus, we have attempted to address three areas in which the earlier review forms have limitations:
We have divided hospital stay into distinct periods of care and then identified the clinical aspects associated with each period—to address the former mixed categories of the type of adverse event.
We have introduced the term “principal problem” to help clarify the review process in order to identify the healthcare management or treatment, or delay or lack of treatment, that led to the adverse event.
We have provided a comprehensive list of contributory factors to include organisational, environmental, and other contextual factors that influence the provision of health care.
THE MODULAR REVIEW FORM (MRF2)
The MRF2 comprises five stages or modules:
Patient information and background to the adverse event
Disability caused by the adverse event
Period of hospitalisation during which the adverse event occurred
Principal problems in the process of care
Causative/contributory factors and preventability of the adverse event
Once an adverse event has been identified following the first stage review (RF1), reviewers are required to complete stages A and B in full. When completing stage A they are required to identify one or more of five periods of care during which adverse event(s) occurred (C1–C5). Next they complete the relevant part(s) of stage C. In so doing, they are required to identify the nature of the underlying cause(s) of the adverse event ranging from errors in diagnosis to mismanagement at the time of discharge. This leads to the completion of one or more of subsections D1–D7. Finally, stage E provides the opportunity for identifying contributory factors and potential preventability. Details of each stage are shown in box 1 and fig 1. The modular structure allows reviewers to select which stages to complete, depending on the focus of the review. The MRF2 is available on the QSHC website (http://www.qshc.com/supplemental).
The objectives of this study were to pilot and evaluate the new review form (MRF2) using an evaluation questionnaire.
PILOTING AND PRELIMINARY EVALUATION OF THE MRF2
After publication of our case record review study,6 a considerable number of clinicians and researchers contacted us to ask for copies of the review forms with a view to conducting studies in their own hospitals. All those who had previously requested the materials were contacted and invited to take part in piloting the MRF2. Of 36 individuals contacted, 20 agreed to take part.
Twelve teams (from the UK (2), Italy (2), France, Spain, Australia (2), New Zealand, Japan and USA (2)) took part in piloting the MRF2. Team members had a range of clinical expertise and specialist interests including epidemiology, paediatrics, nursing, infectious diseases, pharmacy, endocrinology, anaesthetics, nephrology, and emergency medicine. Eight teams sent 44 completed forms and the remaining four provided summaries of 20 adverse events. The number of reviewers per team ranged from 1 to 3 and the number of review forms completed per reviewer ranged from 1 to 6.
Participants were asked to identify 5–10 known adverse events and to review each one using the MRF2. The reviewers were warned that it takes time to understand the review process and, in particular, to interpret definitions correctly. The instruction document provided definitions of important concepts, an outline of the review process, and a list of conditions which reviewers have had difficulty with in previous review studies such as hospital acquired infection, pressure sores, and falls. Reviewers were given the opportunity to contact the research team with queries related to the review process.
After using the new forms, participants were asked to rate them on an evaluation questionnaire consisting of ratings of the statements shown in table 1 together with questions on the strengths and weaknesses of each stage of the review form and the form overall. In addition to this we posed some specific questions on issues of concern. For example: “Did you find the concept of the ‘principal problem’ in the process of care clear and meaningful? If not, do you have any alternative suggestions?” and “Did you find it possible to rate the contributory factors in a case when working from records alone?” Participants were required to either return fully completed review forms or to provide a summary of the adverse events they had reviewed on the evaluation questionnaire.
In most cases the MRF2 was completed adequately and, in some cases, the level of detail was exceptional. Most of the reviewers (7–10 teams) agreed with the 12 positively worded statements regarding the MRF2 (table 1). Only one reviewer gave consistent negative feedback (disagreeing with nine statements). Several useful comments were received on how the review form could be improved and, where possible, the final version has been modified accordingly. A summary of comments is included in box 2.
Box 2 Comments from reviewers
Strengths of the review form
Layout comprehensive and easy to follow.
Comprehensive but lengthy—relatively easy to complete.
The modular structure is definitely helpful in identifying the particular period of care in which problems arose.
Clear separation of different components allows detailed review.
Good basis; good mix of open ended questions and categorisation of events.
Useful for clinicians for risk management.
Strong emphasis on identifying systemic factors and contributory causes.
Easy to complete.
Problems can be identified with greater precision by probing with follow up questions.
Specific and potential to obtain a lot of information.
Limitations of the review form
Layout initially appears confusing.
Too complex—too specific and too exhaustive–need specialist experience to complete and/or considerable dedication to unearth circumstances surrounding an adverse event.
Not enough information for epidemiological analysis but too complicated for clinical use.
Not suitable for root cause analysis.
Some questions are too subjective—will get different answers from different reviewers.
Too time consuming.
Sometimes impossible to answer questions—need more “don’t know” or “data not available” options.
Repetition in some areas.
Three teams directly reported difficulty with the term “principal problem”. Of these, one was uncertain whether the question referred to the “principal problem for admission or with patient”, with two others finding that the principal problem was not sufficiently distinguished from the adverse event itself. The remaining teams reported that, on the whole, they found the term useful.
Five teams found “…it possible to rate the contributory factors in a case when working from records alone” whereas four teams did not. Of the three remaining teams, one lead reviewer left this question blank, another had “intimate knowledge of the adverse event”, and the third thought it was necessary to include someone from the department in the review team. Almost all said it was “…easier to rate contributory factors if you were familiar with the department in which the adverse event occurred”. Comments on the guidance notes, where given, were generally positive but additions and clarifications were requested, particularly for definitions of terms. One team also suggested moving more definitions into the review form itself. As a result of the study we have modified the MRF2 to take account of criticisms and suggestions (table 2).
The review form was generally thought to be comprehensive, well structured, and clear. The new format scored particularly well on its comprehensive nature and its division into modules, although some difficulties with terminology and definitions remained. A few teams reported difficulty with the term “principal problem”, although most found the term useful after some additional clarification had been provided. The definition of an adverse event needs particular emphasis, both on the review form and during the review process. We have since added further explanatory statements and have provided illustrative examples—such as was it a diagnostic error, technical mishap, or failure to monitor?
Identifying contributory factors is difficult but reviewers with detailed knowledge of the type of adverse event or local circumstances can provide useful indications on contributory factors which will form the basis of improvement and prevention strategies. Completion of this and other sections draws on the expertise and experience of the reviewer and, in that sense, their judgement is required. This can be addressed with training and reliability checks.
Devising a review form that is capable of encompassing almost all aspects of hospital care inevitably results in a long document. We have, where possible, tried to reduce the level of detail in the interest of making the whole process more manageable. We purposely kept the guidance notes short and inserted definitions within the review form itself, as some reviewers in our previous study6 found it irksome to consult the manual while completing the form. While the complete form may still look daunting, reviewing a single record only involves completing a relatively small number of specific subsections. The process could be made more efficient with an electronic version of the form. Reviewers may have to adapt the forms to their particular country and circumstances, and perhaps add additional guidance notes appropriate to the local setting. However, results from this pilot study suggest that the underlying structure should prove robust. Reviewers with a particular interest in a certain type of adverse event may wish to include much more detailed questions on some aspects of care.
Our experience of studying adverse events and piloting these forms indicates that training of reviewers remains of vital importance. Clinical ability and acumen, although essential, do not guarantee that the review process will be either understood or adhered to. Preparation by review of trial cases followed by discussion is essential. Without training in identifying the key issues quickly and efficiently, the process can become unduly time consuming. In our experience6,13 most disagreements usually revolve around definitions and terminology rather than differences of opinion on clinical matters.
We acknowledge that the present study is only a preliminary test of the modified review form and that it needs to be compared with previous versions and tried out in larger studies in a variety of settings. As no formal comparison with the old form has been made, we are unable to comment on whether the MRF2 is an improvement on previous versions. Furthermore, we were unable to control for variation in the specific instructions and information that was given to the reviewers by team leaders or the variability of specialties and experience of each review team. However, it is worth acknowledging the practical setting in which these forms may be used. We consider that the positive nature of the present findings suggests that the modified form should be considered for use in future case record reviews.
In our view the full potential of case record review has yet to be explored. Most adverse event reviews conducted so far have been labour intensive, large scale studies. However, we believe that there is considerable potential for small scale studies which may be either local clinical review or formal research. Case record review could be especially valuable for routine review at the local level, particularly when specific types of cases are targeted and with a full exploration of causes and methods of prevention. The modular structure of the present form allows those with limited resources to cut down the review process to its essentials and adapt it to their particular purpose. There is also considerable scope for using record review prospectively in combination with staff interviews. Record review, whether retrospective or prospective, remains one of the primary methods for assessing the incidence of adverse events.10,11
For 25 years a basic form has been used for recording the results of case record review to identify adverse events in hospital practice.
The method has provided epidemiological data but has been of limited value in defining targets for improvement in care
We present a revised form which has a modular structure. Key components define (1) the period of care during which action or inaction led to the adverse event and (2) the type of clinical action or inaction.
A further module offers the opportunity of recording factors that may have contributed to the adverse event.
This revised form received mostly favourable comments when assessed by researchers in 12 other units.
The method needs to be tested in practice; it may be of particular value in institutions that wish to identify areas of improvement based on contributing factors.
As with previous versions of the form, training in completing the form is essential.
The authors thank the Dunhill Medical Trust and the BUPA Foundation for funding this pilot study; and J Minton (UK), C Pearson (UK), J Smit (Australia), M Silver (USA), J M Aranaz (Spain), R Cinotti (Italy), P Michel (France), A Gillian (Australia), M Kelly (USA), D Kunac (New Zealand), F Briganti (Italy), T Ohkubo (Japan), and their colleagues for their time and comments regarding the MRF2.
The appendix is available as a downloadable PDF (printer friendly file).
If you do not have Adobe Reader installed on your computer,
you can download this free-of-charge, please Click here
Files in this Data Supplement:
- [View PDF] - MODULAR REVIEW FORM (MRF2) for Retrospective Case Record Review
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.