Error Review: Can This Improve Reporting Performance?

doi:10.1053/crad.2001.0760

Clinical Radiology

Volume 56, Issue 9, September 2001, Pages 751-754

https://doi.org/10.1053/crad.2001.0760 Get rights and content

Abstract

AIM: This study aimed to assess whether error review can improve radiologists' reporting performance. MATERIALS AND METHODS: Ten Consultant Radiologists reported 50 plain radiographs, in which the diagnoses were established. Eighteen of the radiographs were normal, 32 showed an abnormality. The radiologists were shown their errors and then re-reported the series of radiographs after an interval of 4–5 months. The accuracy of the reports to the established diagnoses was assessed. Chi-square test was used to calculate the difference between the viewings. RESULTS: On re-reporting the radiographs, seven radiologists improved their accuracy score, two had a lower score and one radiologist showed no score difference. Mean accuracy pre-education was 82.2%, (range 78–92%) and post-education was 88%, (range 76–96%). Individually, two of the radiologists showed a statistically significant improvement post-education (P < 0.01,P < 0.05). Assessing the group as a whole, there was a trend for improvement post-education but this did not reach statistical significance. Assessing only the radiographs where errors were made on the initial viewing, for the group as a whole there was a 63% improvement post-education. CONCLUSION: We suggest that radiologists benefit from error review, although there was not a statistically significant improvement for the series of radiographs in total. This is partly explained by the fact that some radiologists gave incorrect responses post-education that had initially been correct, thus masking the effect of the educational intervention. Tudor, G. R. and Finlay, D. B. (2001). Clinical Radiology56, 751–754.

References (24)

CA Daly et al.
Second round cancers: how many were visible on the first round of the UK National Breast Screening Programme, three years earlier?
Clin Radiol
(1998)
PJ Robinson
Radiology's Achilles heel: error and variation in the interpretation of the Rontgen image
Br J Radiol
(1997)
DA Davis et al.
Evidence for the effectiveness of CME. A review of fifty randomised controlled trials
JAMA
(1992)
AD Oxman et al.
No magic bullets: a systematic review of 102 trials of interventions
Can Med Assoc J
(1995)
DA Davis et al.
Changing physician performance: a systematic review of continuing medical education strategies
JAMA
(1995)
JA Espinosa et al.
Reducing errors made by emergency physicians in interpreting radiographs: longitudinal study
BMJ
(2000)
JM de Rijke et al.
A blind review and an informed review of interval breast cancer cases in the Limburg screening programme, the Netherlands
J Med Screen
(2000)
GR Tudor et al.
Is there an improvement in performance when radiographs are re-reported at 24 hours?
Br J Radiol
(1999)
RR Newell et al.
Descriptive classification of pulmonary shadows: revelation of unreliability in roentgenographic diagnosis of tuberculosis
Am Rev Tuberculosis
(1954)
LH Garland
Studies on the accuracy of diagnostic procedures
AJR
(1959)

DL Renfrew et al.

Error in radiology: classification and lessons in 182 cases presented at a problem case conference

Radiology

(1992)

Cited by (19)

Perceptual and Interpretive Error in Diagnostic Radiology—Causes and Potential Solutions
2019, Academic Radiology
Interpretation of increasingly complex imaging studies involves multiple intricate tasks requiring visual evaluation, cognitive processing, and decision-making. At each stage of this process, there are opportunities for error due to human factors including perceptual and ergonomic conditions. Investigation into the root causes of interpretive error in radiology first began over a century ago. In more recent work, there has been increasing recognition of the limits of human image perception and other human factors and greater acknowledgement of the role of the radiologist's environment in increasing the risk of error. This article reviews the state of research on perceptual and interpretive error in radiology. This article focuses on avenues for further error examination, and strategies for mitigating these errors are discussed. The relationship between artificial intelligence and interpretive error is also considered.
Adult chest radiograph reporting by radiographers: Preliminary data from an in-house audit programme
2014, Radiography
Citation Excerpt :
A recent study of the performance of trained reporting radiographers in a structured clinical examination reported a high sensitivity and specificity (95.4% and 95.9% respectively), with the common false negative and false positive errors comparable to those made by radiologists.15 Multidisciplinary meetings (MDM) where radiographs are reviewed by radiologists with the clinicians have become increasingly common.1,40 They also provide a learning environment for radiologist, reporting radiographer and clinician alike.
To examine the adult chest radiograph (CXR) reporting performance of a reporting radiographer in clinical practice using different audit systems; single radiologist and two radiologists, with clinical review of discordant cases.
100 chest radiographs (CXRs) were drawn randomly from a consecutive series of 4800 CXRs which had been reported during a nine month period at a district general hospital by a radiographer after two years of training. Diagnostic outcomes were normal or abnormal, and agreement with the reporting radiographer or not. There was 50% duplication of CXRs reported between three radiologists. Concordance rates were determined for the radiographer-radiologist and inter-radiologist interpretations. Independent clinical review of discordant cases was performed to establish the final diagnosis.
Ninety-nine cases were reviewed, with 40 cases deemed abnormal by at least one radiologist. Consensus was found with the radiographers report in 59 normal and 33 abnormal CXRs reviewed by two radiologists (96.7% and 86.8% respectively). Seven CXR reports were discrepant with clinical review: mediastinal lymphadenopathy was missed by both radiologist and radiographer; linear atelectasis was reported by two radiologists but not the radiographer. Three cases were over-interpreted and on two occasions at least one radiologist agreed with the radiographer. There was very high concordance between the radiographer and each radiologist, 96%, 96% and 92% respectively.
This study suggested that regular audit, which incorporates case note review and discrepant reporting within a multidisciplinary setting, should contribute to safe practice.
Diagnostic decision-making and strategies to improve diagnosis
2013, Current Problems in Pediatric and Adolescent Health Care
Citation Excerpt :
Feedback on performance in both actual and simulated care scenarios offers clinicians the opportunity to learn from mistakes and near-misses. Tudor and Finlay27 demonstrated improvement in diagnostic accuracy among expert radiologists by providing feedback and review of committed errors. Educational efforts aimed at familiarizing clinicians about the sciences of decision-making and applying them more explicitly in clinical practices is crucial.
A significant portion of diagnostic errors arises through cognitive errors resulting from inadequate knowledge, faulty data gathering, and/or faulty verification. Experts estimate that 75% of diagnostic failures can be attributed to clinician diagnostic thinking failure. The cognitive processes that underlie diagnostic thinking of clinicians are complex and intriguing, and it is imperative that clinicians acquire explicit appreciation and application of different cognitive approaches to make decisions better. A dual-process model that unifies many theories of decision-making has emerged as a promising template for understanding how clinicians think and judge efficiently in a diagnostic reasoning process. The identification and implementation of strategies for decreasing or preventing such diagnostic errors has become a growing area of interest and research. Suggested strategies to decrease diagnostic error incidence include increasing clinician's clinical expertise and avoiding inherent cognitive errors to make decisions better. Implementing Interventions focused solely on avoiding errors may work effectively for patient safety issues such as medication errors. Addressing cognitive errors, however, requires equal effort on expanding the individual clinician's expertise. Providing cognitive support to clinicians for robust diagnostic decision-making serves as the final strategic target for decreasing diagnostic errors. Clinical guidelines and algorithms offer another method for streamlining decision-making and decreasing likelihood of cognitive diagnostic errors. Addressing cognitive processing errors is undeniably the most challenging task in reducing diagnostic errors. While many suggested approaches exist, they are mostly based on theories and sciences in cognitive psychology, decision-making, and education. The proposed interventions are primarily suggestions and very few of them have been tested in the actual practice settings. Collaborative research effort is required to effectively address cognitive processing errors. Researchers in various areas, including patient safety/quality improvement, decision-making, and problem solving, must work together to make medical diagnosis more reliable.
Does PACS improve diagnostic accuracy in chest radiograph interpretations in clinical practice?
2012, European Journal of Radiology
Citation Excerpt :
The sensitivity reported in this study varies between 0.51 and 0.86, depending on period and whether uncertain findings were included or excluded. Even though it is desirable with a sensitivity of close to 100%, these results are in line with other studies [20–24]. Eng et al. reported, for example, in a prospective study of a mixed set of images, sensitivities between 0.53 and 0.78 for radiologists, 0.24 and 0.42 for emergency medicine physicians [20].
To assess the impact of a Picture Archiving and Communication System (PACS) on the diagnostic accuracy of the interpretation of chest radiology examinations in a “real life” radiology setting.
During a period before PACS was introduced to radiologists, when images were still interpreted on film and reported on paper, images and reports were also digitally stored in an image database. The same database was used after the PACS introduction. This provided a unique opportunity to conduct a blinded retrospective study, comparing sensitivity (the main outcome parameter) in the pre and post-PACS periods.
We selected 56 digitally stored chest radiograph examinations that were originally read and reported on film, and 66 examinations that were read and reported on screen 2 years after the PACS introduction. Each examination was assigned a random number, and both reports and images were scored independently for pathological findings. The blinded retrospective score for the original reports were then compared with the score for the images (the gold standard).
Sensitivity was improved after the PACS introduction. When both certain and uncertain findings were included, this improvement was statistically significant. There were no other statistically significant changes.
The result is consistent with prospective studies concluding that diagnostic accuracy is at least not reduced after PACS introduction. The sensitivity may even be improved.
In a New Kind of Light: Patient Safety in Pediatric Radiology
2006, Clinical Pediatric Emergency Medicine
Patient safety has typically been underemphasized in pediatric radiology. There are many reasons why this is the case, including the emphasis on technology in the specialty of radiology as well as inherent differences in the physician-patient relationship. However, radiologists have the same responsibility as other pediatric health care providers for protecting the welfare of the children (and families) undergoing imaging evaluation. This responsibility goes beyond the typical focus of safe practice in radiology, which includes radiation protection and administration of intravenous contrast media and sedative agents. Radiologists must work with clinical colleagues to develop a comprehensive safety program that crosses specialties and incorporates the components of contemporary models, including a focus on communication (especially at the local, patient care level), adherence to protocol (minimizing human contribution), and nonpunitive procedures for error reporting and review.
Preoperative evaluation of live renal donors using multislice CT angiography
2005, Clinical Radiology
Citation Excerpt :
As shown in Fig. 5, whereas the number of reporting errors remains fairly constant, the use of CT angiography has increased markedly over the period of the study, and it can be seen that there is a trend towards improved reporting accuracy. This is due largely to increased experience of reporting radiologists and greater awareness of potential reporting pitfalls.7,17,19–24 However, another very important factor has been close liaison with the surgical team, and also clinicoradiological review of errors as they have arisen, which has highlighted imaging shortcomings and brought about modifications to the imaging techniques which have further improved accuracy.24
To determine the accuracy of multidetector row CT renal angiography in the preoperative evaluation of live kidney donors, and to identify potential pitfalls when reporting.
Between July 1998 and June 2003, 74 consecutive live renal donors underwent contrast-enhanced multidetector row CT renal angiography before donor nephrectomy. The operative notes and radiological reports of all cases were reviewed retrospectively. Where a significant discrepancy was identified, the archived images were reviewed by two radiologists in the light of the intraoperative findings.
A total of 12 discrepancies were identified in 11 of the 74 cases (15%). In the preoperative CT angiography reports, 4 accessory arteries, 6 early-branching renal arteries, 1 duplicated renal vein and 1 accessory ureter were not identified; 9 of these were evident on review, but were not detected at the time of reporting. In 3 cases (1 accessory artery, the duplicated renal vein and the accessory ureter), the anomaly was not visible on review of the CT angiographic data, even with the benefit of hindsight. Surgical feedback during the study period resulted in modifications to CT technique and improved performance.
In the majority of cases, there was good correlation between preoperative CT renal angiography and operative findings. Most discrepancies were due either to an oversight by the reporting radiologist, or failure to fully appreciate the potential surgical significance of certain findings. Regular surgical feedback plays a valuable role in improving reporting accuracy and maintaining imaging standards.

View all citing articles on Scopus

^f1: Author for correspondence and guarantor of study: Dr G. R. Tudor, Department of Radiology, Princess of Wales Hospital, Coity Road, Bridgend, S. Wales, CF31 1RQ, U.K.

View full text

Regular ArticleError Review: Can This Improve Reporting Performance?

Abstract

Clin Radiol

Radiology's Achilles heel: error and variation in the interpretation of the Rontgen image

Br J Radiol

Evidence for the effectiveness of CME. A review of fifty randomised controlled trials

JAMA

No magic bullets: a systematic review of 102 trials of interventions

Can Med Assoc J

Changing physician performance: a systematic review of continuing medical education strategies

JAMA

Reducing errors made by emergency physicians in interpreting radiographs: longitudinal study

BMJ

A blind review and an informed review of interval breast cancer cases in the Limburg screening programme, the Netherlands

J Med Screen

Is there an improvement in performance when radiographs are re-reported at 24 hours?

Br J Radiol

Descriptive classification of pulmonary shadows: revelation of unreliability in roentgenographic diagnosis of tuberculosis

Am Rev Tuberculosis

Studies on the accuracy of diagnostic procedures

AJR

Error in radiology: classification and lessons in 182 cases presented at a problem case conference

Radiology

Regular Article
Error Review: Can This Improve Reporting Performance?