Comparison of On-Call Radiology Resident and Faculty Interpretation of 4- and 16-row Multidetector CT Pulmonary Angiography with Indirect CT Venography

doi:10.1016/j.acra.2007.06.030

Academic Radiology

Volume 15, Issue 1, January 2008, Pages 71-76

https://doi.org/10.1016/j.acra.2007.06.030 Get rights and content

Rationale and Objectives

On-call radiology residents frequently interpret computed tomography (CT) pulmonary angiography and CT venography studies outside of routine working hours. The purpose of this study was to compare resident and faculty interpretation concordance rates and to see if concordance rates differed depending on the number of CT detectors used.

Materials and Methods

The study population included 122 consecutive CT pulmonary angiography (CTPA) and CT venography (CTV) examinations performed on a four-row multidetector CT (MDCT) and 125 consecutive CTPA examinations performed using a 16-row MDCT scanner with CTV performed in 124 patients. Preliminary resident reports and final faculty reports were compared. Discrepant cases were independently reviewed by three cardiothoracic radiologists who were unaware of the initial interpretations. Interpretation concordance rates were calculated for both 4- and 16- row MDCT studies and compared using Fisher’s exact test.

Results

Resident and faculty CTPA and CTV interpretations were concordant in 80% of the 4-row cases and 94% of the 16-row cases. When comparing resident interpretation to the final expert reference standard, the corrected resident error rate was 11% and 2% for 4-row CTPA and CTV, respectively and 4% and 2% for 16-row CTPA and CTV, respectively. Overall CTPA and CTV concordance was significantly lower for 4-row MDCT (80% versus 94%, P < .001 [two-sided] by Fisher’s exact test).

Conclusions

Radiology resident interpretation of CTPA and CTV studies demonstrates a high level of agreement with radiology faculty interpretation. Concordance rates are significantly higher for 16-row MDCT than 4-row MDCT which may be due to improved image quality.

Section snippets

Study Population

Patient populations consisted of both emergency room patients and inpatients. All studies were performed outside of routine work hours (weekdays 5 pm to 8 am, weekends, and holidays). The 4-row MDCT study group consisted of 122 consecutive CTPA and CTV studies performed over a 5-month period from March 2001 to July 2001. There were 75 females and 47 males with a mean age of 53 years (range 21–80). The 16-row MDCT group consisted of 125 consecutive CTPA examinations performed over a 3-month

Results

Resident and faculty CTPA and CTV interpretations agreed in 98 of 122 patients (80%) for 4-row MDCT and 118 of 125 patients (94%) using 16-row MDCT (Fig 1). The difference in concordance rates was statistically significant (P < .001 [two-sided] by Fisher’s exact test). Using the final expert panel reference standard, CTPA agreement was significantly higher with 16-row MDCT than the 4-row MDCT (94% versus 83%, P = .0046 [two-sided] by Fisher’s exact test). CTV agreement was 96% with 4-row MDCT

Discussion

Resident interpretation of emergent radiology studies is becoming increasingly significant, particularly in the university hospital environment. A prospective study by Scott et al found that radiology residents are interpreting a diverse and increasing number of emergent diagnostic examinations after hours. In their study, the most common examination performed during off hours was CT (69%) (2). The results of our study show that preliminary interpretations of CTPA and CTV examinations by

Conclusion

Interpretation of CTPA and CTV studies by second, third, and fourth year on-call radiology residents demonstrates a high level of concordance with radiology faculty interpretations, supporting the practice of resident interpretation of emergent CTPA and CTV studies performed outside of routine work hours. Concordance rates may be further improved when MDCT scanners with a greater number of detectors are used.

References (14)

M.C. Albano et al.
Resident interpretation of emergency CT scans in the evaluation of acute appendicitis
Acad Radiol
(2001)
D. Shaham et al.
CT pulmonary angiography for the detection of pulmonary embolism: interobserver agreement between on-call radiology residents and specialists (CTPA interobserver agreement)
Clini Imaging
(2006)
S. Patel et al.
Helical CT for the evaluation of acute pulmonary embolism
AJR Am J Roentgenol
(2005)
J. Scott et al.
On-call services provided by radiology residents in a university hospital environment
Can Assoc Radiol J
(2003)
N.R. Lal et al.
Clinical consequences of misinterpretations of neuroradiologic CT scans by on-call radiology residents
AJNR Am J Neuroradiol
(2000)
W.K. Erly et al.
Radiology resident evaluation of head CT scan orders in the emergency department
Am J Neuroradiol
(2002)
L. Lowe et al.
Nonenhanced limited CT in children suspected of having appendicitis: prospective comparison of attending and resident interpretations
Radiology
(2001)

There are more references available in the full text version of this article.

Cited by (21)

Preserving the educational value of call in a diagnostic radiology residency program
2014, Journal of the American College of Radiology
Citation Excerpt :
At Arkansas Children's Hospital, more than 3,600 preliminary readings have been reviewed since 2010 and accuracy is >95%, with no case resulting in a negative effect on patient care. This rate is similar to that reported in the literature [11-19]. Diagnostic radiology programs face challenges on many fronts.
Our study was designed to determine residents' opinions of the advantages, disadvantages and educational value of a traditional "Tandem Call" (TC) model as compared to night float (NF). Because TC is more representative of adult learning principles and constructivist theory, we hypothesized that resident satisfaction and educational outcomes would demonstrate a preference for, and the educational efficacy of, the TC model.
We surveyed all residents in a university-based radiology residency on their opinions of TC and its educational value. Aggregate data from annual Graduate Medical Education Committee institutional surveys (2008-2012) and annual radiology alumni surveys (2009-2012) were reviewed as measures of satisfaction with TC. Performance on the ABR oral exam was a proxy for educational outcome. Quality data for the year of study and prior years in which TC was in effect were reviewed as a measure of patient safety.
The great majority of respondents attributed confidence/competence on call and added value to their education directly to TC. A majority believed that teamwork required for TC facilitated more positive relationships among residents and more peer teaching. Most said that they would not prefer NF. Almost all believed indirect supervision with attending backup aided in developing confidence in performance. Quality data confirmed a low number of discrepancies between preliminary resident and final attending reads.
TC provides a more consistent call experience throughout residency than NF. TC is valued by residents, facilitates retrieval-based learning and development of independence and efficiency, and parallels essential elements of team-based learning. Quality data suggests that lack of 24-hour attending supervision is not detrimental to patient safety.
Pulmonary Embolism
2012, Emergency Medicine Clinics of North America
Comparison of discrepancy rates in resident and faculty interpretations of On-Call PE CT and V/Q scans: Is one study more reliable during off hours?
2011, Journal of the American College of Radiology
Citation Excerpt :
To our knowledge, there has been no comparison of PE CT and V/Q scans in terms of residents' accuracy in interpretation during off hours. Consistent with the findings of multiple prior studies, we have demonstrated a high level of agreement (95%, κ = 0.77) between preliminary resident interpretations and final faculty member interpretations of PE CT examinations [15-20]. Rufener et al [19] and Cervini et al [18] previously demonstrated 94% and 90% agreement, respectively, between residents' and attending radiologists' interpretations of CT pulmonary angiographic examinations.
There has been no comparison of concordance rates for residents' and faculty members' interpretations of pulmonary embolism (PE) CT and ventilation/perfusion (V/Q) studies. If significantly different rates of agreement are demonstrable, this could influence which test is ordered during off hours. The purpose of this study was to evaluate how the performance of residents compared with that of faculty members in interpreting these two modalities.
Interobserver agreement between preliminary resident reports and final attending radiologist interpretations was calculated retrospectively for PE CT studies (n = 1,179) and V/Q scans (n = 331) interpreted by on-call residents from April 2007 to October 2008. Discordant cases were reviewed for clinical significance and outcomes at 3 months. Interobserver variability was also evaluated relative to residents' training levels and faculty members' years of experience.
Interobserver agreement between faculty members and residents was substantial, at 95% for PE CT (κ = 0.77) and 77.9% for V/Q scans (κ = 0.67). Although changes in interpretation were significantly more common with V/Q scans (22.1% vs 5%; χ² < .0001), the rates of clinically significant discrepancy did not differ significantly between the modalities (2.1% for V/Q vs 1.2% for PE CT; χ² P = .20). The overall discrepancy rate and the rate of clinically significant discrepancy did not correlate with residents' training levels or faculty members' years of experience.
Concordance rates for residents' interpretations of V/Q scans and PE CT studies were high, and discrepancies resulting in changes in patient management were rare for both modalities. Residents' preliminary interpretations of both modalities are reliable and safe for making initial patient management decisions.
Effect of Work Hours, Caseload, Shift Type, and Experience on Resident Call Performance
2010, Academic Radiology
Citation Excerpt :
Weekend overnight shifts are associated with a higher discrepancy rate (OR 1.3 [1.2–1.5], P < .001) than weekdays despite a shorter total workday (15 vs. 24 total hours, respectively). Published trainee-faculty concordance rates for the interpretation of on-call cross-sectional imaging studies have historically been extraordinarily good (9–19). Although encouraging, this low prevalence of discordances can make it difficult to assess the effect of specific variables (eg, changing call type) on resident call performance.
To analyze the independent effects of multiple variables on resident call performance.
Independent radiology resident “on call” cross-sectional imaging interpretation quality assurance (QA) data obtained during a 171-day period at a single tertiary care Level 1 trauma teaching institution was reviewed. Clinically significant resident-faculty discrepancies were compared among three different call types: traditional single-day overnight call (OC, 15 hours/night after 9 daytime hours on weekdays), 7-night nightfloat (NF, 9 hours/night), and weekend day call (WD, 10 hours/day). Logistic regression analyses were performed to evaluate associations.
There were 119 (0.89%) clinically significant resident-faculty discordances from 13,424 cross-sectional interpretations: 56 (0.79%) from 7102 interpretations on 172 OC shifts, 39 (0.85%) from 4567 interpretations on 165 NF shifts, and 24 (1.4%) from 1755 interpretations on 49 WD shifts. Individual residents (n = 20) had a mean discrepancy rate of 0.9% (0.45%–1.9%). Overall, 102 (26.2%) of the shifts had at least one discordance. The following were associated with significantly (P < .001) increased discrepancy rates: junior vs. senior residents (odds ratio [OR] = 1.3 [1.2–1.4]), OC vs. NF (OR = 1.5 [1.3–1.6], WD vs. NF (OR = 1.4 [1.2–1.6]), weekend vs. weekday (OR = 1.3 [1.2–1.4]), and increasing cases/hour (OR = 1.6 [1.5–1.7]). Weekend OC shifts had a higher discrepancy rate (OR 1.3[1.2–1.5], P < .001) than weekday OC shifts despite a shorter workday (15 vs. 24 hours).
Increasing caseload, junior residents, and weekends are associated with a significantly higher discrepancy rate. OC is associated with a significantly higher discrepancy rate than NF. Measured discrepancy rates are low, regardless of call type.
ECG-gated chest CT angiography: Value for atypical chest pain evaluation
2009, Journal de Radiologie
The aim of this study was to evaluate ECG-gated whole chest CTA as a routine triage tool for patients with acute chest pain.
Whole chest CTA with retrospective ECG-gating was performed in 30 patients with acute atypical chest pain. The ten main segments of the coronary arteries, the pulmonary arteries, the aorta, and the myocardium (function, morphology) were independently analyzed by a resident and two senior radiologists. The inter-observer agreement between resident and senior radiologists was calculated. A final diagnosis was determined by consensus.
Thirty patients were included. The coronary artery segments, myocardium and pulmonary arteries were considered analyzable in 84 %, 90 % and 97 % of cases respectively. A final diagnosis for the cause of pain was retained in 19 patients: significant coronary artery stenosis (5), pulmonary embolus (5), aortic dissection (1), hypokinetic cardiomyopathy (2), lung parenchymal abnormalities (5), and hiatus hernia (1). Inter-observer agreement ranged from 0.76 to 1 between senior radiologists and from 0.76 to 1 between resident and senior radiologists. The average time of image interpretation ranged from 14 to 15 minutes.
ECG-gated whole chest CT angiography appears as a promising tool for the evaluation of acute chest pain. Combined evaluation of appearance and function of the myocardium can reveal additional interesting information.
Le but de cette étude était d’évaluer un protocole d’angioscanner thoracique avec synchronisation ECG comme outil de tri des patients présentant une douleur thoracique aigue.
Un angioscanner de l’ensemble du thorax avec synchronisation rétrospective à l’ECG a été réalisé chez 30 patients présentant une douleur thoracique aigue. Les dix segments principaux des artères coronaires, les artères pulmonaires, l’aorte, et le myocarde (morphologie, fonction) ont été analysés de façon indépendante par un interne et deux radiologues seniors. L’agrément interobservateur et un diagnostic final ont été déterminés.
Trente patients ont été inclus. Les artères coronaires, le myocarde et les artères pulmonaires ont été considérés comme analysables dans respectivement 84, 90 et 97 % de cas. Un diagnostic final pour la douleur a été retenu chez 19 patients : sténose coronarienne (5), embolie pulmonaire (5), dissection aortique (1), cardiopathie hypokinétique (2), pathologies pulmonaires (5), hernie hiatale (1). L’agrément interobservateur était entre 0,76 et 1 entre les radiologues seniors et entre 0,76 et 1 entre l’interne et les radiologues seniors. Le temps moyen d’interprétation était de 14 à 15 minutes.
L’angioscanner thoracique avec synchronisation ECG avec analyse combinée de l’aspect et de la fonction du myocarde apparait être un outil intéressant pour l’évaluation des patients présentant une douleur thoracique aigue.
Pulmonary Embolism Detection Using Machine and Deep Learning Techniques
2024, Blockchain and Deep Learning for Smart Healthcare

View all citing articles on Scopus

¹: Funded in part by the RSNA Research Scholar Grant. Funded in part by the GE-AUR Radiology Research Academic Fellowship.

View full text

Original investigationComparison of On-Call Radiology Resident and Faculty Interpretation of 4- and 16-row Multidetector CT Pulmonary Angiography with Indirect CT Venography1

Rationale and Objectives

Materials and Methods

Results

Conclusions

Section snippets

Study Population

Results

Discussion

Conclusion

Acad Radiol

Clini Imaging

Helical CT for the evaluation of acute pulmonary embolism

AJR Am J Roentgenol

On-call services provided by radiology residents in a university hospital environment

Can Assoc Radiol J

Clinical consequences of misinterpretations of neuroradiologic CT scans by on-call radiology residents

AJNR Am J Neuroradiol

Radiology resident evaluation of head CT scan orders in the emergency department

Am J Neuroradiol

Nonenhanced limited CT in children suspected of having appendicitis: prospective comparison of attending and resident interpretations

Radiology

Original investigation
Comparison of On-Call Radiology Resident and Faculty Interpretation of 4- and 16-row Multidetector CT Pulmonary Angiography with Indirect CT Venography1