Survey of Faculty Perceptions Regarding a Peer Review System

doi:10.1016/j.jacr.2013.08.011

Journal of the American College of Radiology

Volume 11, Issue 4, April 2014, Pages 397-401

https://doi.org/10.1016/j.jacr.2013.08.011 Get rights and content

Purpose

Virtually all radiologists participate in peer review, but to our knowledge, this is the first detailed study of their opinions toward various aspects of the process.

Methods

The study qualified for quality assurance exemption from the institutional review board. A questionnaire sent to all radiology faculty at our institution assessed their views about peer review in general, as well as case selection and scoring, consensus section review for rating and presentation of errors, and impact on radiologist performance.

Results

Of 52 questionnaires sent, 50 were completed (response rate, 96.2%). Of these, 44% agreed that our RADPEER-like system is a waste of time, and 58% believed it is done merely to meet hospital/regulatory requirements. Conversely, 46% agreed that peer review improves radiologist performance, 32% agreed that it decreases medical error, and 42% believed that peer review results are valuable to protect radiologists in cases referred to the medical board. A large majority perform all peer reviews close to the deadline, and substantial minorities frequently or almost always select more than one previous examination for a single medical record number (28%), consciously select “less time intensive” cases (22%), and intentionally avoid cases requiring more time to peer review (30%).

Discussion

Almost one-half of respondents agreed that peer review has value, but as currently performed is a waste of time. The method for selecting cases raises serious questions regarding selection bias. A new approach is needed that stresses education of all radiologists by learning from the mistakes of others.

Introduction

The Joint Commission guidelines [1] state that practitioners are expected to “demonstrate knowledge of established and evolving biomedical, clinical, and social sciences, and the application of their knowledge to patient care and the education of others.” At most institutions, ongoing professional practice evaluation of radiologist performance includes a process of peer review based on a template first described by Donnelly [2]. Peer review should provide an unbiased, fair, and balanced evaluation of radiologist performance to identify opportunities for additional education, error reduction, and self-improvement [3]. Ideally, it should be nonpunitive, have minimal effect on workflow, and allow easy participation [3]. Although one article [4] reported that a “significant percentage” of faculty members viewed peer review as a “time-consuming bureaucratic process to create more paperwork” rather than a means to improve medical care, to our knowledge there has been no detailed study of the opinions of radiologists toward various specific aspects of peer review. Therefore, we undertook a study to assess the views of radiologists at a large urban medical center toward our peer review discrepancy system, which has been mandatory for more than 6 years.

Section snippets

Methods

The institutional review board determined that this study qualified for the quality assurance exemption.

A questionnaire was sent to 52 members of the radiology faculty to determine their views about our local peer review system. Very similar to the ACR's RADPEER™ product, it has been in place for more than 6 years and mandates that each radiologist submit a number of cases equal to 2.5% of each radiologist's prior year's volume (with a maximum of 300 cases). Questions for the survey were

Results

Of 52 questionnaires sent, 50 were completed (response rate, 96.2%). Of the 50 respondents, 44% agreed with the statement that peer review as performed using the RADPEER-like process in our department is a waste of time, 58% thought that peer review is done merely to meet hospital/regulatory requirements, and 42% believed that peer review results are valuable to protect radiologists when there is an issue requiring reporting to the local State Board of Registration in Medicine (Table 1). Also,

Discussion

At most institutions, peer review is commonly used for assessing radiologist performance in terms of medical and clinical knowledge and for remediating any deficiencies that are detected. In our study, however, 44% of respondents agreed with the statement that peer review, as done in our institution using a system similar to RADPEER, is a waste of time; 58% agreed it was performed merely to meet hospital/regulatory requirements; and 46% participate in peer review only because they are forced to

Take-Home Points

•
Almost half of radiologists agreed that peer review improves radiologist performance and is valuable to protect radiologists in cases referred to the Medical Board, while almost one-third agreed that it decreases medical error.
•
Conversely, almost half agreed that our RADPEER-like system is a waste of time, and a majority believe it is done merely to meet hospital and regulatory requirements.
•
The method for selecting cases raises serious questions regarding resulting selection bias.
•
A new approach

References (8)

L.F. Donnelly
Performance-based assessment of radiology practitioners: promoting improvement in accordance with the 2007 Joint ommission standard
J Am Coll Radiol
(2007)
J.K.T. Lee
Quality—a radiology imperative: interpretation accuracy and pertinence
J Am Coll Radiol
(2007)
M.J. Halsted
Radiology peer review as an opportunity to reduce errors and improve patient care
J Am Coll Radiol
(2004)
G.J. Butler et al.
The next level of radiology peer review: enterprise-wide education and improvement
J Am Coll Radiol
(2013)

There are more references available in the full text version of this article.

Cited by (56)

Peer learning is both preferable and less expensive than score-based peer review: Initial experience at a tertiary academic center
2024, Clinical Imaging
To examine radiologist experiences and perceptions during a transition from score-based peer review to a peer learning program, and to assess differences in time-cost efficiency between the two models of quality improvement.
Differences in Likert scale survey responses from radiologists (N = 27) in a multispecialty group at a single tertiary academic center before and following intervention were evaluated by Mann-Whitney U test. Multiple variable linear regression analysis assessed independent variables and program preference.
All positive impacts rated significantly higher for the peer learning program. Workflow disruption for the peer learning program rated significantly lower. 70.4 % (19 of 27) preferred the new program, and 25.9 % (7 of 27) preferred the old program. Only the “worth investment” questionnaire score demonstrated a significant correlation to program preference and with an effect that was greatest among all variables (Beta = 1.11, p = 0.02). There was a significantly decreased amount of time per month used to complete peer learning exercises (0.76 ± 0.45 h, N = 27) versus peer review exercises (1.71 ± 1.84 h, N = 34, p = 0.011). The result was a difference of 0.95 ± 1.89 h/month (11.4 ± 22.7 h/year), translating to an estimated direct salary time-cost saving of $1653.68/year/radiologists and a direct productivity time-cost saving of $3469.39/year/radiologist when utilizing the peer learning program.
There was a strongly positive perception of the new peer learning program. There was a substantial implied direct time-cost saving from the transition to the peer learning program.
The peer learning model emphasizes learning from errors via feedback in a non-punitive environment. This model was positively perceived and demonstrated substantial implied direct time-cost saving.
Event-Based Learning and Improvement: Radiology's Move From Peer Review to Peer Learning
2024, Seminars in Ultrasound, CT and MRI
Over the past 15 years, the radiology community has made great progress moving from a system of score-based peer review to one of peer learning. Much has been learned along the way. In peer learning, cases in which learning opportunities are identified are reviewed solely for the purpose of fostering learning and improvement. This article defines peer learning and peer review and emphasizes the difference; looks back at the 20-year history of score-based peer review and transition to peer learning; outlines the problems with score-based peer review and the key elements of peer learning; discusses the current state of peer learning; and outlines future challenges and opportunities.
The Joint Commission's Ongoing Professional Practice Evaluation Process: Costly, Ineffective, and Potentially Harmful to Safety Culture
2024, Journal of the American College of Radiology
To evaluate the estimated labor costs and effectiveness of Ongoing Professional Practice Evaluation (OPPE) processes at identifying outlier performers in a large sample of providers across multiple health care systems and to extrapolate costs and effectiveness nationally.
Six hospital systems partnered to evaluate their labor expenses related to conducting OPPE. Estimates for mean labor hours and wages were created for the following: data analysts, medical staff office professionals, department physician leaders, and administrative assistants. The total number of outlier performers who were identified by OPPE metrics alone and that resulted in lack of renewal, limitation, or revoking of hospital privileges during the past annual OPPE cycle (2022) was recorded. National costs of OPPE were extrapolated. Literature review of the effect of OPPE on safety culture in radiology was performed.
The evaluated systems had 12,854 privileged providers evaluated by OPPE. The total estimated annual recurring labor cost per provider was $50.20. Zero of 12,854 providers evaluated were identified as outlier performers solely through the OPPE process. The total estimated annual recurring cost of administering OPPE nationally was $78.54 million. In radiology over the past 15 years, the use of error rates based on score-based peer review as an OPPE metric has been perceived as punitive and had an adverse effect on safety culture.
OPPE is expensive to administer, inefficient at identifying outlier performers, diverts human resources away from potentially more effective improvement work, and has been associated with an adverse impact on safety culture in radiology.
Interventional Radiology Peer Learning Platform and Adverse Event Reporting (IR-PEER): Initial Experience Implementing a Team-based Novel Peer Learning System in Interventional Radiology
2024, Journal of the American College of Radiology
Although the transition from peer review to peer learning has had favorable outcomes in diagnostic radiology, experience with implementing a team-based peer review system in interventional radiology (IR) remains limited. Peer learning systems benefit diverse IR teams composed of multiple clinical roles and could contribute value in archiving events that have potential educational value. With multiple stakeholder input from clinical roles within the IR division at our institution (ie, radiologic technologists, nurses, advanced practice providers, residents, fellows, and attending physicians), we launched a HIPAA-compliant secure IR complication and learning opportunity reporting platform in April 2022. Case submissions were monitored over the subsequent 24 weeks, with monthly dashboard reports provided to departmental leadership. Preintervention and postintervention surveys were used to assess the impact of the peer learning platform and adverse event reporting in IR (IR-PEER) on perceptions of complication reporting in the IR division across clinical roles. Ninety-two peer learning submissions were collected for a weekly average ± standard error of 3.8 ± 0.6 submissions per week, and an additional 26 submissions were collected as part of the division’s ongoing monthly complication review conference, for a total of 98 unique total case references. A total of 64.1% of submissions (59 of 92) involved a complication and/or adverse event, and 35.9% of submissions (33 of 92) identified a learning opportunity (no complication or adverse event). Nurses reported that IR-PEER made the complication-reporting process easier (P = .01), and all clinical roles reported that IR-PEER improved the overall process of complication reporting. Peer learning frameworks such as IR-PEER provide a more equitable communication platform for multidisciplinary teams to capture and archive learning opportunities that support quality and safety improvement efforts.
Current State of Peer Learning in Radiology: A Survey of ACR Members
2023, Journal of the American College of Radiology
Peer learning (PL) programs seek to improve upon the limitations of score-based peer review and incorporate modern approaches to improve patient care. The aim of this study was to further understand the landscape of PL among members of the ACR in the first quarter of 2022.
Members of the ACR were surveyed to evaluate the incidence, current practices, perceptions, and outcomes of PL in radiology practice. The survey was administered via e-mail to 20,850 ACR members. The demographic and practice characteristics of the 1,153 respondents (6%) were similar to those of the ACR radiologist membership and correspond to a normal distribution of the population of radiologists and can therefore be described as representative of that population. Therefore, the error range for the results from this survey is ±2.9% at a 95% confidence level.
Among the total sample, 610 respondents (53%) currently use PL, and 334 (29%) do not. Users of PL are younger (mode age ranges, 45-54 years for users and 55-64 years for nonusers; P < .01), more likely to be female (29% vs 23%, P < .05), and more likely to practice in urban settings (52% vs 40%, P = .0002). Users of PL feel that it supports an improved culture of safety and wellness (543 of 610 [89%]) and fosters continuous improvement initiatives (523 of 610 [86%]). Users of PL are more likely than nonusers to identify learning opportunities from routine clinical practice (83% vs 50%, P < .00001), engage in programming inclusive of more team members, and implement more practice improvement projects (P < .00001). PL users’ net promoter score of 65% strongly suggests that users of PL are highly likely to recommend the program to colleagues.
Radiologists across a breadth of radiology practices are engaged in PL activities, which are perceived to align with emerging principles of improving health care and enhance culture, quality, and engagement.
Transition From Peer Review to Peer Learning: Lessons Learned
2023, Current Problems in Diagnostic Radiology
Landmark publications, such as To Err is Human, confronted the healthcare community with the egregious toll medical errors played in both patient safety and overall healthcare costs. This heralded a paradigm shift and a call for action by professional organizations to enact methods to ensure physician competency and quality assurance. The American College of Radiology similarly convened a task force to discuss these concerns and how best to address quality assurance in radiology practice, leading to the development of RADPEER, a score-based peer review system. However, critics were quick to point out the deficiencies of this model, highlighting it as punitive and a poor evaluator of physician performance. The recognized deficiencies in score-based peer review prompted the pursuit of an alternate model that would instead emphasize learning and improvement. Peer learning was proposed and highlighted the necessity of an inclusive and collaborative environment where colleagues could discuss case errors as learning opportunities without fear of punitive consequence. This paper explores peer learning, its benefits and challenges, as well as how to identify specific learning opportunities by utilizing case examples.

View all citing articles on Scopus

View full text

Original articleSurvey of Faculty Perceptions Regarding a Peer Review System

Purpose

Methods

Results

Discussion

Introduction

Section snippets

Methods

Results

Discussion

Take-Home Points

J Am Coll Radiol

J Am Coll Radiol

J Am Coll Radiol

J Am Coll Radiol

Original article
Survey of Faculty Perceptions Regarding a Peer Review System