The psychometric properties of Observer OPTION5, an observer measure of shared decision making

doi:10.1016/j.pec.2015.04.010

Patient Education and Counseling

Volume 98, Issue 8, August 2015, Pages 970-976

https://doi.org/10.1016/j.pec.2015.04.010 Get rights and content

Highlights

•
We assessed the psychometric properties of a 5-item measure of SDM, Observer OPTION⁵.
•
OPTION⁵ was used to rate clinical encounters from two patient decision aid trials.
•
OPTION⁵ produced valid and reliable scores in this sample.
•
Raters reported lower cognitive burden when using OPTION⁵ compared to OPTION¹².
•
OPTION⁵ is a brief, psychometrically sound observer measure of SDM.

Abstract

Objectives

Observer OPTION⁵ was designed as a more efficient version of OPTION¹², the most commonly used measure of shared decision making (SDM). The current paper assesses the psychometric properties of OPTION⁵.

Methods

Two raters used OPTION⁵ to rate recordings of clinical encounters from two previous patient decision aid (PDA) trials (n = 201; n = 110). A subsample was re-rated two weeks later. We assessed discriminative validity, inter-rater reliability, intra-rater reliability, and concurrent validity.

Results

OPTION⁵ demonstrated discriminative validity, with increases in SDM between usual care and PDA arms. OPTION⁵ also demonstrated concurrent validity with OPTION¹², r = 0.61 (95%CI 0.54, 0.68) and intra-rater reliability, r = 0.93 (0.83, 0.97). The mean difference in rater score was 8.89 (95% Credibility Interval, 7.5, 10.3), with intraclass correlation (ICC) of 0.67 (95% Credibility Interval, 0.51, 0.91) for the accuracy of rater scores and 0.70 (95% Credibility Interval, 0.56, 0.94) for the consistency of rater scores across encounters, indicating good inter-rater reliability. Raters reported lower cognitive burden when using OPTION⁵ compared to OPTION¹².

Conclusions

OPTION⁵ is a brief, theoretically grounded observer measure of SDM with promising psychometric properties in this sample and low burden on raters.

Practice implications

OPTION⁵ has potential to provide reliable, valid assessment of SDM in clinical encounters.

Introduction

Brief patient reported measures are the most scalable means of assessing the level of shared decision making (SDM) in routine clinical encounters. However, patient reports can be subject to halo, leniency and gratitude biases, which tend to provide unvarying high scores (ceiling effects) that make it difficult to discriminate between high and low achievement of SDM [1]. Observer (direct) measures, where trained assessors directly observe behavior via, for example, recorded (audio, video, transcript) encounters, are theoretically less prone to these effects, and can be triangulated with patient reported measures to produce more comprehensive assessment [2].

There are several observer measures of SDM, ranging 7–70 items [3], [4], [5], [6], [7], [8], [9], with variable levels of reliability and largely unexamined validity [10]. Observer OPTION¹² (OPTION¹²) [6] is the most commonly-used observer measure. OPTION¹² is a one-dimensional scale, consisting of 12 items, which focuses specifically on clinician behavior. While discriminative validity of OPTION¹² is satisfactory, inter-rater reliability has been mixed [11]. In addition, OPTION¹² has also been criticized for it lack of focus on the elicitation of patient preference. Some specified behaviors are rarely observed (Item 2, Item 3 and Item 10). In addition, assessor burden is high due to the high number of items, which may not occur in a linear fashion during a medical encounter. In hindsight, Elwyn et al. [12] believe OPTION¹² contained some items depicting an idealized form of SDM that is unrealistic in the real world, and missed items that address other core aspects, such as dealing with patient preferences [13].

Observer OPTION⁵ was developed as a five-item measure to address issues with OPTION¹² (http://www.optioninstrument.org/) with fewer items and a focus on the assessment of patient preferences [12]. OPTION⁵ is based on Collaborative Deliberation, a conceptual model describing the process of patients considering alternative health care options, in collaboration with clinicians and others. While existing theories of health care communication focus on decision making of a single individual, the model of Collaborative Deliberation was an attempt to develop a model that considers a collaborative effort. The model consists of five core dimensions: (1) constructive interpersonal engagement, (2) recognition of alternative actions, (3) comparative learning, (4) preference construction and elicitation, and (5) preference integration [14]. The goal was to provide a more efficient measure, that focused more on the core components of SDM while retaining good psychometric properties.

This study aimed to assess the psychometric qualities of OPTION⁵ using video and audio recorded clinical encounters from two trials, the Osteoporosis Choice Randomized Trials [15] and the Chest Pain Choice Trial [16].

Section snippets

Study design

Data was analyzed from two trials assessing the impact of patient decision aids (PDAs) used during the medical encounter: the Osteoporosis Choice Randomized trial [15] and the Chest Pain Choice trial [16]. In the Osteoporosis Choice trial, postmenopausal women in primary care aged 50 years or older, at risk for osteoporotic fractures, and eligible for bisphosphonate therapy were randomized to an intervention or usual care arm. The World Health Organization's Fracture Risk Assessment Tool (FRAX^®

Results

Across the randomized trials, 151 patients received PDA interventions and 160 received usual care. Participant characteristics are outlined in Table 1.

Discussion

In a sample of audio and video data from clinical encounters in two randomized trials, OPTION⁵ produced valid and reliable estimates of SDM. OPTION⁵ demonstrated discriminative validity, concurrent validity with OPTION¹², intra-rater reliability and promising inter-rater reliability at the item level. Raters welcomed the improved differentiation between target behaviors and the reduced cognitive burden of the tool compared to the 12-item version.

We acknowledge several study limitations. New

Conflicts of interest

No external support was received for this work. Glyn Elwyn has received funding from the Informed Medical Decisions Foundation, Boston, MA, USA, and provides ad hoc consulting to Emmi Solutions, Chicago, IL, USA. MRG was supported by CTSA Grant Number TL1 TR000137 from the National Center for Advancing Translational Science (NCATS). This manuscript's contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH. No other authors have

Acknowledgments

We would like to acknowledge Dr. Annie Le Blanc and Dr. Megan Branda for their comments on the study design, and Angela Sivly for her work as an OPTION⁵ rater.

References (26)

P. Butow et al.
Shared decision making coding systems: how do they compare in the oncology context?
Patient Educ Couns
(2010)
P. Guimond et al.
Validation of a tool to assess health practitioners’ decision support and communication skills
Patient Educ Couns
(2003)
D. Stacey et al.
Audit and feedback using the brief Decision Support Analysis Tool (DSAT-10) to evaluate nurse-standardized patient encounters
Patient Educ Couns
(2008)
I. Scholl et al.
Measurement of shared decision making: a review of instruments
Z Evid Fortbild Qual Gesundhwes
(2011)
G. Elwyn et al.
Using a talk model of shared decision making to propose an observation-based measure: observer OPTION 5 Item
Patient Educ Couns
(2013)
R.E. Glasgow et al.
Pragmatic measures: what they are and why we need them
Am J Prev Med
(2013)
G. Elwyn et al.
Collaborative deliberation: a model for patient care
Patient Educ Couns
(2014)
V.M. Montori et al.
Use of a decision aid to improve treatment decisions in osteoporosis: the osteoporosis choice randomized trial
Am J Med
(2011)
P.A. Harris et al.
Research electronic data capture (REDCap) – a metadata-driven methodology and workflow process for providing translational research informatics support
J Biomed Inform
(2009)
L.E. Eberly et al.
Estimating Bayesian credible intervals
J Stat Plan Inference
(2003)

A.R. Feinstein et al.

High agreement but low Kappa: I. The problems of two paradoxes

J Clin Epidemiol

(1990)

D.L. Streiner et al.

Health measurement scales: a practical guide to their development and use

(2003)

J. McCroskey et al.

Self report as an approach to measuring communication competence

Commun Res Rep

(1988)

Cited by (102)

Current level of shared decision-making in hepatobiliary surgical oncology (SAPACHA)
2024, HPB
Shared decision-making (SDM) may improve patient autonomy and health outcomes. This study assessed the level of SDM at both ends of the spectrum of hepatobiliary surgery to identify needs and opportunities for improvement.
A mixed-methods study was performed. Consultations regarding surgery for perihilar cholangiocarcinoma (pCCA) or hepatocellular adenoma (HCA) were prospectively included between September 2020 and December 2022. The level of patient involvement in treatment decision-making was assessed objectively by analysis of audio-recorded consultations using the OPTION-5 instrument. The perceived level of SDM was appreciated by patients (SDM-Q-9) and surgeons (SDM-Q-doc) through questionnaires. Higher scores indicated higher levels of SDM. Outcomes were compared between patient groups and two focus groups were held.
Ten pCCA- and nine HCA-patients were included in the quantitative part of the study. Median OPTION-5, SDM-Q-9 and SDM-Q-doc scores were 35% (IQR: 25–45%), 86% (IQR: 76–96%), and 73% (IQR: 71–78%), respectively. SDM-Q-9 scores among HCA-patients (79% [IQR: 71–82%]) were significantly lower than in pCCA-patients (96% [IQR: 93–100%], p < 0.001). In focus groups, patients reported a lack of information, support, and expressed positive attitudes towards decision support tools (DSTs).
Patient involvement and information provision among HPB-surgical patients show room for improvement, particularly for HCA-patients. DSTs may be helpful for this purpose.
Shared decision making between community therapists and Latinx caregivers during evidence-based practice delivery in publicly-funded children's mental health services
2023, Patient Education and Counseling
This observational study examined shared decision-making (SDM) with caregivers of Latinx youth within the delivery of multiple evidence-based practices (EBPs) in community mental health services. Study aims were to (1) Characterize therapist use of SDM strategies and (2) Describe the types of treatment decisions that were the focus of therapist use of SDM.
The OPTION instrument was used to measure SDM in 210 audio-recorded therapy sessions with 62 community therapists and 109 Latinx caregivers; frequency and mean ratings of OPTION items were examined. Qualitative analysis on the descriptions of treatment decisions being deliberated was also conducted.
Results revealed that therapists used at least one SDM step in most sessions (N = 192; 91.43%) with a mean composite score of 32.78 (SD=17.79; range: 6.25–81.25). Four superordinate categories of decisions were: (1) Treatment planning, (2) Evidence-based Parenting Strategies, (3) Addressing Youth Functioning, and (4) Addressing Family Psychosocial Needs.
Findings suggest that community therapists serving Latinx families are naturalistically engaging in SDM steps about a variety of decisions during most EBP sessions, but only at modest levels.
Evaluation of the shared decision-making process scale in cancer screening and medication decisions
2023, Patient Education and Counseling
Examine reliability and validity of the Shared Decision-Making (SDM) Process scale for cancer screening and medication decisions.
Secondary data analysis from 6174 participants who made decisions about cancer screening (breast, colon or prostate) or medication (menopause, depression, hypertension or high cholesterol). Key measures included the SDM Process scale, decisional conflict, decision regret, and decision quality. Construct validity was examined by testing whether higher SDM Process scores were associated with lower regret, lower decisional conflict and higher decision quality. Meta-analyses summarized data across studies. Some studies assessed the scale’s reliability.
Average SDM Process scores ranged from 1.2 to 2.5. There was a moderate-to-large, positive association between scores and lack of decisional conflict (cancer screening: d=0.61, CI(0.38, 0.84), p < .001; medications: d=0.36, CI(0.29, 0.44), p < .001). High scores were associated with lower decision regret (cancer screening: d=−0.24, CI(−0.37, −0.11), p < .001; medications: d=−0.30, CI(−0.40,−0.20), p < .001). There was no relationship with decision quality. Retest reliability was acceptable (ICC>0.7) for seven of eight clinical samples.
The SDM Process scale demonstrated construct validity and retest reliability in cancer screening and medication decisions.
The validated SDM Process scale is a short, patient reported metric to evaluate the current state of SDM.
Shared decision making: A novel approach to personalized treatment in obstructive sleep apnea
2022, Sleep Medicine: X
Citation Excerpt :
To the best of our knowledge, our study is the first using video recording to describe phenotyping by SDM between doctors and patients with newly diagnosed OSA. A high degree of SDM was found both when assessed by self-report and by rater based scoring compared to studies of non-OSA populations [42,43]. A study completed in German primary care practices regarding type 2 diabetes, chronic back pain, depression and other diseases, found an “Observer OPTION [5]” mean total score of 11.843.
i) To describe a novel approach of phenotyping by shared decision making (SDM) in obstructive sleep apnea (OSA) discharge consultations ii) to describe correlation between patient and observer based evaluations of SDM and iii) to describe treatment adherence.
Consecutive patients referred to the otorhinolaryngology department at Akershus University Hospital with suspected OSA between 2015 and 2016 participated. Patients with body mass index >30 were oversampled. Four male communication-trained doctors aged from 30 to 60 years participated. SDM was evaluated by modified content analysis and by the CollaboRATE self-report questionnaire and the “Observer OPTION (Young et al., 2008) [5]” rating scale. Positive airway pressure (PAP) treatment adherence and weight reduction was assessed by interview at six year follow-up.
Eighteen consultations were video filmed. The content analysis revealed that the patient perspectives only briefly were explored. PAP was chosen by 17 of 18 patients. Median CollaboRATE questionnaire score was 29 (26, 30). Mean OPTION (Young et al., 2008) [5] score was 65.6 (SD 6.6, range 55–80). The correlation between SDM assessed by CollaboRATE self-report and by the “Observer OPTION (Young et al., 2008) [5]” rating scale was low (Pearson's r = 0.09). At follow up, 11 patients (64.7%) were PAP adherent and no one achieved 10% weight loss.
Despite a high degree of SDM compared to studies of non-OSA populations, the sub-optimal exploration of the patient perspective by communication-trained doctors precluded identification of patients willing to cope actively. SDM assessed by self-report and by a rating scale may represent two different constructs. PAP adherence was good.
Effectiveness of interprofessional shared decision-making training: A mixed-method study
2022, Patient Education and Counseling
This study evaluated the learning effects and examined the participants’ perceptions of an interprofessional shared decision-making (IP-SDM) training program.
This mixed-method study used a quasi-experimental pretest–posttest design in the quantitative phase and semi-structured interviews in the qualitative phase. The 6-week curriculum design, based on Kolb’s experiential learning cycle, consisted of two simulated objective structured clinical examinations with standardized patients and blended teaching methods through various course modules.
A total of 39 multidisciplinary healthcare personnel completed the 6-week training program, and 32 of them participated in qualitative interviews. The IP-SDM training program effectively improved the SDM process competency of the participants from the perspectives of the participants, standardized patients, and clinical teachers. The interviews illustrated how the curriculum design enhanced learning; the effectiveness results indicated improvements in learners’ attitude, knowledge, skills, and teamwork.
This IP-SDM training program improved multidisciplinary healthcare personnel’s competency, self-efficacy, and intention to engage in IP-SDM.
Applying Kolb's experiential learning cycle and blended teaching methods to develop and implement the IP-SDM training program can improve multidisciplinary healthcare personnel’s knowledge, attitude, skills, and teamwork in IP-SDM.
Bringing personal perspective elicitation to the heart of shared decision-making: A scoping review
2022, Patient Education and Counseling
Proponents of shared decision-making (SDM) advocate the elicitation of the patient’s perspective. This scoping review explores if, and to what extent, the personal perspectives of patients are elicited during a clinical encounter, as part of a SDM process. We define personal perspective elicitation (PPE) as: the disclosure (either elicited by the clinician or spontaneously expressed by the patient) of information related to the patient’s personal preferences, values and/or context.
A search was conducted in five literature databases from inception dates up to July 2020, to identify empirical studies about SDM (with/without SDM instrument).
The search identified 4562 abstracts; 263 articles were read in full text, resulting in 99 included studies. Studies reported low levels of PPE. Integration of personal perspectives into the conversation or a future care plan was largely absent. The majority of the discussed content related to physical health, while social and psychological topics were mostly unaddressed.
PPE occurs on a very low level in efforts to achieve SDM according to evaluation studies.
PPE is advocated but rarely achieved in SDM evaluation studies. Causes should be identified, followed by designing interventions to improve this aspect of SDM.

View all citing articles on Scopus

View full text

Medical Decision MakingThe psychometric properties of Observer OPTION5, an observer measure of shared decision making

Highlights

Abstract

Objectives

Methods

Results

Conclusions

Practice implications

Introduction

Section snippets

Study design

Results

Discussion

Conflicts of interest

Acknowledgments

Patient Educ Couns

Patient Educ Couns

Patient Educ Couns

Z Evid Fortbild Qual Gesundhwes

Patient Educ Couns

Am J Prev Med

Patient Educ Couns

Am J Med

J Biomed Inform

J Stat Plan Inference

J Clin Epidemiol

Health measurement scales: a practical guide to their development and use

Self report as an approach to measuring communication competence

Commun Res Rep

Medical Decision Making
The psychometric properties of Observer OPTION⁵, an observer measure of shared decision making