Methodological approaches for assessing response shift in longitudinal health-related quality-of-life research
Introduction
Research on social factors and health relies heavily on the measurement of perceived quality of life (QOL). It is founded, however, on assumptions about the stability of intra-individual standards which may not be valid. As people can vary, so can their values. This variability may reflect informative shifts in an individual's internal standards, in values and priorities, or in the conceptualization of perceived QOL, in addition to changes in actual health state. This `response shift' phenomenon is of fundamental importance to social and medical science.
As an illustration, the results of a psychosocial randomized trial (Schwartz, in press) will be described, the objective of which was to compare the effectiveness of a coping skills group intervention as compared to a peer telephone support intervention for people with multiple sclerosis. These results motivated a quest for methods which measure response shift. The group intervention consisted of a comprehensive package which included cognitive, behavioral and supportive techniques aimed at helping patients to accommodate to their progressive disease trajectory. The peer telephone support intervention involved less intensive contact, no in-person meetings and was non-directive. Given the chronic and non-fatal nature of this autoimmune disease, patients had a prognosis for a normal lifespan with periods of relapses and progressive disability (Lechtenberg, 1988).
Two-year follow-up data suggested that despite significant deterioration in neuropsychological performance, neurological disability and self-efficacy function, participants in the group intervention reported reduced psychosocial role limitations, decreased use of negative coping strategies and enhanced well-being. In contrast, peer support recipients did not report significant changes in psychosocial role performance, coping, or well-being. Thus, although both groups exhibited significant deterioration in clinical measures of function, the well-known disparity between patient-reported and clinical measures of function was largest in the coping skills group.
The apparent separation of physical functioning and psychological well-being may represent `response shift phenomenon.' Our working definition of response shift refers to a change in the meaning of one's self-evaluation of a target construct as a result of: (a) a change in the respondent's internal standards of measurement (i.e. scale recalibration); (b) a change in the respondent's values (i.e. the importance of component domains constituting the target construct) or (c) a redefinition of the target construct (i.e. reconceptualization). Indeed, qualitative interviews with participants revealed that they felt that meeting people in the group who were worse or better off than they were led them to change their internal standards of how badly off they themselves were. Additionally, they felt that the group intervention helped them to reconsider the goals that were important and feasible to them (i.e. changes in values), and to learn that it was possible to have a reasonable QOL even with a worse condition (i.e. reconceptualization of QOL). This example illustrates how a psychosocial intervention might teach response shift.
The concept of response shift has its foundation in research on educational training interventions (Howard et al., 1979a, Howard et al., 1979b, Howard et al., 1979c, Howard et al., 1981) and organizational change (Golembiewski et al., 1976). Whereas Howard and colleagues defined response shift in terms of changes in internal standards of measurement, Golembiewski et al. introduced the component of changes in conceptualization and internal standards. Investigators from the discipline of management sciences described a typology of change which distinguished alpha, beta and gamma shifts, referring to objective changes, change in internal standards and reconceptualizations, respectively (Armenakis, 1988). The working definition adds changes in values as another component that is relevant to a change in the meaning of one's self-evaluation. While changes in values were inherent in the Golembiewski description of reconceptualization, the working definition adopted in this paper includes this as a separate third component that is relevant to the change in the meaning of one's self-evaluation. Making it a distinct third aspect will thus highlight its importance and emphasizes the need to measure it carefully (Sprangers and Schwartz, 1999). Finally, this working definition avoids the use of the terms `beta' and `gamma' shift since these terms do not reveal their content. It would be preferable to use the more transparent terms of change in internal standards, change in value and reconceptualization.
Response shift is important to consider in treatment evaluations, especially insofar as it may serve to attenuate or to exaggerate estimates of treatment effects as patients adapt to treatment toxicities or disease progression over time. Methods which assess response shift will not only be necessary to reveal unbiased treatment effects, but will also be useful for examining the impact of illness over time. The purpose of the present work is to discuss existing methods or those which can be developed to address the different components of response shift. As will be noted below, these approaches operationalize more than one aspect of response shift, a measurement challenge which may reflect an inherent inter-connection between internal standards, values and conceptualization. This inter-connection will be discussed in greater depth after the methods have been presented. Our goal is to encourage and facilitate research on this construct.
Section snippets
Methods
Several of the methods which will be described are protocols we are suggesting for adapting existing tools so that they can assess one or more of the three aspects of response shift. Others are new methods developed by the authors and for which empirical substantiation is currently underway. Finally, we also describe approaches which would influence the design of new studies or the analysis of existing data sets. Including outcome tools, design approaches and statistical methods in the
Discussion
We have discussed a number of methods which could be adapted or further developed to investigate response shift. The more established methods (e.g. those originating from educational or organizational change) have been discussed more critically than the relatively new methods because empirical evidence is available on the former to highlight their drawbacks. Consequently, the discussion of the respective methods has been necessarily unbalanced. Because very few methods have been designed
Acknowledgements
We would like to acknowledge the invaluable contribution of the following social and medical scientists who participated in the Response Shift Workshop which was held in Boston, December 1996 and funded by Agency for Health Care Policy and Research (1 RO1 HSO8582-01A1) to CES and by a matched contribution from Frontier Science. These participants included: Achilles Armenakis, Ph.D.; Katy Benjamin, SM, MSW; Lawren Daltroy, Dr.PH; Susan Folkman, Ph.D.; Rick Gibbons, Ph.D.; Maureen Wilson
References (88)
- et al.
Quality of life: a dynamic construct
Social Science and Medicine
(1997) - et al.
Questioning patient satisfaction: an empirical investigation in two outpatient clinics
Social Science and Medicine
(1997) - et al.
Individual quality of life in patients undergoing hip replacement
The Lancet
(1992) Response shift bias: a challenge to the assessment of patients' quality of life in cancer clinical trials
Cancer Treatment Reviews
(1996)Measurement of health state utilities for economic appraisal
J Health Econ
(1986)- et al.
Quality of life in hypertensives and normotensives in Israel: introducing a self-structured subjective measure
A review of research on the change typology
Research in Organizational Change and Development
(1988)- et al.
An algorithm for assessing factor structure congruence
Educational and Psychological Measurement
(1977) - et al.
A comparison of two methods for evaluating the similarity of factor analysis solutions
- et al.
Interpreting the measurement of change in organizational research
Personnel Psychology
(1979)
Journal of Rheumatology
Re-examining quality of life of hypertensives: a new self-structured measure
American Journal of Hypertension
The significance of congruence coefficients: a comment and statistical test
Journal of Management
Methodological considerations in the evaluation of a teacher-training program
Journal of Educational Psychology
Application of hierarchical linear models to assessing change
Psych. Bull.
Individual quality of life in the healthy elderly
Quality of Life Research
Detecting scale recalibration in survey research: a laboratory investigation
Group and Organization Studies
Quality of life in cancer patients: an hypothesis
Journal of Medical Ethics
The Pattern of Human Concerns
The measurement of patient satisfaction
Journal of Public Health Medicine
Experimental Designs
On the foundations of hysteresis in economic systems
Economics and Philosophy
Fates worse than death: the role of valued life activities in health-state evaluations
Health Psychology
The method of paired comparisons
Estimation of time response curves and their confidence bands
Biometrics
Analysis of change: modeling individual growth
A threshold estimation problem for processes with hysteresis
Statistics and Probability Letters
Measuring change and persistence in human affairs: types of change generated by OLD designs
J Applied Behavioral Science
An experimental approach to evaluate methods to detect scale recalibration
Educational and Psychological Measurement
The retrospective pretest in an educational training context
Journal of Experimental Education
Influence of objective measures on self-reports in a retrospective pretest–posttest design
Journal of Experimental Education
Response shift bias: a source of contamination of self-report measures
Journal of Applied Psychology
Internal invalidity in pretest–posttest self-report evaluations and a re-evaluation of retrospective pretests
Applied Psychological Measurement
The feasibility of informed pretests in attenuating response-shift bias
Applied Psychological Measurement
Influence of subject response style effects on retrospective measures
Applied Psychological Measurement
Internal invalidity in studies employing self-report instruments. A suggested remedy
Journal of Educational Measurement
Individual quality of life and clinical trials
Quality of Life Newsletter
LISREL V: Analysis of Linear Structural Relationships by Maximum Likelihood and Least Square Methods
Prospect theory: an analysis of decision under risk
Econometrica
The distribution of chance congruence coefficients from simulated data
Psychometrika
Random-effects models for longitudinal data
Biometrics
Multiple Sclerosis Fact Book
Retrospective versus actual pre-course self-assessments
Evaluation Health Profession
Cited by (650)
How did the COVID-19 pandemic influence students’ career expectations in the hotel and tourism industry?
2023, Journal of Hospitality, Leisure, Sport and Tourism EducationExpanding and explaining symptoms in knee osteoarthritis trajectories: fluctuations, flares, and future directions
2023, Osteoarthritis and CartilageResponse shift results of quantitative research using patient-reported outcome measures: a descriptive systematic review
2024, Quality of Life Research