Article Text

PDF

Use of discrete choice experiments to elicit preferences
  1. M Ryan, MRC senior research fellow and reader in health economics1,
  2. A Bate, research assistant1,
  3. C J Eastmond, consultant rheumatologist2,
  4. A Ludbrook, senior research fellow1
  1. Health Economics Research Unit, University of Aberdeen, Foresterhill, Aberdeen AB25 2ZD, UK
  2. Department of Rheumatology, University of Aberdeen
  1. Dr M Ryan m.ryan{at}abdn.ac.uk

Abstract

This paper considers the application of discrete choice experiments for eliciting preferences in the delivery of health care. Drawing upon the results from a recently completed systematic review, the paper summarises the application of this technique in health care. It then presents a case study applying the technique to rheumatology outpatient clinics. 200 patients were questioned about the importance of six attributes: staff seen (junior doctor or specialist nurse); time in waiting area; continuity of contact with same staff; provision of a phone-in/advice service; length of consultation; and change in pain levels. The systematic review indicated that discrete choice experiments have been applied to a wide number of areas and a number of methodological issues have been addressed. Consistent with this literature, the case study found evidence of both rationality and theoretical validity of responses. The approach was used to establish the relative importance of different attributes, how individuals trade between these attributes, and overall benefit scores for different clinic configurations. The value of attributes was estimated in terms of time, and this was converted to a monetary measure using the value of waiting time for public transport. Discrete choice experiments represent a potentially useful instrument for eliciting preferences. Future methodological work should explore issues related to the experimental design of the study, methods of data collection and analysis, and satisfaction with the economic axioms of the instrument. Collaborative work with psychologists and qualitative researchers will prove useful in this research agenda.

  • discrete choice experiments
  • patient preference
  • decision making
  • patient-caregiver communication

Statistics from Altmetric.com

Key messages

  • Discrete choice experiments are potentially useful for eliciting preferences in the delivery of health care.

  • Discrete choice experiments allow estimation of the relative importance of different aspects of care, the trade offs between these aspects, and the total satisfaction or benefit respondents derive from health care services. Monetary values of attributes may be indirectly estimated by including waiting time as an attribute.

  • The technique has the potential to address a number of common issues facing the NHS.

  • Future methodological work should explore issues related to the experimental design of the study, methods of design and analysis of the data, and the underlying economic axioms of the instrument.

  • Collaborative work with psychologists and qualitative researchers will prove useful in addressing the research agenda.

Recent years have seen an increased use of discrete choice experiments (DCEs; also known as conjoint analysis) as a technique for eliciting preferences. This paper considers what we know to date about the application of DCEs in health and identifies important areas for future research. The technique is described and its use in health economics is considered. The results from a recently completed systematic review of the technique are summarised and a case study from an outpatient rheumatology clinic is presented which demonstrates both the standard approach to conducting a DCE and its potential uses. Methodological questions that need to be addressed are discussed.

Discrete choice experiments

Discrete choice experiments are based on the premise that, firstly, any good or service can be described by its characteristics (or attributes) and, secondly, the extent to which an individual values a good or service depends upon the nature and levels of these characteristics. The technique involves presenting individuals with choices of scenarios described in terms of characteristics and associated levels. For each choice they are asked to choose their preferred scenario. Response data are modelled within a benefit (or satisfaction) function which provides information on whether or not the given characteristics are important; the relative importance of characteristics; the rate at which individuals are willing to trade between characteristics; and overall benefit scores for alternative scenarios.14

DISCRETE CHOICE EXPERIMENTS IN HEALTH CARE

Ryan et al5 have systematically reviewed the application of DCEs in health care. The technique was initially applied in an economic evaluation framework in an attempt to go beyond health outcomes and to take account of “non-health outcomes” and “process attributes” in the delivery of health care.4 Following this, the technique has been applied to address a wide range of issues including estimation of benefits within health technology assessments; analysis of patient/consumer and professional decision making; and developing prioritisation frameworks.5

The increased number of applications has been accompanied by investigation of methodological and theoretical issues. Few difficulties have been reported when answering DCEs and the technique has been well received by policy makers.5 Validity has been addressed at a number of levels. High levels of internal validity—that is, results consistent with a priori expectations—have been recorded and convergent validity—that is, results move in line with those of other instruments measuring the same construct—has been demonstrated with respect to standard gamble and willingness to pay.5 The technique has been shown to be relatively insensitive to both the ordering and levels of attributes.6 At the theoretical level three key axioms underlying the technique—completeness, stability, and rationality—have been investigated with encouraging results.79

Case study: preferences for a specialist nurse in the provision of rheumatology care

BACKGROUND

This application considers patient preferences for potential benefits from a change in the organisation of service delivery. The setting is a rheumatology outpatient clinic but the approach has general applications. The issue arose from difficulties in meeting the demand for outpatient services against a background of changes in junior doctors' hours and constrained resources. One possible solution was the introduction of a specialist nurse practitioner to hold review clinics for patients with stable rheumatoid arthritis.

The DCE approach was chosen to value benefits because it allows consideration of health outcomes (such as pain levels) and process attributes (such as waiting time and continuity of care). More traditional means of measuring benefits have concentrated on improvements in health outcomes using either hard clinical outcomes or Quality Adjusted Life Years.1011

SUBJECTS AND METHODS

The study population comprised patients attending the rheumatology outpatient clinics at Aberdeen Royal Infirmary. A researcher (AB) attended clinics to collect the data on various days of the week and at various times. Two hundred questionnaires were handed out and the researcher was available to answer questions. This number was sufficient to carry out statistical analysis on both the attributes and subgroups (see below).1213

The attributes for the study were determined by reviewing the literature on extending the role of the specialist nurse in rheumatology.10111415 These are shown in table 1 together with the levels assigned to them (and their coding for the regression analysis). The attributes chosen relate to both health outcomes and process attributes. Levels were chosen to reflect both the current situation with doctor led care and the likely situation if nurse led care was introduced.

Table 1

Attributes and levels

The attributes and levels chosen resulted in 192 (24 × 31 × 41) possible clinic configurations. The experimental design package speed was used to reduce these to a manageable number while still being able to infer benefit scores for all possible configurations.16 A main-effects linear model was assumed. This technique resulted in a design of 16 orthogonal scenarios (absence of multicollinearity between scenarios) which were converted into eight discrete choices and an attempt was made to maintain orthogonality in differences. Respondents were presented with these eight choices and, for each, asked whether they preferred clinic A or B (see table 2 for an example of a discrete choice).

Table 2

Example of choice question

From the response data the following equation was estimated:

ΔB = α1ΔStaff + α2ΔWait + α3ΔContinuity + α4ΔPain + α5ΔPhone + α6ΔLength + e where ΔB is the change in benefit in moving from clinic A to clinic B and the independent variables are the differences in the attributes of the two clinics, as defined in table 1. Response data were analysed in limdep17 using a random effects probit regression model (to take account of multiple observations from respondents). A general to specific approach was used. The general model included all the attributes and the specific model included only those significant at the 5% level. From the above equation the following were investigated:

  • the relative importance of the attributes (as indicated by the significance of the coefficients α and their size);

  • how individuals trade between these attributes—that is, the rate at which they give up one unit of an attribute for an increase in another. This is shown by the ratio of the coefficients—for example, α12 shows how much waiting time an individual is willing to trade to have their most preferred or least preferred member of staff;

  • benefit (satisfaction) scores for alternative ways of providing the services.

For more details on the underlying economics of the model the reader is referred to Louviere et al.1

Information was also collected on respondents' age, sex, length of time attending clinic and time since last appointment, current physical and mental health rating, time to complete the questionnaire, and ease of completion (on a scale from 1 to 5, where 1 = very easy and 5 = very difficult). For policy purposes it may be useful to know how preferences vary across individuals. Response data were analysed separately according to length of time attending the rheumatology clinic: first time attendees; 1 week–12 months; 13–36 months; 37–96 months, and 97+ months.

“Rationality” of responses was assessed by including two dominant options—that is, choices where one option was “better” than another on all levels. Respondents were expected to choose the “better” configurations; those who “failed” one test were assumed to have done so through random error whereas those who “failed” both tests were defined as “irrational” and were dropped from the regression analysis.

RESULTS

Of the 200 questionnaires distributed, 189 (94.5%) were returned. The characteristics of the respondents are shown in table 3. The higher prevalence of women in the study sample is not surprising since rheumatoid arthritis is more prevalent in women. Of the 189 respondents, 41 “failed” one rationality test and 15 “failed” both tests. Given that the questionnaire was distributed in a clinic setting (reducing the chance of the sample being biased through non-response by those not understanding the questionnaire as can be the case for postal methods), the high response rate, and the high proportion of older respondents, these rationality results are encouraging.

Table 3

Characteristics of respondents

Estimated coefficients and their standard errors for the estimated benefit equation are shown in table 4. The results from the specific model are discussed here. Given the coding in table 1, the positive signs indicate that patients prefer continuity in staffing, reduced pain levels, and the introduction of a phone-in/advice line service. The negative sign on “wait” indicates that patients prefer to have shorter waiting times. These results support the theoretical validity of the technique.

Table 4

Regression results from discrete choice experiment

Introducing a phone-in/advice line service increases benefit by 0.97 and having staff continuity increases benefit by 0.45. Although waiting time has the smallest coefficient, it must be noted that this attribute is measured in minutes. While a change in waiting time of 1 minute may not be as important as a marginal change in any of the other three attributes, assuming a linear utility function, the change in benefit resulting from a 30 minute change is equal to 0.477 (0.0159*30), which outweighs the benefit of a marginal change in both staff continuity and improvement in pain.

The value of individual attributes can be estimated in terms of the time respondents are willing to give up for that attribute. For example, respondents are willing to wait 61 minutes longer in the waiting area if a phone-in/advice line is available (0.974/0.0159) and 29 minutes longer in the waiting room to have staff continuity (0.454/0.0159). This implies that a phone-in service is twice as important as staff continuity (also indicated by the relative size of the coefficients). The monetary value patients would be willing to pay for improvements in these attributes could be estimated by assuming that waiting time in clinics has a similar value to waiting time for public transport, which has been estimated to be £0.15 per minute.18 This implies a marginal value of £9.15 per visit for the introduction of a phone-in service and £4.35 per visit for staff continuity.

Benefit scores and time values are presented in table 5 for different ways of providing the service. Example 1 illustrates the case when moving from the current situation in Aberdeen to a hypothetical clinic that also provides staff continuity and a phone-in/advice line. The benefit score is 1.4279, with an associated value of time of 90 minutes. A benefit score and time value greater than zero indicates that patients would benefit from this change. Thus, assuming other attributes remain constant, the introduction of these services would benefit patients. The second example highlights moving from a situation where staff continuity is provided (with small improvements in pain) to a drop in outpatient clinic where there is no waiting time and no staff continuity. This benefit score is negative (–0.1358) with an associated value of time of –8 minutes, indicating that the lower waiting time is not sufficient to compensate for the loss of staff continuity. These conclusions are echoed by the implied monetary value of the attributes. At a value of £0.15 per minute the implied value of the move in example 1 is £13.50 per visit and £–1.28 per visit in example 2, indicating that respondents would be willing to pay to avoid this move.

Table 5

Estimated benefit scores

Generally, a negative benefit score means that patients are worse off from the service change, a benefit score of zero would leave overall benefit unchanged, and a benefit score greater than zero implies an increase in benefit. Policy makers should attempt to reach the highest benefit score within the available budget.

Table 6 shows the results for the segmented model. All five groups valued staff continuity and the introduction of a phone-in/advice service. Waiting time was important to all groups other than first time attendees. This latter group also preferred to see a specialist nurse (as indicated by the positive sign on the coefficient). Pain was only important to the group who had been attending for 1–12 months. The results indicate that different groups may have different preference structures. Future work should investigate the reasons for these differences.

Table 6

pecific regression results from segmented model

Discrete choice experiments: research issues for the future

The case study illustrates how DCEs can be used to address policy relevant issues. Given the absence of a monetary attribute, willingness to pay was implied from the value of reducing waiting time for public transport. The evaluation of staff continuity and phone-in services are likely to be important in the coming years.1920 However, a number of issues are raised in the design and analysis of DCE studies of which the reader should be aware.

EXPERIMENTAL DESIGN

Experimental methods employed in the health economics literature have mainly been modifications of standard linear experimental designs. Such designs result in a number of orthogonal scenarios. For example, in the above case study an orthogonal fractional factorial design was created using speed.16 When using DCEs these scenarios must be placed into choice sets which are orthogonal in differences, have minimum overlap, and be balanced (occurring an equal number of times).2122 Pairing these choice sets in order to maintain these properties can be challenging. Various methods are currently adopted, including pairing the choices manually and checking the statistical properties; comparing each scenario with the same base comparator; and the use of computer software. Future work should examine the sensitivity of results to the experimental design employed.

DATA COLLECTION AND METHODS OF ANALYSIS

Once an efficient choice set has been devised, a number of methods exist for eliciting responses. While simple binary response models, analysed using random effects probit and logit models, have proved the most common approach, future studies should consider offering respondents more than two options. While more information is obtained by the researcher, issues are raised concerning the method of analysis. The standard approach to analysing such data is conditional probit or logit models. Although the latter is computationally easier, it is known to violate the assumption of independence compared with irrelevant alternatives assumption (IIA)—that is, the ratio of probabilities for any two alternatives is assumed to be independent of the attribute levels in the third alternative. Furthermore, multinomial logit models do not account for multiple observations from individuals. Revelt and Train23 have developed the random parameter logit model to take account of these limitations and future research should explore these modelling techniques with a DCE framework.24

ESTIMATION OF BENEFIT/UTILITY SCORES

For simplicity the model used in this study assumed an additive linear relationship between the choice of clinic and attribute levels. The additive assumption implies that there are no interaction terms between attributes—that is, the level at which one attribute is set does not effect preferences for another. The linear assumption implies that the effects of the attributes on choice do not change as the level of that attribute changes—that is, each additional unit change in waiting time has the same marginal effect on choice. Research from outside health economics has shown that alternatives to the linear additive model seldom result in a significantly better fit.45 Future work applying DCEs to health care should explore going beyond the linear additive model. Such modelling needs to be built into the experimental design of the study and requires data to be collected on a larger choice set.

Given that cost was not included as an attribute in this study, a benefit score was estimated for different ways of providing the service. Environmental economists have developed a similar scoring method termed the “attractiveness index”.2627 While this index is potentially useful at the policy level, questions have been raised concerning its properties—is it cardinal or ordinal? Future work should explore this.

This study also valued attributes in terms of waiting time. Users of health care may be more used to trading this attribute and future work should explore its use in estimating the value of alternative ways of providing a service. From such data it is possible to estimate the monetary value of time. The method adopted here assumed that the value of time when waiting for public transport is the same as waiting time in a healthcare setting. Such an assumption should be tested.

ESTIMATING WILLINGNESS TO PAY

Including cost as an attribute in DCEs allows willingness to pay (a monetary measure of benefits) to be indirectly estimated.4 The contingent valuation method, which has most commonly been used to elicit maximum willingness to pay,28 has been shown to be subject to a number of problems including lack of scope sensitivity,29 strategic biases,30 and warm glow.31 Future work should examine the extent to which DCEs overcome these limitations.3233

TESTING THEORETICAL AXIOMS

This study tested for rationality of responses by including dominance tests. A number of issues are raised here. Firstly, when considering responses, research should investigate the reasons for “irrational responses”. There is a growing literature from both psychology and economics indicating that apparently “irrational” responses can be rationally explained,83435 and future work should explore “irrational” responses in more detail. Qualitative research techniques will prove useful here.36 Secondly, it may be argued that conventional dominance test are “easy” to satisfy and that different results regarding rationality may be reached if different tests are employed. Alternative definitions of rationality should be explored. Possibilities include testing for transitivity and Sen's contraction (α) and expansion (β) properties. Such tests are arguably more demanding than dominance tests and are therefore more difficult to satisfy.79

DCEs also assume a compensatory decision making process—that is, when presented with choice sets, respondents consider all attributes. Tests of this axiom have been limited to examining whether individuals always make decisions according to the best level of a given attribute. However, such a decision making process may be simply a result of the attribute levels presented in the experiment. Alternative methods of examining the trading axiom should be examined. The psychology literature indicates that respondents often employ simplifying heuristics (decision making rules) in decision making, employing “fast and frugal heuristics”.37 Such decision making heuristics may also explain apparent irrationalities.38 Future work should explore the compensatory decision making assumption in detail, with consideration given to the relationship of cognitive strategies to the complexity of the choice sets presented.

Conclusions

This paper has considered the role of DCEs when eliciting preferences in the delivery of health care. While DCEs have been applied in a number of healthcare settings and potentially offer useful information to aid decision making, methodological issues should continue to be addressed. Important areas of future research relate to experimental design, alternative methods of data collection and analysis, and investigation of the underlying axioms of economic theory. Collaborative work with psychologists and qualitative researchers will prove useful when investigating these issues.

Acknowledgments

The authors would like to thank the individuals who responded to the questionnaire, colleagues who allowed their patients to be included in the study, and anonymous referees for comments on a previous draft. Financial support was received from the Medical Research Council (MRC) and the Chief Scientist Office of the Scottish Executive Health Department (SEHD). The views expressed in the paper are those of the authors.

References

View Abstract

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.