Do ordering effects matter in willingness-to-pay studies of health care?

https://doi.org/10.1016/S0167-6296(02)00003-6Get rights and content

Abstract

Willingness-to-pay studies are increasingly being used in the evaluation of health care programmes. However, there are methodological issues that need to be resolved before the potential of willingness-to-pay can be fully exploited as a tool for the economic evaluation of health care programmes. Of particular methodological interest are the consequences of varying the order in which willingness-to-pay questions are presented to respondents in contingent valuation studies. This paper examines the possibility of ordering effects in willingness-to-pay studies in health care. That is, when asking willingness-to-pay questions about three health care programmes within a single survey, does the order the programmes are presented have an impact on the reported willingness-to-pay? Ordering effects are observed in the ranking of the programmes, in the proportion of zero values reported and in the WTP for one of the programmes. The results suggest that the best explanation for the ordering effects is one of fading glow, whereby the first programme in any sequence captures much of the utility associated with giving.

Introduction

Research on eliciting willingness-to-pay (WTP) values in order to evaluate projects has been extensive in environmental economics and is growing in health economics Diener et al., 1998, Klose, 1999, Smith, 2000. In 1993, the National Oceanic and Atmospheric Administration (NOAA) published guidelines for the use of contingent valuation (CV) studies of environmental resource damage. However, the publication of these guidelines has not ended the debate about the ‘best’ survey design, particularly in the health care field. For that reason, research is ongoing into the reliability of CV studies and most of the methodological issues raised in the environmental field remain when CV studies are used in health care.

This paper explores the possibility of ordering effects in CV studies of health care programmes as part of a larger research project (EuroWill)1 to examine CV studies in the health care field. This paper asks, “does the WTP for a programme change when the programme is in a different position in a sequence of programmes?” This question is answered by surveying two samples and reversing the order of the programmes between the two samples. This paper builds on previous, methodologically diverse, research on ordering effects in the environmental field Gorden, 1969, Brookshire et al., 1981, Boyle et al., 1990, Boyle et al., 1993, Bateman and Langford, 1997 and in the health care field Kartmann et al., 1996, Halvorsen, 1996. The results from previous literature have been mixed, with support both for and against ordering effects. However, this literature is sparse with only a small number of published studies having examined whether an ordering effect arises when CV questions are asked of multiple programmes in a single survey instrument.

Section snippets

Conceptual issues and hypotheses

Why elicit WTP for several programmes within one survey? There are three reasons. First, the NOAA guidelines recommend that respondents be reminded of substitutes when reporting WTP values because, “if individuals fail to consider seriously the public or private goods that might be substitutes for the resources in question, their responses to questions in a CV survey may be unrealistically large” (Arrow et al., 1993). Second, if WTP values are to be used to aid resource allocation then values

Sampling

The survey was carried out in the largely rural Western Health Board region of Ireland, which contains a population of approximately 350,000 people. The sample design was based on a two-stage clustered sample using the Electoral Register as a population frame. The data generated from the electoral register was re-weighted on the basis of the principal economic status of head of household, household composition and sub-regional classification to make it representative of the overall population

Results

Table 2 describes how respondents ranked the programmes in the two samples. The cancer programme was ranked most important by a larger percentage of respondents than the other two programmes in both samples. It was more difficult to decide the ranking between the other two programmes. We concluded that hearts was ranked higher than community care because a larger proportion ranked it second most important.

As Table 2 indicates, there was a difference in the proportions in the rankings between

Discussion

This paper confirms the existence of ordering effects in CV studies in health care. Asking people to put a monetary WTP value on a sequence of health care programmes will likely yield different results depending on the order of the programmes because answers to earlier questions will affect responses to later questions. We found that the fading glow hypothesis best explains the observed ordering effects. The fading glow refers to the utility respondents gain from saying that they are willing to

Conclusion

This paper has examined how people’s WTP for three health care programmes is related to the order in which the programmes are presented. If WTP estimates are to be used to inform policy decisions, then the degree of bias associated with the estimation procedure needs to be made explicit. If the value of a subsequent programme is influenced by the value given to a previous one then an important axiom, the independence of irrelevant alternatives, is being violated.

Ordering effects are observed in

Acknowledgements

The authors would like to thank Ian Bateman, Graham Loomes, Martin Dooley, David Feeny, participants at the EuroWill Dublin Conference, and participants at the Irish Economic Association Meetings, May 2000. The authors would like to thank James Williams and the Economic and Social Research Institute for assistance with the survey. Cam Donaldson holds the Svare Chair in Health Economics and is an Alberta Heritage Senior Scholar, Canadian Institutes of Health Research Senior Investigator and

References (29)

  • K.J. Boyle et al.

    The role of question order and respondent experience in contingent-valuation studies

    Journal of Environmental Economics and Management

    (1993)
  • Brookshire, D.S., d’Arge, R.C., Schulze, W.D., Thayer, M.A., 1981. Experiments in valuing public goods. In: Smith, V.K....
  • A. Diener et al.

    Health care contingent valuation studies: a review and classification of the literature

    Health Economics

    (1998)
  • P. Dolan

    The nature of individual preferences: a prologue to Johannesson, Jonsson and Karlsson

    Health Economics

    (1997)
  • Cited by (56)

    • What creates heterogeneity in ballot order effects? Evidence from Korea's local elections of education superintendent

      2017, Electoral Studies
      Citation Excerpt :

      Relevant to this study, literature on survey design has long been aware that survey respondents are more likely to select response choices presented earlier and that the magnitude of such effects is varied across demographic groups (Krosnick, 1999; MacKenzie and Podsakoff, 2012). For example, education level (McClendon, 1986; Narayan and Krosnick, 1996; Holbrook et al., 2007), cognitive sophistication level (Krosnick and Alwin, 1987), attitude and knowledge (Auspurg and Jäckle, 2015), and previous experience (Stewart et al., 2002) are suggestive of factors moderating the response-order effects. Much of the same logic seems applicable to voting.

    • Exploiting order effects to improve the quality of decisions

      2014, Patient Education and Counseling
      Citation Excerpt :

      For example, subjects tended to view and choose ads in the Yellow Pages that were at the top of the alphabetical list [13] and choose candidates listed at the top of electoral ballots [25]. Research in economics points to a warm (or fading) glow effect in the way information influences people's values [26], which can go on to influence peoples choices [27,28]. There is limited evidence on the influence of order effects in the design of health education materials, despite a recognition that such cognitive biases can impact people's ability to process content-related information [29].

    • Determinants of willingness-to-pay for water pollution abatement: A point and interval data payment card application

      2012, Journal of Environmental Management
      Citation Excerpt :

      Section 5 presents the results and section 6 concludes. Results from payment card applications that allow for point responses at the right tail of the WTP distribution are typically estimated using the classical interval data model (Morey et al., 1997; O’Shea et al., 2001; Stewart et al., 2002; Huhtala and Pouta, 2008; O’Shea et al., 2008; Barton et al., 2010; Serneels et al., 2010; Lindhjem and Navrud, 2011; Moore et al., 2011). However, articles do not provide information on how point data are treated.

    • Putting different price tags on the same health condition: Re-evaluating the well-being valuation approach

      2011, Journal of Health Economics
      Citation Excerpt :

      Future research should therefore return to instrument for both health and income in each of the tested well-being equations. Moreover, given that the listed health conditions used in this study are self-reported, it is possible that our estimates will be biased from well-known ‘ordering’ effects, i.e. the estimated health effects may vary according to the order of the survey questions (see, e.g., Stewart et al., 2002; Paloyo, 2009), which implies that the SPs presented in this paper should be treated with care. In conclusion, the relatively stable internal rankings across well-being measures suggest that the various measures of life experience could all be used to derive a consistent ordinal ranking of the impacts of health conditions on people's quality of life—assuming that cardinalization is not of policy concern.

    • Individual responsibility and health-risk behaviour: A contingent valuation study from the ex ante societal perspective

      2011, Health Policy
      Citation Excerpt :

      Second, despite of CVM being a natural way to measure preferences within managed competition, is has also limitations. It has been argued that ordering effects could influence WTPs [30]. We first asked WTP for the independent health-risk behavioural disease and then WTP for the dependent health-risk behavioural disease.

    View all citing articles on Scopus
    View full text