Do ordering effects matter in willingness-to-pay studies of health care?
Introduction
Research on eliciting willingness-to-pay (WTP) values in order to evaluate projects has been extensive in environmental economics and is growing in health economics Diener et al., 1998, Klose, 1999, Smith, 2000. In 1993, the National Oceanic and Atmospheric Administration (NOAA) published guidelines for the use of contingent valuation (CV) studies of environmental resource damage. However, the publication of these guidelines has not ended the debate about the ‘best’ survey design, particularly in the health care field. For that reason, research is ongoing into the reliability of CV studies and most of the methodological issues raised in the environmental field remain when CV studies are used in health care.
This paper explores the possibility of ordering effects in CV studies of health care programmes as part of a larger research project (EuroWill)1 to examine CV studies in the health care field. This paper asks, “does the WTP for a programme change when the programme is in a different position in a sequence of programmes?” This question is answered by surveying two samples and reversing the order of the programmes between the two samples. This paper builds on previous, methodologically diverse, research on ordering effects in the environmental field Gorden, 1969, Brookshire et al., 1981, Boyle et al., 1990, Boyle et al., 1993, Bateman and Langford, 1997 and in the health care field Kartmann et al., 1996, Halvorsen, 1996. The results from previous literature have been mixed, with support both for and against ordering effects. However, this literature is sparse with only a small number of published studies having examined whether an ordering effect arises when CV questions are asked of multiple programmes in a single survey instrument.
Section snippets
Conceptual issues and hypotheses
Why elicit WTP for several programmes within one survey? There are three reasons. First, the NOAA guidelines recommend that respondents be reminded of substitutes when reporting WTP values because, “if individuals fail to consider seriously the public or private goods that might be substitutes for the resources in question, their responses to questions in a CV survey may be unrealistically large” (Arrow et al., 1993). Second, if WTP values are to be used to aid resource allocation then values
Sampling
The survey was carried out in the largely rural Western Health Board region of Ireland, which contains a population of approximately 350,000 people. The sample design was based on a two-stage clustered sample using the Electoral Register as a population frame. The data generated from the electoral register was re-weighted on the basis of the principal economic status of head of household, household composition and sub-regional classification to make it representative of the overall population
Results
Table 2 describes how respondents ranked the programmes in the two samples. The cancer programme was ranked most important by a larger percentage of respondents than the other two programmes in both samples. It was more difficult to decide the ranking between the other two programmes. We concluded that hearts was ranked higher than community care because a larger proportion ranked it second most important.
As Table 2 indicates, there was a difference in the proportions in the rankings between
Discussion
This paper confirms the existence of ordering effects in CV studies in health care. Asking people to put a monetary WTP value on a sequence of health care programmes will likely yield different results depending on the order of the programmes because answers to earlier questions will affect responses to later questions. We found that the fading glow hypothesis best explains the observed ordering effects. The fading glow refers to the utility respondents gain from saying that they are willing to
Conclusion
This paper has examined how people’s WTP for three health care programmes is related to the order in which the programmes are presented. If WTP estimates are to be used to inform policy decisions, then the degree of bias associated with the estimation procedure needs to be made explicit. If the value of a subsequent programme is influenced by the value given to a previous one then an important axiom, the independence of irrelevant alternatives, is being violated.
Ordering effects are observed in
Acknowledgements
The authors would like to thank Ian Bateman, Graham Loomes, Martin Dooley, David Feeny, participants at the EuroWill Dublin Conference, and participants at the Irish Economic Association Meetings, May 2000. The authors would like to thank James Williams and the Economic and Social Research Institute for assistance with the survey. Cam Donaldson holds the Svare Chair in Health Economics and is an Alberta Heritage Senior Scholar, Canadian Institutes of Health Research Senior Investigator and
References (29)
- et al.
Sequencing and nesting in contingent valuation studies
Journal of Environmental Economics and Management
(1995) - et al.
Sequencing and valuing public goods
Journal of Environmental Economics and Management
(1998) - et al.
A note on the warm glow of giving and scope sensitivity in contingent valuation studies
Journal of Economic Psychology
(2000) - et al.
An independent sample of yea-saying and starting point bias in dichotomous-choice contingent valuation
Journal of Environmental Economics and Management
(1995) - et al.
Valuing public goods: the purchase of moral satisfaction
Journal of Environmental Economics and Management
(1992) The contingent valuation method in health care
Health Economics
(1999)Impure altruism and donations to public goods: a theory of warm-glow giving
Economic Journal
(1990)- Arrow, K., Solow, R., Portney, P.R., Leamer, E.E., Radner, R., Schuman, H., 1993. Report of the National Oceanic and...
- Bateman, I.J., Langford, I.H., 1997. Budget constraint, temporal and ordering effects in contingent valuation studies....
- et al.
Species substitution and question sequencing in contingent valuation surveys evaluating the hunting of several species of wildlife
Leisure Science
(1990)
The role of question order and respondent experience in contingent-valuation studies
Journal of Environmental Economics and Management
Health care contingent valuation studies: a review and classification of the literature
Health Economics
The nature of individual preferences: a prologue to Johannesson, Jonsson and Karlsson
Health Economics
Cited by (56)
What creates heterogeneity in ballot order effects? Evidence from Korea's local elections of education superintendent
2017, Electoral StudiesCitation Excerpt :Relevant to this study, literature on survey design has long been aware that survey respondents are more likely to select response choices presented earlier and that the magnitude of such effects is varied across demographic groups (Krosnick, 1999; MacKenzie and Podsakoff, 2012). For example, education level (McClendon, 1986; Narayan and Krosnick, 1996; Holbrook et al., 2007), cognitive sophistication level (Krosnick and Alwin, 1987), attitude and knowledge (Auspurg and Jäckle, 2015), and previous experience (Stewart et al., 2002) are suggestive of factors moderating the response-order effects. Much of the same logic seems applicable to voting.
Exploiting order effects to improve the quality of decisions
2014, Patient Education and CounselingCitation Excerpt :For example, subjects tended to view and choose ads in the Yellow Pages that were at the top of the alphabetical list [13] and choose candidates listed at the top of electoral ballots [25]. Research in economics points to a warm (or fading) glow effect in the way information influences people's values [26], which can go on to influence peoples choices [27,28]. There is limited evidence on the influence of order effects in the design of health education materials, despite a recognition that such cognitive biases can impact people's ability to process content-related information [29].
Determinants of willingness-to-pay for water pollution abatement: A point and interval data payment card application
2012, Journal of Environmental ManagementCitation Excerpt :Section 5 presents the results and section 6 concludes. Results from payment card applications that allow for point responses at the right tail of the WTP distribution are typically estimated using the classical interval data model (Morey et al., 1997; O’Shea et al., 2001; Stewart et al., 2002; Huhtala and Pouta, 2008; O’Shea et al., 2008; Barton et al., 2010; Serneels et al., 2010; Lindhjem and Navrud, 2011; Moore et al., 2011). However, articles do not provide information on how point data are treated.
Putting different price tags on the same health condition: Re-evaluating the well-being valuation approach
2011, Journal of Health EconomicsCitation Excerpt :Future research should therefore return to instrument for both health and income in each of the tested well-being equations. Moreover, given that the listed health conditions used in this study are self-reported, it is possible that our estimates will be biased from well-known ‘ordering’ effects, i.e. the estimated health effects may vary according to the order of the survey questions (see, e.g., Stewart et al., 2002; Paloyo, 2009), which implies that the SPs presented in this paper should be treated with care. In conclusion, the relatively stable internal rankings across well-being measures suggest that the various measures of life experience could all be used to derive a consistent ordinal ranking of the impacts of health conditions on people's quality of life—assuming that cardinalization is not of policy concern.
Individual responsibility and health-risk behaviour: A contingent valuation study from the ex ante societal perspective
2011, Health PolicyCitation Excerpt :Second, despite of CVM being a natural way to measure preferences within managed competition, is has also limitations. It has been argued that ordering effects could influence WTPs [30]. We first asked WTP for the independent health-risk behavioural disease and then WTP for the dependent health-risk behavioural disease.
Distal order effects in stated preference surveys
2011, Ecological Economics