Background: Key issues in the quality of care for people with cataracts in the UK include hospital waiting lists, complication rates from surgery, and the use of junior surgeons. The main objective of this study was to investigate the relative importance that older people attach to these factors when given theoretical choices over options for cataract surgery.
Method: A systematic sample of 194 individuals aged 60–84 years on a general practice register in Nottingham were invited to take part in an interview based survey. Respondents ranked 11 “cataract surgery packages” containing different waiting list lengths, complication risks, and surgeon grades. Conjoint analysis was performed to determine the relative importance of these factors for individuals and for the group as a whole.
Results: Of the 194 subjects invited to participate, 146 (72%) completed the interview. For the group as a whole the “averaged importance” of the factors was: complication risk 45.8%; waiting time 41.1%, surgeon grade 13.1%. Analysis of importance scores for individuals showed that some were particularly concerned about complication risk while others were more concerned about waiting times. There was a strong negative correlation between importance scores for these factors (Spearman’s rho –0.78, p<0.001).
Conclusions: Most respondents thought that either risk of damage to sight and/or waiting time were important, while surgeon grade was relatively unimportant. The findings show that some potential cataract patients prefer a greater risk of complication combined with a short wait than a low complication rate and a longer wait.
Statistics from Altmetric.com
The recent Bristol Royal Infirmary Inquiry criticised health professionals in the NHS for not being sufficiently open with patients1 and stated that “patients should be able to gain access to information about the relative performance of a hospital or a particular service or consultant unit”.1 The inquiry showed that doctors were not following Department of Health advice on consent in terms of providing “sufficient information in a way that [patients] can understand about the proposed treatments, the alternatives and any substantial risks, so that they can make a balanced judgement”.2 Providing patients with sufficient information to enable them to make informed choices is an essential aspect of promoting patient autonomy. Following on from the Bristol Royal Infirmary Inquiry, it is important for health professionals to question current practices of information giving when seeking informed consent and to consider whether they need to provide more information to allow patients to make informed decisions.
One area where this point may be particularly relevant is with respect to surgical complication rates for junior surgeons. Surgeons in training are less skilled than their fully trained colleagues and are likely to have higher complication rates, even when supervised.3,4 This applies to cataract surgery as well as other operative procedures. Currently it is standard practice to withhold information on surgeon grade when counselling patients for surgery in the NHS, even when this is the dominant risk factor for poor outcome.
Important sight threatening complications following cataract surgery are retention of lens matter, retinal detachment, endophthalmitis, cystoid macula oedema, and corneal decompensation. An important risk factor for the development of these complications is rupture of the posterior lens capsule with prolapse of the vitreous into the anterior chamber. This occurs more often with surgeons in training.3–5
The Department of Health has recently recommended that operating lists for surgeons in training should be separated from service lists to speed up the throughput of cataract surgery.6 We believe that many patients would be unlikely to opt for cataract surgery on a training list if the full implications of a having a junior surgeon were understood. Nevertheless, the introduction of training lists provides an opportunity to offer patients other potential benefits, such as shorter waiting time, in exchange for increased risk from surgery. We do not know, however, whether patients would be prepared to opt for an increased risk in exchange for a shorter waiting time.
One way of assessing how patients may be prepared to trade one thing off against another is by using conjoint analysis. This is a well established market research technique that is increasingly being applied to healthcare research.5–22 It allows the relative importance of different factors to be assessed23 and shows what features individuals are prepared to trade to gain what they think is most important. There are several approaches to conjoint analysis.7 We opted for a conjoint analysis ranking approach which has been shown to be manageable for respondents7 and to have high levels of consistency,7,17 reliability,13 internal validity,7,13,18 and construct validity.13
This study investigated the preferences of a group of older people for different options for cataract surgery. The objectives were to determine:
the relative importance of complication risk from surgery, surgeon grade, and waiting time;
whether some respondents would prefer a shorter waiting time and a higher complication risk to a longer waiting time and a low complication risk;
whether preferences were associated with the sociodemographic characteristics of respondents.
The study was approved by the Nottingham University Medical School Ethics committee and was conducted in the year 2000.
Development of the interview schedule and ranking cards
In order to develop an interview schedule for the conjoint analysis it was necessary to develop a series of cards that participants ranked in order of preference (see fig 1 for an example). To help inform this task we performed a pilot study with 10 patients aged 60–84, each chosen from a different age-sex band of a general practice list in Nottingham, UK. We asked about the respondents’ understanding of cataracts, elicited views on different factors relevant to cataract treatment, and piloted a mock ranking task. The interviews suggested that respondents had a good understanding of cataract, mainly through personal contacts. They also suggested that waiting time, complication risk, and surgeon grade were the most important factors relevant to cataract treatment. Some participants had difficulty considering more than three factors at once so we decided only to include these factors in our study.
The following approach was taken in deciding on factor levels for the ranking cards used in the study:
Waiting times were chosen at 4, 8, and 16 months, reflecting the variation that can be found across the NHS at the time of study design.
Three levels of complication rate were chosen (1%, 5%, and 10%) on the basis of the published literature.3–5 Piloting suggested that the most meaningful way to express this was “risk that your sight could be worse than before the operation”.
Surgeon grade was divided into junior and consultant surgeons as pilot interviews suggested that this terminology was the most widely understood.
With these factor levels it is possible to generate 18 different scenarios (2 surgeon grade × 3 waiting time × 3 for risk = 18). Rather than asking respondents to rank all of these scenarios we used a factorial design (termed “orthoplan” in SPSS23) to randomly generate an “orthogonal array” of nine cataract surgery packages (the minimum number needed to exceed the number of factors sufficiently to allow for error degrees of freedom). In addition, two “holdout” packages23 were generated. The holdout packages described two scenarios that were different from the nine that constituted the orthoganol array and were generated by the same programme. They were ranked along with the other cards, but were not included in the main analysis. Instead, they were used for internal validity checks whereby the position that participants ranked these packages was compared with the position predicted by the conjoint analysis. The different cataract surgery packages were put on cards using 16 point type (fig 1).
Members of the public aged 60–84 years (who are all at risk of cataract)24–26 were systematically selected for inclusion in the survey from the general practice register. Those over 84 years were excluded because of the difficulties of surveying the very old. Using a random starting point, every seventh person was selected from the age-sex register (which was ordered by date of birth). The names of those selected where then checked by the general practitioners for suitability for inclusion. Exclusion criteria were: patients no longer registered with the practice; anticipated communication problems such as profound hearing loss; dementia and other serious mental illnesses.
An invitation letter was sent to people in the sample and respondents were given the choice of being interviewed in their home or at the general practice. The same interviewer (MAR) conducted all the interviews.
Conduct of the interviews
Before completing the ranking exercise the nature of the task was explained and the interviewer checked that participants had a reasonable understanding of cataract. The likely impact of cataract on participants was illustrated with the aid of a cataract simulator (see Acknowledgments for details). Participants were then presented with the 11 cards containing the different cataract surgery packages and were asked to rank these in order of preference to determine the relative importance of complication risk, surgeon grade, and waiting time. The participants were then asked sociodemographic questions in order to see whether their preferences could be explained in terms of any of the following: age, sex, educational history, occupation (whether employed or retired), current driving status, “average” time spent watching TV or reading, regular participation in activities requiring good vision (as defined by the respondent), and cataract history. We were particularly interested to see whether some patients—for example, those regularly participating in activities requiring good vision—would attach greater importance to certain factors.
Analysis of data
The first 10 interviews in the main study (and one in 10 of the subsequent interviews) were audiotaped and then checked (by AJA) for consistency of interviewing style and potential interviewer bias. No problems were found. The interviews were recorded on a precoded interview script, the entry of which was double checked.
Data were analysed using SPSS for Windows version 9.0. Conjoint analysis was performed using the “conjoint” procedure in spss categories. This procedure takes the rankings of the different cards for each participant and, through a set of linear regressions, generates utility scores for each factor level. The utility scores are the coefficients derived from the regression analyses. They are expressed on a common scale, so the relative importance of each factor can be expressed in percentage terms. This is done by taking the range of utility scores for any factor (highest minus lowest), dividing this by the sum of all the utility ranges, and multiplying by 100. A worked example is shown in the Appendix. Utility scores and importance scores were calculated for individual respondents, and mean utility and importance scores were calculated for the group as a whole.
Internal validity checks were performed as follows. Firstly, from the utility scores for each factor level the computer program calculated the utility of each card for each respondent. Using Kendall’s tau statistics, the ranking of the cards according to the computer generated utility scores was correlated with the way in which respondents actually ranked the cards. A similar approach was taken with the “holdout” cards in that Kendall’s tau statistics were used to determine the correlation between the observed ranking of the “holdout” cards with that predicted by the utility scores for these cards.
Correlations between importance scores for the different factors were investigated using Spearman’s rho correlation (two tailed).
Potential differences in the age and sex of responders and non-responders were explored using the Mann-Whitney U test and χ2 test, respectively. Potential associations between sociodemographic factors and importance scores for the different factors were assessed using non-parametric statistical tests (Mann-Whitney U test and Kruskal-Wallis one way analysis of variance).
Two hundred and six patients were selected from the 1445 patients aged 60–84 years on the practice register. Twelve patients were excluded (three had died since the sample frame was created, five had dementia, three had severe communication problems, and one was psychotic). A total of 194 were therefore contacted, of which 150 (77.3%) were interviewed and 146 (72.2%) completed the interview fully.
There was no significant difference between responders and non-responders in age or sex. Responders were similar to both Nottingham27 and British28 populations for age and sex. The percentage of responders aged 65–84 years who were employed (5.3%) was similar to national figures (4.1%)29 for those over 65 in employment. The prevalence of cataract in our sample was comparable to other white community based studies.24–26 The Jarman deprivation score30 for the practice was −2.7, which is close to the national average.
The median (interquartile range) age of responders was 69.5 years (64.75–75.0); 67 (44.7%) of the responders were male, 87 (58%) said that they were current car drivers, and 88 (58.7%) said that they participated in regular activities requiring good vision. In terms of the age at which respondents finished full time education, 119 (79.3%) were aged 14–16, 11 (7.3%) were aged 17–19, and 20 (13.3%) were aged 20 or over.
A summary of the overall results of the conjoint analysis is shown in table 1. In line with expectations it can be seen that the least preferable attribute levels were associated with the most negative utility scores. There was a fairly wide range of utility scores for complication risk and waiting time and this is reflected in the high “mean importance” scores calculated for these factors. In contrast, there was a relatively small difference in utility scores between the two grades of surgeon. For the group as a whole, complication rate was the most important attribute, closely followed by waiting time.
While it is standard practice to present “mean importance” scores for the overall sample (as shown in table 1), these scores were not normally distributed. We plotted histograms of importance scores for individuals and these showed that some respondents had high scores for risk while others had high scores for waiting time (figs 2 and 3). Spearman’s rho showed a strong negative correlation between importance scores for these factors (–0.78, p<0.001).
Internal validity checks showed that observed rankings for the nine cataract surgery packages and rankings predicted by utility scores from the conjoint analysis were highly correlated (Kendall’s tau = 0.944, p = 0.0002). The observed rankings for the two “holdout cards” were perfectly correlated with ranking predicted by the conjoint analysis (Kendall’s tau = 1.00).
In conjoint analysis studies, respondents sometimes appear to prefer a poor feature such as a long waiting time. Some researchers exclude such “inconsistent” responders.15 In our study there were three respondents who preferred a long waiting time, three who preferred a high complication rate, and eight who preferred a junior surgeon. The analyses were re-run with these respondents excluded. This made minimal difference to the findings, so the results are not presented here.
We undertook a number of univariate analyses to see if there were associations between respondents’ characteristics and their importance scores for each of the factors. The only statistically significant associations were related to importance scores for risk:
The higher the age of leaving full time education, the greater the importance score for risk (Kruskal-Wallis test for age of leaving full time education grouped into 14–16 years, 17–19 years, and 20 years and older: χ2 = 7.4, p = 0.025).
Respondents stating that they were current drivers or that they regularly undertook activities requiring good vision had higher importance scores for risk (Mann-Whitney U test: 1265, p = 0.044 and 1231, p = 0.042, respectively).
Summary of main findings
This study investigated the preferences of members of the general public aged 60–84 years for potential options for cataract surgery. Complication rate appeared to be the most important factor, closely followed by waiting time. By comparison, surgeon grade was relatively unimportant. Individual respondents attached different levels of importance to complication rates and waiting times. The importance scores for these factors were strongly negatively correlated.
Strengths and weaknesses of the study
Conjoint analysis is a powerful technique for eliciting public preferences for health care.7 In our study we had a reasonable response rate and most of the respondents were able to do the ranking task. The model developed for assessing the utility of different cataract surgery packages showed high levels of internal validity.
Overall, the study provides an example of the successful use of a ranking approach to conjoint analysis in health care, and one could envisage the approach being used in other areas of health services research. Nevertheless, it should be recognised that the alternative approach of presenting participants with a series of pairwise choices may be more appropriate in modelling the types of choices that are normally presented to patients.7
Our study involved participants from a single general practice in Nottingham. While the characteristics of participants were similar to national figures, the results may not be generalisable to other areas.
Implications of the study
This study has potentially important implications for the quality of care of patients with cataract in health services that have long waiting lists for surgery. The fact that some respondents rated waiting time as more important than complication risk suggests that the wait for surgery is a major factor for these individuals. The fact that surgeon grade was not ranked as particularly important suggests that the public do not realise that junior surgeons have markedly higher complication rates than most consultants. Few patients would opt for a junior surgeon rather than a competent consultant if they knew about the likely differences in complication rates. Nevertheless, our findings suggest that there could be a solution to dealing with the potential difficulties of being open with patients about the complication rates of junior surgeons.
On the basis of our results we propose that, if patients with cataract are offered an operation on a junior surgeon operating list, they should have a choice in this matter. One way of encouraging some patients to make this choice would be to have shorter waiting times for these operating lists. Our findings suggest that, if offered this option, some people would be prepared to opt for a higher complication risk in exchange for a shorter waiting time.
We recognise that this is a radical proposal, but it is one that is consistent with being open with patients and with offering them choices in their health care. We believe that it would be inappropriate to have major differences between waiting times for junior and consultant surgeon operating lists as this might appear to be coercing patients into opting for treatment by a junior surgeon. Nevertheless, analysis of utility scores from respondents involved in our study suggests that, even with a 4 month difference in waiting time, some patients might opt for a higher surgical risk (see Appendix).
In introducing any system of shorter waiting times for junior surgery operating lists, careful attention would need to be paid to informed consent. One would need to recognise that some patients—for example, those who regularly participate in activities requiring good vision—might be particularly keen to avoid complications.
This study has shown that, when older people were presented with a range of different cataract surgery packages, respondents thought that risk of damage to sight and/or waiting times were of greatest importance and surgeon grade was relatively unimportant. These findings indicate that some potential cataract patients prefer a greater risk of complications combined with a short wait to a low complication rate and a longer wait.
When presented with different options for cataract surgery, a group of older people attached particular importance to complication risk and waiting time for surgery.
Some respondents were most concerned about complication risk while others were more concerned about waiting time.
Respondents who stated that they were current drivers or that they regularly undertook activities requiring good vision were more likely than others to attach particular importance to complication risk.
When discussing potential options for cataract surgery, some people may be prepared to opt for a higher complication risk (such as that associated with operation by a junior surgeon) in exchange for a shorter waiting time.
A worked example is shown of how importance scores are calculated and how the utility of different combinations of factor levels can be calculated.
The utility scores generated by conjoint analysis are expressed on a common scale and the relative importance of each factor can therefore be expressed in percentage terms. The relative importance of the different factors is calculated using the range of utility scores for each factor. While the computer program does this automatically, it can also be done manually. This is illustrated below with respect to the importance score for waiting time for one of the respondents in the study (table 2). The calculation is based on taking the range of utility scores for waiting time (highest minus lowest), dividing this by the sum of all the utility ranges, and multiplying by 100:
Please note that the importance scores in table 1 are the means of the importance scores for all respondents and are not calculated directly from the mean utility scores.
The utility of any combination of factors can be calculated as follows:
Utility = the sum of the utilities of all the factor levels plus the constant
Table 3 illustrates this for a number of different combinations of factor levels in relation to the respondent whose data are shown in table 2. It can be seen that the respondent would marginally prefer a 4 month wait and a 10% risk of sight being worse after surgery than an 8 month wait with a 1% risk of sight being worse after surgery. The former option would be much preferable to a 1% risk and a 16 month wait.
We are grateful to Toby Gosden, Lindsay Groom, Judy Jones, Ciaran O’Neill, Carol Coupland, Clive Richards and the referees of this paper for providing helpful comments and advice. The study was funded by the Nottingham Health Authority. The cataract simulator can be obtained from Visual Impairment North-East (Registered charity No. 518308. c/o Mr F Hodson, Pearey House, Preston Park, North Shields NE29 9JR, UK).
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.