BACKGROUND

There is an increasing move towards clinical decision making that engages the patient.^1,2 This has resulted in the increasing use of decision aids to support both clinicians and patients in making decisions³—for example, the development of interactive computer programs and video disks for patients which incorporate a presentation of the clinical evidence and the likely effects of alternative treatments such as that developed for men with prostate symptoms.^4–7 Other examples include decisions about breast cancer treatment⁸ and audiotapes and booklets for a range of clinical decisions including anticoagulation for atrial fibrillation (AF).⁹

One means of involving patients in clinical decision making is the use of decision analysis^10–12 which attempts to create a rational framework for evaluating complex medical decisions and to provide a systematic way of integrating potential outcomes with probabilistic information. This recognises the importance of patients' views in arriving at the “optimal” treatment decision, but also that patients may be ill equipped to integrate their values with complex medical information in order to make a fully informed decision for themselves. Thus, the quality of care provided to patients is inevitably influenced by the quality of clinical decision making and the degree to which patients' values are successfully incorporated into treatment decisions.

Atrial fibrillation is a cardiac condition common in the elderly and is known to be associated with a greatly increased risk of stroke. Several randomised controlled trials, a pooled analysis, and a meta-analysis have shown that treatment of selected patients in AF with warfarin significantly reduces the risk of stroke.^13,14 However, treatment with warfarin is known to have adverse effects—most notably, the increased risk of suffering a major bleed. In addition, patients' quality of life may be diminished on warfarin therapy by having to undergo frequent blood tests, bruising easily, and controlling their lifestyles. This has resulted in uncertainty over its appropriate use,¹⁵ with considerable variation in treatment rates and in the content of available guidelines.^16,17 Decision analysis seemed an appropriate tool to apply to this clinical problem.

A decision analysis was undertaken, incorporating a systematic literature review and appraisal and patient utility elicitation using the standard gamble method to develop evaluative guidelines, using a Markov decision analysis to model the decision of whether or not to use warfarin (“treat” or “do not treat”) for patients in AF. This showed that the decision to treat is highly sensitive to the disutility of warfarin therapy, and that this varied widely between patients.^18,19

We set out to develop a means of incorporating the decision analysis model, or its output, into clinical practice in the form of an interactive decision aid for use in individual clinical consultations (the Decision Analysis in Routine Treatment Study (DARTs) tool). Decision analytical models have been used to aid clinical decision making in a number of settings.^20,21 More recently there have been attempts to engage patients in the process of shared decision making using decision analytical approaches—for example, in the fields of prenatal screening (Hilary Bekker, personal communication), oophorectomy at the time of hysterectomy,²² and antihypertensive therapy (Tom Fahey, personal communication). To the best of our knowledge, none have sought to apply a decision analytical approach to shared decision making involving both the patient and the responsible clinician; we set out to develop a tool that could be used directly within consultations between the patient and the responsible clinician.

Using decision analysis based information to inform the clinical decision for individual patients involves three components: assessing the utility that individual patients ascribe to the various relevant health states in the decision analysis; presenting information on the risks and benefits of treatment to patients and clinicians to enable them to process this information in a meaningful way; and reflecting back the results of the decision analysis to inform patients' and clinicians' decision making. This paper describes the methods and primary results of the developmental process and the resulting prototype decision aid.

The research was iterative, involving several interwoven components with two main phases (fig 1). The main objective of the development phase was to turn a previously developed decision analytical model (see below) into a computerised decision support tool beneficial to patients and clinicians alike. In so doing, we sought:

• to explore the desire for detailed information on risks and benefits in patients and GPs;

• to explore means of presenting information on risks and benefits of treatment to elderly patients;

• to assess the feasibility of taking elderly respondents through a computerised decision tool, including a standard gamble utility assessment exercise. The training and testing phase sought:

• to assess initial acceptability of the tool to patients and clinicians;

• to identify areas for further tool refinements;

• to understand the training needs of clinicians and the time needed to apply the tool;

• to inform the design of a subsequent intervention study.

THE UNDERLYING MODEL

Before describing the methods used in the current study, a brief description is given of the previously developed decision analysis model that formed the basis for the computerised tool.¹⁸ Briefly, we used a Markov decision analysis to model the decision on warfarin treatment of patients in AF using a systematic literature review and appraisal, supplemented by additional research. The “optimal” strategy for any particular patient is the decision (treat or do not treat) that yields the highest expected utility over his/her remaining life expectancy.

In keeping with clinical practice, we modelled the treatment decision for 12 months, after which time the decision was reassessed—that is, the model was rerun using the updated information. Data on effectiveness of warfarin, absolute risk of stroke, risk of recurrent stroke, outcome of stroke, and risk of major (non-cerebral) bleed were derived from systematic review, with point estimates used in the model. A meta-analysis undertaken for the development of the decision analysis showed no significant effect of aspirin over placebo.¹⁸ Furthermore, subsequent meta-analyses have largely confirmed that warfarin is more effective than aspirin.¹⁴ We therefore decided to limit our decision analysis to the decision on whether to anticoagulate with warfarin, given the clear benefit of warfarin over aspirin, with aspirin as the treatment to apply if the shared decision is not to take warfarin.

A utility assessment exercise was carried out using the standard gamble method.¹⁹ The descriptions of the health states were adapted from those used in previous research²³ following discussion with clinical experts. Further details of the decision analysis model, the meta-analysis, the clinical data used to populate it, and the standard gamble exercise are published elsewhere.^18,19

Figure 2 provides a schematic account of the decision analysis model that underlies the DARTS tool. Data items listed as “variables” refer to those that vary from patient to patient according to clinical evidence (risk of bleed and life expectancy in the case of risk of stroke) or patient preferences (in the case of utility values). Those listed as “parameters” are data items that are assumed to be constant across all patients (based on available data).

METHODS

The methods for each phase are described below (further details are available from the authors).

Development phase

The development phase consisted of four stages: (1) 10 GPs with no specialised interest in AF underwent a semi-structured interview; (2) draft risk presentation materials were piloted in a group of 20 elderly respondents not in AF; (3) four patient focus groups were conducted involving patients with (n=10) and without (n=11) AF; and (4) the prototype computerised tool was tested for clarity, visual presentation, and acceptability in a sample of six volunteer elderly patients, none of whom were in AF.

GP interviews and focus groups were recorded and transcribed verbatim and analysed using framework analysis.²⁴ In the remaining stages the comments and reactions of the participants were recorded by two researchers and the notes discussed and combined. At each stage, key points of relevance to the design of the tool were identified and summaries produced which were circulated to the research team and discussed in regular project meetings.

Training and testing phase

After attending a one hour introductory session, four GPs and three practice nurses were recruited to take part in: (1) a half day training programme during which members of the research team acted as “dummy” patients; (2) a further half day training session consisting of simulated consultations using actors experienced in assuming a patient role for training purposes; and (3) a refresher course lasting 30–40 minutes prior to a brief feasibility study in which clinicians carried out consultations in their own practice with patients in AF in order to support decisions on whether to treat with warfarin. Patients received an information leaflet prior to the consultation with their GP which introduced them to the increased risk of stroke in AF and the value of warfarin in reducing this risk.

A variety of methods were used in this phase including audio and video taping of consultations and semi-structured interviews with clinicians and patients. We sought to discover from the patients (including actor patients) how well they thought the clinician had conducted the consultation, their level of understanding of the tool, and suggestions for potential improvements. After each stage, GPs and nurses were interviewed to explore any problems that arose, their level of confidence in using the tool, and potential improvements to it.

RESULTS

Development phase

The results of the GP interviews (to be reported in detail elsewhere) revealed uncertainty about appropriate use of warfarin in patients in AF together with apparent gaps in knowledge. Readily accessible information on the evidence base and the risks and benefits to individual patients would generally be welcomed, although some clinicians felt that sharing this might be alarmist to patients. While older people, both with and without AF, rarely requested numerical risk information spontaneously when asked about their expectations of the consultation, when presented with such information they generally welcomed it. These results emphasised that the tool should meet the needs of the GPs in terms of bringing the research evidence to the consultation in a readily accessible format, but allow flexibility in the means of presenting the information to patients.

Several methods of presenting the risk of stroke were piloted including bar charts, pie charts, “smiley faces”, and non-representational proportionate figures. Respondents generally preferred graphical to numerical information alone, with an overwhelming preference for the “smiley face” representation of risk, a method that has been used in other settings.⁹

The prototype tool was well received and none of the volunteers had a problem using a computer in a consultation. However, the hypothetical nature of the standard gamble component required careful explanation. It was crucial to make this aspect of the tool quite distinct from the component which presents their own risks and personal clinical information.

The findings of this phase were encouraging in terms of the acceptability of the tool in this age group. Throughout the development phase, issues were raised which led to further ongoing iterative refinement of the tool and which also helped to identify issues for the training of users of the tool (table 1).

DARTS tool

The prototype at the end of this development process consists of three components, described in greater detail below:

• the derivation of patients' values for the relevant health states associated with stroke and treatment with warfarin using a standard gamble method (the standard gamble component)¹⁹;

• the presentation of risk information for an individual patient using the Framingham stroke risk equation in conjunction with estimates of the effectiveness of warfarin at reducing this risk and the associated risk of major bleed (the risk factor component)¹⁸;

• the decision making process: the discussion of the patient's views and a consideration of the prediction of the Markov decision model (the decision making component).

Delphi 4 was chosen as the programming language and the software runs on a laptop PC in a Windows environment.

Standard gamble component

Standard gamble exercises set out to elicit patients' preferences for health outcomes by means of asking them hypothetical questions.²⁵ The purpose of standard gamble questions is to find the point of indifference or “equivalence” between option A (an intermediate health outcome for sure) and option B (some chance of a better health outcome (in this case normal health) and some chance of a worse outcome (in this case death)). If the end point states (in this case, normal health and death) are assigned values of 1 and 0 respectively, the indifference value of p may be taken as the individual's utility valuation of the intermediate state.

The standard gamble component consists of a series of four questions of the type described above and makes use of an iterative procedure to reach the point of indifference. The four health states to be valued are: mild stroke, severe stroke, stomach bleed, and treatment with warfarin (the screen image for this stage is shown in fig 3). The two stroke states (mild and severe) are assessed against normal health and death, while treatment with warfarin and stomach bleed are assessed against normal health and mild stroke. The utility values are calculated on a scale from 0 (dead) to 1 (normal health) directly for the stroke states, and indirectly using a two part procedure known as “chaining” for the other states.

The chances of normal health and death are initially set at 50 in 100 and patients are then asked whether they would prefer option A, option B, or cannot choose between them. The chances of normal health and death then change until the point of indifference is found.

Risk factor component

The second component presents patients with information on their absolute risk of stroke on warfarin and with no treatment, and their risk of a gastrointestinal (stomach) bleed on warfarin. Baseline risks of stroke were assessed with a variant of the Framingham stroke risk equation using the patient's age, sex, systolic blood pressure, history of cardiovascular disease, diabetes, left ventricular hypertrophy, treatment for hypertension, and smoking history. Our published decision analysis used the version of the Framingham stroke risk equation published in 1991 as this was the only community cohort study we uncovered that allowed a calculation of an individual's absolute risk of stroke.²⁶

However, while it is appropriate for the purpose of development of guidelines on the use of warfarin, this version has problems when used as a tool for predicting benefit from risk reduction by blood pressure control. It yields a counterintuitive result that initiating treatment with antihypertensive therapy appears to increase an individual's risk of stroke even when a reduction in blood pressure is achieved. This is likely to be due to confounding in the original Framingham cohort whereby those on antihypertensive therapy are likely to be those with more severe hypertension (or at least detected hypertension) than those found to be hypertensive but not receiving treatment. We discussed this with the authors of the Framingham study who kindly supplied us with an alternative specification of their risk equation, based on a Weibull model, which did not have this property and which we have adopted for the decision support tool (D'Agostino, personal communication).

Figure 4 shows the data input form and output information for a 75 year old man. Following piloting, a number of features were built into the decision support tool which were additional to the original decision analysis model. Figure 4 shows a “target risk” box alongside the data on systolic blood pressure and smoking behaviour which appears whenever the patient is hypertensive (systolic blood pressure of ≥160) and/or smokes, and allows the impact of modifying these risk factors to be considered. Furthermore, the 5 year risk button calculates the risk of stroke both on and off warfarin over the next 5 years, rather than 12 months in the default.

Decision making component

The third component of the tool helps clinicians and patients to reach a shared decision using a structured process. After completion of the risk factor component, patients are first asked for their preference for treatment with warfarin over no treatment, given their knowledge of the risks and benefits of treatment and having reflected on their values for the health states in the standard gamble. Having expressed their initial preference, patients are then given the “optimal” prediction from the Markov decision analysis model based on their individual risks and benefits and their health state values derived in the standard gamble.

Patients with the modifiable risk factors of hypertension and/or smoking are also presented with the “optimal” decision based on their reduced risk of stroke were they to achieve their target of reducing blood pressure, smoking cessation, or both. This allows discussion as to whether warfarin therapy would still be indicated over no treatment if they were able to reduce their risk of stroke by other means.

The route through this section thereafter depends on whether or not the patient has modifiable risk factors and the match between their initial preference for treatment and the prediction of the model. All patients are given further opportunities to express their views on treatment with warfarin over no treatment before a final decision is reached. When patients do not want to go on warfarin or are undecided, aspirin is discussed as an alternative. The final decision is then “logged”, together with any agreed plans to target modifiable risk factors. If patients remain uncertain at the end of the initial consultation, there is an option to defer the decision and return for a further consultation following reflection.

A print out offers the patient and clinician a record of the key features of the consultation, including the current stroke risk profile and agreed plans to target modifiable risk factors. The risk of stroke is shown both on warfarin and with no treatment, as is the risk of gastrointestinal (stomach) bleed on warfarin. Five year risk of stroke can also be included if the clinician and patient so wish. Where appropriate, the effect of reducing modifiable risk factors on stroke risk is also presented. Finally, the print out records the shared decision and agreed plan for subsequent review.

Training and testing phase

At the end of the training phase clinicians felt comfortable using the tool and were prepared to use it with patients. Nonetheless, use of the tool required fairly intensive guidance in certain aspects, in particular the importance of remaining “neutral” during the application of the standard gamble component. A standardised explanation in the form of a suggested “script” was prepared. In contrast, some felt the initial sequence of screens in the decision making component was too directive and requested more flexibility in this part of the tool.

The time taken to complete the tool in the consultations with actor patients ranged from 18 to 45 minutes (n=7). The implications of this were discussed in the context of primary care based consultations. It was clear that the tool would require dedicated time for its use.

In the feasibility phase 10 patients (seven men) of mean age 72 (range 61–86) in AF who were not currently taking warfarin completed a consultation lasting between 25 and 50 minutes. No patient expressed a dislike of the computer tool and most were very positive. Eight understood it well, felt it had been very well explained, and reported no important problems with the consultation. One patient was not able to discuss the details of the consultation, and one stated that he “didn't feel convinced” by it and didn't feel fully informed of the side effects of warfarin. In both the testing and feasibility phases some concern was expressed that the tool might be seeking to “persuade” patients to accept warfarin. There was also some confusion about the relationship between digoxin and warfarin—two respondents stated after the consultation that they were happy with the drug they were taking for controlling their irregular heart rate (digoxin) and didn't want to change.

The 10 consultations took on a number of different patterns and the final decisions of patients did not necessarily conform to the predictions of the Markov model, hence expected utility theory. Three patients did not proceed to the decision component as their risk of stroke was considered too low for them to benefit from taking warfarin. This is because the risks of cerebral haemorrhage may be greater than the reduction in risk from anticoagulation.¹⁸ Of the remaining seven, the final decision concurred with the prediction of the Markov model in four cases and differed in three. Table 2 gives a brief summary of the seven consultations, including the prediction of the Markov model and the final decision on whether or not the patient wanted to be treated with warfarin.

DISCUSSION

This paper has described an iterative approach to the development of a decision analysis based decision support tool and the outcome of that development programme, including initial pilot experience of use with patients. Such an iterative approach has been used previously—for example, in the development of a (non-decision analysis based) decision support tool for patients with breast cancer²⁷—and there are a number of reasons we felt that design to be particularly appropriate here. We were developing a tool based on a technique—namely, decision analysis—with which patients and clinicians alike would be unfamiliar and possibly find difficult conceptually. In requiring the patient to answer hypothetical questions, the standard gamble utility component in particular requires a major departure from the usual interaction between doctors and their patients.

The potential benefit of using a decision analytical approach to clinical decision making is that it might reduce the cognitive burden on the patient, an important factor in light of the evidence that individuals are known to make systematic mistakes when dealing with probabilistic information. Expected utility based decision analysis models such as ours provide a mechanism for integrating patient preferences with probabilistic information in a systematic and explicit way. However, as our pilot results show, the decisions patients actually make will not necessarily concur with the “optimal” decision yielded by the model. This is an area worthy of further exploration as more evidence becomes available during subsequent phases of research.

Clearly, the ultimate success of the development process can only be determined by a robust evaluation of the decision support tool such as is planned for subsequent phases of research. The difficulties with the evaluation of decision support tools have been well discussed by Entwhistle,²⁸ but there remain considerable challenges in assessing the benefits of such interventions. Entwhistle's overview argues that the outcome measures depend to an extent upon the intended outcome of the decision aid itself and its underlying theoretical basis.

While our tool incorporates a decision analysis (and hence draws upon expected utility theory), we recognise that “better” decisions are not necessarily those that coincide with the predictions of expected utility theory.¹¹ Nor can we measure “better” decisions simply in terms of traditional clinical outcomes (such as strokes prevented, warfarin induced bleed) as patients may have considerations other than reducing their risk of suffering an adverse event (such as the quality of life associated with treatment itself). Nonetheless, the impact of decision support tools on health outcomes is clearly an important consideration which has implications for the achievement of public health goals.²⁹

A number of “patient” scales have been developed for evaluating decision support tools underpinned by different notions of “better” decision making—for example, decision conflict is defined as a state of uncertainty about a course of action. O'Connor and colleagues used this framework in developing their decision conflict scale, a 16 item instrument which sets out to explore the patient's uncertainty about choosing among alternatives, the factors contributing to uncertainty, and perceived effective decision making.³⁰ Other proxies used for better decisions include patient satisfaction³¹ and lack of anxiety surrounding decision making.³²

Furthermore, there are other potential benefits of such tools, such as clinical and patient behaviour change and compliance. Perhaps an obvious way in which the impact of decision tools might be assessed is in how they affect patient behaviour and the utilisation of services. This issue is discussed by O'Connor and colleagues in their systematic review of decision tools, which concluded that the effects on utilisation varied depending on the clinical problem addressed.³ In trials involving decisions about major surgery such as coronary revascularisation, prostatectomy, and mastectomy, decision aids have reduced the preference for the more intensive treatment by 21–42%. On the other hand, no significant effects on utilisation or uptake were found with decisions involving screening for breast cancer genes, prenatal testing, and hormonal replacement therapy. Such effects and their direction are likely to depend not only on patient preferences, but also on underlying levels of (over or under) utilisation.

In the case of warfarin therapy for AF, there is evidence of potential undertreatment in terms of the number of strokes potentially prevented, but this also needs considering in the context of the effects of treatment (involving monitoring, clinic visits, and potential lifestyle changes) on quality of life. It is interesting to note that, of the seven patients who went through the tool in our pilot study, the final decision of five of them was not to be treated with warfarin, indicating that, for them, the benefits of treatment did not outweigh the costs.

CONCLUSIONS

The DARTs project was a development and feasibility study with the aim of producing a prototype decision aid for anticoagulation of patients with AF to prevent stroke. It sought to develop a tool to support “better” shared decisions in an area where the implementation of the evidence base has been criticised. There is an increasing recognition of the need to develop decision making in the face of uncertainty which incorporates the patient. This tool is one means of doing so, with an additional benefit that it brings the evidence base to the consultation, giving ready access to it, not only for the patient, but also for the clinician.

We have shown that it is feasible to take elderly patients through a computerised decision support tool, but it requires time and expertise to use. Patient numbers are such that, for now, it is most likely that the tool will best be used by a small number of clinicians who have developed experience of its use and can maintain their skills. The next planned stage is an efficacy study to assess whether the tool can work in ideal circumstances before a pragmatic randomised controlled trial of effectiveness is undertaken. This will involve three arms comparing the explicit tool as described here, an implicit tool (which uses the personalised risk factor component and a foreshortened shared decision making component, but excludes the standard gamble and decision analysis), and evidence based guidelines. In view of the findings of the testing and development phases regarding training needs and the time to deliver the tool, these will be delivered in a central clinic setting by trained clinicians following referral by the patient's GP, with ongoing management returning to the referring GP with a recommendation for treatment. The primary outcome measure will be the decision conflict scale,³⁰ but we will also assess secondary outcomes including maintenance of the decision, warfarin control, and clinical outcomes.²⁸

In a resource constrained healthcare system, the benefits of tools designed to promote informed or shared decision making have to be evaluated relative to those of other healthcare interventions.²⁹ While progress has been made towards developing patient centred scales such as the decision conflict scale, further research is required into how the wider resource implications of decision support tools are to be assessed. There may be major tensions between individually informed treatment decisions and, for example, public health or health service goals. While further discussion of these issues is beyond the scope of this paper, the increasing development and use of decision support tools and the shift towards patient empowerment raise a multitude of questions worthy of detailed research in an area in which there are few robust studies to date.