Article Text
PDF
Abstract
Introduction There is an increasing need to assess the value of safety improvements to society. Concerns exist, however, as to what extent standard health economic methods appropriately reflect this value because these methods do not typically incorporate the non-health or extra-consequentialist value of avoiding healthcare incidents, which may—for example, be associated with a decreased trust of patients and citizens in healthcare systems and providers.
Objectives (1) To identify health and non-health attributes of safety from the literature and (2) to prioritise those that are considered most important by healthcare decision-makers and could be included in a subsequent conjoint analysis to determining the relative value of safety interventions and the willingness to pay of decision-makers.
Methods A literature review and 25 semistructured interviews have been conducted with healthcare decision-makers experienced in safety management, considering a general healthcare, Methicillin-resistant Staphylococcus aureus (MRSA) and sharps injuries context.
Results The literature review showed that in addition to likelihood of an incident and its direct medical and cost consequences, factors such as preventability, dread, controllability and trust in safety devices or systems affect the value of safety and decision-makers' willingness to pay. The interview results consistently indicated that “preventability of healthcare incidents”, “health consequences”, “financial consequences” and “trust in safety systems/devices” are the most important attributes across all contexts. In addition, context-specific attributes were identified.
Conclusion A set of four common and two context-specific attributes, including health and non-health aspects of safety, was identified. The next step is to attaching appropriate levels to these attributes and to incorporate them into a series of case studies among various groups of decision-makers, healthcare professionals, patient groups and the general public.
- Safety
- economic evaluation
- conjoint analysis
- research methodology
- decision-making
Statistics from Altmetric.com
Several studies have shown that healthcare is a “risky business”,1–9 as illustrated by estimates of adverse event rates in hospitals of between 1.6% and 11.7% in the UK and 3.7%–17.7% in the USA, and that healthcare incidents draw heavily on health resources.10–14 In the UK—for example, healthcare incidents cost the NHS up to £2 billion for additional hospital days.7 Furthermore, the NHS pays out approximately £400 million a year in settlements of clinical negligence claims and has a potential liability of approximately £2.4 billion for existing and expected claims. Apart from these medical and legal costs, there is also a heavy toll in human costs—for example, feelings of guilt, fear and isolation that affect those who are harmed, those who care for them, as well as those who have caused the harm.15
Because considerable amounts of resources have been allocated to monitoring and improving safety in healthcare,16 there is an increasing need to assess the value of safety improvements to society. Gray9 and Warburton16 have described the potential of standard methods for economic evaluation for this purpose. Concerns exist, however, as to what extent these standard methods of economic evaluation, assuming risk-neutrality and focussing on the direct medical costs and consequences of healthcare incidents, appropriately reflect the value of improved safety. The standard approach—for example, would overlook the fact that even in the absence of any actual adverse event happening to a particular patient, the mere notion of potentially compromised healthcare safety can lead to a loss of confidence in health organisations and their professional staff as well as to suboptimal relationships between professionals and patients. This may subsequently lead to a reduced possibility of achieving expected outcomes and to economic and social costs.17 Thus, there might be an additional (extra-consequentialist and non-health) value to improved safety over and above the direct health and economic gains associated with the actual prevention of a particular healthcare incident.
Yet, to consider how best to estimate the value of safety improvements, it is necessary to define safety and its attributes. Therefore, this study first aims to identify definitions and attributes of safety and adverse events as described in various areas of the literature including healthcare, ecological and environmental risk analysis and economic research. The second objective is to prioritise the identified attributes of safety for a general healthcare context as well as for the specific contexts of MRSA infections (exemplar of patient safety) and needlestick injuries (exemplar of staff safety) from the perspective of healthcare decision-makers. Based on the results from the literature and pilot interviews, recommendations are provided for future research into methods for the economic evaluation of safety improvements in healthcare.
Methods
Literature review
The literature was reviewed to identify definitions and attributes of safety and adverse events. A range of online databases was searched encompassing literature relating to healthcare, medicine, health and safety, ecological and environmental risk studies, economics, environmental economics and health economics to identify relevant papers (eg, Medline, Embase and Scopus). The search was limited to English literature published between 1966 and 2008. Standard search strings were devised comprising a combination of either medical subject headings or a range of text words. Publications were identified by their title and abstract. When items were considered relevant, full text copies were retrieved for more detailed appraisal.
Interviews
After receiving ethical clearance from the university's ethics committee, semistructured interviews were performed with healthcare professionals who are experienced in healthcare budgeting and decision-making at hospital trust level or similar. The aim of these interviews was to prioritise attributes of safety in different healthcare contexts, that is, a general healthcare context, the context of needlestick injuries18 ,19 and MRSA infections.20 ,21
Reasons to limit this series of interviews to healthcare professionals with experience in decision-making were mainly pragmatic, that is, approaching professionals who are presently working in the field of healthcare safety and budgeting would likely create a relatively engaged and homogeneous study sample. In addition, because the respondents were possibly more numerate than average, they would be more likely to retrieve and use appropriate numerical principles in the ranking exercises and therefore be less susceptible to potential framing effects as compared to less numerate individuals.22
Volunteers were recruited from the National Patient Safety Agency and from NHS hospitals. Because sample sizes in qualitative research are not a matter of numbers or convenience, but should be strategically focussed to collect the most appropriate and sufficiently “rich data”, the number of interviews was not determined beforehand and recruitment continued until saturation of information occurred. Potential participants were informed about the purpose and procedures of the interview and a meeting was scheduled after written informed consent had been obtained.
Before the interview, participants were sent a written version of the interview structure and a sheet providing definitions and examples of the attributes identified from the literature. Dreadfulness—for example, was described as “a subjective measure of the extent to which you perceive the incident itself or its consequences to be unpleasant, awful or frightful for the affected person(s)”; preventability was described as “the extent to which a particular healthcare incident is currently considered to be avoidable by using the safety systems and devices in place, and/or common sense”, and voluntariness was defined as “the extent to which an individual, who can be considered capable of controlling his/hers own exposure to health and safety to a certain extent, puts him/herself consciously in a situation that increases the probability of a certain healthcare incident to happen”.
Furthermore, short descriptions of the healthcare contexts to be considered (ie, general, MRSA and needlestick) were provided on three separate sheets, with each sheet addressing three topics: (1) facts and figures on the likelihood of healthcare incident(s), (2) their potential consequences and associated costs and (3) ways to prevent the incident(s) and/or treat the health consequences.23–26
At the start of the interview, respondents were given the opportunity to clarify potential issues regarding the attributes and health context descriptions. After confirming they were ready to start the actual interview, respondents were asked to think out loud about which attributes should be taken into consideration when allocating resources to patient safety interventions for a particular healthcare context. The first context to consider was the general healthcare context. After some time for free deliberation (up to 15 min), they were asked to try to translate their thoughts into a more quantitative interpretation by two exercises. In the first exercise,27 respondents had to allocate a fixed budget of 100 points over a predefined set of attributes. The second exercise27 involved the prioritisation of the six most important attributes in a hierarchical way with the relative importance of each attribute for resource allocation descending from 1 to 6. In this exercise respondents were allowed to include attributes that were not part of the literature-based set provided for exercise one when they felt these would be more appropriate for that particular context. The same procedure was then repeated for the MRSA and needlestick contexts.
Data analysis
The data gathered from the interviews were processed using a spreadsheet (Microsoft Excel 2000). Demographic data of the participants (including age, sex, professional background, current job title, number of years experience with budgeting in healthcare and number of years experience with budgeting in healthcare safety), number of points allocated to each predefined attribute (exercise 1) and the number of top six rankings (exercise 2) were analysed by descriptive statistics (ie, frequencies, means and standard deviations or medians and interquartile ranges, when appropriate). The total number of top three ranks for each attribute was calculated for every context specifically, by simply summing the number of first, second and third position ranks received by each attribute out of the six hierarchically prioritised attributes provided by the individual respondents (theoretical maximum op top three ranks is 25 and minimum is 0). In addition, it was explored (1) whether a set of attributes exists that is of relatively high importance in all three contexts and (2) whether there might by any divergence between subgroups of respondents—for example, as determined by their professional background or number of years of experience.
Results—literature review
Definitions and attributes of safety
Most safety enhancement is concerned with reducing the probability or severity of healthcare accidents. The Institute of Medicine defined patient safety as (1) “freedom from accidental injury”, (2) “medical practice consistent with current medical knowledge and best practice” and (3) “responsiveness to customer-specific values, expectations and preferences”.6 The UK's Department of Health refined these definitions to, “If safety can be defined as freedom from accidental injury, then an adverse event can be defined as an injury caused by medical management rather than by the underlying disease or medical condition of the patient, and a preventable adverse event can be defined as an adverse event attributable to error. Finally, a negligent adverse event can be defined as a subset of a preventable adverse event that satisfies legal criteria used in determining negligence”.8
It is important to realise, however, that safety is not necessarily synonymous with eliminating all potential adverse events. Where adverse events can be seen as a “failure” of an individual or a system to provide an intended “safe” treatment, healthcare safety rather refers to policies or institutional practices that are implemented based on explicit risk assessments but accepts a certain degree of risk associated with them. As such, an adverse event resulting from a medication error—for example, giving the incorrect dose of morphine, is different from an adverse outcome that is the result of a planned treatment strategy that to a certain extent is inherently risky even if the procedure is carried out in accordance with the institutional safety policy, especially if it concerns complex procedures. Or, as Sir Cyrill Chantler described it, “Medicine used to be simple, ineffective and relatively safe. Now it is complex, effective but potentially dangerous”. On a similar note, Barr (Barr DP. Hazards of modern diagnosis and therapy: the price we pay. J Am Med Assoc 1955;159:1452-6.) defined adverse events as “the price to pay for modern diagnosis and therapy methods”.
When further zooming in on the impact of adverse events, we find that minor adverse events are defined as those events that typically have a clinical impact involving fear, discomfort and/or pain, whereas major adverse events likely involve exposure to unnecessary and highly dangerous risks that end up causing serious harm or injury. Both types of adverse events, however, may generate important press releases, legal and emotional effects, and influence citizen's perceptions of the quality of medical care they receive and their trust in the work performed by health professionals.17 Finally, adverse events may happen to patients and to healthcare providers or others.
Studies on risk perception demonstrated that people have a broad conception of risk, which is qualitative and complex and brings considerations such as uncertainty, dread, catastrophic potential, controllability, voluntariness, equity, risk to future generations, and so forth, into the equation.28 ,29 This “contextualist conception” of risk and safety places probabilities and consequences on the list of relevant attributes along with the before mentioned contextual parameters and postulates that risk and safety are characterised by some combination of these attributes.30–32 It further suggests that attributes of interest for the non-market good “safety” could include the characteristics of hazards as perceived by consumers.28 ,33 ,34 Following Lancaster's theory of demand, the perceived characteristics of hazards could enter into consumer utility functions when evaluating safety trade-offs and become partial determinants of the willingness to pay (WTP) for safety, next to the more traditional socioeconomic variables as emphasised in the normative economic literature.34–36
In summary, the published scientific literature on healthcare and environmental safety revealed the following attributes of safety28–30 ,37:
Likelihood of the healthcare incident (eg, the number of minor and major adverse events per year associated with a certain procedure);
Financial consequences (eg, to the patient, the care provider, the care organisation(s) involved, the healthcare system and society);
Health consequences (eg, risk of death for the average person and/or for the persons at highest risk);
Time between exposure and health effects (eg, immediate or delayed occurrence of consequences);
Voluntariness of being in the “risky” situation (eg, the extent to which an individual may be able to control one's own exposure to health and safety risks);
Preventability of the healthcare incident (eg, scientific understanding of the root causes of a particular hazard and how to eliminate these);
Dreadfulness of the incident and its consequences/the catastrophic potential of a particular hazard;
Controllability of the healthcare incident and its consequences (eg, scientific understanding of the health effects and how to deal with these);
Trust in the safety systems/devices to manage the risk;
Equity of the risk for a particular healthcare incident among the total population.
This list indicates that the potential benefits from improved safety are not solely consequential to the reduced likelihood of experiencing a healthcare incident and its associated direct medical and cost implications. According to Niven (2002),38 this explains why so little economic evaluations of safety interventions have been successful in adequately supporting decision-making regarding healthcare safety. Moreover, because valuing benefits is well recognised as more problematic than valuing costs, this may further explain the overreliance on costing studies that exists in safety decision-making and research.38
Results—interviews
Study sample
The study sample consisted of 25 healthcare professionals (mean age 45.2 years (SD 3.7 years); 76% females) from different backgrounds (eg, public health, nursing, management and administration), but all with a specific interest in safety. They had on average 10.3 (4.9) years experience in healthcare budgeting and 5.7 (3.2) years in safety-related budgeting. Most (n=22; 88%) of respondents gained this experience as board member of an NHS hospital or foundation trust, often as patient safety manager (n=15; 68%).
Relative importance of attributes
Of the 10 predefined attributes, “preventability” is considered to be most important, with a median score of 30 points and a mean (SD) of 30.2 (15) points out of a theoretical maximum of 100 points. The attributes “probability”, “dreadfulness”, “health consequences”, “controllability”, and “trust” all showed medians of 10 points and are ordered on relative importance by their corresponding means (SD) of 12.5 (13.4), 12.2 (11.9), 11.1 (10.5), 8.9 (9.3) and 7.8 (8.5). The attributes “financial consequences” and “impact on equity” both scored medians of five points and means (SD) of 7.2 (6.8) and 4.9 (5.9), respectively, whereas the attributes “timing” and “voluntariness” returned medians of zero and corresponding means (SD) of 2.8 (3.8) and 2.4 (2.9) points (see table 1).
Allocation of a fixed 100-points budget over the 10 predefined attributes
Completeness and prioritisation of attributes by context
General healthcare context
Seventy-six per cent of respondents regarded the predefined set of attributes as complete for application to a general healthcare context. Attributes that were suggested as additions or substitutions for some of the predefined attributes include “adverse effects on individual staff involved in the occurrence of an incident” and “adverse effects on the profession as a whole and professional values of staff” and “perceived risk of the incident”. Other aspects that should perhaps not be considered attributes in themselves, but which may emphasise the breadth of the cost attribute, are “cost of preventing the incident happening” and “ease of implementing new safety devices or systems”.
The predefined attributes “voluntariness” and “timing” were considered to be redundant by ≥50% of respondents and were most likely to be sacrificed for the new attributes mentioned above.
Eighteen respondents (72%) ranked the attribute “preventability” at place one, and three persons ranked “dreadfulness” highest. Rankings 2–6 showed a more scattered pattern. In table 2, the total number of top three rankings is presented for each attribute. The six attributes with the most top three ranks, as shown in figure 1, are preventability (n=24), likelihood of the incident (n=12), dreadfulness (n=10), controllability (n=9), health consequences (n=8) and trust (n=6).
Overview of total number of top three rankings by context
Overview of the six attributes with the highest total number of top three rankings by context. MRSA, methicillin-resistant Staphylococcus aureus.
MRSA context
Seventy-two per cent of respondents consider the predefined attributes to be complete for the MRSA context. The same new attributes have been suggested for this context as for the general healthcare context. However, they appear in the top six ranking more often again at the cost of the attributes “timing” and “voluntariness”.
With 52% of respondents (n=18) considering preventability as the most important attribute of safety, this attribute again dominates the rankings, although not a strong as in the general healthcare context. The attribute “trust in safety systems or devices” is ranked highest by 24% (n=6) respondents. Furthermore, the attributes “health consequences”, “financial consequences”, “adverse effects on individual staff”, and “ease of implementing safety systems/devices” received number one rankings (table 2). The six of attributes with the most top 3 rankings are preventability (n=20), controllability (n=15), trust (n=12), health consequences (n=12), likelihood of the incident (n=5), and financial consequences (n=4) (figure 1).
Sharps injuries context
Sixty-eight per cent of respondents regard the predefined set of attributes to be complete for the sharps injuries context. The same new attributes have been suggested in this context; however, these were suggested to replace “timing” and “dreadfulness”, whereas “voluntariness” gets relatively more weight (table 2). Nevertheless, “preventability” received the most number one rankings (n=13), and the top six attributes includes to a large extent the same attributes as found for the general healthcare and MRSA contexts (see figure 1), being preventability (n=20), voluntariness (n=11), controllability (n=10), trust in safety systems/devices (n=9), health consequences (n=6) and financial consequences (n=5).
Discussion
Following concerns as to what extent standard methods of economic evaluation appropriately reflect the value of improved safety, we aimed to identify the health and non-health attributes of healthcare safety and to gain understanding of which potential attributes are considered important by healthcare decision-makers in various health and safety contexts. The literature review yielded 10 potential attributes of safety and showed that in addition to likelihood of an incident and health consequences, such factors as uncertainty, dread, controllability and voluntariness are likely to have an impact on the value of improved safety and WTP for that. Approximately 70% of the respondents considered the literature-based set of attributes to be complete, and although additional attributes were suggested, these did eventually not reach enough top rankings to be included in the overall top six of attributes. The interviews consistently showed that “preventability of the healthcare incident”, ”health consequences”, “financial consequences” and “trust in safety systems/devices” are regarded of relatively high importance across all contexts. In addition, two context-specific attributes were defined for each context.
From our relative homogeneous sample, no apparent divergence was observed between subgroups of respondents as—for example, determined by their professional background, years of experience, age or sex. Therefore, this set of attributes seems valid for use in a conjoint analysis to determining the cost–benefit of safety interventions and WTP (or willingness to invest) of healthcare and safety decision-makers. Nevertheless, other stakeholder groups within the healthcare safety field, such as patient groups, frontline healthcare staff or the general public, may have other opinions on the relative importance of the various safety attributes. Future studies should therefore investigate to what extent the results of this study apply to other groups to ultimately base decision-making on the combined views of all relevant stakeholders. Focus groups with participants from different stakeholder groups might be a useful method for investigating this, not in the least because previous studies showed that a group deliberation process helps participants to correct potential misconceptions about safety hazards and come to a common understanding regarding the relative importance of safety characteristics.37
Once a set of attributes is agreed on by all stakeholders, conjoint analysis could be used to estimate the value of safety improvements. Because conjoint analysis allows for assessment of multiple dimensions of healthcare innovation, it is increasingly suggested as an alternative to the standard economic evaluation, which is usually based on one single measure of outcome.39 In the field of risk assessment and diagnostics—for example, conjoint analysis has already shown that both physicians and the general public attach significant utility to aspects of diagnostics other than their pure clinical value.40–42 When applying conjoint analysis to a healthcare safety-specific context then, based on the results of this study, one could define attributes and associated levels to be traded off by a healthcare decision-maker. Appendix 1 illustrates this for an hypothetical programme to manage hospital-acquired MRSA infections.
Because cost is included as an attribute, this conjoint analysis would also allow for indirectly estimating WTP for improvements in individual attributes.39 Estimates of WTP for improvements in particular attributes of safety are of great importance for health policy because they permit individual preferences to be expressed in the non-market domain. As such, they can be used to provide guidelines for structuring the NHS budget. Traditionally, however, the means for improving healthcare safety, while affecting health, have come from sources that are not necessarily incorporated in the health budget. Thus, a first use of estimates of WTP for risk reduction is to improve the allocation of the budget between health and non-health components. Second, within the health domain, one (eg, health insurers or procurement agencies) may want to trade off attributes against each other. For example, the potential preventive benefits of a particular medical device may be traded off against curative benefits of another. However, because consumers cannot be forced to take advantage of preventive devices made available by industry or others, considerations of relative effectiveness need to be complemented by WTP estimates indicating whether individuals at risk value these devices to a sufficient degree as to actually take advantage of them.
Clearly, the definition of attributes and choice of levels needs careful consideration and should be pilot tested and validated before it can be used—for example, in the format of a self-administered discrete choice experiment. As such, we do not suggest the example described in Appendix 1 to be a final result of this study but merely as an illustration of our proposed approach. The next step should be to define levels for each attribute that are understandable, plausible and tradable to people.39 For example, levels of health consequences may be as simple as “alive” and “death”, or refer to specific stages of severe, moderate or minor health consequences. For attributes as “controllability”, “preventability” and “voluntariness” however, defining appropriate levels may prove more challenging. Furthermore, indirectly estimating WTP via a cost attribute should be undertaken cautiously—for example, as regards ordering of the cost levels because this may in itself influence the WTP estimate.43–45
If safety attributes are to be considered in addition to expected health gain, then we need to understand and be explicit about the trade-off of such attributes against simple health maximisation. The role and influence of non-health benefits on WTP has only rarely been studied,46 and only very few WTP studies conducted in the health sector have used their results in a cost–benefit analysis. Yet, this is a necessary next step to inform resource allocation decisions in the context of a finite budget. Methods for aggregating sample WTP values to provide an estimate of population level benefit that could be fed into cost–benefit analysis have recently been presented47 and non-health benefits can in principle be incorporated in the standard quality-adjusted life year framework. Although this has, as yet, not been done,46 it would facilitate a more meaningful comparison with other healthcare interventions, including those outside of the patient safety domain. This notion is of utmost importance because concentrating on safety only (or any other single niche of medicine) will compromise overall population health if it comes at the expense of more threatening health problems. As pointed out by Woolf,48 people are less likely to die of an overdose of warfarin (a lapse in safety) than of not receiving warfarin at all (a lapse in quality). Therefore, resources allocated to improve safety should be proportionate to the resources dedicated to improve other aspects of quality of care. By developing and improving on a common metric, which captures all relevant costs and consequences of a safety intervention to contrast its relative impact to that of preventive services and other quality improvements, we can support the pursuit of efficient resource allocation and optimise population health. After all, “patients deserve far more than not to be harmed by their physicians”.48
In conclusion, the identification of a common set of health and non-health attributes of safety may be considered as an important step forward to applying a formal conjoint analysis in the field of healthcare safety. The next step is to attaching appropriate levels to these safety attributes and to incorporate them into a series of surveys among various groups of decision-makers, healthcare professionals, patient groups and the general public. The combined survey results may guide structuring healthcare budgets and allocating scarce resources between health and non-health components with the aim to improve healthcare safety in the most cost-effective way.
Acknowledgments
The authors would like to thank all respondents for the time and effort they put into the interviews, and Drs Beverly Norris, Tanya Huehns and Ewen Cummins from the National Patient Safety Agency of England and Wales for their intellectual contributions and critical appraisal of this research from the start. Furthermore, we would like to thank two anonymous reviewers for their insightful comments on earlier draughts of the paper. The opinions expressed in the paper are the sole responsibility of the authors.
Appendix 1 Applying conjoint analysis to investigate the value of a hypothetical programme to manage hospital-acquired MRSA infections
When considering a hypothetical programme to manage hospital-acquired MRSA infections, based on the results of this study, one could think of the following attributes and levels:
Attribute 1: “The likelihood that a patient admitted to your hospital will acquire an MRSA infection equals [xx]%” (level values may range from 1% to 100%).
Attribute 2: “For the majority of patients that acquire an MRSA infection in your hospital, the health consequences can be described as […]” (levels may be described as “no health consequences” or “an asymptomatic infection” or “an symptomatic infection that is successfully treated” or “an symptomatic infection leading to permanent impairment” or “death”).
Attribute 3a: “The total financial consequences of one MRSA infection to society are circa £[xx]” (level values may range from £0 to £ 1,000,000).
Attribute 3b: “The total cost to be paid from your organisational budget per MRSA infection are £[xx]” (level values may range from £0 to £1,000,000).
Attribute 4: “When the current safety devices and systems available in your hospital are optimally utilised [xx]% of MRSA infections could be prevented” (level values may range from 0% to 100%).
Attribute 5: “When the treatment and management options currently available in your hospital to control spread of MRSA are optimally utilised, the controllability of MRSA could be described as […]” (levels may be low, medium or high).
Attribute 6: “The trust that citizens have in the safety measures applied in your hospital to prevent and manage MRSA can be described as […]” (levels may be low, medium or high).
The analysis would subsequently involve the estimation of a utility function of the form:
where ΔBenefits is the change in utility in moving from one safety scenario to another—for example, as the result of the new safety programme to manage MRSA.
The relative importance of the different attributes can then be given by αj (j=1, 2, 3, 4, 5, 6).39 The ratio of the parameters would show the trade-offs between the attributes.
Consequently, α2/α3b indirectly estimates the decision-makers' marginal WTP for an improvement in health outcomes, α4/α3b the marginal WTP for one level increase in “preventability”, etc.
References
Footnotes
Competing interests None.
Ethics approval This study was conducted with the approval of the Brunel University Ethics Committee.
Provenance and peer review Not commissioned; externally peer reviewed.