Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Since the earliest days of the COVID-19 vaccines, voluntary uptake of the vaccine by healthcare workers has been below optimal levels in many countries and across different healthcare systems.1–5 Health systems have implemented diverse initiatives to both increase healthcare worker motivation to vaccinate and close intention–behaviour gaps. Despite substantial effort to promote COVID-19 vaccination among healthcare workers in the USA, for example, nearly a third were not fully vaccinated as of September 2021.6 Many employers and governments have since considered or implemented vaccine mandates,7–9 with steep penalties for non-vaccination including termination.10 Notably, mandates for employees of hospitals, health systems and other healthcare entities receiving federal funds were recently upheld by the US Supreme Court.11
In this pressing policy context, scientists from several disciplines (behavioural science, improvement science, delivery science) are looking to evidence from other employee vaccination and occupational health initiatives to inform ongoing efforts to promote healthcare worker COVID-19 vaccination. Prior efforts to increase influenza vaccination uptake among healthcare workers at the health system level are an important part of that evidence. In this issue of BMJ Quality & Safety, Liaqat and colleagues12 analyse health system responses to performance-based financial incentives for healthcare worker influenza vaccination that were introduced by England’s National Health Service (NHS) in 2016/2017. Using data from eight vaccination seasons that span the period before and after the introduction of these incentives, the authors demonstrate in a convincing manner that the pay-for-performance scheme led to an increase in influenza vaccination rates among frontline staff. However, they also provide compelling evidence that the incentives had ‘threshold’ effects—vaccination rates tended to cluster just above the 70%–80% thresholds established by the NHS to receive incentive payments.
These threshold effects have important implications for the design and implementation of performance-based financial incentives. First and foremost, the authors note that an incentive scheme that produces threshold effects may be suboptimal if improvements are clustered around the threshold versus realised across the distribution of system-level performance. Second, the authors make a useful distinction between competing mechanisms that could produce threshold effects: an ‘effort’ mechanism, in which health systems change their practices and policies in order to move the needle on the incentivised metric (ie, vaccination coverage among frontline staff), versus a ‘manipulation’ mechanism, in which gaming occurs through changing numerators or denominators in order to meet thresholds. Third, the authors review three policy options for performance-based financial incentives given their robust finding of threshold effects. Policymakers can: (1) recognise but ultimately ignore threshold effects, given the overall improvement in performance metrics that they bring about; (2) try to improve the design (and, we would argue, the implementation) of performance incentive schemes to maximise impact and reduce unintended consequences; or (3) abandon performance-based incentives in favour of other strategies (eg, non-incentivised behavioural interventions, mandates).
As behavioural scientists who have designed and evaluated various interventions to change behaviours relevant to infectious disease prevention,13–16 we find the large effect of performance-based financial incentives on vaccination rates as well as the evidence of threshold effects to be highly noteworthy. However, we also found ourselves wanting to ‘look under the hood’ of their analyses and to think more critically about their policy implications. Broadly, the questions raised by this study (which apply to COVID-19 vaccination also) fall under three categories: What strategies were used to achieve higher vaccination rates? Were there unintended consequences of incentivising influenza vaccination alone? How might we improve the design of these pay-for-performance schemes?
How exactly do hospital trusts achieve higher employee vaccination rates in response to incentives?
Liaqat et al find very compelling evidence of threshold effects in response to the NHS performance-based incentive schemes to increase healthcare worker influenza vaccination. According to the effort hypothesis, this suggests that hospital trusts are doing something to change employer behaviour. While the specifics of those somethings are understandably not within the scope of the original analysis, we are very interested in what exactly the trusts are doing to meet the incentive thresholds. Are the staff being offered incentives to get vaccinated? Are additional vaccine clinics being scheduled by hospital trusts? Are soft or hard mandates being introduced? Understanding the specific approaches that hospital trusts use—that is, the results of their ‘effort’—is a crucial next step in this quality improvement inquiry, no less important than documenting the overall impact of the performance incentives programme and the threshold effects it generated. From a behavioural science perspective, this can provide timely and useful insights to employers and policymakers on how to achieve higher vaccination rates.
Documenting the programmes and policies that hospital trusts implemented in response to performance incentives would allow for a robust comparison of trusts that did versus did not meet thresholds, and of trusts that just met thresholds versus exceeded them by a comfortable margin. The quality improvement literature offers many examples of rigorous documentation of employee vaccination initiatives,17 18 including COVID-19 vaccination.19 20 Such an investigation could also help confirm the effort hypothesis, uncover additional mechanisms driving the threshold effects, and inform ongoing quality improvement efforts around employee influenza vaccination.
Are there unintended consequences to pursuing a specific quality target?
A common concern about pay-for-performance schemes is that they may reduce the attention that organisations devote to outcomes that are not incentivised, and thereby result in a deterioration in those outcomes.21 Such a possibility is not examined by Liaqat and colleagues, who focus solely on influenza vaccination rates among frontline staff. While this concern may be most warranted when incentives focus on specific care quality metrics, it remains possible that hospital trusts which allocated greater resources to increasing influenza vaccination rates may have also reduced resources allocated to employee benefit programmes or to other activities that affected employee or patient health outcomes. For example, since vaccination against both influenza and COVID-19 is important at this time, offering incentives for achieving influenza vaccination targets only may shift resources or attention away from COVID-19 vaccination efforts (or vice versa). It would be reassuring if additional research were to show that there are few unintended consequences to pay-for-performance schemes like the one studied here. More generally, this points to the value of adopting integrated approaches that incentivise influenza vaccination and other important health behaviours. Additionally, learning about how the provider organisations spend the performance payments—which are typically £1–£2 million per organisation—would also be useful, as these payments could be used to improve employee well-being or health system performance.
Have opportunities to improve incentives design truly been ‘exhausted’?
In their discussion of policy implications, Liaqat et al note that options to improve the design of incentives ‘seem to have been exhausted’. We disagree. While the authors’ concerns about limits to incentive amounts that society is willing to tolerate and the dangers of overly complex incentive schemes are very well taken, we also think the recent literature on incentives suggests several alternative designs that would be both feasible and potentially effective while minimising threshold effects. In particular, different approaches to goal setting seem particularly promising. Prior studies informed by goal setting theory22 have demonstrated that personalised or self-set goals are both more motivating and preferred by individuals; it is reasonable to assume that these principles might apply at the organisational level as well. NHS could suggest a performance threshold of, for example, 10 percentage points above the previous year’s performance for those trusts that were below 50%, and 5 percentage points above the previous year’s performance for those trusts that were above 50%. Hospital trusts could then use that guidance to set their own ‘personalized’ threshold goal, an approach that has been shown to be successful for individual physical behaviour change.23–25 While this approach might still generate threshold effects at each trust’s target amount, the increases in employee vaccination coverage should be realised across the full range of coverage rates rather than clumping at the higher end of the distribution. It is important to note here that trust-set goals are different from the partial payment thresholds that were evaluated in Liaqat et al. We suspect that partial payments for lower targets did not produce threshold effects (in contrast to the strong threshold effects for the full payment targets) due to lower salience and reduced motivation to receive a small, ‘partial’ reward as opposed to a larger reward. Other design innovations that could be trialled here include social proof and other normative interventions that helped health trusts place their employee vaccination rates in the context of peers.26 27
Performance-based financial incentive schemes have been used to influence the behaviour of healthcare providers in numerous settings, with the ultimate goal of improving the quality of healthcare in an efficient manner. Liaqat et al demonstrate that for the case of influenza vaccination among frontline staff, providing rewards conditional on achieving specific targets resulted in a large increase in vaccination rates while also leading some health systems to curtail their effort once they reach the incentivised threshold. Policymakers may well find such a trade-off to be acceptable, particularly if performance-based incentives result in ambitious targets being met. This work is relevant for the COVID-19 pandemic, as there is a vital need for approaches that can increase vaccination rates among employees of health systems and other organisations. Aside from the overall effects of performance-based incentives, understanding the specific demand-side and supply-side actions that employers take when they are offered such incentives (as well as the effectiveness of those actions) is vital for furthering the evidence base on how we can achieve increases in vaccination rates. Given the presence of threshold effects, policymakers should also consider alternative designs of performance-based incentives that do not leave low performers behind but continue to motivate high performers to achieve targets. The behavioural science toolkit offers many compelling strategies to boost quality improvement efforts and increase the impact of pay-per-performance schemes.
Patient consent for publication
This study does not involve human participants.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Commissioned; internally peer reviewed.