You are here

  • Home
  • Archive
  • Volume 31, Issue 8
  • Cost-effectiveness of a quality improvement project, including simulation-based training, on reducing door-to-needle times in stroke thrombolysis

Article Text

Article menu
Original research
Cost-effectiveness of a quality improvement project, including simulation-based training, on reducing door-to-needle times in stroke thrombolysis
  1. Soffien Chadli Ajmi1,2,
  2. Martin W Kurz1,3,
  3. Hege Ersdal2,4,
  4. Thomas Lindner4,5,
  5. Mayank Goyal6,
  6. S Barry Issenberg7,
  7. Corinna Vossius8
  1. 1 Department of Neurology, Stavanger University Hospital, Stavanger, Norway
  2. 2 Faculty of Health Sciences, Universitetet i Stavanger, Stavanger, Norway
  3. 3 Institute of Clinical Medicine, University of Bergen, Bergen, Norway
  4. 4 Department of Anaesthesiology and Intensive Care, Stavanger University Hospital, Stavanger, Norway
  5. 5 The Regional Centre for Emergency Medical Research and Development, Stavanger, Norway
  6. 6 Department of Radiology and Clinical Neurosciences, University of Calgary, Calgary, Alberta, Canada
  7. 7 The Gordon Centre for Research in Medical Education, University of Miami Miller School of Medicine, Miami, Florida, USA
  8. 8 Centre for Age-Related Medicine, Stavanger University Hospital, Stavanger, Norway
  1. Correspondence to Dr Soffien Chadli Ajmi, Department of Neurology, Stavanger University Hospital, Stavanger, Norway; soffiena{at}yahoo.com

Abstract

Background Rapid revascularisation in acute ischaemic stroke is crucial to reduce its total burden including societal costs. A quality improvement (QI) project that included streamlining the stroke care pathway and simulation-based training was followed by a significant reduction in median door-to-needle time (27 to 13 min) and improved patient outcomes after stroke thrombolysis at our centre. Here, we present a retrospective cost-effectiveness analysis of the QI project.

Methods Costs for implementing and sustaining QI were assessed using recognised frameworks for economic evaluations. Effectiveness was calculated from previously published outcome measures. Cost-effectiveness was presented as incremental cost-effectiveness ratios including costs per minute door-to-needle time reduction per patient, and costs per averted death in the 13-month post-intervention period. We also estimated incremental cost-effectiveness ratios for a projected 5-year post-intervention period and for varying numbers of patients treated with thrombolysis. Furthermore, we performed a sensitivity analysis including and excluding costs of unpaid time.

Results All costs including fixed costs for implementing the QI project totalled US$44 802, while monthly costs were US$2141. We calculated a mean reduction in door-to-needle time of 13.1 min per patient and 6.36 annual averted deaths. Across different scenarios, the estimated costs per minute reduction in door-to-needle time per patient ranged from US$13 to US$29, and the estimated costs per averted death ranged from US$4679 to US$10 543.

Conclusions We have shown that a QI project aiming to improve stroke thrombolysis treatment at our centre can be implemented and sustained at a relatively low cost with increasing cost-effectiveness over time. Our work builds on the emerging theory and practice for economic evaluations in QI projects and simulation-based training. The presented cost-effectiveness data might help guide healthcare leaders planning similar interventions.

  • simulation
  • quality improvement
  • cost-effectiveness
  • medical emergency team

Data availability statement

All data deemed relevant to the study are included in the article or uploaded as online supplemental information. Additional supporting data are available on reasonable request. Data are available upon reasonable request.

https://doi.org/10.1136/bmjqs-2021-013398

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Stroke is the second leading cause of death and disability worldwide.1 Despite the current declining incidence of stroke, population growth and ageing are likely to contribute to an increase of people at risk of stroke.2 Reported costs of post-stroke care range from US$883 to US$4850 per patient per month.3 In a policy statement by the American Heart Association, an increase in stroke prevalence is expected to cause direct annual stroke-related medical costs to increase from US$71.55 billion in 2012 to US$183.13 billion in 2030.4 The economic burden of stroke is thus substantial and will likely increase.

    In acute ischaemic stroke, rapid revascularisation through intravenous thrombolysis alone or in combination with endovascular treatment is crucial to improve outcomes.5–9 Minimising treatment delays is thus important to reduce the overall burden of stroke. Several authors have reported different methods, including the use of simulation-based training, to reduce treatment delay in stroke thrombolysis.10–13 Multicomponent quality improvement (QI) initiatives and developing efficient stroke care pathways are among the grade 1A recommendations in the 2019 American Stroke Association guidelines.14 However, there are few reports that describe the cost-effectiveness of implementing and sustaining QI initiatives in stroke thrombolysis.15 16 This is also true for simulation-based training interventions. Even though simulation-based training is widely used and resource intensive, examples of economic evaluations are scarce.17 18 Frameworks for cost-effectiveness analyses exist both for QI and simulation-based training.19 20 As few examples of their practical application exist, a formal cost-effectiveness analysis of a QI project including simulation-based training could contribute as a practical example in both fields and assist healthcare leaders to prioritise similar initiatives.

    At Stavanger University Hospital, developing an efficient stroke care pathway for stroke thrombolysis, together with in situ simulation-based team training sessions, was followed by a considerable reduction in median door-to-needle times (27 to 13 min) and improved patient outcomes (6.89 deaths averted) 13 months post-intervention.21 In the present study, we aim to retrospectively determine the costs of implementing and sustaining the QI initiative (including costs of simulation-based training) and relate the costs to the observed effects in a cost-effectiveness analysis. In addition, we aim to estimate potential future cost-effectiveness over a period of 5 years post-intervention. Our primary outcome measures are costs per minute reduction in door-to-needle time per patient and costs per averted death post-intervention, compared with baseline performance.

    Methods

    Setting and context

    Stavanger University Hospital is one of the largest stroke centres in Norway serving a population of 365 000. Approximately 850 patients are admitted annually with criteria-based activation of the acute stroke team (typically patients with a suspicion of acute stroke eligible for revascularisation). Of patients with a diagnosis of acute ischaemic stroke, 20% to 30% receive intravenous thrombolysis. All consecutive patients with a suspicion of stroke having received intravenous thrombolysis are prospectively included in our local thrombolysis registry. The registry contains multiple variables including patient demographics, risk factors, stroke severity, all relevant time points, 90-day all-cause mortality and modified Rankin Scale at baseline, and 3 months post-stroke.

    A QI project aiming to reduce door-to-needle times in stroke thrombolysis at our centre in order to improve patient outcomes was started in February 2017.21 Interventions included assessing readiness for QI, surveying all stroke team members, implementing relevant changes to streamline the stroke care pathway, communicating changes to stakeholders and planning in situ simulation-based team training sessions to replicate the in-hospital stroke care pathway from door to needle. Consecutive patients treated with intravenous thrombolysis 3 years pre-intervention were compared with patients treated 13 months post-intervention. A total of 399 patients pre-intervention and 190 post-intervention were included in our analysis. In the 13-month post-intervention period, a total of 26 simulation-based training sessions were completed. The outcome measures included door-to-needle times and 90-day all-cause mortality. Statistical process control charts were used as the primary means of analysis. An X-bar chart indicated a reduction in mean door-to-needle time of 13.1 min post-intervention while a variable life adjusted display chart for mortality indicated that 6.89 deaths were averted post-intervention.22 23 Further details of our study setting, context and a description of the QI interventions including simulation workflow, outcome measures and results are found in our previously published QI report.21

    Framework for cost analysis

    The method for assessing cost-effectiveness is based on several frameworks for economic analysis of both QI and simulation-based training interventions.17–20 24 25 The frameworks we followed most closely to determine costs was Levin’s ‘ingredients method’ and Zendejas et al’s framework for costing in simulation-based medical education.20 25 According to Levin, the idea behind the ‘ingredients method’ is that every intervention uses so called ‘ingredients’ or resources that have a cost, and if the ingredients can be identified and their costs ascertained, we can estimate the total cost of the intervention. Levin’s framework thus involves reflecting on the process (ie, asking “What was being attempted?”, “How and why did the outcomes occur?” and “Which actions were involved?”) to identify all ingredients involved in achieving the outcomes. Reflecting on the process of change and understanding why outcomes occur are integral parts of QI methodology and thus Levin’s framework was considered appropriate for determining the costs of our project. Zendejas et al’s framework couples Levin’s ingredients framework with specific cost components identified through a review of simulation-based medical education interventions. Zendejas et al termed the specific cost components ‘cost-ingredients’. Zendejas et al provide a list of cost-ingredient categories, examples of cost-ingredients in each category and steps for assigning monetary values to cost-ingredients. Zendejas et al’s framework was deemed particularly useful for costing the simulation-based training intervention. The main steps thus included reflecting on the process to identify and categorise a list of all relevant cost-ingredients, assigning monetary values to each ingredient and combining outcome data with costs to calculate cost-effectiveness (figure 1).

    Figure 1
    Figure 1

    Method used to evaluate cost-effectiveness of quality improvement on process and outcome measures in stroke thrombolysis*. *Blue boxes: previously published analysis; green boxes: current analysis. †Aim: reduce door-to-needle time and improve patient outcomes. ‡Planning quality improvement: September 2016 to January 2017. Implementing, sustaining and assessing the outcome measures: February 2017 to March 2018. Created by the authors.

    Methods for cost analysis

    The cost analysis involved reviewing the primary drivers for QI (key elements contributing directly to achieve the aim) and the sequence of improvement (ie, QI education, understanding the problem, developing/testing changes and sustaining the gains). Actions involved were converted to specific cost-ingredients according to Levin and Zendejas et al’s frameworks. In situ simulation-based training sessions were listed as a separate action and converted to specific cost-ingredients based on the suggestions in table 2 of the Zendejas et al framework.20 Each cost-ingredient was labelled as fixed (costs for implementing the QI project) or monthly (recurring costs for sustaining the QI project).

    After deriving all cost-ingredients, we assigned monetary values to each ingredient. For personnel costs, we identified all involved professions and assigned an hourly rate for each profession (online supplemental table 1). Based on these rates, we calculated hourly rates for meetings with different team constellations (online supplemental table 2). To estimate the number of hours spent, we reviewed all documents used in planning the project (project plans, driver diagrams etc) in addition to emails and calendars which included most meetings involving members of the QI team. For the QI education and train-the-trainer course, we used the course schedule to estimate time spent.26 27 Whenever written information on time spent was lacking, the programme directors arrived at an estimate after a discussion regarding the workload of the intervention in question (eg, estimating time spent doing a literature research pre-intervention). In these cases, we aimed to err on the side of caution, making sure to include all hours at the risk of overestimating rather than underestimating time spent.

    Supplemental material

    All unpaid time was included and costed as opportunity costs. Unpaid time was defined as personnel time spent for QI without causing any direct or extra personnel costs. Unpaid time was split into two categories based on whether it was performed during working hours (eg, a nurse spending time for QI in between patients) or outside of normal working hours (ie, voluntary spare time work). For example, time spent for employees already on-call, participating in a simulation session, was categorised as unpaid time within working hours. Unpaid time within working hours was costed using the same hourly rates as for paid time. Unpaid time spent outside regular working hours was assigned a monetary value of one-third of the hourly rates. This rate is based on previous work applying a similar proxy to cost leisure time.28 29

    For all other costs (ie, equipment, materials and subscriptions), we collected actual costs from receipts and forms for travel expenses. As most of the time spent for planning, implementing and sustaining the QI project was spent at our hospital, we did not include facility costs. We did not include the costs of depreciation of hospital equipment (computers, beds, sheets etc) as the impact of our interventions on this was probably negligible. Furthermore, potential additional healthcare system costs (eg, additional CT scans, price of additional thrombolytic medication, ICU stays etc) was not included. As our interventions primarily reduced treatment time, without altering the inclusion/exclusion criteria for thrombolysis, we did not expect significant changes in use of imaging or thrombolytic medication. Deaths averted might have led to increased healthcare system costs (eg, hospital bed days, ICU stays, other treatments etc), but the effect of this is more difficult to predict and might be partly offset by an expected overall reduction in morbidity and adverse events because of shortening the time to treatment.30 Costing was performed from the perspective of the healthcare sector as we did not include costs of informal healthcare (eg, unpaid caregiver time costs) and non-health care sectors (eg, cost of social services, housing improvements etc).31 All costs were adjusted for inflation, expressed in 2019 prices, and converted from Norwegian kroner to US dollars as per 1 November 2019. Input data depicting cost details adjusted for inflation and presented in their original currency are found in online supplemental table 3.

    Supplemental material

    Outcome measures

    The primary outcome measures were costs per minute reduced in door-to-needle time per patient, and costs per averted death. For costs per minute reduced, we presented the incremental cost-effectiveness ratio (ICER) as the total costs (fixed and monthly costs) divided by the number of minutes reduced for each patient treated with intravenous thrombolysis in the post-intervention period. Under the assumption that the monthly number of minutes saved was constant throughout the study period, we also presented monthly ICERs for door-to-needle time reduction throughout the post-intervention period. These were presented both with and without costs of unpaid time. For costs per death averted, we presented the ICER as total cost divided by deaths averted in the post-intervention period. Under the same assumption as for costs per minute reduced, we presented the monthly costs per death averted with and without costs of unpaid time. We also projected annual incremental cost effectiveness ratios over a 5-year period by estimating future fixed and recurring costs under the assumptions of unchanged fixed costs and sustained effectiveness 5 years post-intervention. In addition, through determining the minutes reduced and deaths averted per patient treated in the post-intervention period, we presented estimated ICERs for varying numbers of patients treated (50–350).

    Sensitivity analysis

    All the aforementioned outcome measures are presented including and excluding costs of all unpaid time as part of our sensitivity analysis. In addition, we present costs per minute reduced and deaths averted, during the post-intervention period, excluding costs of unpaid time within working hours (ie, time that, depending on the context, can be reallocated to QI without causing any extra costs (scenario 3)).

    Results

    Fixed costs for implementing the QI project amounted to US$44 802, while monthly costs were US$2141 when all costs were included (scenario 1, table 1). Personnel costs were the largest contributor, comprising 97.6% of fixed costs and 95.9% of monthly costs (online supplemental table 3). The estimated costs per minute reduction in door-to-needle time per patient was US$29, and the estimated costs per averted death was US$10 543 post-intervention when all costs were included (scenario 1). The cost per minute reduction and averted deaths post-intervention for all scenarios in the sensitivity analysis are shown in table 2. Monthly ICERs during the post-intervention period, including (scenario 1) and excluding (scenario 2) costs of unpaid time, are presented in figure 2.

    View this table:
    Table 1

    Costs (US$) of different components of a QI project designed to improve door-to-needle time in stroke thrombolysis

    View this table:
    Table 2

    Sensitivity analysis: costs (US$) per minute door-to-needle time reduced and deaths averted for different scenarios in a QI project designed to improve door-to-needle time in stroke thrombolysis

    Figure 2
    Figure 2

    Monthly incremental cost-effectiveness ratios during the post-intervention period of a QI project designed to improve door-to-needle times in stroke thrombolysis. Costs per (A) minute door-to-needle time reduced per patient and (B) death averted. Scenario including all costs is represented with triangles; scenario excluding cost of unpaid time is represented with circles. Created by the authors.

    Estimated annual ICERs for a projected 5-year post-intervention period including (scenario 1) and excluding (scenario 2) costs of unpaid time are presented in figure 3. Estimated costs per minute reduced in the fifth year was US$15 including and US$3 excluding costs of unpaid time (figure 3A). Estimated costs per death averted in the fifth year was US$5449 including and US$1137 excluding costs of unpaid time (figure 3B). Estimated ICERs for different numbers of patients treated in the post-intervention period are presented both including and excluding costs of unpaid time in figure 4A,B.

    Figure 3
    Figure 3

    Estimated yearly incremental cost-effectiveness ratios of a QI project designed to improve door-to-needle times in stroke thrombolysis during a projected 5-year post-intervention period. Costs per (A) minute door-to-needle time reduced per patient and (B) death averted. Scenario including all costs is represented with triangles; scenario excluding cost of unpaid time is represented with circles. Created by the authors.

    Figure 4
    Figure 4

    Estimated incremental cost-effectiveness ratios 1 year post-intervention of a QI project designed to improve door-to-needle times in stroke thrombolysis with varying numbers of patients treated with thrombolysis annually. Costs per (A) minute door-to-needle time reduced per patient and (B) death averted. Scenario including all costs is represented with triangles; scenario excluding cost of unpaid time is represented with circles. Created by the authors.

    Discussion

    The estimated fixed costs of implementing a QI project aiming to reduce treatment times in stroke thrombolysis at our centre were US$44 802 while monthly costs were US$2141. This included simulation-based training sessions. The cost of reducing the door-to-needle time by 1 min per patient was US$29 when all costs were included, and US$13 when unpaid time was excluded, while the costs of averting a single death were US$10 543 and US$4679, respectively. As effects were maintained throughout the 13-month study period, cost-effectiveness increased over time.

    Whether an investment is deemed cost-effective in a specific context depends on what the society in question is willing to pay for the observed effects. With regards to deaths averted, the most common method in healthcare is considering the societies’ willingness to pay for the quality-adjusted life years (QALYs) gained. There is no official willingness to pay threshold for QALYs in Norway, but a recent Norwegian health technology assessment suggests a threshold for patients who had a stroke of approximately 385 000 NOK per QALY (approximately US$43 000).32–34 The maximum cost scenario with costs of averting a single death in our study amounted to US$10 543. This is the equivalent of the willingness to pay for approximately 0.25 QALYs in Norway. Thus, to consider the QI project cost-effective with regards to deaths averted, each death averted in our population must have produced on average a minimum of 0.25 QALYs. The number of QALYs a single death averted produced cannot be derived directly from our data. For comparability, however, the aforementioned Norwegian health technology assessment estimated an average absolute loss of QALYs due to stroke of 5.5 (life expectancy of 12.7 QALYs minus a prognosis with stroke of 7.2 QALYs) in 70-year-old patients.34 It is thus likely that each averted death in our population (with a median age of 71) resulted in a lot more than the cost-effectiveness threshold of 0.25 QALYs.21 The value of 1 min door-to-needle time saved per patient is more complex to estimate as the effect is a potential reduction of unknown size in morbidity and mortality. A study of patients treated with intravenous thrombolysis in an Australian and Finnish cohort provides some perspective as every minute reduction in onset-to-treatment time produced on average 1.8 days of additional healthy life.9 In comparison, reducing the door-to-needle time by 1 min came at a cost of US$29 (maximum cost scenario) in our study. Even if we were not able to assign monetary values to the observed effects of this QI project (as in a cost–benefit analysis), and the effect estimates from other populations are not directly comparable, these numbers indicate that the presented costs are likely justifiable.18

    As far as we know, there are no reports assessing the cost-effectiveness of a simulation-based training intervention in stroke thrombolysis. A recent Chinese study on the cost-effectiveness of a multifaceted QI project in acute ischaemic stroke concluded that their interventions were highly cost-effective.15 However, the interventions were very different from ours as most interventions were directed at different steps of the in-hospital stroke care pathway including discharge (rather than improving stroke thrombolysis treatment). This study is thus too different from ours to directly compare. In a recent systematic review including cost analyses of seven large-scale implementations of healthcare simulation-based training programmes, only two studies (related to new-born care) compared costs to effectiveness data, and both concluded with cost-effectiveness.35–37 In 2010, Cohen et al reported cost savings from reduced catheter-related bloodstream infections after simulation-based skill training in central venous catheter insertion for intensive care residents in the USA.38 Furthermore, a study from Ireland introducing a paediatric Medical Emergency Team accompanied by weekly in situ simulation training reported the total annual cost of training to be £74 250 and savings from reduced paediatric intensive care bed days to be £801 600 per annum.39 These studies are from different contexts with different applications of simulation-based medical education and thus comparability is limited. These examples, however, imply that QI projects and simulation-based training interventions can be cost-effective and thus support our findings.

    The cost-effectiveness analysis presented here included effectiveness with regards to door-to-needle time reduction and deaths averted. There are additional aspects of effectiveness to consider: First, in our centre the QI project is still (2021) ongoing with possible residual effects that are not apparent from the 13-month post-intervention period. Under the important assumption that effects are sustainable for longer than 13 months with repeated simulation sessions, the costs of implementation are probably distributed over a longer timeframe potentially resulting in an even higher cost-effectiveness over time. An estimate of the cost-effectiveness over a longer timeframe is shown in figure 3. Second, there are several intangible effects that warrant mentioning, such as capacity building (both for QI and simulation-based training), high self-perceived usefulness scores from participants attending simulation sessions, patient participation (patients who had a previous stroke participated as simulated patients and helped shape the stroke care pathway from a patient perspective) and the potential for discovering safety threats through diagnostic simulation.40 The latter might also be useful for adapting the procedure according to context changes. For example, during the 2020 COVID-19 pandemic, the stroke simulation scenarios at our centre were adapted to specifically educate personnel in treating patients with acute stroke with communicable diseases.41 Similar QI projects might thus be beneficial for both patients and personnel with effects that are not apparent through the tangible effects presented here.

    Our study has several limitations. First, we performed a sensitivity analysis with different scenarios for costs but did not perform a similar analysis with regards to effects. Including the variation from the estimates of effect in the primary outcome measures would have resulted in wider ranges for the presented outcome measures. Second, although causality between the QI project and the effects is likely and theoretically plausible, there are important limitations (eg, due to the pre–post design) that are discussed in detail in our previous publication.21 Third, as all the interventions were implemented over a short period of time, we cannot conclude which part produced the observed effects. It is thus possible that the same effects could be achieved with a less costly or different set of interventions. Fourth, costing unpaid time outside of working hours (voluntary spare time) is a potentially complex calculation, while our proxy of one-third of hourly rates has been used in other studies, this estimate has limitations as it does not necessarily reflect everyone’s valuation of spare time. Regardless of its monetary value, accounting for spare time spent for QI is nonetheless important. Fifth, from the perspective of the healthcare sector, there might be costs related to averting deaths and costs related to non-healthcare sectors (which would be important from the societal perspective) that were not included. Sixth, optimally, a cost-effectiveness study should be performed prospectively.20 This was a retrospective study, and although most costs were accounted for through written sources, we cannot rule out that some costs might have been missed. Seventh, with regards to future projections, our QI project is a complex intervention with many variables potentially affecting future sustainability of effects. Results from the analysis of cost-effectiveness beyond the 13-month post-intervention period must therefore be interpreted with caution. Finally, in terms of direct costs, a life-saving intervention is rarely a cost-saving intervention. Following the QI intervention at our centre, in addition to a reduction in mortality, we did see a reduction in patients with the highest degree of morbidity (assessed using the modified Rankin Scale).21 In our population, this might have lessened some of the direct costs, but deaths averted would have led to higher direct costs. Therefore, cost-effectiveness must be viewed together with each society’s willingness to pay for averting a death, which in part reflects the opportunity costs in that society (ie, whether the investment could be better spent averting deaths in some other field) rather than direct cost savings.

    With regards to generalisability, there are several matters to consider. First, generalisability of the QI project and ensuing outcomes, which this cost-effectiveness analysis is based on, depends on several contextual factors. A formal analysis of contextual factors at our centre, using the Model for Understanding Success in Quality, and a detailed discussion of how these might have affected our results are found in the previously published QI report.21 42 The most important facilitating contextual factors were previous improvement efforts and having all stroke team members represented in the QI team thus increasing ownership within the different professions. Considering these factors along with known successful features of simulation-based medical education programmes might help guide the QI project’s translation to other institutions.43 Second, we might assume that cost-effectiveness is approximately proportional to the number of patients treated with intravenous thrombolysis post-intervention. In our study, 190 patients were treated with intravenous thrombolysis in the 13-month post-intervention period. Thus, centres treating more patients might achieve a higher cost-effectiveness in the same amount of time and vice versa. This is, however, only true within certain limits as several factors might affect the effectiveness of the interventions in centres with a higher or lower volume than ours. An estimate of the effect of varying numbers of annual thrombolysis treatments is shown in figure 4. Third, the hourly rates and the discussion concerning willingness to pay is primarily applicable to a Norwegian setting. Fourth, generalisability of the different scenarios in the sensitivity analysis depends largely on the institution’s flexibility with regards to reallocating tasks for personnel on duty. For example, at our centre, the costs presented in scenario 3 (excluding costs of unpaid time within working hours) most closely resembles the true costs as participants in the simulation-training sessions were already on paid duty but could participate without causing any extra direct costs for the institution through reallocating tasks. We have supplied data with regards to hours spent for different professions, numbers of patients treated and different scenarios in a sensitivity analysis. Generalisability might thus increase through adjusting these parameters to local conditions.

    There is a strong emphasis in stroke care to reduce treatment times and multicomponent QI programmes to safely reduce treatment times are recommended.14 We know that every minute counts in stroke care, but publications on costs of making every minute count are largely lacking. This is particularly true for interventions including simulation-based training.17 We have provided a practical example of an economic evaluation in a quality improvement project including simulation-based training. The cost-effectiveness data might be useful for program directors and policy-makers planning to implement similar interventions within existing budget constraints.

    Conclusions

    We have shown that a QI project aiming to improve stroke thrombolysis treatment at our centre can be implemented and sustained at a relatively low cost with increasing cost-effectiveness over time. Our work builds on the emerging theory and practice for economic evaluations in QI projects and simulation-based training. The presented cost-effectiveness data might help guide healthcare leaders planning similar interventions.

    Data availability statement

    All data deemed relevant to the study are included in the article or uploaded as online supplemental information. Additional supporting data are available on reasonable request. Data are available upon reasonable request.

    Ethics statements

    Patient consent for publication

    Acknowledgments

    The authors would like to acknowledge Linn Haraldseid for acquisition and registration of data and Jan Terje Kvaløy for aiding with statistical analysis. The authors would also like to thank the clinical lead and all fellows that participated in the Scottish Quality and Safety Fellowship Programme Cohort 9 for providing quality improvement education and guiding our project. The authors would additionally like to acknowledge the cooperation of the local patient organization for stroke victims (LHL), all members of the stroke team in the study hospital and members of the quality improvement team for their engagement. Furthermore, the authors would like to thank the reviewers and editorial team for their constructive and helpful feedback in preparing this manuscript.

    References

    1. ↵
      1. GBD 2016 Stroke Collaborators
      . Global, regional, and national burden of stroke, 1990–2016: a systematic analysis for the global burden of disease study 2016. Lancet Neurol 2019;18:439–58.doi:10.1016/S1474-4422(19)30034-1 pmid:http://www.ncbi.nlm.nih.gov/pubmed/30871944
      OpenUrlCrossRefPubMed
    2. ↵
      2. Gorelick PB
      . The global burden of stroke: persistent and disabling. Lancet Neurol 2019;18:417–8.doi:10.1016/S1474-4422(19)30030-4 pmid:http://www.ncbi.nlm.nih.gov/pubmed/30871943
      OpenUrlCrossRefPubMed
    3. ↵
      2. Rajsic S ,
      3. Gothe H ,
      4. Borba HH , et al
      . Economic burden of stroke: a systematic review on post-stroke care. Eur J Health Econ 2019;20:107–34.doi:10.1007/s10198-018-0984-0 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29909569
      OpenUrlCrossRefPubMed
    4. ↵
      2. Ovbiagele B ,
      3. Goldstein LB ,
      4. Higashida RT , et al
      . Forecasting the future of stroke in the United States: a policy statement from the American Heart Association and American Stroke Association. Stroke 2013;44:2361–75.doi:10.1161/STR.0b013e31829734f2 pmid:http://www.ncbi.nlm.nih.gov/pubmed/23697546
      OpenUrlAbstract/FREE Full Text
    5. ↵
      2. Emberson J ,
      3. Lees KR ,
      4. Lyden P , et al
      . Effect of treatment delay, age, and stroke severity on the effects of intravenous thrombolysis with alteplase for acute ischaemic stroke: a meta-analysis of individual patient data from randomised trials. Lancet 2014;384:1929–35.doi:10.1016/S0140-6736(14)60584-5 pmid:http://www.ncbi.nlm.nih.gov/pubmed/25106063
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      2. Hacke W ,
      3. Kaste M ,
      4. Bluhmki E , et al
      . Thrombolysis with alteplase 3 to 4.5 hours after acute ischemic stroke. N Engl J Med 2008;359:1317–29.doi:10.1056/NEJMoa0804656 pmid:http://www.ncbi.nlm.nih.gov/pubmed/18815396
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      2. Rodrigues FB ,
      3. Neves JB ,
      4. Caldeira D , et al
      . Endovascular treatment versus medical care alone for ischaemic stroke: systematic review and meta-analysis. BMJ 2016;353:i1754.doi:10.1136/bmj.i1754 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27091337
      OpenUrlAbstract/FREE Full Text
    8. ↵
      1. National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group
      . Tissue plasminogen activator for acute ischemic stroke. N Engl J Med 1995;333:1581–8.doi:10.1056/NEJM199512143332401 pmid:http://www.ncbi.nlm.nih.gov/pubmed/7477192
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      2. Meretoja A ,
      3. Keshtkaran M ,
      4. Saver JL , et al
      . Stroke thrombolysis: save a minute, save a day. Stroke 2014;45:1053–8.doi:10.1161/STROKEAHA.113.002910 pmid:http://www.ncbi.nlm.nih.gov/pubmed/24627114
      OpenUrlAbstract/FREE Full Text
    10. ↵
      2. Xian Y ,
      3. Smith EE ,
      4. Zhao X , et al
      . Strategies used by hospitals to improve speed of tissue-type plasminogen activator treatment in acute ischemic stroke. Stroke 2014;45:1387–95.doi:10.1161/STROKEAHA.113.003898 pmid:http://www.ncbi.nlm.nih.gov/pubmed/24713527
      OpenUrlAbstract/FREE Full Text
    11. ↵
      2. Kamal N ,
      3. Smith EE ,
      4. Jeerakathil T , et al
      . Thrombolysis: Improving door-to-needle times for ischemic stroke treatment – a narrative review. Int J Stroke 2018;13:268–76.doi:10.1177/1747493017743060 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29140185
      OpenUrlPubMed
    12. ↵
      2. Kurz MW ,
      3. Ospel JM ,
      4. Advani R , et al
      . Simulation methods in acute stroke treatment. Stroke 2020;51:1978–82.doi:10.1161/STROKEAHA.119.026732
      OpenUrlCrossRefPubMed
    13. ↵
      2. Goyal M ,
      3. Ospel JM
      . Stroke systems of care. Stroke 2020;51:1928–31.doi:10.1161/STROKEAHA.120.030288
      OpenUrl
    14. ↵
      2. Powers WJ ,
      3. Rabinstein AA ,
      4. Ackerson T , et al
      . 2018 guidelines for the early management of patients with acute ischemic stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 2018;49:e46–99.doi:10.1161/STR.0000000000000158 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29367334
      OpenUrlAbstract/FREE Full Text
    15. ↵
      2. Pan Y ,
      3. Zhang L ,
      4. Li Z , et al
      . Cost-effectiveness of a multifaceted quality improvement intervention for acute ischemic stroke in China. Stroke 2020;51:1265–71.doi:10.1161/STROKEAHA.119.027980 pmid:http://www.ncbi.nlm.nih.gov/pubmed/32019480
      OpenUrlPubMed
    16. ↵
      2. Yarnoff B ,
      3. Khavjou O ,
      4. Elmi J , et al
      . Estimating costs of implementing stroke systems of care and data-driven improvements in the Paul Coverdell National Acute Stroke Program. Prev Chronic Dis 2019;16:E134.doi:10.5888/pcd16.190061 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31580797
      OpenUrlPubMed
    17. ↵
      2. Lin Y ,
      3. Cheng A ,
      4. Hecker K , et al
      . Implementing economic evaluation in simulation-based medical education: challenges and opportunities. Med Educ 2018;52:150–60.doi:10.1111/medu.13411 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28949032
      OpenUrlPubMed
    18. ↵
      2. Maloney S ,
      3. Haines T
      . Issues of cost-benefit and cost-effectiveness for simulation in health professions education. Adv Simul 2016;1:13.doi:10.1186/s41077-016-0020-3
      OpenUrl
    19. ↵
      2. Tolsgaard MG ,
      3. Tabor A ,
      4. Madsen ME , et al
      . Linking quality of care and training costs: cost-effectiveness in health professions education. Med Educ 2015;49:1263–71.doi:10.1111/medu.12882 pmid:http://www.ncbi.nlm.nih.gov/pubmed/26611191
      OpenUrlPubMed
    20. ↵
      2. Zendejas B ,
      3. Wang AT ,
      4. Brydges R , et al
      . Cost: the missing outcome in simulation-based medical education research: a systematic review. Surgery 2013;153:160–76.doi:10.1016/j.surg.2012.06.025 pmid:http://www.ncbi.nlm.nih.gov/pubmed/22884087
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      2. Ajmi SC ,
      3. Advani R ,
      4. Fjetland L , et al
      . Reducing door-to-needle times in stroke thrombolysis to 13 min through protocol revision and simulation training: a quality improvement project in a Norwegian stroke centre. BMJ Qual Saf 2019;28:939–48.doi:10.1136/bmjqs-2018-009117 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31256015
      OpenUrlAbstract/FREE Full Text
    22. ↵
      2. Provost LP ,
      3. Murray S
      . The health care data guide: learning from data for improvement. San Francisco, Calif.: Jossey-Bass, 2013.
    23. ↵
      2. Lovegrove J ,
      3. Valencia O ,
      4. Treasure T , et al
      . Monitoring the results of cardiac surgery by variable life-adjusted display. Lancet 1997;350:1128–30.doi:10.1016/S0140-6736(97)06507-0 pmid:http://www.ncbi.nlm.nih.gov/pubmed/9343500
      OpenUrlCrossRefPubMedWeb of Science
    24. ↵
      2. Brown SES ,
      3. Chin MH ,
      4. Huang ES
      . Estimating costs of quality improvement for outpatient healthcare organisations: a practical methodology. Qual Saf Health Care 2007;16:248–51.doi:10.1136/qshc.2006.019232 pmid:http://www.ncbi.nlm.nih.gov/pubmed/17693669
      OpenUrlAbstract/FREE Full Text
    25. ↵
      2. Levin H ,
      3. McEwan P
      . Cost-effectiveness analysis: methods and applications. 2nd ed. Thousand Oaks: Sage, 2001.
    26. ↵
      1. NHS Education for Scotland
      . Scottish quality and safety fellowship programme. Available: https://learn.nes.nhs.scot/814/quality-improvement-zone/learning-programmes/scottish-quality-and-safety-sqs-fellowship-programme; [Accessed 14 Jul 2020].
    27. ↵
      Stavanger Acute Medicine Foundation for Education and Research – Train the trainer course. Available: https://www.safer.net/kurs/#ttt [Accessed 14 Jul 2020].
    28. ↵
      2. Wimo A ,
      3. Jönsson L ,
      4. Fratiglioni L , et al
      . The societal costs of dementia in Sweden 2012 – relevance and methodological challenges in valuing informal care. Alzheimers Res Ther 2016;8:59.doi:10.1186/s13195-016-0215-9 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27986093
      OpenUrlPubMed
    29. ↵
      2. Johannesson M ,
      3. Borgquist L ,
      4. Jönsson B , et al
      . The costs of treating hypertension—an analysis of different cut-off points. Health Policy 1991;18:141–50.doi:10.1016/0168-8510(91)90095-F pmid:http://www.ncbi.nlm.nih.gov/pubmed/10112585
      OpenUrlCrossRefPubMed
    30. ↵
      2. Fonarow GC ,
      3. Zhao X ,
      4. Smith EE , et al
      . Door-to-needle times for tissue plasminogen activator administration and clinical outcomes in acute ischemic stroke before and after a quality improvement initiative. JAMA 2014;311:1632–40.doi:10.1001/jama.2014.3203 pmid:http://www.ncbi.nlm.nih.gov/pubmed/24756513
      OpenUrlCrossRefPubMed
    31. ↵
      2. Sanders GD ,
      3. Neumann PJ ,
      4. Basu A , et al
      . Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: second panel on cost-effectiveness in health and medicine. JAMA 2016;316:1093–103.doi:10.1001/jama.2016.12195 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27623463
      OpenUrlCrossRefPubMed
    32. ↵
      Helse- og omsorgsdepartementet (the Norwegian Ministry og health and care services). Verdier I pasientens helsetjeneste. Melding om prioritering. Available: https://www.regjeringen.no/no/dokumenter/meld.-st.-34-20152016/id2502758/; [Accessed 14 Jul 2020].
    33. ↵
      2. Magnussen JAM ,
      3. Granaas T ,
      4. Magelssen M , et al
      . Severity of illness and priority setting in Norway. Summary of a report from a working group 2015.
    34. ↵
      2. Ormstad SS LU ,
      3. Chudasama KK ,
      4. Frønsdal KB
      . Prehospital CT for early diagnosis and treatment of suspected acute stroke or severe head injury, 2019. ISBN: 978-82-8406-005-7.
    35. ↵
      2. Hippe DS ,
      3. Umoren RA ,
      4. McGee A , et al
      . A targeted systematic review of cost analyses for implementation of simulation-based education in healthcare. SAGE Open Med 2020;8:205031212091345.doi:10.1177/2050312120913451 pmid:http://www.ncbi.nlm.nih.gov/pubmed/32231781
      OpenUrlPubMed
    36. ↵
      2. Manasyan A ,
      3. Chomba E ,
      4. McClure EM , et al
      . Cost-effectiveness of essential newborn care training in urban first-level facilities. Pediatrics 2011;127:e1176–81.doi:10.1542/peds.2010-2158 pmid:http://www.ncbi.nlm.nih.gov/pubmed/21502223
      OpenUrlAbstract/FREE Full Text
    37. ↵
      2. Vossius C ,
      3. Lotto E ,
      4. Lyanga S , et al
      . Cost-effectiveness of the "helping babies breathe" program in a missionary hospital in rural Tanzania. PLoS One 2014;9:e102080.doi:10.1371/journal.pone.0102080 pmid:http://www.ncbi.nlm.nih.gov/pubmed/25006802
      OpenUrlCrossRefPubMed
    38. ↵
      2. Cohen ER ,
      3. Feinglass J ,
      4. Barsuk JH , et al
      . Cost savings from reduced catheter-related bloodstream infection after simulation-based education for residents in a medical intensive care unit. Simul Healthc 2010;5:98–102.doi:10.1097/SIH.0b013e3181bc8304 pmid:http://www.ncbi.nlm.nih.gov/pubmed/20389233
      OpenUrlCrossRefPubMedWeb of Science
    39. ↵
      2. Theilen U ,
      3. Fraser L ,
      4. Jones P , et al
      . Regular in-situ simulation training of paediatric medical emergency team leads to sustained improvements in hospital response to deteriorating patients, improved outcomes in intensive care and financial savings. Resuscitation 2017;115:61–7.doi:10.1016/j.resuscitation.2017.03.031 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28359769
      OpenUrlPubMed
    40. ↵
      2. McGaghie WC ,
      3. Draycott TJ ,
      4. Dunn WF , et al
      . Evaluating the impact of simulation on translational patient outcomes. Simul Healthc 2011;6 Suppl:S42–7.doi:10.1097/SIH.0b013e318222fde9 pmid:http://www.ncbi.nlm.nih.gov/pubmed/21705966
      OpenUrlCrossRefPubMed
    41. ↵
      2. Kurz MW ,
      3. Ospel JM ,
      4. Daehli Kurz K , et al
      . Improving stroke care in times of the COVID-19 pandemic through simulation: practice your protocols! Stroke 2020;51:2273–5.doi:10.1161/STROKEAHA.120.030091 pmid:http://www.ncbi.nlm.nih.gov/pubmed/32432995
      OpenUrlPubMed
    42. ↵
      2. Kaplan HC ,
      3. Provost LP ,
      4. Froehle CM , et al
      . The Model for Understanding Success in Quality (MUSIQ): building a theory of context in healthcare quality improvement. BMJ Qual Saf 2012;21:13–20.doi:10.1136/bmjqs-2011-000010 pmid:http://www.ncbi.nlm.nih.gov/pubmed/21835762
      OpenUrlAbstract/FREE Full Text
    43. ↵
      2. McGaghie WC ,
      3. Issenberg SB ,
      4. Cohen ER , et al
      . Translational educational research: a necessity for effective health-care improvement. Chest 2012;142:1097–103.doi:10.1378/chest.12-0148 pmid:http://www.ncbi.nlm.nih.gov/pubmed/23138127
      OpenUrlCrossRefPubMedWeb of Science

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Twitter @AjmiSoffien

    • Contributors SCA is the corresponding author. SCA, MWK, HE and CV contributed to the conceptualisation, design of the study in addition to analysis and interpretation of data for the work. TL, MG and SBI contributed to the analysis and interpretation of data for the work. SCA drafted the work and MWK, HE, CV, TL, MG and SBI contributed with multiple critical revisions. SCA is responsible for the overall content as guarantor.

    • Funding This study was funded by Universitetet i Stavanger (Safer Healthcare Grant (Number: Not applicable)).

    • Competing interests SCA is a research fellow funded by a Safer Healthcare Grant (University Research Fund). Otherwise, the authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors. MG has a consulting agreement with Mentice. The remaining authors report no disclosures.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

    Linked Articles