You are here

  • Home
  • Archive
  • Volume 29, Issue 2
  • Effect of hands-on interprofessional simulation training for local emergencies in Scotland: the THISTLE stepped-wedge design randomised controlled trial

Article Text

Article menu

Download PDFPDF

Original research
Effect of hands-on interprofessional simulation training for local emergencies in Scotland: the THISTLE stepped-wedge design randomised controlled trial
  1. Erik Lenguerrand1,
  2. Cathy Winter2,
  3. Dimitrios Siassakos3,
  4. Graeme MacLennan4,
  5. Karen Innes4,
  6. Pauline Lynch5,
  7. Alan Cameron6,
  8. Joanna Crofts2,
  9. Alison McDonald4,
  10. Kirsty McCormack4,
  11. Mark Forrest4,
  12. John Norrie7,
  13. Siladitya Bhattacharya8,
  14. Tim Draycott1,2
  1. 1 Translational Health Sciences, University of Bristol, Bristol, UK
  2. 2 Department of Women’s Health and Children’s Health, Southmead Hospital, North Bristol NHS Trust, Westbury on Trym, UK
  3. 3 Institute for Women’s Health, UCL, London, United Kingdom
  4. 4 Centre for Healthcare Randomised Trials, Health Services Research Unit, University of Aberdeen, Aberdeen, UK
  5. 5 Maternity Unit, Ninewells Hospital and Medical School, NHS Tayside, Dundee, UK
  6. 6 Ian Donald Fetal Medicine Centre, Queen Elizabeth University Hospital, NHS Greater Glasgow and Clyde, Glasgow, UK
  7. 7 Edinburgh Clinical Trials Unit, Centre for Population Health Sciences, Usher Institute, University of Edinburgh, Edinburgh, UK
  8. 8 Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK
  1. Correspondence to Dr Erik Lenguerrand, Translational Health Sciences, University of Bristol, Bristol BS8 1TH, UK; erik.lenguerrand{at}bristol.ac.uk

Abstract

Objective To assess whether the implementation of an intrapartum training package (PROMPT (PRactical Obstetric Multi-Professional Training)) across a health service reduced the proportion of term babies born with Apgar score <7 at 5 min (<75mins).

Design Stepped-wedge cluster randomised controlled trial.

Setting Twelve randomised maternity units with ≥900 births/year in Scotland. Three additional units were included in a supplementary analysis to assess the effect across Scotland. The intervention commenced in March 2014 with follow-up until September 2016.

Intervention The PROMPT training package (Second edition), with subsequent unit-level implementation of PROMPT courses for all maternity staff.

Main outcome measures The primary outcome was the proportion of term babies with Apgar<75mins.

Results 87 204 eligible births (99.2% with an Apgar score), of which 1291 infants had an Apgar<75mins were delivered in the 12 randomised maternity units. Two units did not implement the intervention. The overall Apgar<75mins rate observed in the 12 randomised units was 1.49%, increasing from 1.32% preintervention to 1.59% postintervention. Once adjusted for a secular time trend, the ‘intention-to-treat’ analysis indicated a moderate but non-significant reduction in the rate of term babies with an Apgar scores <75mins following PROMPT training (OR=0.79 95%CI(0.63 to 1.01)). However, some units implemented the intervention earlier than their allocated step, whereas others delayed the intervention. The content and authenticity of the implemented intervention varied widely at unit level. When the actual date of implementation of the intervention in each unit was considered in the analysis, there was no evidence of improvement (OR=1.01 (0.84 to 1.22)). No intervention effect was detected by broadening the analysis to include all 15 large Scottish maternity units. Units with a history of higher rates of Apgar<75mins maintained higher Apgar rates during the study (OR=2.09 (1.28 to 3.41)) compared with units with pre-study rates aligned to the national rate.

Conclusions PROMPT training, as implemented, had no effect on the rate of Apgar <75mins in Scotland during the study period. Local implementation at scale was found to be more difficult than anticipated. Further research is required to understand why the positive effects observed in other single-unit studies have not been replicated in Scottish maternity units, and how units can be best supported to locally implement the intervention authentically and effectively.

Trial registration number ISRCTN11640515.

  • intrapartum emergencies
  • training course
  • apgar score
  • stepped-wedge trial
  • obstetrics
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjqs-2018-008625

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    INTRODUCTION

    Maternity care in the UK could be safer.1–3 Perinatal outcomes are substantially worse than in countries of similar GDP4 and more than 50% of adverse pregnancy outcomes are considered preventable.5–7

    Training for intrapartum emergencies has been recommended to improve care3–6 8–10 and mandated by litigation authorities11 12 but data regarding effective intrapartum training are conflicting. Some training programmes have been associated with significant safety improvements,13–18 whereas other initiatives have reported either no change16 19 or a deterioration in outcomes.15 Observational studies of a multiprofessional intrapartum emergencies training course for local maternity staff—PRactical Obstetric Multi-Professional Training—(PROMPT, Bristol, UK) have previously demonstrated improved compliance with clinical standards,13 17 reductions in clinical error14 and sustainable improvements in perinatal outcomes.20 These positive effects were initially observed in a single maternity unit but have since been repeated across other international settings.21–23 Although these early data are extremely encouraging, it is important to determine whether these improvements could be replicated at scale across a national health system using a robust study design such as a randomised controlled trial (RCT).

    With over 50 000 births annually,24 Scotland provided an ideal setting to investigate the effect of implementing PROMPT training at national scale. Only 3 of the 15 Scottish maternity units (≥900 births annually—2012) had previously undertaken PROMPT training. Moreover, all NHS maternity outcomes are routinely collated centrally by the Information Statistics Division Scotland (ISD), providing a robust data set for analysis. Finally, the proportion of term babies with Apgar <7 at 5 min (Apgar<75mins),25 an outcome that has previously been shown to be reduced following PROMPT training, was higher in Scotland than in other settings13 (1.19% between 2004 and 2009, data from ISD Scottish Morbidity Record (SMR 02).

    The aim of this study was to conduct an RCT to investigate whether the implementation of an intrapartum emergencies training package across all large maternity units within an entire health service could reduce the proportion of term babies with an Apgar score <75mins.

    METHODS

    The protocol for this trial has been published previously.26

    Intervention

    The intervention was the PROMPT training package (Second edition), which included a 2 day PROMPT Train the Trainers (T3) programme (TIDieR checklist, online supplementary appendix 1), with subsequent unit-level implementation of local PROMPT courses for all maternity staff. PROMPT courses are ‘in-house’, adaptable multiprofessional training days which cover the management of core obstetric emergencies, including postpartum haemorrhage, sepsis and shoulder dystocia as well as fetal monitoring in labour and team-working. Each participating maternity unit identified a multiprofessional ‘in-house’ training team (recommended: two midwives, one obstetrician and one anaesthetist) to attend the T3 programme at The Scottish Clinical Simulation Centre in Larbert, before the start of their allocated intervention step. The T3 programme included a demonstration ‘PROMPT Course’ and a T3 day to guide ‘in house’ trainers in how to use the PROMPT Course-in-a-Box (Second Edition) and set up local PROMPT courses.27 28 The ‘in-house’ trainers were asked to implement local PROMPT courses, aiming to train all their maternity staff within 12 months of commencing local training.

    Supplemental material

    Study design, setting and participants

    We planned a cross-sectional design stepped-wedge cluster randomised controlled superiority trial (SW-RCT). This was viewed as the safest, most pragmatic and ethical option, with the additional rigour of randomisation.

    The study comprised a control period when none of the units received training, three intervention periods, and at least two follow-up periods (figure 1). Four maternity units were randomised to each intervention step. Each period lasted 6 months to allow all 12 units to receive their intervention within 18 months. This would reduce potential contamination between trained and untrained units (rotation of trained staff between units) within the pragmatic funding and time constraints.

    Figure 1
    Figure 1

    Consort flow diagram of the Thistle Stepped-Wedge trial. A more detailed Consort flow diagram is provided in online supplementary appendix 4.

    Maternity units in NHS Scotland with ≥900 births/year (n=15) that had not previously participated in the PROMPT T3 programme were eligible (n=12) and invited to participate through the Scottish Committee of the Royal College of Obstetricians and Gynaecologists. The three units that had already been conducting PROMPT training prior to this study were included in a supplementary preplanned analysis to assess the intervention at national scale. Units with <900 births/year were excluded (<6% of births between 01/01/2000 and 30/09/2013), as their staffing structures may have differed from the larger units. Babies born preterm, at home, at other hospitals and then transferred to a participating unit were all excluded, as were those confirmed to have an intrauterine death prior to labour. Babies born by elective caesarean section (CS) were also excluded as their care was not impacted by the intrapartum training provided by PROMPT. The first four participating maternity units attended a T3 programme in March 2014. The 6-month period preceding the start of the first intervention period (‘Intervention 1’) (01/10/2013–01/04/2014) was identified as the control period (‘Control’). The study ended with the completion of the last follow-up period (‘Follow-up period 2’) on the 30/09/2016.

    Randomisation, concealment and blinding

    One eligible maternity unit that had attended PROMPT T3 training before the start of the study but had not commenced ‘in-house’ local PROMPT courses, was allocated to period 1 for inclusion in the study. The remaining participating units were allocated to an intervention period using the ‘imbalance statistic’ method29 30 to balance units by size (small, medium, large annual births). An independent statistician from the Centre for Healthcare Randomised Trials (CHaRT) randomly selected an allocation sequence list from a subset with the most desirable balance properties using computer-generated random numbers. CHaRT then informed the nominated in-house training coordinator from each unit of the date that their team had been allocated to attend the T3 programme. Participating units that were unable to start training at their allocated time slot were manually reassigned to another period, maintaining balance in the size of the maternity units in each period.

    Pregnant women were not made aware of maternity unit participation in the study, and any additional training that each unit would normally undertake continued during the study period. Because of the nature of the intervention and study design, the staff and research teams could not be blinded.

    Primary outcome

    The primary outcome was the Apgar score of eligible term infants measured at 5 min after birth. An Apgar score <75mins is associated with poor long-term outcomes,31 32 can be improved by best care13 and is independent of maternal demographics.31 No secondary outcomes were investigated.

    Data sources

    Birth information was obtained from Information Services Division (ISD) Maternity Inpatient and Day Case dataset (SMR 02). Data were accessed remotely using the National Services Scotland Safe Haven (Access permission: XRB13180). Clinical and individual information were extracted to obtain the date of birth (used to identify the study period to which a birth should be linked), Apgar score at 5 min and variables required to identify the relevant population of term infants such as the type of birth (vaginal, emergency or elective CS), gestation (<37,≥37 weeks), place of birth (at home, transfer from a non-NHS unit, transfer from a NHS unit, in the unit of interest) and intrauterine deaths.

    Each maternity unit provided process information about their local PROMPT courses using a standardised data collection sheet (online supplementary appendix 2). Each unit’s training coordinator supplied the dates of their local courses, and anonymised data on numbers of attendees by grade and clinical role. They also submitted course programmes that provided information on the actual course content and degree of PROMPT implementation, and therefore the fidelity of the intervention at unit level. The training coordinators were also requested to provide some baseline data on local staff composition and volume, with the aim of determining the cumulative proportion of staff trained in each unit during the study period.

    Statistical analysis

    Assuming four maternity units were randomised at each intervention period, with three intervention periods (steps 1–3) and two follow-up periods (steps 4 and 5) each comprising 6 months, an average cluster size of 1200 births and an intracluster correlation coefficient of 0.1, we estimated we could detect a relative reduction of 35% or more in the Apgar<75mins (1.18%–0.77%) with 80% power and two-sided alpha of 5%.33 The 12 eligible Scottish maternity units had a rate of Apgar<75mins at term of 1.18% in 2010, and a monthly average of 200 births per maternity unit, that is, around 1200 births per period duration.

    We compiled the prevalence rates of Apgar<75mins by maternity unit and/or time periods as the number of eligible babies with Apgar<75mins divided by the total number of eligible births.

    Births recorded between 01/01/2000 and 30/09/2013 were used to identify the pre-study context, that is, unit preintervention Apgar scores (online supplementary appendix 3) and birth volumes. An initial intention-to-treat (ITT) analysis was undertaken across the 12 randomised units. Exposure to the intervention was defined using the randomisation schedule. Units that were unable to attend the T3 programme, comply with the randomisation schedule or were unable to implement any local training during the study period, were analysed according to their allocated intervention step. We investigated the intervention effect (exposed/unexposed to PROMPT) using logistic mixed regression (with random effects for clustering of births within unit), adjusted for time-modelled as a categorical variable (five dummy variables for the six periods (table 1) with the control period as the reference) to account for a potential confounding effect of an underlying time-trend in the outcome.34 35 This model provided the time-averaged intervention effect.33 35 36 To account for heterogeneity in the pre-study Apgar rates between units and the unit size, a factor used in the minimisation randomisation, this main analysis was also adjusted for pre-study maternity unit size (small <2000, medium 2000–5000 (reference), large >5000 births/year), as well as status of pre-study Apgar score<75mins rate (below, above, at national average (reference), online supplementary appendix 3). This analysis was conducted on complete-cases but was then repeated to account for the 866 infants (0.7%) with missing Apgar scores. All births with missing score were first considered to be comparable to those with a ‘high’ Apgar score (good to excellent baby condition) and assigned into the Apgar≥75mins group (Imputation 1). The 164 babies with missing scores admitted to a neonatal care unit and/or who received some form of resuscitation other than facial oxygen were then assigned to the Apgar score<75mins group; the remaining 702 births with missing data were assigned to the Apgar score≥ 75mins group (Imputation 2).

    View this table:
    Table 1

    Prevalence rate of Apgar score<75mins during the study period*

    To account for departures from the randomisation schedule, an ‘as-implemented’ analysis was conducted replicating the above analyses but using the date of the first local training to define exposure to the intervention, that is, the time periods during which births were actually ‘exposed and unexposed’ to the intervention (figure 2). Data from randomised units that did not implement the intervention were considered as unexposed throughout the study.

    Figure 2
    Figure 2

    Apgar<75 mins prevalence rates by maternity units and study time periods, randomisation schedule and actual intervention implementation.

    To provide an assessment of the intervention effect across the health service, all 15 Scottish maternity units with ≥900 births/year, including the three non-randomised units that had implemented PROMPT training prior to this study, were analysed in an exploratory analysis using the ‘as-implemented’ approach. Data from these three units were considered as exposed for all time periods. We finally modelled a categorical variable (not exposed, exposed ≤6 months, 6–12, 12–18, 18–24 or ≥24 months) in the ‘as-implemented’ analyses to explore the effect of the duration of exposure to the intervention.36

    Log-likelihood ratio (LR) test was performed to assess the comparisons (using two-sided p value).

    RESULTS

    Between 2000 and September 2013, the period preceding the study, the overall prevalence rate of Apgar<75mins across all 15 participating and non-participating large maternity units ≥900 births/year was 1.16%, increasing from 1.06% in 2000 to 1.23% in 2013 (risk ratio: 1.17 95% CI (1.02 to 1.34), p=0.022) (online supplementary appendix 3-figure 1). Four maternity units (U4, U5, U6, U15) had consistently lower rates of Apgar<75mins compared with the overall 2000–2013 rate observed across all 15 units (0.92% (0.88 to 0.96)); five units (U2, U3, U7, U8, U9) had higher rates of Apgar<75mins(1.51% (1.44 to 1.57)) (online supplementary appendix 3-figure 2). The rates observed in the remaining units were not different from the overall rate. Based on this pre-study information, units were classified for the following analyses as having a status of ‘low’, ‘normal’ or ‘high’ pre-study prevalence rate of Apgar<75mins.

    A CONSORT diagram of the SW-RCT is presented in figure 1 and online supplementary appendix 4.

    There were 123 943 eligible births in the 15 main Scottish maternity units over the study duration; 123 077 were assigned an Apgar5mins score (99.3%) and 1765 infants were scored an Apgar<75mins.

    Three of the 15 maternity units had already commenced PROMPT training prior to the study. Ten of the 12 randomised units implemented the intervention; the remaining 2 declined to participate during the study period. There was a total of 87 204 eligible births (including 99.2% with an assigned Apgar score) and 1291 infants with an Apgar<75mins in the 12 randomised maternity units.

    Figure 2 describes the delivery of the intervention as initially planned and also as actually implemented, including the starting point, length of time to implement local courses and timings for each training session. Units 1–4, 5–8 and 9–12 were randomised to intervention period 1, 2 and 3, respectively. There was considerable variation in the implementation of the local PROMPT courses at unit level in terms of volume of training sessions delivered (online supplementary appendix 5 table 1) and adherence to the randomisation schedule (figure 2).

    Data on local training submitted by the participating units demonstrated significant differences in local implementation, including the content of courses. All units included elements of team working in their courses, but only three included fetal monitoring sessions, which is considered a core component of PROMPT training and related to Apgar scores. Shoulder dystocia training, another core element of PROMPT, was also not included in some programmes (online supplementary appendix 5 table 2). Participating units provided information on numbers of staff trained during their local courses but were unable to provide baseline unit-level staffing numbers, and therefore, we were not able to determine the cumulative proportion of staff trained by unit during the study period.26 The overall Apgar<75mins prevalence rate observed during the study period in the 12 randomised maternity units was 1.49%: increasing from 1.32% pre-intervention to 1.59% post-intervention (unadjusted test for trend of odds, χ2=6.4, p=0.01) (table 1). This underlying time trend was adjusted for in the different analyses.

    Effect of the intervention

    The ITT analysis showed a moderate but non-significant reduction in the proportion of babies with an Apgar score<75mins after the implementation of the intervention (OR=0.79 95%CI(0.63 to 1.01), table 2). Consistent results were observed in the sensitivity analyses. No intervention effect was found when exposure to the intervention was defined according to its actual implementation rather than the randomisation schedule (OR=1.01 (0.84 to 1.22), table 2). The duration of exposure to the intervention did not affect the outcome (ORexposed≤6months vs not exposed=1.08 (0.88 to 1.32), ORexposed6-12months vs not exposed=0.93 (0.73, 1.19), ORexposed12-18months vs not exposed=1.02 (0.75 to 1.37), ORexposed18-24months vs not exposed=1.18 (0.82 to 1.71), ORexposed≥24months vs not exposed=1.12 (0.66 to 1.87); overall LR test χ2=3.7, p=0.86). There was also no evidence of an intervention effect when the analyses were extended across all 15 large maternity units (table 2).

    View this table:
    Table 2

    PROMPT training effectiveness (OR and associated 95% CI); main analysis and sensitivity analyses*

    Effect of time-trend, pre-study Apgar<75mins rate and maternity unit size

    A time-trend was observed in the ITT analyses, with higher rates of Apgar<75mins towards the end of the study, compared with the beginning (table 2). This trend was no longer evident in the ‘as-implemented’ analyses (overall LR test χ2=3.1, p=0.69), with numerous births in the same time period being reclassified as unexposed rather exposed (and vice-versa), including births which occurred in units 1 and 9 being fully reclassified as unexposed during all time periods.

    The rate of term babies with an Apgar score<75mins remained higher in units with a pre-study Apgar rate higher than the national rate compared with units with a pre-existing rate comparable to the national rate (OR=2.09 (1.28 to 3.41), table 2). Units with a low pre-study rate had comparable Apgar score<75mins rates during the study period compared with units with an average pre-study rate (OR=0.72 (0.40 to 1.29)). There was no evidence of interaction between the intervention and the pre-study Apgar rate (overall LR test: In ITT χ2=0.9, p=0.64, ‘as-implemented’12units χ2=4.2, p=0.12, ‘as-implemented’15units χ2=3.4, p=0.18).

    There was no evidence of an association between maternity unit size and the rate of babies born with an Apgar score <75 mins (table 2).

    DISCUSSION

    There was no reduction in the proportion of term births with an Apgar score <75mins following PROMPT training (OR=1.01 (0.84 to 1.22)) in Scotland. The rate of Apgar score<75mins continued to increase (1.32% preintervention to 1.59% postintervention) and was higher in units with a high baseline rate of Apgar<75mins (OR=2.09 (1.28 to 3.41). The implementation of the intervention was very heterogeneous between units, with significant variation in the content of local courses, number of courses implemented and also the number of staff trained.

    This study is the first to evaluate implementation of a multiprofessional training intervention for obstetric emergencies at national level. The clinical data were collected prospectively and nationally, independent of this study, limiting potential reporting bias and reducing the burden of data collection.

    We employed a SW-RCT37 design to evaluate an intervention with prior evidence of benefit. This design has been widely used in the context of limited delivery resources to evaluate educational or public health training programmes that are anticipated to be beneficial and, therefore, to prevent ethical objections.38 In our study, it was deemed unethical to use a parallel (cluster) RCT design as there was insufficient equipoise for the intervention, and no alternative training could be proposed to the control group.15 16 19 However, the SW-RCT design proved to be less robust than originally envisaged when used to study a training intervention.39 While most units were keen to participate, adherence to the randomisation schedule was variable. Some units were able to commence training immediately after their T3 session and before their allocated step, whereas others started several months after the allocated starting point. We performed an ITT approach using the dates at which maternity units were randomised to commence training, irrespective of when they actually started local training, to define the ‘control/non-exposed’ and ‘intervention/exposed’ groups. We also performed a sensitivity analysis in the form of an ‘as-implemented’ analysis. We do not think that the ITT approach accurately captured the local implementation of the intervention. Therefore, we consider the ‘as-implemented’ approach to be the most accurate analysis method for the effect of intervention. While both ITT and ‘as-implemented’ analyses led to the same conclusion, the extent of the estimated risk was different between these two approaches, most likely because the definition of exposure to the intervention was fundamentality different between the two approaches and because of the pre-study Apgar<75mins status of the unit. A number of births have been alternatively considered as ‘unexposed’ in one approach but ‘exposed’ in the other approach (or vice versa) for the same time period. Moreover, units with low and average baseline rates of Apgar<75mins status—a high predictor of current Apgar<75mins rate—had the largest time departure from the planned implementation schedule, and therefore the largest intervention exposure misclassification in the ITT analysis compared with the ‘as-implemented’ approach. Other unknown factors might also explain the change in the effect size between the two approaches. This raises two important methodological considerations when modelling findings from SW-RCT. We agree with other methodologists that it is important to consider the impact of time trend on the estimated effect33 35–37 and the necessity for time adjustment in the primary analysis. More importantly, our findings suggest that ITT analysis should not be considered in isolation when writing the statistical analysis plan of a SW-RCT. Sensitivity analysis such as our ‘as-implemented’ approach should also be employed, particularly for complex interventions for example, training.

    We had initially envisaged the SW-RCT design as a pragmatic design to assess PROMPT at larger scale, within a reasonable timeframe and with restricted resources. However, the intervention implementation was challenging and the restrictions on logistics, time and funding meant that we were unable to provide ongoing implementation support to the participating units, other than by phone or email. Furthermore, maternity units were often unable to commence the intervention as planned, either starting earlier or later than the randomisation schedule. Finally, the content and frequency of local courses was heterogeneous. We recommend that future SW-RCT studies should consider a feasibility stage and a parallel process evaluation, as recommended for complex interventions,40 41to explore potential barriers and facilitators to randomisation and intervention implementation. The theoretical savings in time, resources and staffing that a SW-RCT appeared to offer, might be limited, especially when dealing with an intervention requiring behavioural changes or authentic implementation in busy health systems. Therefore, SW-RCT designs may not have that many advantages for this type of study.

    We were also unable to provide funding to support maternity units to collate their baseline staffing composition and volume: the cumulative proportion of staff trained over the duration of the study could not be derived as initially planned.26 Moreover, to reduce the burden of increased workload on already busy clinical staff, we requested only limited process data. Therefore, our investigation of the reasons underlying the delay, frequency, duration and content of local intervention implementation was limited, as was our investigation into the extent of the ‘intervention penetration’; we could not test the intervention effect according to the proportion of staff trained at different time points. We recommend that funding should be sought to assess local intervention penetration in future studies of training; in particular, any SW-RCT evaluating training effectiveness should allocate resources to enable the measurement of the proportion of staff trained.

    PROMPT training has been associated with improvements in outcomes in other contexts and associated with sustained local multiprofessional training where more than 90% of staff trained annually.13 22 23 It is unlikely that any of the participating Scottish units were able to train 90%–95% of their staff. However, a higher level of implementation support was also provided by the central PROMPT team in some of these settings compared with the THISTLE units. Other national and international maternity units that have successfully implemented PROMPT took 1 year to train all of their staff and another year to demonstrate a difference in outcomes.13 21 The PROMPT intervention involves behavioural changes and more research is required to understand the support needed by maternity units to implement local PROMPT authentically.

    The substantial variation in the implementation of the PROMPT intervention by the Scottish units in this study may have affected the intervention authenticity and, therefore, our findings most likely reflect the effectiveness of the intervention as implemented in Scotland, rather than the efficacy that can be achieved if PROMPT is implemented exactly as recommended and has been observed among other early adopter sites. There may be several reasons for the variation in intervention authenticity; there were simultaneous and sometimes contradictory national training initiatives for example, a national training programme for Fetal Monitoring in Labour,42 was introduced during the study period. In addition, although recommendations were made during the T3 programme to guide participating teams on the core components to include in their local multiprofessional PROMPT courses, units were allowed flexibility to facilitate implementation in their local context, to enhance ownership of the training. Furthermore, while we recommended start dates for local implementation for all participating units and that they should aim to train all of their maternity staff within 12 months, we could not mandate this or provide any further implementation support. Therefore, we were not able to ensure total authenticity of local implementation and in particular, the omission of the PROMPT fetal monitoring elements by some units is likely to have had a major impact on the primary outcome.

    The use of Apgar scores to determine the quality of intrapartum care has been criticised.43 However, an Apgar score <75mins is associated with poor long-term outcomes,31 32 can be improved by best care13 and is independent of maternal demographics.31 It would have been useful to have investigated other outcomes that have been improved with PROMPT for example, brachial plexus injuries20 and future studies should consider more than a single measure of improvement.

    Reducing avoidable harm in maternity care is an urgent priority and improved training for intrapartum care is at least part of the solution, but it must be both effective and sustainable. Unlike some other less-effective interventions15 that appeared to show early promise, this research has not shown any evidence of harm related to the PROMPT intervention. Not all training for obstetric emergencies is effective,44 however well intentioned, but the implementation of PROMPT at unit level has been associated with statistically significant, clinically important improvements in perinatal and maternal outcomes in different resource settings. This includes reductions in brachial plexus injuries,14 20 23 encephalopathy or low 5 min Apgar score,13 23 composite neonatal outcomes45 as well as reductions in maternal deaths in Zimbabwe21 and in the Philippines.46

    Health and social care systems continue to be challenged by the problem that initially promising interventions often prove difficult to replicate and scale. The dynamic interplay between intervention and context means that it is often difficult to separate intervention from context.47 48 Transplanting a programme in its entirety from one setting to another is rarely straightforward: the features of context (infrastructure, organisational systems, values, skills) may need to be reproduced too.49–51 A recent ethnographic study conducted in the maternity unit where PROMPT was created have identified how this intervention and the unit contextual features shaped each other and how this process was influenced by the unit's structural conditions, such as staffing levels and physical environment.52 The PROMPT team are now exploring a social science approach to scale up and spread, including social franchising.53 A parallel process study, THISTLE Plus,54 separately funded, has also been implemented in line with MRC recommendations55 and will provide further insight into local barriers and facilitators that impacted the intervention implementation in Scotland.

    It is essential that we investigate how best to scale effective interventions, particularly for maternity care where the costs of litigation are unsustainable,56 the human costs of preventable harm at birth notwithstanding. This study demonstrates that implementation can be difficult at local level and suggests that implementation support is required to ensure that the intervention is authentically adopted and effective. Training is not free,57 however, effective training is very cost effective. Improving perinatal outcomes can substantially reduce litigation costs.23 56

    The rate of Apgar<75mins in Scotland went up throughout the study. This increase is likely to be multifactorial. The rate of prematurity—a key risk factor for low Apgar score—, as well as the rates of intrauterine death and of elective CS—two other exclusion factors for our studied sample—remained stable between the pre-study period and the study period (7% vs 8 %, 0.3% vs 0.3% and 14% vs 15%, respectively). This is suggesting that the studied sample has retained some comparability over time with regard to our exclusion criteria and/or factors related to low Apgar score status. Other factors such as maternal occupation or education status could have changed over time; the strength of their relationship with the Apgar score status is complex, depending, for example, on the mode of delivery, that is, absent or marginal in vaginal delivery but higher in CS delivery.31 Changes in quality of care could explain the observed increase in low Apgar score and further work is now required to understand and reverse this trend. Research and clinical investigations in units which have maintained low rates of Apgar<75mins would provide insights into this and could help to understand the ‘active ingredients’ of effective training, as well as strategies for the successful implementation of effective interventions at scale. Finally, it is important for clinical researchers to mobilise evidence up to policy makers. For example, system actors like state insurers are not patient safety organisations, but they do have a role in reducing, particularly aligning and focussing the system on improvement56 58 and initiatives should be based on the evidence base for clinically effective improvement interventions.

    Conclusion

    PROMPT training, as implemented, had no effect on the rate of Apgar <75mins in Scotland during the study period. Local implementation was found to be more difficult than anticipated and further research is required to understand why the positive effects observed in other single-unit studies have not been replicated in Scottish maternity units during the THISTLE Study.

    Acknowledgments

    The authors thank the CSO for funding the THISTLE study and the staff of the Scottish maternity units for their participation and continued support with the THISTLE Study. We would like to acknowledge the independent members of the Trial Steering Committee for their participation and continued support with the THISTLE Study. We also wish to thank Joannes Kerssens, Lizzie Nicholson, Mark Macartney, Carole Morris and Katrina Smith from eDRIS, ISD for continued support with the data management and Tracey Davidson, Bev Smith, Becky Bruce, Maria Ntessalen and Gladys McPherson based at CHaRT, University of Aberdeen for their logistic support. Finally, we would like to thank Dr Perla Marang van de Mheen for her insightful contribution in the interpretation of our results.

    References

    1. ↵
      Royal college of obstetricians and gynaecologists. Each baby counts 2018 progress report. London: RCOG, 2018
    2. ↵
      2. Magro M
      . Five years of cerebral palsy claims. A thematic review of NHS Resolution data. London: NHS Resolution, 2017.
    3. ↵
      2. Draper ES ,
      3. Gallimore ID ,
      4. Kurinczuk JJ , et al
      . MBRRACE-UK Perinatal mortality surveillance report, UK Perinatal deaths for births from january to december 2016. Leicester: The infant mortality and morbidity studies, department of health sciences, University of Leicester, 2018
    4. ↵
      2. O'Neill O
      . Safe births: Everybody’s business. An independent inquiry into the safety of maternity services in england. London: King’s Fund, 2008.
    5. ↵
      2. Lewis G
      . The Confidential Enquiry into Maternal and child health (CEMACH). The Seventh Report on Confidential Enquiries into Maternal Deaths in the United Kingdom. London: CEMACH, 2007.
    6. ↵
      1. Maternal and Child Health Research Consortium
      . Maternal and Child Health Research Consortium. Confidential Enquiry into Stillbirths and Deaths in Infancy. London 2001.
    7. ↵
      1. NHS Litigation Authority
      . Ten years of maternity claims. An analysis of nhs litigation authority data. London: NHS LA, 2012.
    8. ↵
      1. NHS Litigation Authority
      . Cnst maternity clinical risk management standards. London: NHS LA, 2006.
    9. ↵
      In: Royal College of OBSTETRICIANS and GYNAECOLOGISTS, Royal College of MIDWIVES, Royal College of ANAESTHETISTS, . Standards for Maternity Care. Report of a Working Party. London RCOG Press; 2008. et al.
    10. ↵
      1. Victorian Managed Insurance Authority (VMIA)
      . Annual report. Melbourne VMIA; 2018.
    11. ↵
      1. NHS Litigation Authority
      . Clinical negligence scheme for trusts – maternity clinical risk management standards. London: NHS LA, 2010.
    12. ↵
      1. NHS Resolution
      . CNST Scheme 2017. London: NHS Resolution, 2017.
    13. ↵
      2. Draycott T ,
      3. Sibanda T ,
      4. Owen L , et al
      . Does training in obstetric emergencies improve neonatal outcome? BJOG: Int J O&G 2006;113:177–82.doi:10.1111/j.1471-0528.2006.00800.x
      OpenUrl
    14. ↵
      2. Draycott TJ ,
      3. Crofts JF ,
      4. Ash JP , et al
      . Improving neonatal outcome through practical shoulder dystocia training. Obstet Gynecol 2008;112:14–20.doi:10.1097/AOG.0b013e31817bbc61
      OpenUrlCrossRefPubMedWeb of Science
    15. ↵
      2. MacKenzie IZ ,
      3. Shah M ,
      4. Lean K , et al
      . Management of shoulder dystocia: trends in incidence and maternal and neonatal morbidity. Obstet Gynecol 2007;110:1059–68.doi:10.1097/01.AOG.0000287615.35425.5c
      OpenUrlCrossRefPubMed
    16. ↵
      2. Nielsen PE ,
      3. Goldman MB ,
      4. Mann S , et al
      . Effects of teamwork training on adverse outcomes and process of care in labor and delivery: a randomized controlled trial. Obstet Gynecol 2007;109:48–55.doi:10.1097/01.AOG.0000250900.53126.c2
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      2. Siassakos D ,
      3. Hasafa Z ,
      4. Sibanda T , et al
      . Retrospective cohort study of diagnosis-delivery interval with umbilical cord prolapse: the effect of team training. BJOG 2009;116:1089–96.doi:10.1111/j.1471-0528.2009.02179.x
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      2. Sørensen JL ,
      3. Løkkegaard E ,
      4. Johansen M , et al
      . The implementation and evaluation of a mandatory multi-professional obstetric skills training program. Acta Obstet Gynecol Scand 2009;88:1107–17.doi:10.1080/00016340903176834
      OpenUrlCrossRefPubMed
    19. ↵
      2. Fransen AF ,
      3. van de Ven J ,
      4. Schuit E , et al
      . Simulation-based team training for multi-professional obstetric care teams to improve patient outcome: a multicentre, cluster randomised controlled trial. BJOG 2017;124.doi:10.1111/1471-0528.14369
    20. ↵
      2. Crofts JF ,
      3. Lenguerrand E ,
      4. Bentham GL , et al
      . Prevention of brachial plexus injury-12 years of shoulder dystocia training: an interrupted time-series study. BJOG 2016;123:111–8.doi:10.1111/1471-0528.13302
      OpenUrl
    21. ↵
      2. Crofts JF ,
      3. Mukuli T ,
      4. Murove BT , et al
      . Onsite training of doctors, midwives and nurses in obstetric emergencies, Zimbabwe. Bull World Health Organ 2015;93:347–51.doi:10.2471/BLT.14.145532
      OpenUrl
    22. ↵
      2. Shoushtarian M ,
      3. Barnett M ,
      4. McMahon F , et al
      . Impact of introducing practical obstetric multi-professional training (PROMPT) into maternity units in Victoria, Australia. BJOG 2014;121:1710–8.doi:10.1111/1471-0528.12767
      OpenUrlCrossRefPubMed
    23. ↵
      2. Weiner CP ,
      3. Collins L ,
      4. Bentley S , et al
      . Multi-professional training for obstetric emergencies in a US hospital over a 7-year interval: an observational study. J Perinatol 2016;36:19–24.doi:10.1038/jp.2015.136
      OpenUrl
    24. ↵
      2. Scotland ISD
      . Births in Scottish Hospitals: ISD Scotland, 2016. Available: http://www.isdscotland.org/Health-Topics/Maternity-and-Births/Births/ [Accessed 02 Dec 2016].
    25. ↵
      2. Apgar V
      . The Newborn (APGAR) Scoring System: Reflections and Advice. Pediatr Clin North Am 1966;13:645–50.doi:10.1016/S0031-3955(16)31874-0
      OpenUrlCrossRefPubMed
    26. ↵
      2. Lenguerrand E ,
      3. Winter C ,
      4. Innes K , et al
      . THISTLE: trial of hands-on Interprofessional simulation training for local emergencies: a research protocol for a stepped-wedge clustered randomised controlled trial. BMC Pregnancy Childbirth 2017;17:294.doi:10.1186/s12884-017-1455-9
    27. ↵
      1. PROMPT Maternity Foundation
      . Trainer's Manual. London: Royal College of Obstetricians and Gynaecologists, 2012.
    28. ↵
      1. PROMPT Maternity Foundation
      . Course Manual. Cambridge: Cambridge University Press, 2012.
    29. ↵
      2. Carter BR ,
      3. Hood K
      . Balance algorithm for cluster randomized trials. BMC Med Res Methodol 2008;8:65.doi:10.1186/1471-2288-8-65
    30. ↵
      2. Raab GM ,
      3. Butcher I
      . Balance in cluster randomized trials. Stat Med 2001;20:351–65.doi:10.1002/1097-0258(20010215)20:3<351::AID-SIM797>3.0.CO;2-C
      OpenUrlCrossRefPubMedWeb of Science
    31. ↵
      2. Odd DE ,
      3. Doyle P ,
      4. Gunnell D , et al
      . Risk of low Apgar score and socioeconomic position: a study of Swedish male births. Acta Paediatr 2008;97:1275–80.doi:10.1111/j.1651-2227.2008.00862.x
      OpenUrlPubMed
    32. ↵
      2. Odd DE ,
      3. Lewis G ,
      4. Whitelaw A , et al
      . Resuscitation at birth and cognition at 8 years of age: a cohort study. Lancet 2009;373:1615–22.doi:10.1016/S0140-6736(09)60244-0
      OpenUrlCrossRefPubMedWeb of Science
    33. ↵
      2. Hussey MA ,
      3. Hughes JP
      . Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials 2007;28:182–91.doi:10.1016/j.cct.2006.05.007
      OpenUrlCrossRefPubMedWeb of Science
    34. ↵
      2. Barker D ,
      3. D'Este C ,
      4. Campbell MJ , et al
      . Minimum number of clusters and comparison of analysis methods for cross sectional stepped wedge cluster randomised trials with binary outcomes: A simulation study. Trials 2017;18:119.doi:10.1186/s13063-017-1862-2
    35. ↵
      2. Hemming K ,
      3. Taljaard M ,
      4. Forbes A
      . Analysis of cluster randomised stepped wedge trials with repeated cross-sectional samples. Trials 2017;18:101.doi:10.1186/s13063-017-1833-7
    36. ↵
      2. Nickless A ,
      3. Voysey M ,
      4. Geddes J , et al
      . Mixed effects approach to the analysis of the stepped wedge cluster randomised trial-Investigating the confounding effect of time through simulation. PLoS One 2018;13:e0208876.doi:10.1371/journal.pone.0208876
    37. ↵
      2. Hemming K ,
      3. Haines TP ,
      4. Chilton PJ , et al
      . The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. BMJ 2015;350:h391.doi:10.1136/bmj.h391
    38. ↵
      2. Brown CA ,
      3. Lilford RJ
      . The stepped wedge trial design: a systematic review. BMC Med Res Methodol 2006;6:54.doi:10.1186/1471-2288-6-54
    39. ↵
      2. Handley MA ,
      3. Schillinger D ,
      4. Shiboski S
      . Quasi-experimental designs in practice-based research settings: design and implementation considerations. J Am Board Fam Med 2011;24:589–96.doi:10.3122/jabfm.2011.05.110067
      OpenUrlAbstract/FREE Full Text
    40. ↵
      2. Craig P ,
      3. Dieppe P ,
      4. Macintyre S , et al
      . Developing and evaluating complex interventions: new guidance, 2006. Available: https://www.mrc.ac.uk/documents/pdf/complex-interventions-guidance/
    41. ↵
      2. Kristunas CA ,
      3. Hemming K ,
      4. Eborall HC , et al
      . The use of feasibility studies for stepped-wedge cluster randomised trials: protocol for a review of impact and scope. BMJ Open 2017;7:e017290.doi:10.1136/bmjopen-2017-017290
    42. ↵
      About the Fetal Monitoring programme. Available: https://www.e-lfh.org.uk/programmes/electronic-fetal-monitoring [Accessed 11/05/2018].
    43. ↵
      2. Pyykönen A ,
      3. Gissler M ,
      4. Jakobsson M , et al
      . Determining obstetric patient safety indicators: the differences in neonatal outcome measures between different-sized delivery units. BJOG 2014;121:430–7.doi:10.1111/1471-0528.12507
      OpenUrl
    44. ↵
      2. Draycott T
      . Not all training for obstetric emergencies is equal, or effective. BJOG 2017;124.doi:10.1111/1471-0528.14391
    45. ↵
      2. Shoushtarian M ,
      3. Barnett M ,
      4. McMahon F , et al
      . Impact of introducing Practical Obstetric Multi-Professional Training (PROMPT) into maternity units in Victoria, Australia. 121. BJOG: An International Journal of Obstetrics & Gynaecology, 2014: 1710–8.doi:10.1111/1471-0528.12767
      OpenUrl
    46. ↵
      1. Republic of the Philippines Phillippine Information Agency
      . PROMPT implementation reduces maternal, neonatal mortality Quezon City: Republic of the Philippines, 2018. Available: http://pia.gov.ph/news/articles/1005787 [Accessed 08 May 2018].
    47. ↵
      2. Hawe P ,
      3. Shiell A ,
      4. Riley T
      . Theorising interventions as events in systems. Am J Community Psychol 2009;43:267–76.doi:10.1007/s10464-009-9229-9
      OpenUrlCrossRefPubMedWeb of Science
    48. ↵
      2. Wells M ,
      3. Williams B ,
      4. Treweek S , et al
      . Intervention description is not enough: evidence from an in-depth multiple case study on the untold role and impact of context in randomised controlled trials of seven complex interventions. Trials 2012;13.doi:10.1186/1745-6215-13-95
    49. ↵
      2. Bauman LJ ,
      3. Stein REK ,
      4. Ireys HT
      . Reinventing fidelity: The transfer of social technology among settings. Am J Community Psychol 1991;19:619–39.doi:10.1007/BF00937995
      OpenUrlCrossRefPubMedWeb of Science
    50. ↵
      2. Dixon-Woods M ,
      3. McNicol S ,
      4. Martin G
      . Overcoming challenges: Lessons from the Health Foundation’s improvement programme evaluations and relevant literature to improving quality. London: Health Foundation, 2012.
    51. ↵
      2. Dixon-Woods M
      . The problem of context in quality improvement: Perspectives on context. London: Health Foundation, 2014: 87–101.
    52. ↵
      2. Liberati EG ,
      3. Tarrant C ,
      4. Willars J , et al
      . How to be a very safe maternity unit: An ethnographic study. Soc Sci Med 2019;223:64–72.doi:10.1016/j.socscimed.2019.01.035
      OpenUrlCrossRefPubMed
    53. ↵
      2. Greenhalgh T ,
      3. Papoutsi C
      . Spreading and scaling up innovation and improvement. BMJ 2019;365:l2068.doi:10.1136/bmj.l2068
    54. ↵
      2. Collins K ,
      3. Moore A ,
      4. Winter C , et al
      . Understanding the implementation of obstetric emergency training: The THISTLE-Plus study. RCOG World Congress. Cape Town, South Africa: BJOG, 2017: 43–4.
    55. ↵
      2. Moore GF ,
      3. Audrey S ,
      4. Barker M , et al
      . Process evaluation of complex interventions: Medical Research Council guidance. BMJ 2015;350:h1258.doi:10.1136/bmj.h1258
    56. ↵
      2. Draycott T ,
      3. Sagar R ,
      4. Hogg S
      . The role of insurers in maternity safety. Best Pract Res Clin Obstet Gynaecol 2015;29.doi:10.1016/j.bpobgyn.2015.07.002
    57. ↵
      2. Yau CWH ,
      3. Pizzo E ,
      4. Morris S , et al
      . The cost of local, multi-professional obstetric emergencies training. Acta Obstet Gynecol Scand 2016;95:1111–9.doi:10.1111/aogs.12956
      OpenUrl
    58. ↵
      2. Weinstein L
      . A multifacited approach to improve patient safety, prevent medical errors and resolve the professional liability crisis. Am J Obstet Gynecol 2006;194:1160–5.doi:10.1016/j.ajog.2005.12.024
      OpenUrlPubMed

    Footnotes

    • Contributors All members of the THISTLE group were involved in the development of the study and contributed its implementation. CW, DS, JC and TD developed and coordinated the implementation of the intervention. EL designed and conducted the statistical analysis with the support of GM and JN. All authors contributed to the data interpretation, manuscript writing and review and approved its final version. SB is the chief investigator and SB and TD are the guarantors. EL, CW, GM, DS,PL, JN, SB and TD acquired the funding.

    • Funding The article outlines independent research funded by the Chief Scientist Office (CZH/4/893).

    • Disclaimer The views expressed are those of the authors and the study sponsor and funders have had no role in the study design, collection, management analysis and interpretation of data, writing of the report and decision to submit the report for publication.

    • Competing interests EL is an employee of the University of Bristol which receives funding from PROMPT charity to pay part of EL’s salary. CW is seconded from North Bristol NHS Trust as the Lead Research Midwife to the PROMPT charity. DS is an invited member of GLOBE, an initiative funded by Ferring. TD is a Trustee of the PROMPT Maternity Foundation charity, which provides PROMPT Training. He is a consultant for Limbs & Things and Laerdal and paid speaker for Ferring Pharmaceuticals. CW, DS and JC are members of the PROMPT Maternity Foundation charity. The remaining authors have no competing interests.

    • Patient consent for publication Not required.

    • Ethics approval Ethics approval was obtained: REC reference 13/NS/0111.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data availability statement The study data are saved on the National Services Scotland Safe Haven, only accessible by ISD trained and approved member and will be archived as per ISD regulations. Further information on how to access these data and use the Safe Haven are available here: http://www.isdscotland.org/Products-and-Services/EDRIS/.

    Linked Articles