You are here

  • Home
  • Online First
  • Quality and reporting of large-scale improvement programmes: a review of maternity initiatives in the English NHS, 2010–2023

Article Text

Article menu
Original research
Quality and reporting of large-scale improvement programmes: a review of maternity initiatives in the English NHS, 2010–2023
  1. James McGowan1,
  2. Bothaina Attal1,
  3. Isla Kuhn1,2,
  4. Lisa Hinton3,
  5. Tim Draycott4,
  6. Graham P Martin1,
  7. Mary Dixon-Woods1
  1. 1The Healthcare Improvement Studies Institute (THIS Institute), Department of Public Health and Primary Care, University of Cambridge, Cambridge, UK
  2. 2School of Clinical Medicine, University of Cambridge, Cambridge, UK
  3. 3Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
  4. 4Department of Women's Health, North Bristol NHS Trust, Westbury on Trym, Bristol, UK
  1. Correspondence to Professor Mary Dixon-Woods, THIS Institute, University of Cambridge School of Clinical Medicine, Cambridge, UK; md753{at}medschl.cam.ac.uk

Abstract

Background Large-scale improvement programmes are a frequent response to quality and safety problems in health systems globally, but have mixed impact. The extent to which they meet criteria for programme quality, particularly in relation to transparency of reporting and evaluation, is unclear.

Aim To identify large-scale improvement programmes focused on intrapartum care implemented in English National Health Service maternity services in the period 2010–2023, and to conduct a structured quality assessment.

Methods We drew on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidance to inform the design and reporting of our study. We identified relevant programmes using multiple search strategies of grey literature, research databases and other sources. Programmes that met a prespecified definition of improvement programme, that focused on intrapartum care and that had a retrievable evaluation report were subject to structured assessment using selected features of programme quality.

Results We identified 1434 records via databases and other sources. 14 major initiatives in English maternity services could not be quality assessed due to lack of a retrievable evaluation report. Quality assessment of the 15 improvement programmes meeting our criteria for assessment found highly variable quality and reporting. Programme specification was variable and mostly low quality. Only eight reported the evidence base for their interventions. Description of implementation support was poor and none reported customisation for challenged services. None reported reduction of inequalities as an explicit goal. Only seven made use of explicit patient and public involvement practices, and only six explicitly used published theories/models/frameworks to guide implementation. Programmes varied in their reporting of the planning, scope and design of evaluation, with weak designs evident.

Conclusions Poor transparency of reporting and weak or absent evaluation undermine large-scale improvement programmes by limiting learning and accountability. This review indicates important targets for improving quality in large-scale programmes.

  • health services research
  • healthcare quality improvement
  • health policy
  • obstetrics and gynecology
  • womens health

Data availability statement

Data are available upon reasonable request. Please contact the first author.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjqs-2023-016606

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    WHAT IS ALREADY KNOWN ON THIS TOPIC

    • Large-scale improvement programmes are a key strategy for addressing unwarranted variations in quality and safety of care, but their impact is mixed and often limited.

    • Previous research suggests a number of features of improvement programmes that need to be optimised, but how well these quality criteria are routinely met remains unknown.

    WHAT THIS STUDY ADDS

    • Many large-scale maternity improvement initiatives in the English National Health Service—including some major national programmes of the last decade—lack an evaluation report.

    • Where an evaluation report was available, quality and design of programmes against prespecified criteria was highly variable, often demonstrating significant weaknesses.

    HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

    • Poor transparency of reporting and weak evaluation in large-scale improvement programmes undermine learning and accountability; explicit attention to features of quality is necessary to improve the design, conduct and impact of large-scale programmes.

    Introduction

    Variations in quality and safety of healthcare have remained troubling and persistent across health systems globally. Efforts to address these challenges often take the form of large-scale improvement programmes,1–4 including, for example, multiorganisational collaborative approaches, major initiatives commissioned by policy and professional bodies, implementation programmes and research projects. These programmes are variably effective, with often disappointing results.5–10 Clarity is, however, now emerging on some of the key features of ‘what good looks like’ for such programmes.4 10–13 In this article, we report a study that both identifies large-scale improvement initiatives in a clinical area experiencing major patient safety challenges, and offers a structured quality assessment of improvement programmes where an evaluation report was retrievable.

    The available literature suggests that a number of features are especially important in large-scale improvement programmes. First, such programmes should be well specified and reported14–16 to ensure shared understanding of what the programme comprises and its mechanisms.2 10 17 A second feature of high-quality improvement programmes is that the interventions they use and their delivery should be supported by best available clinical evidence.11 18–21 Third, high-quality programmes should recognise and meet the requirements for implementation support in participating organisations.2 22–25 Such support needs to be sensitive to the highly heterogenous nature of local capability, which has been implicated in variable responses to improvement programmes,2 10 16 22–24 with lower performing organisations having distinctive support needs.11 13 25–31 Fourth, consistent with published policy objectives,32–34 programmes should explicitly address inequalities between socioeconomic and ethnic groups.35 Fifth, patient and public involvement (PPI) has an important role in enhancing the impact of improvement efforts.36–42 Sixth, improvement programmes benefit from use of formal published theories, models and frameworks from implementation science to guide their work.43 Finally, an important feature of good improvement programmes is a commitment to sound evaluation,11 20 21 including, where possible, assessment of effectiveness, process evaluation and economic evaluation.44–46

    The extent to which large-scale improvement programmes routinely meet these seven criteria is unclear. Many programmes, including those commissioned or delivered by national-level organisations, are conducted in a context where expectations of some features (eg, programme specification) may be insufficiently explicit, and incentives for high-quality evaluation and reporting may be lacking.14 15

    English National Health Service (NHS) maternity services are an important example of where quality problems are especially prominent in public discourse47–49 arising from high-profile organisational failures,50–53 evidence of persistent unwarranted variation in outcomes,54–56 rising clinical negligence claims,49 culture and workforce challenges57 58 and inequalities linked to socioeconomic status and ethnicity.33 35 59–63 These challenges have not yet proved tractable, despite multiple large-scale improvement initiatives.64 65 Maternity services are therefore an important setting in which to assess quality of large-scale improvement programmes, particularly in relation to their reporting and evaluation.

    We aimed to identify large-scale improvement programmes that had been implemented in English NHS maternity services between 2010 and 2023 and, for those with an available evaluation report, to conduct a structured quality assessment based on the selected features identified above.

    Methods

    Design

    Our design was a review with two components: a search for large-scale improvement initiatives implemented in English NHS maternity services in the period 2010–2023 and, for those that met definitional criteria as improvement programmes and had a retrievable evaluation report, a structured quality assessment. We focused specifically on programmes that primarily addressed quality or safety of intrapartum care, which has been consistently implicated in variations in adverse clinical outcomes in maternity care.66–68 Initiatives primarily focused on antenatal or postnatal care were therefore not in scope.

    Our initial exploratory work found that the majority of maternity improvement initiatives were not research projects and had not been reported in the academic literature; relevant information was mostly available in diverse sources such as websites, policy documents and programme reports. To ensure that both our search and our assessment of programmes was nonetheless structured and systematic, our review was informed by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension for Scoping Reviews guidance69 (online supplemental material A). We also completed all five steps in the Arksey and O’Malley framework,70 although not sequentially.

    Supplemental material

    As our review was not intended as a full scoping review of research, we did not register the review in an online database; we did, however, produce a protocol that was used to guide the conduct of the review (online supplemental material B).

    Supplemental material

    Eligibility criteria

    We developed prospective criteria to guide the identification of eligible improvement programmes and sources of evidence.

    Identification of improvement programmes

    Initial scoping identified that a large number of highly heterogeneous improvement efforts had taken place in English NHS maternity services in the period we were studying. The following types of initiatives, strategies and interventions were excluded from our review: incident investigation and inspection programmes; national clinical audits and confidential enquiries (eg, the Mothers and Babies: Reducing Risk through Audits and Confidential Enquiries - MBRRACE programme)71; organisational restructuring, major system change and service transformation programmes; health technology assessments and trials of digital technologies without an existing evidence base; clinical guidelines or recommendations without an accompanying implementation programme; and single-site quality improvement projects. Additionally, programmes implemented in clinical specialties other than maternity, outside the English NHS, or implemented in full before 2010 were excluded.

    Only initiatives that met our definitional criteria as improvement programmes, had an evaluation report available and focused on intrapartum care were eligible for quality assessment. For this purpose, we defined ‘improvement programmes’ as encompassing a set of planned activities applied at a scale larger than local quality improvement projects, and requiring participation of more than one organisation or clinical service (see box 1 for our full definition).1–3 72

    Box 1

    Definition of ‘improvement programme’

    For the purpose of our study, we defined a healthcare improvement programme as a set of planned activities:

    • Seeking to address a known quality or safety deficit; or seeking to implement evidence-based recommendations or standards of care or practice.

    • Implemented at scale, that is, in two or more healthcare organisations or clinical services.

    • With the characteristics of an organised programme, for example, with a structured set of goals, resources, a programme team and report.

    • Primarily concerned with improving clinical care quality or safety including structures, processes or outcomes.

    Only improvement programmes where an evaluation report could be retrieved were included in our quality assessment, since exploratory work indicated it would not be possible to make reliable judgements about programme quality and reporting without such a report. We defined an ‘evaluation report’ as a published assessment of programme design, implementation or outcomes, including formative and/or summative evaluation activities.44 We classified reports as ‘retrievable’ where they were available for full-text review following their identification in search results, or were otherwise publicly available (eg, published on organisational websites). In determining eligibility for quality assessment, evaluation reports were included without regard to where they had been published (eg, in academic or grey literature) or to the design and quality of the evaluation.

    To ensure comprehensive quality assessment of programmes where an evaluation report was available, we supplemented our analysis of evaluation reports with available information from policy reports, programme reports, website entries, peer-reviewed research articles, reviews and study protocols. We excluded editorials, viewpoints, commentaries and letters, non-English language articles and sources published before 2010. Consistent with our focus on intrapartum care, we also excluded sources that primarily addressed quality or safety of care in the antenatal or postnatal periods, or that had only limited focus on intrapartum care (eg, the Getting It Right First Time - GIRFT programme).

    Information sources and search strategy

    Information sources

    We searched two research databases, determining that this number was both proportionate to our aims and consistent with published guidance regarding the conduct of scoping studies.73 Database searches were performed in MEDLINE via Ovid and CINAHL via Ebsco from 1 January 2010 to 8 February 2023. These databases were chosen because of their high subject relevance to maternity care. Subject headings (eg, Medical Subject Headings) and free text search terms and synonyms were included. We did not apply restrictions to publication type. Filters were applied for ‘England’ or ‘NHS’, adapted from Ayiku et al.74 The database searches were designed and performed by a health librarian (IK) in collaboration with JM.

    A series of structured searches were performed to identify literature or other information relevant to maternity improvement programmes in Google, Google Scholar and websites of national organisations active in UK maternity quality and safety, supplemented by purposive hand searches. Online searches were based on the search strings ‘maternity safety programme’, ‘maternity quality programme’ and ‘maternity improvement programme’, and were performed independently by two researchers (JM and BA).

    Search strategy

    Our search strategy was based on a modified ‘PICOS’ framework:

    • Patient population—women receiving NHS care during the intrapartum period.

    • Intervention—quality and safety improvement programmes.

    • Comparison—not applicable.

    • Outcome—clinical and other outcomes related to quality and safety of maternity care.

    • Setting—NHS maternity services in England.

    Four studies identified during preliminary scoping searches were used as ‘golden bullets’75 to assess and improve the sensitivity and specificity of the search strings in identifying relevant literature.76–79 We supplemented structured searches with forward and backward citation tracking of a purposively selected sample of included studies to improve the sensitivity of the search.

    The full list of information sources, search strategies for database and grey literature searches and search record templates are provided in online supplemental material C.

    Supplemental material

    Eligibility screening

    Bibliographic database search results were deduplicated in EndNote, imported into Rayyan80 and screened on the basis of title and abstract. Screening of search results from non-bibliographic sources was performed onscreen by JM and BA; for each search, the first 100 (Google Search and organisational websites) and 500 (Google Scholar) search results were screened and unique sources identified by consensus. Screening of all search results and full-text sources to identify improvement programmes eligible for assessment and relevant sources of evidence was performed independently by two researchers (JM and BA); disagreements regarding the eligibility of both programmes and sources were resolved by discussion.

    Data categories and charting

    Data categories

    For improvement programmes eligible for quality assessment, we charted data under the seven categories in table 1. These categories correspond to selected features of quality we had identified from the wider improvement literature, as well as identifying basic programme characteristics.

    View this table:
    Table 1

    Data charting framework

    View this table:
    Table 2

    Multi-item checklist for assessment of programme specification (modified TIDieR checklist)

    View this table:
    Table 3

    Multi-item checklist for assessment of programme evaluation

    Our assessment of programmes was supported by published standards in two areas. First, we supported our assessment of programme specification (category 1 in the charting framework) and implementation support (category 3) by using a modified Template for Intervention Description and Replication (TIDieR) checklist,81 which was adapted to apply to improvement programmes (table 2).

    Second, our assessment of programme evaluation (category 7 in the charting framework) was supported by a multi-item checklist informed by UK government guidance on programme evaluation (the ‘Magenta Book’)44 (table 3).

    Charting process

    The charting process was supported by a tool built in Microsoft Excel that mapped to the charting framework in table 1 (online supplemental material D). Consistent with scoping review methodology,70 we developed the data items for extraction into the charting framework iteratively, modifying them as new data were identified and analysis progressed. The charting tool was piloted using a small sample (n=2) of sources and amended prior to formal charting. Two researchers (JM and BA) independently charted all data for six of the seven data categories; data relating to use of theories, models and frameworks were charted by JM. Disagreements in assessment gradings were resolved by discussion.

    Supplemental material

    Appraisal of evidence and reporting of findings

    Though conducting a quality assessment of eligible programmes (based on the seven features in table 1) was a key objective of our analysis, we did not seek to review evidence of effectiveness of programmes, nor did we aim to conduct an appraisal of the methodological quality of individual sources of evidence, as these were not goals of our study. Consistent with the norm in scoping reviews,70 we also did not formally aggregate or synthesise evidence, instead developing summaries of the data organised by the charting framework. Key findings and themes are reported under the categories of this framework in the Results section82 and are summarised in a series of supplemental tables.

    Patient and public involvement

    Patients and the public were not involved in the design or conduct of the review.

    Results

    We identified 1434 records via bibliographic databases and searches of the grey literature and other sources. From these, 93 full-text sources were retrieved and assessed for eligibility, including one evaluation report that was not initially available, but was subsequently retrieved through personal correspondence.83 The process by which sources of evidence were identified and screened to determine their eligibility is summarised in our PRISMA diagram (figure 1).

    Figure 1
    Figure 1

    Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram (adapted from Page et al [109]). NHS, National Health Service.

    Following full-text eligibility screening, we found 38 improvement initiatives (reported across 50 sources) that could not be included in the quality assessment. These initiatives are documented, together with exclusion reasons, in online supplemental material E. Of these, 14 initiatives—including most major initiatives in English maternity services that were implemented during the time period under study, such as the NHS England’s Maternity and Neonatal Safety Improvement Programme and the Maternity Safety Support Programme—lacked a retrievable evaluation report. A further 13 initiatives did not meet our definitional criteria as improvement programmes (eg, because they did not report implementation in two or more clinical services). We also excluded nine initiatives that focused primarily on antenatal or postnatal care from the quality assessment, since our scope was limited to intrapartum care. Two other initiatives that had not been implemented in the relevant setting or time period were also excluded at this stage.

    Supplemental material

    We identified 15 initiatives implemented in maternity services in England between 2010 and 2023 that had a principal focus on intrapartum care, met our definitional criteria as improvement programmes and had a retrievable evaluation report. These 15 initiatives were included in the structured quality assessment and were reported across 43 sources of evidence. Of these sources, grey literature constituted the majority (24). Peer-reviewed academic journal articles constituted 19 sources, with nine taking the form of original research articles reporting findings of evaluations.30 76–79 84–87 Other article types included study protocols (n=5),88–92 quality improvement reports (n=3)93–95 and review articles (n=2).96 97

    We report the findings of our quality assessment of the 15 programmes organised around the seven features of ‘what good looks like’ below, with additional data in online supplemental material F.

    Supplemental material

    Feature 1: quality of programme specification

    Reporting of the basic characteristics of the programmes included in our quality assessment was reasonable (online supplemental material F: table 1). For example, it was possible to identify the clinical setting, target recipients, programme type and source of funding for all 15 programmes. Information on the scale of the programme was available for 14 of the 15 programmes, and on time period of implementation for 13 programmes.

    Beyond these basic characteristics, completeness of programme description varied markedly between programmes and was generally of low quality (online supplemental material F: table 2). No programme described all 10 items of the modified TIDieR checklist in full. Five of the included programmes provided no description for half of the checklist items. Two programmes (Saving Babies’ Lives Care Bundle and Prevention of cerebral palsy in pre-term Labour - PReCePT) provided full description of six checklist items, while three programmes (Maternity Incentive Scheme, Maternity Safety Training Fund and the Safer Births Project) did not describe any checklist item in full.

    Completeness of description also varied between checklist items. Goals (described in full for 11 programmes) were generally well described, as were mechanisms of action and theories of change (described in full for eight programmes). In contrast, frequency, duration and time period of local implementation, local tailoring and modifications, processes for assessing or maintaining fidelity and outcome of fidelity assessment were described poorly. For example, no programmes offered a full description of frequency, duration and time period of local implementation or of local tailoring and modifications, and only three programmes offered any level of description for the latter item. Items relating to fidelity were particularly poorly described; only two programmes (Saving Babies’ Lives Care Bundle and PReCePT) offered any level of description of a fidelity assessment outcome. Only the Saving Babies’ Lives Care Bundle provided a full description of both the processes and outcome of fidelity assessment.

    Feature 2: use of evidence-based interventions

    Programmes varied in the extent to which they explicitly based their improvement interventions on evidence. Of the 15 programmes we assessed, only eight reported that their interventions were developed with explicit reference to published evidence (online supplemental material F: tables 3 and 4). Of these, most (n=6) based their interventions on recommendations from national clinical guidance or quality standards, for example, those published by Royal Colleges and the National Institute for Health and Care Excellence. Two programmes (PReCePT and Perinatal Excellence to reduce injury in preterm birth - PERIPrem) cited a range of studies in support of their interventions, including systematic reviews and meta-analyses, randomised controlled trials and cohort studies.79 95 98 99 The remaining seven programmes did not cite the evidence base for interventions used, if any.

    Feature 3: description of implementation support for services

    Completeness of description of the implementation support available to clinical services (final column, online supplemental material F: table 2) was poor. Only three programmes (Obstetric anal sphincter injury Care Bundle - OASI-CB, PReCePT and PERIPrem) were judged to have described implementation support and activities in full. Whether or not implementation support had been provided was unclear for seven programmes, as no level of description was offered.

    We did not identify any programme report that described customisation of implementation support for challenged services. The PReCePT programme did offer enhanced implementation support to a subset of participating services, though the authors did not report whether this was targeted specifically at challenged services.100 Recruitment to the Labour Ward Leadership programme was partially informed by Care Quality Commission (CQC) inspection reports, with ‘Trusts with identified problems’ given priority.83 Two programmes91 100 101 accounted for variation in local service contexts in their evaluation methods by using, among other criteria, CQC inspection ratings to inform sampling of study participants. The interim evaluation of the Maternity Incentive Scheme102 stated that additional support was provided to Trusts that were not in compliance with all 10 incentivised actions to enable them to ‘achieve full compliance’, but we did not find evidence that this support was targeted to support challenged services. In contrast, NHS trusts in ‘special measures’ and those in receipt of support from national regulators were specifically excluded from participating in the Each Baby Counts Learn & Support programme.103 An incidental finding was the potential for programmes to exacerbate inequalities of resourcing between services, as noted by a PERIPrem report (parentheses added):

    In Trusts where there was a pre-existing QI (quality improvement) culture and a desire to embed new practice and change, implementation was easier. Those with active hospital QI teams were able to access additional support and training for the project.104

    Feature 4: commitment to reducing inequalities

    Although we identified several evaluation reports that adjusted for socioeconomic status, ethnicity or levels of deprivation in their analysis,78 79 84 95 100 101 105 we did not find any examples among the 15 programmes we assessed that identified the reduction of health or care inequalities as an explicit goal (online supplemental material F: table 4).

    Feature 5: patient and public involvement

    Of the 15 programmes we assessed, only seven made explicit use of PPI practices. These included five that demonstrated comprehensive attention to involvement at all stages of the programme (including design, development, implementation, evaluation and dissemination). The remaining eight programmes did not mention of the role of patients or the public (online supplemental material F: table 4).

    Feature 6: use of formal published theories, models or frameworks

    A minority (n=6) of assessed programmes made explicit use of formal published theories, models or frameworks from implementation science to guide programme implementation or evaluation (online supplemental material F: tables 4 and 5).106 We identified 12 documented examples of use of theories, models or frameworks from four of Nilsen’s proposed five categories,107 including implementation theories (n=6), classic theories (n=2), determinant frameworks (n=2) and evaluation frameworks (n=2).

    Feature 7: programme evaluation

    Programmes varied in their reporting of the planning, scope, design and conduct of evaluation (online supplemental material F: table 6). Evidence of a prospective evaluation plan (eg, in the form of a published protocol) was identified for seven of the 15 programmes, and four programmes conducted a pilot that informed programme development or implementation.

    The evaluations we assessed were dominated by weak designs, often relying on post hoc methods of data collection, such as self-report questionnaires and staff surveys. Of the 19 peer-reviewed research articles identified (which collectively reported on nine programmes), five76–79 84 reported evaluations of effectiveness, but only two studies reported on cost-effectiveness.78 79 Only three evaluation reports76 84 100 (including one preprint100) employed a randomised design. Three evaluations employed quasiexperimental approaches.78 79 92 Only four articles reported findings from process evaluations, implementation research or qualitative studies.30 85–87

    Discussion

    Large-scale improvement programmes have been a key strategy for addressing quality deficits in healthcare globally, but the programmes assessed in our review of one exemplar clinical area often fell short on key features of quality. Though a large number of improvement initiatives have been undertaken in a 13-year period in maternity services, a particularly challenged clinical specialty in the English NHS, many—including a large number of major national programmes of the last decade—did not meet a basic requirement of a retrievable evaluation report. This represents a major threat to learning and accountability. Among programmes with a focus on intrapartum care that offered some form of evaluation report and could be assessed, there was considerable variability and very often evident flaws in transparency and quality of programme specification, use of evidence-based interventions, implementation support, PPI, use of formal published theories, models and frameworks, and evaluation. Notably, no programme that we quality assessed had explicitly set reduction of inequality as a goal. Our findings are unlikely to be unique to maternity settings or to the English NHS. These findings, and the methods used to generate them, are likely to be of relevance to many other clinical areas targeted by large-scale improvement programmes in healthcare settings internationally.

    A first step in improving the quality of improvement programmes is to ensure that they are sufficiently well specified to permit identification of their components and the mechanisms through which they work, not least so that they can be scaled with fidelity if shown to be effective,10 14 108 and modified or abandoned if not effective. Programmes included in our quality assessment demonstrated highly variable completeness of programme description, with some demonstrating weaknesses across several items of a modified TIDieR checklist.81 We also identified an important lack of transparency in reporting relating to the implementation support available to participating services, despite its recognised role in effective improvement,25 29 including in maternity care.30 100 Crucially, despite recurrent reported organisational degradations in NHS maternity services, we were unable to identify any examples of adaptation of implementation specifically to account for the context of challenged services, with one programme actively excluding challenged units.103 The extent to which programmes explicitly grounded their interventions in evidence or drew on theories, models or frameworks from implementation science43 to guide implementation and evaluation was also highly variable, with many examples of poor reporting practice.

    Second, a commitment to evaluation and public reporting of all findings should be seen as fundamental to high-quality commissioning of large-scale improvement programmes in healthcare. Many (14) of the initiatives we identified—including some of the high-profile national maternity programmes of the last decade—lacked a retrievable evaluation report. In these cases, it is not possible to determine whether the programmes worked (made a difference to outcomes) and should be scaled, to assess whether these programmes represent a good use of resources, or to identify how programme design or implementation might have been improved. Even where evaluation reports were available, they often demonstrated substantial weaknesses. Future programme design and delivery should be organised to facilitate the use of rigorous evaluation designs that allow reliable assessments to be made about effectiveness and cost-effectiveness, as well as what works, what doesn’t and why across diverse clinical settings.44

    Third, despite repeated policy commitments to improve equity,32 34 our findings add to concerns about inequalities in NHS maternity care.33 35 59 We found no examples of improvement programmes included in our quality assessment that identified the reduction of health and care inequalities as an explicit goal. Indeed, there was some evidence of programme design having potential to contribute to widening inequalities between high-performing and low-performing services, which is likely to impede efforts aimed at improving equity.33 Finally, despite the emphasis placed by national policy on including those who use maternity services in the design and delivery of improvement programmes,41 42 PPI appeared to be lacking in over half of the programmes we assessed. Improving the impact of large-scale improvement efforts on the quality and safety of care in future will likely require these gaps between these enshrined policy objectives and programme design to be closed.

    Strengths and limitations

    Our adaptation of the principles of scoping review methodology was successful in addressing our aims, given that most programmes we identified were not research projects and therefore unsuitable for full scoping or systematic review designs. The design of our search strategy was improved by specialist librarian input, by pilot testing the search strings to test their capacity to identify relevant literature, by selecting databases with relevant scope and by supplementing our structured searches with extensive hand searches. We sought to lend rigour to the charting process by requiring that assessment decisions be agreed by two researchers, by pilot testing the data charting tool to improve its reliability and by basing our assessment criteria, where possible, on published standards relating to programme specification81 and evaluation.44

    Our selection of the TIDieR checklist81—developed to improve published descriptions of healthcare interventions, including complex interventions—on which to base our assessments of reporting was appropriate given the sociotechnical nature of improvement programmes. Our modified checklist is likely to be valuable in future studies to enable better description of features specific to improvement programmes. Relatedly, our definition of ‘improvement programme’ offers clarity to those engaged in programme commissioning, design and evaluation regarding specific aspects of these interventions that distinguish them from other types of improvement intervention in healthcare.

    We acknowledge several limitations. It is possible that relevant sources of evidence relevant to eligible programmes were missed by our search strategy. The list of quality features we identified was necessarily selective and may not be comprehensive. It remains possible that researcher-related factors may have biased our assessments in some non-transparent way. Practical considerations meant our search was limited to programmes implemented in England since 2010, meaning that potentially important learning from programmes implemented elsewhere in the UK and prior to 2010 was excluded. Restriction of scope to intrapartum care may have excluded some programmes with different characteristics of quality and reporting.

    Conclusions

    Transparent reporting and high-quality evaluation are critical to learning and accountability in healthcare systems facing persistent quality of care challenges. This review of large-scale maternity improvement programmes in the English NHS since 2010 has identified widespread poor practice in programme design, transparency of reporting and evaluation. These findings are both cause for concern and unlikely to be unique to this clinical setting. Our study suggests important targets for improving the design, delivery, evaluation and reporting of large-scale programmes in healthcare to maximise their impact on quality and safety, ensure accountability, including for how resources are used, and better aggregate learning to improve care for patients.

    Data availability statement

    Data are available upon reasonable request. Please contact the first author.

    Ethics statements

    Patient consent for publication

    References

    1. ↵
      1. Ovretveit J,
      2. Gustafson D
      . Using research to inform quality programmes. BMJ 2003;326:759–61. doi:10.1136/bmj.326.7392.759
      OpenUrlFREE Full Text
    2. ↵
      1. Benn J,
      2. Burnett S,
      3. Parand A, et al
      . Studying large-scale programmes to improve patient safety in whole care systems: challenges for research. Social Science & Medicine 2009;69:1767–76. doi:10.1016/j.socscimed.2009.09.051
      OpenUrl
    3. ↵
      1. Ovretveit J
      . Evaluation of quality improvement programmes. Qual Saf Health Care 2002;11:270–5. doi:10.1136/qhc.11.3.270
      OpenUrlAbstract/FREE Full Text
    4. ↵
      1. Dixon-Woods M,
      2. McNicol S,
      3. Martin G
      . Ten challenges in improving quality in Healthcare: lessons from the health foundation’s programme evaluations and relevant literature. BMJ Qual Saf 2012;21:876–84. doi:10.1136/bmjqs-2011-000760
      OpenUrlAbstract/FREE Full Text
    5. ↵
      1. Peden CJ,
      2. Stephens T,
      3. Martin G, et al
      . Effectiveness of a national quality improvement programme to improve survival after emergency abdominal surgery (EPOCH): a stepped-wedge cluster-randomised trial. Lancet 2019;393:2213–21. doi:10.1016/S0140-6736(18)32521-2
      OpenUrlCrossRefPubMed
    6. ↵
      1. Power M,
      2. Tyrrell PJ,
      3. Rudd AG, et al
      . Did a quality improvement collaborative make stroke care better? A cluster randomized trial. Implement Sci 2014;9:40. doi:10.1186/1748-5908-9-40
      OpenUrlCrossRefPubMed
    7. ↵
      1. Bion J,
      2. Richardson A,
      3. Hibbert P, et al
      . Matching Michigan: a 2-year stepped Interventional programme to minimise central venous catheter-blood stream infections in intensive care units in England. BMJ Qual Saf 2013;22:110–23. doi:10.1136/bmjqs-2012-001325
      OpenUrlAbstract/FREE Full Text
    8. ↵
      1. Benning A,
      2. Ghaleb M,
      3. Suokas A, et al
      . Large scale organisational intervention to improve patient safety in four UK hospitals: mixed method evaluation. BMJ 2011;342:d195. doi:10.1136/bmj.d195
    9. ↵
      1. Benning A,
      2. Dixon-Woods M,
      3. Nwulu U, et al
      . Multiple component patient safety intervention in English hospitals: controlled evaluation of second phase. BMJ 2011;342:d199. doi:10.1136/bmj.d199
    10. ↵
      1. Lenguerrand E,
      2. Winter C,
      3. Siassakos D, et al
      . Effect of hands-on Interprofessional simulation training for local emergencies in Scotland: the THISTLE stepped-wedge design randomised controlled trial. BMJ Qual Saf 2020;29:122–34. doi:10.1136/bmjqs-2018-008625
      OpenUrlAbstract/FREE Full Text
    11. ↵
      1. Dixon-Woods M
      . How to improve healthcare improvement-an essay by Mary Dixon-woods. BMJ 2019;367:l5514. doi:10.1136/bmj.l5514
    12. ↵
      1. Brandrud AS,
      2. Nyen B,
      3. Hjortdahl P, et al
      . Domains associated with successful quality improvement in healthcare – a nationwide case study. BMC Health Serv Res 2017;17:648. doi:10.1186/s12913-017-2454-2
    13. ↵
      1. Vindrola-Padros C,
      2. Ledger J,
      3. Hill M, et al
      . The Special Measures for Quality and Challenged Provider Regimes in the English NHS: a rapid evaluation of a national improvement initiative for failing healthcare organisations. Int J Health Policy Manag 2022;11:2917–26. doi:10.34172/ijhpm.2022.6619
      OpenUrlCrossRefPubMed
    14. ↵
      1. Jones EL,
      2. Lees N,
      3. Martin G, et al
      . How well is quality improvement described in the perioperative care literature? A systematic review. Jt Comm J Qual Patient Saf 2016;42:196–206. doi:10.1016/s1553-7250(16)42025-8
      OpenUrlCrossRefPubMed
    15. ↵
      1. Jones EL,
      2. Dixon-Woods M,
      3. Martin GP
      . Why is reporting quality improvement so hard? A qualitative study in perioperative care. BMJ Open 2019;9:e030269. doi:10.1136/bmjopen-2019-030269
    16. ↵
      1. Skivington K,
      2. Matthews L,
      3. Simpson SA, et al
      . A new framework for developing and evaluating complex interventions: update of medical research Council guidance. BMJ 2021;374:n2061. doi:10.1136/bmj.n2061
    17. ↵
      1. Back J,
      2. Ross AJ,
      3. Duncan MD, et al
      . Emergency Department escalation in theory and practice: A mixed-methods study using a model of organizational resilience. Ann Emerg Med 2017;70:659–71. doi:10.1016/j.annemergmed.2017.04.032
      OpenUrlPubMed
    18. ↵
      1. Shojania KG,
      2. Grimshaw JM
      . Evidence-based quality improvement: the state of the science. Health Aff (Millwood) 2005;24:138–50. doi:10.1377/hlthaff.24.1.138
      OpenUrlAbstract/FREE Full Text
    19. ↵
      1. Hempel S,
      2. Bolshakova M,
      3. Turner BJ, et al
      . Evidence-based quality improvement: a scoping review of the literature. J Gen Intern Med 2022;37:4257–67. doi:10.1007/s11606-022-07602-5
      OpenUrlCrossRefPubMed
    20. ↵
      1. Marshall M,
      2. Pronovost P,
      3. Dixon-Woods M
      . Promotion of improvement as a science. Lancet 2013;381:419–21. doi:10.1016/S0140-6736(12)61850-9
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      1. Dixon-Woods M,
      2. Martin GP
      . Does quality improvement improve quality Future Hosp J 2016;3:191–4. doi:10.7861/futurehosp.3-3-191
      OpenUrlAbstract/FREE Full Text
    22. ↵
      1. Carter P,
      2. Ozieranski P,
      3. McNicol S, et al
      . How collaborative are quality improvement collaboratives: a qualitative study in stroke care. Implement Sci 2014;9:32. doi:10.1186/1748-5908-9-32
      OpenUrlCrossRefPubMed
    23. ↵
      1. Dixon-Woods M,
      2. Leslie M,
      3. Tarrant C, et al
      . Explaining Matching Michigan: an ethnographic study of a patient safety program. Implement Sci 2013;8:70. doi:10.1186/1748-5908-8-70
      OpenUrlCrossRefPubMed
    24. ↵
      1. Stephens TJ,
      2. Peden CJ,
      3. Pearse RM, et al
      . Improving care at scale: process evaluation of a multi-component quality improvement intervention to reduce mortality after emergency abdominal surgery (EPOCH trial). Implementation Sci 2018;13:142. doi:10.1186/s13012-018-0823-9
      OpenUrl
    25. ↵
      1. Greenhalgh T,
      2. Robert G,
      3. Macfarlane F, et al
      . Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 2004;82:581–629. doi:10.1111/j.0887-378X.2004.00325.x
      OpenUrlCrossRefPubMedWeb of Science
    26. ↵
      1. Kaplan HC,
      2. Brady PW,
      3. Dritz MC, et al
      . The influence of context on quality improvement success in health care: a systematic review of the literature. Milbank Q 2010;88:500–59. doi:10.1111/j.1468-0009.2010.00611.x Available: http://doi.wiley.com/10.1111/milq.2010.88.issue-4
      OpenUrlCrossRefPubMedWeb of Science
    27. ↵
      1. Dixon-Woods M
      . The problem of context in quality improvement London The Health Foundation; 2014.
    28. ↵
      1. Howarth E,
      2. Devers K,
      3. Moore G, et al
      . Contextual issues and qualitative research, in Challenges, solutions and future directions in the evaluation of service innovations in health care and public health. NIHR Journals Library, 2016.
    29. ↵
      1. Greenhalgh T,
      2. Papoutsi C
      . Spreading and scaling up innovation and improvement. BMJ 2019;365:l2068. doi:10.1136/bmj.l2068
    30. ↵
      1. Redwood S,
      2. Pithara-McKeown C,
      3. Stone T, et al
      . Scaling up an intervention to protect Preterm infants from neurodevelopmental disabilities — findings from a qualitative process evaluation comparing standard with enhanced quality improvement support packages for maternity units in England. Implement Sci 2023;18:19. doi:10.1186/s13012-023-01275-2
    31. ↵
      1. Vindrola-Padros C,
      2. Ledger J,
      3. Barbosa EC, et al
      . “The implementation of improvement interventions for “low performing” and “high performing” organisations in health, education and local government: a phased literature review”. Int J Health Policy Manag 2022;11:874–82. doi:10.34172/ijhpm.2020.197
      OpenUrl
    32. ↵
      1. NHS England
      . Equity and equality. Guidance for local maternity systems. London: NHS England and NHS Improvement, 2021.
    33. ↵
      1. Care Quality Commission
      . Safety, equity and engagement in maternity services. 2021.
    34. ↵
      1. NHS England
      . NHS pledges to improve equity for mothers and babies and race equality for staff. 2021.
    35. ↵
      1. Jardine J,
      2. Walker K,
      3. Gurol-Urganci I, et al
      . Adverse pregnancy outcomes attributable to socioeconomic and ethnic inequalities in England: a national cohort study. Lancet 2021;398:1905–12. doi:10.1016/S0140-6736(21)01595-6
      OpenUrlCrossRefPubMed
    36. ↵
      1. Greenhalgh T,
      2. Jackson C,
      3. Shaw S, et al
      . Achieving research impact through co-creation in community-based health services. Milbank Q 2016;94:392–429. doi:10.1111/1468-0009.12197
      OpenUrlPubMed
    37. ↵
      1. Ostrom E,
      2. Parks RB,
      3. Whitaker GP, et al
      . The public service production process: A framework for analyzing police services. Policy Studies Journal 1978;7:381. doi:10.1111/j.1541-0072.1978.tb01782.x Available: https://onlinelibrary.wiley.com/toc/15410072/7/s1
    38. ↵
      1. Langley J,
      2. Wolstenholme D,
      3. Cooke J
      . Collective making’ as knowledge mobilisation: the contribution of participatory design in the Co-creation of knowledge in Healthcare. BMC Health Serv Res 2018;18:585. doi:10.1186/s12913-018-3397-y
    39. ↵
      1. Robert G,
      2. Locock L,
      3. Williams O, et al
      . Co-producing and Co-designing. Cambridge: Cambridge University Press, 2022. doi:10.1017/9781009237024
    40. ↵
      1. Robert G,
      2. Donetto S,
      3. Williams O
      . Co-designing Healthcare services with patients. In: Loeffler E, Bovaird T, eds. The Palgrave Handbook of Co-production of Public Services and Outcomes. Cham, Switzerland: Palgrave Macmillan Cham, 2021: 313–333. doi:10.1007/978-3-030-53705-0
    41. ↵
      1. Greenhalgh T,
      2. Hinton L,
      3. Finlay T, et al
      . Frameworks for supporting patient and public involvement in research: systematic review and Co-design pilot. Health Expect 2019;22:785–801. doi:10.1111/hex.12888
      OpenUrlCrossRefPubMed
    42. ↵
      1. Mockford C,
      2. Staniszewska S,
      3. Griffiths F, et al
      . The impact of patient and public involvement on UK NHS health care: a systematic review. Int J Qual Health Care 2012;24:28–38. doi:10.1093/intqhc/mzr066
      OpenUrlCrossRefPubMedWeb of Science
    43. ↵
      1. Eccles MP,
      2. Mittman BS
      . Welcome to implementation science. Implementation Sci 2006;1. doi:10.1186/1748-5908-1-1
    44. ↵
      1. HM Treasury
      . Magenta Book: Central Government guidance on evaluation 2020. London,
    45. ↵
      1. Portela MC,
      2. Pronovost PJ,
      3. Woodcock T, et al
      . How to study improvement interventions: a brief overview of possible study types. BMJ Qual Saf 2015;24:325–36. doi:10.1136/bmjqs-2014-003620
      OpenUrlAbstract/FREE Full Text
    46. ↵
      1. Parry G,
      2. Coly A,
      3. Goldmann D, et al
      . Practical recommendations for the evaluation of improvement initiatives. Int J Qual Health Care 2018;30:29–36. doi:10.1093/intqhc/mzy021
      OpenUrlCrossRefPubMed
    47. ↵
      1. Parliamentary and Health Service Ombudsman
      . Spotlight on maternity care: your stories, your rights. Manchester; 2023.
    48. ↵
      1. House of Commons Health and Social Care Committee
      . The safety of maternity services in England, Fourth Report of Session 2021–22. 2021.
    49. ↵
      1. Yau CWH,
      2. Leigh B,
      3. Liberati E, et al
      . Clinical negligence costs: taking action to safeguard NHS Sustainability. BMJ 2020;368:m552. doi:10.1136/bmj.m552
    50. ↵
      1. Kirkup B
      . The report of the Morecambe Bay investigation. 2015.
    51. ↵
      1. Kirkup B
      . Reading the signals. maternity and neonatal services in East Kent – the report of the independent investigation. 2022.
    52. ↵
      1. Ockenden D
      . Findings, conclusions and essential actions from the independent review of maternity services at the Shrewbury and Telford hospital NHS trust. Our Final Report 2022.
    53. ↵
      1. NHS England
      . Independent Review of Maternity Services at Nottingham University Hospitals, . 2022Available: https://www.england.nhs.uk/midlands/publications/independent-review-of-maternity-services-at-nottingham-university-hospitals/
    54. ↵
      1. Care Quality Commission
      . The state of health care and adult social care in England 2021/22. 2022.
    55. ↵
      1. Proudlove NC,
      2. Goff M,
      3. Walshe K, et al
      . The signal in the noise: robust detection of performance “outliers” in health services. Journal of the Operational Research Society 2019;70:1102–14. doi:10.1080/01605682.2018.1487816
      OpenUrl
    56. ↵
      1. Geary R,
      2. Knight H,
      3. Carroll F, et al
      . A step-wise approach to developing indicators to compare the performance of maternity units using hospital administrative data. BJOG 2018;125:857–65. doi:10.1111/1471-0528.15013 Available: https://obgyn.onlinelibrary.wiley.com/toc/14710528/125/7
      OpenUrlPubMed
    57. ↵
      1. Care Quality Commission
      . Maternity survey. 2023.
    58. ↵
      1. Dixon-Woods M
      . Learning from maternity service failures at East Kent hospitals. BMJ 2022;379:o2755. doi:10.1136/bmj.o2755
    59. ↵
      1. Knight M,
      2. Bunch K,
      3. Patel R, et al
      . Saving Lives, Improving Mothers’ Care Core Report - Lessons learned to inform maternity care from the UK and Ireland Confidential Enquiries into Maternal Deaths and Morbidity 2018-20. National Perinatal Epidemiology Unit, University of Oxford Oxford, 2022.
    60. ↵
      1. Birthrights
      . Systemic racism, not Brokenbodies. an inquiry into racial injustice and human rights in UK maternity care; 2022.
    61. ↵
      1. Gohir S
      . INVISIBLE, maternity experiences of Muslim women from Racialised minority communities, A summary report. 2022.
    62. ↵
      1. Peter M,
      2. Wheeler R
      . The Black Maternity Experiences Survey. a nationwide study of black women’s experiences of maternity services in the United Kingdom. 2022.
    63. ↵
      1. House of Commons Women and Equalities Committee
      . Black maternal health, Third Report of Session 2022–23. 2023.
    64. ↵
      1. Department of Health and Social Care
      . Safer Maternity Care, Progress Report 2021. London, 2021.
    65. ↵
      1. Department of Health, Safer Maternity Care
      . The National Maternity Safety Strategy - Progress and Next Steps. London, 2017.
    66. ↵
      1. NHS Litigation Authority
      . Ten Years of Maternity Claims: An Analysis of NHS Litigation Authority Data. London: NHS Litigation Authority, 2018.
    67. ↵
      1. NHS Resolution
      . The second report: the evolution of the Early Notification Scheme. 2022.
    68. ↵
      1. NHS Resolution
      . Five years of Cerebral Palsy claims, A thematic review of NHS Resolution data. London, 2017.
    69. ↵
      1. Tricco AC,
      2. Lillie E,
      3. Zarin W, et al
      . PRISMA extension for Scoping reviews (PRISMA-SCR): checklist and explanation. Ann Intern Med 2018;169:467–73. doi:10.7326/M18-0850
      OpenUrlCrossRefPubMed
    70. ↵
      1. Arksey H,
      2. O’Malley L
      . Scoping studies: towards a methodological framework. International Journal of Social Research Methodology 2005;8:19–32. doi:10.1080/1364557032000119616
      OpenUrl
    71. ↵
      1. Knight M,
      2. Felker BK,
      3. Patel A, et al
      . Saving Lives, Improving Mothers’ Care Core Report - Lessons learned to inform maternity care from the UK and Ireland Confidential Enquiries into Maternal Deaths and Morbidity 2019-21. Oxford: National Perinatal Epidemiology Unit, University of Oxford, 2023.
    72. ↵
      1. Mohr JJ,
      2. Batalden PB
      . Improving safety on the front lines: the role of clinical microsystems. Qual Saf Health Care 2002;11:45–50. doi:10.1136/qhc.11.1.45
      OpenUrlAbstract/FREE Full Text
    73. ↵
      1. Peters M,
      2. Godfrey C,
      3. McInerney P, et al
      . Chapter 11: Scoping reviews section 11.2.5 search Strategy(2020 version). In: Aromataris E, Munn Z, eds. JBI Manual for Evidence Synthesis. JBI, 2020. doi:10.46658/JBIRM-190-01
    74. ↵
      1. Ayiku L,
      2. Levay P,
      3. Hudson T, et al
      . The MEDLINE UK filter: development and validation of a geographic search filter to retrieve research about the UK from OVID MEDLINE. Health Info Libr J 2017;34:200–16. doi:10.1111/hir.12187
      OpenUrlCrossRefPubMed
    75. ↵
      1. Zwakman M,
      2. Verberne LM,
      3. Kars MC, et al
      . Introducing PALETTE: an Iterative method for conducting a literature search for a review in palliative care. BMC Palliat Care 2018;17:82. doi:10.1186/s12904-018-0335-z
    76. ↵
      1. Ismail KMK,
      2. Kettle C,
      3. Macdonald SE, et al
      . Perineal assessment and repair longitudinal study (PEARLS): a matched-pair cluster randomized trial. BMC Med 2013;11:209. doi:10.1186/1741-7015-11-209
    77. ↵
      1. Rahman N,
      2. Vinayakarao L,
      3. Pathak S, et al
      . Evaluation of training programme uptake in an attempt to reduce obstetric anal sphincter injuries: the SUPPORT programme. Int Urogynecol J 2017;28:403–7. doi:10.1007/s00192-016-3158-6
      OpenUrl
    78. ↵
      1. Widdows K,
      2. Roberts SA,
      3. Camacho EM, et al
      . Stillbirth rates, service outcomes and costs of implementing NHS England’s Saving Babies' Lives care bundle in maternity units in England: a cohort study. PLoS ONE 2021;16:e0250150. doi:10.1371/journal.pone.0250150
    79. ↵
      1. Edwards HB,
      2. Redaniel MT,
      3. Sillero-Rejon C, et al
      . National PReCePT Programme: a before-and-after evaluation of the implementation of a national quality improvement programme to increase the uptake of magnesium sulfate in preterm deliveries. Arch Dis Child Fetal Neonatal Ed 2023;108:342–7. doi:10.1136/archdischild-2022-324579
      OpenUrlAbstract/FREE Full Text
    80. ↵
      1. Ouzzani M,
      2. Hammady H,
      3. Fedorowicz Z, et al
      . Rayyan—a web and mobile app for systematic reviews. Syst Rev 2016;5:210. doi:10.1186/s13643-016-0384-4
    81. ↵
      1. Hoffmann TC,
      2. Glasziou PP,
      3. Boutron I, et al
      . Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ 2014;348:bmj.g1687. doi:10.1136/bmj.g1687
    82. ↵
      1. Pope C,
      2. Mays N
      . Qualitative Research in Healthcare, 4th edn. 2020: Wiley Blackwell,
    83. ↵
      1. NHS England (Avoiding Term Admissions into Neonatal units programme), Royal College of Midwives, Royal College of Obstetricians and Gynaecologists
      . Labour ward leaders working together for safe care. pilot programmes: evaluation report; 2017.
    84. ↵
      1. Gurol-Urganci I,
      2. Bidwell P,
      3. Sevdalis N, et al
      . Impact of a quality improvement project to reduce the rate of obstetric anal sphincter injury: a multicentre study with a stepped-wedge design. BJOG: Int J Obstet Gynaecol 2020;128:584–92.
      OpenUrl
    85. ↵
      1. Lau YZ,
      2. Widdows K,
      3. Roberts SA, et al
      . Assessment of the quality, content and perceived utility of local maternity guidelines in hospitals in England implementing the Saving Babies' Lives care bundle to reduce stillbirth. BMJ Open Qual 2020;9:e000756. doi:10.1136/bmjoq-2019-000756
    86. ↵
      1. Bidwell P,
      2. Sevdalis N,
      3. Silverton L, et al
      . Women’s experiences of the OASI care bundle; a package of care to reduce severe perineal trauma. Int Urogynecol J 2021;32:1807–16. doi:10.1007/s00192-020-04653-2
      OpenUrl
    87. ↵
      1. Bidwell P,
      2. Thakar R,
      3. Gurol-Urganci I, et al
      . “Exploring clinicians' perspectives on the 'Obstetric Anal Sphincter Injury Care Bundle' national quality improvement programme: a qualitative study”. BMJ Open 2020;10:e035674. doi:10.1136/bmjopen-2019-035674
    88. ↵
      1. Bick DE,
      2. Kettle C,
      3. Macdonald S, et al
      . Perineal assessment and repair longitudinal study (PEARLS): protocol for a matched pair cluster trial. BMC Pregnancy Childbirth 2010;10:10. doi:10.1186/1471-2393-10-10
    89. ↵
      1. Bidwell P,
      2. Thakar R,
      3. Sevdalis N, et al
      . A multi-centre quality improvement project to reduce the incidence of obstetric anal sphincter injury (OASI): study protocol. BMC Pregnancy Childbirth 2018;18:331. doi:10.1186/s12884-018-1965-0
    90. ↵
      1. Widdows K,
      2. Reid HE,
      3. Roberts SA, et al
      . Saving Babies’ Lives project impact and results evaluation (SPiRE): a mixed methodology study. BMC Pregnancy Childbirth 2018;18:43. doi:10.1186/s12884-018-1672-x
    91. ↵
      1. Edwards H,
      2. Redaniel MT,
      3. Opmeer B, et al
      . Evaluating an enhanced quality improvement intervention in maternity units: PReCePT trial protocol. BMJ Open Qual 2021;10:e001204. doi:10.1136/bmjoq-2020-001204
    92. ↵
      1. O’Brien S,
      2. Lenguerrand E,
      3. Jordan S, et al
      . Simulation training for operative vaginal birth evaluation: study protocol for an observational stepped-wedge interrupted time-series study (STROBE). BMC Pregnancy Childbirth 2019;19:109. doi:10.1186/s12884-019-2222-x
    93. ↵
      1. Burhouse A,
      2. Lea C,
      3. Ray S, et al
      . Preventing cerebral palsy in preterm labour: a multiorganisational quality improvement approach to the adoption and spread of magnesium sulphate for neuroprotection. BMJ Open Qual 2017;6:e000189. doi:10.1136/bmjoq-2017-000189
    94. ↵
      1. Edwards K,
      2. Impey L
      . Extreme preterm birth in the right place: a quality improvement project. Arch Dis Child Fetal Neonatal Ed 2020;105:445–8. doi:10.1136/archdischild-2019-317741
      OpenUrlAbstract/FREE Full Text
    95. ↵
      1. Glover Williams A,
      2. Tuvey S,
      3. McBain H, et al
      . Perinatal excellence to reduce injury in preterm birth (PERIPrem) through quality improvement. BMJ Open Qual 2022;11:11. e001904. doi:10.1136/bmjoq-2022-001904
      OpenUrl
    96. ↵
      1. Robertson L,
      2. Knight H,
      3. Prosser-Snelling E, et al
      . Each Baby Counts in 2018 – lessons learned and future directions. Obstetrics, Gynaecology & Reproductive Medicine 2018;28:253–7. doi:10.1016/j.ogrm.2018.07.004
      OpenUrl
    97. ↵
      1. Jurczuk M,
      2. Bidwell P,
      3. Gurol-Urganci I, et al
      . The OASI care bundle quality improvement project: lessons learned and future direction. Int Urogynecol J 2021;32:1989–95. doi:10.1007/s00192-021-04786-y
      OpenUrl
    98. ↵
      1. West of England Academic Health Science Network (AHSN)
      . PReCePT – preventing cerebral palsy in preterm babies, Available: https://www.weahsn.net/our-work/transforming-services-and-systems/precept
    99. ↵
      1. West of England Academic Health Science Network (AHSN)
      . PERIPrem. n.d. Available: https://www.weahsn.net/our-work/transforming-services-and-systems/periprem
    100. ↵
      1. Edwards HB,
      2. Redaniel MT,
      3. Sillero-Rejon C
      . Evaluation of standard and enhanced quality improvement methods to increase the uptake of magnesium sulfate in preterm deliveries for the prevention of neurodisability (PReCePT study): a cluster randomized controlled trial. Obstet Gynecol [Preprint]. doi:10.1101/2022.05.20.22275244
    101. ↵
      1. Grimwood T,
      2. Snell L
      . The Maternity Safety Training Fund: An Evaluation. University of Cumbria, 2018.
    102. ↵
      1. NHS Resolution
      . Maternity Incentive Scheme – an interim evaluation. 2020.
    103. ↵
      1. Royal College of Obstetricians and Gynaecologists
      . Each Baby Counts + Learn & Support. Final Evaluation Report. London, 2022.
    104. ↵
      1. McBain H,
      2. Tuvey S,
      3. Williams AG, et al
      . Barriers and Enablers to the implementation of a perinatal care bundle: the perinatal excellence to reduce injury in premature birth (PERIPrem) project. [Preprint] 2022. doi:10.21203/rs.3.rs-1550325/v1
    105. ↵
      1. Widdows K,
      2. Roberts S,
      3. Camacho E, et al
      . Evaluation of the implementation of the Saving Babies’ Lives Care Bundle in early adopter NHS Trusts in England. Manchester, UK: Maternal and Fetal Health Research Centre, University of Manchester, 2018.
    106. ↵
      1. Dadich A,
      2. Piper A,
      3. Coates D
      . Implementation science in maternity care: a Scoping review. Implement Sci 2021;16:16. doi:10.1186/s13012-021-01083-6
    107. ↵
      1. Nilsen P
      . Making sense of implementation theories, models and frameworks. Implement Sci 2015;10:53. doi:10.1186/s13012-015-0242-0
    108. ↵
      1. Renwick S,
      2. Hookes S,
      3. Draycott T, et al
      . PROMPT Wales project: national scaling of an evidence-based intervention to improve safety and training in maternity. BMJ Open Qual 2021;10:e001280. doi:10.1136/bmjoq-2020-001280
    109. ↵
      1. Page MJ,
      2. McKenzie JE,
      3. Bossuyt PM, et al
      . The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:71.:n71. doi:10.1136/bmj.n71
      OpenUrlCrossRefPubMed

    Supplementary materials

    Footnotes

    • Twitter @jgmcgowan, @ilk21, @LisaHinton4, @graham_p_martin, @MaryDixonWoods

    • Contributors MD-W is the guarantor of the study. Conceptualisation: JM, MD-W. Data curation: JM, BA. Formal analysis: JM, BA. Funding acquisition: JM, MD-W. Investigation: JM, BA, IK. Methodology: JM, BA, IK, LH, TD, GPM, MD-W. Project administration: JM, IK. Resources: JM, IK, MD-W. Software: JM, IK. Supervision: GPM, MD-W. Validation: LH, TD, GPM, MD-W. Visualisation: JM. Writing—original draft: JM, MDW. Writing—review and editing: all authors.

    • Funding The Health Foundation's grant to THIS Institute.

    • Competing interests TJD: Research and Innovation lead for PROMPT Maternity Foundation and has a part-time appointment at NHS Resolution where he leads on a Safety Action within the Maternity Incentive Scheme.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.