Exploring the sustainability of quality improvement interventions in healthcare organisations: a multiple methods study of the 10-year impact of the ‘Productive Ward: Releasing Time to Care’ programme in English acute hospitals

Background The ‘Productive Ward: Releasing Time to Care’ programme is a quality improvement (QI) intervention introduced in English acute hospitals a decade ago to: (1) Increase time nurses spend in direct patient care. (2) Improve safety and reliability of care. (3) Improve experience for staff and patients. (4) Make changes to physical environments to improve efficiency. Objective To explore how timing of adoption, local implementation strategies and processes of assimilation into day-to-day practice relate to one another and shape any sustained impact and wider legacies of a large-scale QI intervention. Design Multiple methods within six hospitals including 88 interviews (with Productive Ward leads, ward staff, Patient and Public Involvement representatives and senior managers), 10 ward manager questionnaires and structured observations on 12 randomly selected wards. Results Resource constraints and a managerial desire for standardisation meant that, over time, there was a shift away from the original vision of empowering ward staff to take ownership of Productive Ward towards a range of implementation ‘short cuts’. Nonetheless, material legacies (eg, displaying metrics data; storage systems) have remained in place for up to a decade after initial implementation as have some specific practices (eg, protected mealtimes). Variations in timing of adoption, local implementation strategies and contextual changes influenced assimilation into routine practice and subsequent legacies. Productive Ward has informed wider organisational QI strategies that remain in place today and developed lasting QI capabilities among those meaningfully involved in its implementation. Conclusions As an ongoing QI approach Productive Ward has not been sustained but has informed contemporary organisational QI practices and strategies. Judgements about the long-term sustainability of QI interventions should consider the evolutionary and adaptive nature of change processes.


AbstrAct
Background The 'Productive Ward: Releasing Time to Care' programme is a quality improvement (QI) intervention introduced in English acute hospitals a decade ago to: (1) Increase time nurses spend in direct patient care. (2) Improve safety and reliability of care.
(3) Improve experience for staff and patients. (4) Make changes to physical environments to improve efficiency. Objective To explore how timing of adoption, local implementation strategies and processes of assimilation into day-to-day practice relate to one another and shape any sustained impact and wider legacies of a large-scale QI intervention. Design Multiple methods within six hospitals including 88 interviews (with Productive Ward leads, ward staff, Patient and Public Involvement representatives and senior managers), 10 ward manager questionnaires and structured observations on 12 randomly selected wards. Results Resource constraints and a managerial desire for standardisation meant that, over time, there was a shift away from the original vision of empowering ward staff to take ownership of Productive Ward towards a range of implementation 'short cuts'. Nonetheless, material legacies (eg, displaying metrics data; storage systems) have remained in place for up to a decade after initial implementation as have some specific practices (eg, protected mealtimes). Variations in timing of adoption, local implementation strategies and contextual changes influenced assimilation into routine practice and subsequent legacies. Productive Ward has informed wider organisational QI strategies that remain in place today and developed lasting QI capabilities among those meaningfully involved in its implementation. Conclusions As an ongoing QI approach Productive Ward has not been sustained but has informed contemporary organisational QI practices and strategies. Judgements about the long-term sustainability of QI interventions should consider the evolutionary and adaptive nature of change processes.

bAckground
There is a need for greater insight into the assimilation of quality improvement (QI) interventions into day-to-day healthcare practice and their sustained impact. 1 2 With rare exceptions, there are few studies of the sustainability of such change interventions in healthcare organisations. [3][4][5] The 'Productive Ward: Releasing Time to Care' programme is a QI intervention which aims to give ward staff the tools, skills and time needed to implement local improvements in order to: (1) Increase time nurses spend in direct patient care.
(2) Improve safety and reliability of care. (3) Improve experience for staff and patients. (4) Make changes to physical environments to improve efficiency. The rationale for Productive Ward was strongly marketed as empowering frontline staff, especially nurses, to 'take back control' of their wards and-through efficiency savings-'release time to care'.
The NHS Institute for Innovation and Improvement in England developed Productive Ward in 2005/2006 for application in hospitals and wards, and was first implemented in 2007. 6 7 Productive Ward is a self-directed QI toolkit consisting of three foundational modules ('Well Organised Ward', 'Knowing How We are Doing' and 'Patient Status At a Glance') and eight modules dealing with specific ward processes (eg, shift handovers, meals and medicines rounds). 8 The Table 1 Potential compatibility gaps Transformation The adopting hospital modifies its functioning to fit the assumptions behind Productive Ward and the actual use of Productive Ward does not significantly differ from its intended use. Customisation

Original research
Involves both adapting Productive Ward and adjusting ward processes. Loose coupling Productive Ward is adopted only superficially, in a ritualistic way with the functioning of the hospital remaining largely unaffected.

Co-optation
Productive Ward becomes captured and distorted to serve the interests of the most powerful stakeholders. modules were supplemented by guides for ward, project and executive leaders to aid implementation. Guidance included encouragement to think about sustainability from an early stage, 8 although a specific 'sustainability tool' was not developed until after the initial launch. As a large-scale QI intervention Productive Ward has three distinctive features. First, it underwent a systematic and extensive design and development process prior to widespread adoption. 7 It was designed through collaboration with industry partners, drew on social movement theory, and was developed incrementally through piloting and refining modules in collaboration with hospitals. 7 Second, Productive Ward was adopted very rapidly and widely after being formally launched in January 2008. In May 2008, the government invested £50 million to support the implementation of Productive Ward in all acute hospitals in England. 8 By May 2012 it was reported that 70% of all acute wards in the UK were implementing Productive Ward. 9 Third, Productive Ward is remarkable with respect to the scale of the claims made regarding its (potential and achieved) impact. This included a report predicting that a £270 million benefit would result from implementing Productive Ward across acute hospitals in England by March 2014. 10 However, a decade after the initial development of Productive Ward-and despite its widespread adoption and ongoing use in several other countries-there remains little robust evidence of its impact on acute wards in England. Literature reviews either focus on implementation strategies 11 or highlight a lack of clarity between activities, perceived impact and measurable outcomes. 12 Our overall research aim is to explore the impacts and wider legacies of the Productive Ward since its introduction over a decade ago. Responding to a call for more 'theoretically informative improvement research' 13 we seek to add to knowledge relating to the implementation and assimilation of QI interventions into routine day-to-day practice, and their sustainability in terms of maintenance of benefits, continuation of activities and continued capacity manifesting at individual, ward and/or hospital level. 14

Methods
We conducted a multiple methods study in six case studies comprising purposively sampled acute hospitals. We used multiple sources of information to arrive at 'best guess' dates for all adopting English acute hospitals to sample hospitals estimated to have adopted Productive Ward in different years, with the aim of attaining maximum variation. We also sampled for diversity with respect to region, type and size (number of wards) of hospital. Data collection during the period March 2017 to February 2018 comprised a total of: 88 semistructured interviews (with Productive Ward leads, ward staff, Patient and Public Involvement representatives and senior managers); structured observations to note material legacies and observable processes on 12 randomly selected wards (two in each hospital) using a template based on the goals of Productive Ward; and 10 ward manager questionnaires on processes not observable without intrusion, also based on Productive Ward goals. Eligible wards were those that had implemented the three Productive Ward foundation modules and at least one process module. To analyse the interview data we used the Framework method. 15 Initial themes were developed from: the theoretical literature (see below); the topic guide (itself reflecting theoretical and empirical literature); familiarity with interviews; and inductive coding of four transcripts.
We retrospectively analysed the adoption, implementation, assimilation and legacies of Productive Ward in each of our case studies. In doing so we purposefully distinguished between implementation (formal strategies to promote the integration of interventions into existing practices) and assimilation (the informal process by which, over time, new ideas become part of routine ways of doing things). We categorised the strategies used to implement Productive Ward in each hospital using a published framework which provided a comprehensive categorised list of published implementation strategies 16 and compared the execution of these strategies against Productive Ward guidance to assess fidelity. We then drew on potential 'compatibility gaps' between the assumptions underlying the design and implementation of managerial innovations and the characteristics of adopting organisations 3 17 18 and the definitions in table 1-as adapted from Kislov 3to characterise the interplay between the hospitals and the Productive Ward over time.
Descriptive case studies were written based on the overall framework. Documentary, observational and ward manager questionnaire data were included at this stage. During analysis, within cases we looked for concordances and differences between interviewees' accounts and between data sets and treated these as data. For example, interviews with lower grades of staff were sometimes at odds with programme leaders' accounts of their involvement; or observations of wards sometimes belied interview accounts of legacy. We conducted a cross-case analysis of our six case studies, linking legacies to the processes of adoption, implementation and assimilation.

results
Below we work backwards by first setting out the legacies of the Productive Ward as we observed them in

Original research
the six hospitals at the time of our fieldwork (up to 10 years postadoption). We then explain the variation in these legacies by considering the ways in which the Productive Ward was assimilated into routine practice, and how different implementation strategies and timing of adoption may have shaped these long-term organisational processes.

legacies of the Productive Ward Material legacies
In each of the six hospitals we observed material legacies of the Productive Ward which remained up to 10 years after initial implementation. Notwithstanding the differing forms of assimilation of the programme in the six hospitals-and how these evolved over time (see below)-we noted that displays of ward metrics data, use of 'safety crosses' (to show daily patient harm incidents per month) and storage systems were commonplace on the 12 wards we studied. While maintaining these outward signs of Productive Ward, in day-to-day practice legacies had sometimes become disconnected from their original purpose. For instance, safety crosses were not always clear, up to date or on public display and 'Knowing How We are Doing' boards could include data that were difficult to interpret, often out of date and seemingly rarely discussed by ward teams.

Process legacies
Our structured observations found that 7 of 18 potential process legacies arising from the modules were still present on all 12 wards (for example, the menu process being conducted outside of mealtimes and the presence of a system for flagging patients who had missed observations). In contrast, three of these potential legacies were not evident on any of the 12 wards; none displayed a 'ward vision', written standard operating procedures or an up-to-date 'Visit Pyramid' (a graphic display indicating predetermined visits by executive and senior staff which are focused on Productive Ward progress). Ward manager responses to our questionnaires suggested several other standardised processes were commonly still in place (for example, equipment being in the right place and ready to use). The extent to which these processes solely originated with-and have been maintained by-Productive Ward was sometimes unclear. The Productive Ward resources (ie, the guide booklets and toolkit itself) are now rarely used. Nonetheless, staff who had been meaningfully involved in the initial implementation of Productive Ward (mostly Productive Ward leads, ward managers and their deputies) identified wider legacies, such as using Plan-Do-Study-Act improvement cycles, having a 'lean' mindset regarding cutting out waste and improving flow, and giving ward staff a greater voice in QI.  Table 2 (table 2). By the end of the decade 'customization' was a more accurate description of how the programme had been assimilated into routine organisational practices in these sites. In Hospitals E and F we found that Productive Ward modules and tools were being used superficially, in a ritualistic way (if at all), with the functioning of the hospitals remaining largely unchanged (what Kislov et al would refer to as 'loose-coupling' 3 ). Such 'loose coupling' had implications for the relatively limited nature and scale of the legacies we observed in Hospitals E and F.
More nuanced were the stories of the remaining two hospitals. Hospitals C and D adapted Productive Ward significantly while retaining some of the tools; Hospital C also retained the 'look' of Productive Ward with its trademark graphic of a house built of (modular) 'bricks'. Hospital C had radically customised Productive Ward following the implementation period and assimilated the programme in this adapted form with other relevant QI programmes to avoid duplication; modules were modified and new modules added over time to meet new priorities and this continued to operate at the time of our fieldwork (although with some evidence that this may have been something of a paper exercise performed by ward managers on the wards we studied). In Hospital D, which merged its nascent Productive Ward programme with 'Leanbased' training it had already commissioned, we found organisational-level legacies of Productive Ward (use of data, availability of training, influence of ongoing QI strategies) but little involvement of lower staff bands in Productive Ward-related activities. We found locating Hospitals C and D within Kislov et al's framework 3 problematic and return to this issue in our discussion.

Implementing Productive Ward
Fidelity to implementation guidance Implementation of the Productive Ward was supported by detailed guides for ward, project and executive leaders. We found a consistent failure to extend strategic engagement beyond nursing in their steering groups (see table 3). The-typically-2-year, time-limited availability of funding was usually insufficient to enable hospital-wide implementation in the way that was intended by its designers (particularly in larger organisations with high numbers of wards, see table 4). There were three key areas in which the hospitals differed. First, hospital A was the only hospital to follow guidance on how it selected early 'showcase wards' by inviting and assessing applications based on a 'sustainability guide'; the five remaining hospitals either relied on nomination by senior nurses or There was considerable variation between wards in the same hospital in terms of implementation. Attitudes of ward managers, and the degree to which they involved staff in any meaningful way, was one issue. Another was that implementation efforts become laboured and Productive Ward teams overstretched as they either began to run out of time to implement all the mandatory modules in their targeted number of wards before their funding ceased or progressed to the more challenging wards and/or time-consuming modules.

Timing of adoption and local implementation strategies
Beyond fidelity to the guidance, we found variations in the implementation strategies used by the six hospitals. Applying Waltz et al's categorisation revealed that some elements of local approaches to implementation were common, (for example, 'providing interactive assistance' 16 ) but there were also important differences. First, there was variation in terms of dedicated resources. Timing of adoption was significant in this regard (table 4). Hospitals A, B and C (earlier adopting organisations) received government funding towards salaries for dedicated Productive Ward leads, materials, backfill for ward managers (Hospitals B and C) and expert training; they also had access to supportive networks. In contrast, Hospital F adopted Productive Ward after such funding and networks had ceased. There was wide variation in the numbers of wards in the hospital's roll-out plans (13 in hospital C to 72 in hospital A, table 4) and the number of modules Original research each ward was expected to implement, and therefore the workload for Productive Ward leads. Whether there was a dedicated lead with time to monitor and encourage ongoing efforts shaped staff engagement over time.
Second, hospitals A, B, C, D and E purchased expert training and support from external providers. In most hospitals training to managers of two to three 'showcase' wards was provided by Productive Ward leads and facilitators. But in Hospital B-a small Trust-8 of the 10 ward managers received the expert training directly. It is likely that the relatively high proportion of wards receiving expert training in Hospital B was the reason it stood out in that, first, ward managers continued to implement further modules beyond initial support, and second, it had a notable legacy in terms of staff continuing to make changes according to the underlying principles of Productive Ward. Hospitals A and C-2 years into implementation, in parallel with a reduction in funding-changed their strategy, moving from providing intensive, tailored training to small cohorts of wards towards mass training to ward representatives, module by module. Poor reach of training to all levels of staff in Hospitals E and F meant few staff were demonstrating 'Productive Ward thinking' at the time of our fieldwork.
A third key difference was how hospitals evaluated implementation and sustainability, and set up systems to measure outcomes. All hospitals were poor at collating ward-level 'before implementation' measures and cost data. After implementation began Hospital A employed a data manager who designed a wardlevel data dashboard for capturing monthly outcome measures (although these data were not used strategically at the hospital level). The two hospitals that stood out in terms of evaluative strategies were those where the Productive Ward team were integrated within an existing QI (Hospital D) or change management (Hospital F) team. These hospitals were also notable in forming strategic relationships with central services to aid implementation. Hospital D attempted to build in sustainability from the start of their implementation strategy, using a local system of monthly spot checks, audit sheets, required actions and reporting; in Hospital F ward-level monthly reports included identification of risks to sustainability. Despite this, significant events in each of the two hospitals (changes in the senior executive team in Hospital D and a move to a new hospital location in Hospital F) meant that in both cases Productive Ward-as an active programmeeffectively ceased shortly after the implementation period.
The fourth difference in implementation approach was the involvement of patients. While patients were represented on the Productive Ward steering group in Hospitals A, C and F, there was minimal involvement of inpatients on wards in all hospitals. Hospital A, which had a relatively strong history of patient involvement, stood out in how it harnessed patient/ public input.

dIscussIon
In seeking to address calls to examine both the 'extent, nature (and) impact of adaptations to … programmes once implemented' 19 and to 'yield new theoretical insights applicable to a broader range of settings', 13 we have explored (although retrospectively) how adoption, implementation and assimilation processes shaped variations in the legacies of a QI programme in six hospitals over a 10-year period. Our findings emphasise how sustainability is best considered as an evolutionary and adaptive process of change, rather than a stable state. 14 For the most part, Productive Ward was designed to address the needs of nurses working on wards. The components we typically observed as having been sustained (Well Organised Ward and Patient Status At a Glance boards) were those used-and found useful-by a majority of nursing staff. Productive Ward did also develop QI capacity among those staff meaningfully involved in its implementation. We also found evidence of how the programme helped shape contemporary hospital-wide QI strategies. However, we found several examples of 'improvements' being present on wards but only applied in a superficial and ritualistic way (particularly Hospitals E and F). For example, while Productive Ward was undoubtedly part of an emerging movement to make metrics visible and public-through Knowing How We are Doing boards and Safety Crosses-to avoid stasis hospitals need to regularly retrain staff in their use and take advantage of technological developments if intended wider benefits were to be realised on an ongoing basis.
When considering our empirical findings in the light of earlier accounts 3 17-19 we would propose three refinements to existing theories. First, we agree with Addicott et al 18 that there can be 'temporal flow' between the compatibility gap typology originally developed by Lozeau et al; as Kislov et al later noted rather than 'distinct independent categories' the four types can indeed 'represent stages of the same process'. 3 We would argue that there is now a similar need in recent categorisations of implementation strategies 16 20 to better recognise an emergent, temporal element and how these interact with assimilation processes to shape legacies of large-scale QI programmes. To illustrate, over time we noted-both within and between our six hospitals-a shift away from a longer-term vision of empowering ward teams to take ownership of the programme (by implementing the modules themselves-'transformation') towards a narrower, solely efficiency-based view of the goals of Productive Ward (to be achieved through top-down standardisation and performance measures-'loosecoupling', 'co-optation'); this may explain the limited long-term legacies we observed in the later adopting ► In focusing exclusively on ward and nursing processes, the original framing and format of the programme would not meet current demands for multidisciplinary teamwork and system transformation.
► Involving wider ward teams from the start could have helped mitigate risks to sustainability posed by staff turnover. Ensure adequate resourcing for task completion and reflection ► Funding was needed both to release staff from ward duties (so they could carry out Productive Ward activities) and to enable them to reflect on experiential learning in relation to the underlying principles of the programme (so they could then go on to apply them in changing contexts).
► The typical 2-year funding period was, in most cases, insufficient to enable implementation of all the modules as intended. ► A dedicated member of staff was needed to coordinate activities and ongoing training within organisations, as well as demonstrating organisational commitment to the programme. This role was key to realising sustained impacts.
Focus on quality not quantity ► Take time to implement foundational modules fully and share lessons learnt and outcomes.
► As implementation progressed programme leads tended to run out of time and energy, and in later wards 'solutions' were imposed by programme leads and/or ward managers without giving other staff opportunities to identify solutions. This led to a sense of 'being done to' (rather than shared ownership).
► Focus on outcomes achieved or lessons learnt, rather than simply recording progress through a staged programme.
Less may be more ► Productive Ward would have benefited from being more focused. The modules that were most frequently implemented were: the 'Foundation modules' (Well Organised Ward; Patient Status at a Glance; Knowing How We are Doing); followed by four of the process modules (Medicines; Meals; Patient Observations; and Shift Handovers).
► The remaining four process modules were rarely implemented in any meaningful way.
Play the long game ► Adequate before/after and longitudinal data are required to demonstrate impact. ► Efficient local systems for enabling measurement of impacts and costs (initial and over time) should be built in from the start. ► The reputation of the Productive Ward suffered to some degree from overclaiming benefits at a national level without a sufficiently longitudinal or robust evidence base.
► Build considerations of sustainability and develop related guidance into the initial design and testing of a programme.
Adaptability ► QI programmes need to be flexible. A programme such as Productive Ward needs to be able to absorb and adapt to changing organisational or system priorities so that relevant learning and resources can be applied to new priorities, rather than entirely new QI programmes being designed to replace or run alongside existing initiatives.
► Having guidance and toolkits available online where they can be revised and redirected, rather than hard copies of modules, would better support this.
Involve patients and carers as partners ► Involve patients, carers and the public. Although guidance suggested roles for patients, visitors and patient representatives at ward and hospital level, such involvement was generally low to non-existent. hospitals who-with fewer resources-appeared to begin from this narrower view. Such shifts and decisions manifested themselves in a range of implementation 'short cuts' motivated by time constraints and a managerial desire for standardisation. These short cuts seemed to occur where there were conflicting rationales for the programme among different stakeholders; for example, where a managerial efficiency agenda took precedence over 'releasing time to care' on wards. There was often tension between wards' desire to find their own solutions to their problems, and hospital managers' desire for standard practices and infrastructure that made it easier for staff to work across a hospital and which could deliver economies of scale (for example, when purchasing equipment). One such 'short cut'-in hospitals where Productive Ward was relatively under-resourced-saw ward teams excluded from implementation of the programme. For example, in Hospital F facilitators-rather than ward staff-did much of the work including imposing 'solutions'. Where-as intended-whole ward teams had been trained in the principles underlying Productive Ward (Hospital B and initially in Hospital A) we found that staff continued to apply these principles to their QI work even as organisational contexts changed over time. This finding has parallels with that of Jones et al who identified 'collective change agency' as one way of closing a compatibility gap between a QI intervention and adopting hospital. 13 19 More positive legacies were apparent at the time of our fieldwork in those hospitals (A, B, C and D) which enacted (and importantly maintained) higher fidelity to empowering ward-based teams as a fundamental aspect of the recommended implementation approach.
Second, Lozeau et al have previously raised a possible contradiction between a rhetoric of empowerment (as in the Productive Ward) and command-and-control procedures for auditing performance data. 17 Addicott Original research et al highlighted empirically how 'top down' distortions through 'performance management systems set up by the remote centre' limited the intended impact of managed clinical networks for cancer. 18 In contrast, local Productive Ward leads and managers were important actors who developed their own ward-level monthly implementation reports, as well as local systems of regular audits and monitoring. Rather than distorting the Productive Ward, such locally developed audits-in the absence of remote 'top down' guidance or performance measures-did appear to support sustainability. By contrast, it seemed that when ward staff became aware that no one locally was paying attention, work on the programme quickly ceased. This observation contributes to Kislov et al's call for better understanding of how local leaders can balance sensible adaptation of QI interventions with the risk of distortion of core elements of a programme such as Productive Ward. 3 Third, Jones et al have previously suggested that 'loose-coupling' between a QI intervention and an adopting hospital can occur when implementation stalls because of simultaneous improvement initiatives overburdening staff. 19 While this was certainly true in some of the hospitals we studied, the resilience of the QI strategy which was informed by Productive Ward in Hospital C lay in how it absorbed several overlapping initiatives which might otherwise have been competing against each other for limited attention and resources. To a lesser extent we saw similar patterns in Hospitals A and D. Our findings suggest a need to develop and validate a more nuanced classification of assimilation processes than that previously forwarded. 3 17-19 Intelligent 'customization' of the programme at an early stage (Hospitals C and D) gave way to more evolutionary forms of 'loose-coupling' where hospitals took account of the assumptions behind Productive Ward to make a wider, positive impact, even though the programme was no longer being used as intended. These were markedly different forms of assimilation to those observed in Hospitals E and F. Table 5 summarises the implications arising from of our findings for the design and delivery of future large-scale QI programmes.
There are several limitations to our study. First, given the retrospective nature of the fieldwork it was sometimes challenging to trace possible legacies of Productive Ward since one or more other QI interventions had often been implemented concurrently or subsequently. Furthermore, wards and hospitals varied with respect to the number of staff available who had been in post during implementation of the programme. Second, assimilation was a challenging concept to explore retrospectively over such a lengthy time period. Lastly, much of our data come from two randomly selected wards in our case study hospitals. Ward practices, perhaps especially QI work, are strongly influenced by ward managers. A different picture of QI work may have emerged at the time of our data collection had we studied other wards.

conclusIons
We found that Productive Ward has had a lasting impact on specific ward practices; some material and processual changes have remained in place for up to a decade after initial implementation. In this widest sense five of our six case studies could be described as having seen some form of sustained impact from the Productive Ward over the last decade. However, as an ongoing QI approach-continually used to identify and improve problem areas-Productive Ward has been less successful. The resources available at the point of adoption and the-closely related-issue of how the Productive Ward was then implemented locally, shaped the evolving forms of assimilation into routine practice over time; these and wider contextual changes largely determined the legacies of the programme.