• Leadership
• Evaluation methodology
• Patient safety

The Measurement and Monitoring of Safety Framework (MMSF) is a conceptual model to guide organisations in assessing safety. Developed from a synthesis of research literature and case studies from healthcare and other industries,1 2 the MMSF comprises five dimensions of safety, each with an accompanying fundamental safety question (see table 1). The MMSF is a departure from prescriptive and top-down patient safety interventions; it encourages discussion, reflection and learning to improve patient safety. The MMSF tells us there is no one single measure of safety, and that safety measurement and monitoring is complex and multifaceted.

The MMSF supports healthcare organisations to combine both qualitative and quantitative safety intelligence. It supports them to consider how to use more intangible types of safety intelligence (including what is heard, observed and perceived) alongside hard safety metrics. Martin et al 3 have referred to this as ‘soft intelligence’. Their study, which involved interviewing 107 senior managers in the English National Health Service showed that senior managers valued ‘soft intelligence’ but struggled with how to access and translate it into useful forms of knowing. Their findings showed the value of processes and behaviours aimed at questioning and disrupting assumptions about quality, safety and organisational performance: Such approaches value the seeking out and hearing of multiple voices and this in turn supported sense-making.

The impact and challenges of implementing MMSF in nine healthcare organisations in England and Scotland (three regional improvement bodies and six front-line settings) have previously been described.4 The study identified that MMSF creates a common language for safety. MMSF also changes the mindset of healthcare professionals working in different settings and at different levels of the system, often making them more inquisitive about present and future levels of safety. Chatburn et al 4 identified that MMSF did not always lead to broader changes in practice, with some regions only focusing on one dimension of the Framework. Where committed leaders understood MMSF, this enabled front-line staff to explore the Framework and they were better able to translate the concepts to their local setting. As a result, MMSF implementation was more successful. Apart from this study, there is little research evaluating the implementation and impact of MMSF. Therefore, the study by Goldman et al in this issue of the journal is a welcome addition.

The Canadian Learning Collaborative evaluation describes the experience of implementing MMSF in a learning collaborative with 11 healthcare teams drawn from a range of care settings including emergency, surgery, medicine, cardiology, psychiatry, supportive living and long-term care settings, as well as in in person and remote care programmes. The evaluation of the Canadian Learning Collaborative provides deeper insights into how to introduce and implement MMSF. The evaluation involved interviews with 36 collaborative participants, observations at five sites and learning sessions, and a document review (comprising materials from the learning sessions, and those teams had developed). The findings show:

• MMSF changes how healthcare teams think, interact and practice safety measurement and monitoring.

• It takes time for healthcare teams to familiarise themselves with MMSF concepts and language.

• It is important to empower healthcare staff and to enable them to innovate when applying MMSF in their healthcare setting.

• The collaborative model combined with coaching in between learning sessions was vital to reframing teams’ understanding of safety.

• An intervention like MMSF cannot be isolated from the implementation context: in some organisations, introducing MMSF created dissonance, especially where there were misalignments with existing safety and quality improvement processes.

The Canadian authors conclude that introducing MMSF requires extensive discussion, coaching and opportunities for healthcare staff to experiment and practise using MMSF. MMSF does rewire how healthcare professionals think about safety, moving the focus from solely on past harm and reliability to broader consideration of safety in the ’here and now’ and in the future.

As acknowledged by the study’s authors, one limitation with their study is that their evaluation started towards the end of the Learning Collaborative. In the future, longitudinal studies of MMSF implementation experience and studies which measure spread and scale up more systematically are needed to further develop the evidence base on MMSF implementation.

Building on the earlier findings of Chatburn et al,4 the Canadian Learning Collaborative evaluation provides further evidence that organisations implementing MMSF need to:

1. Have supportive and engaged leaders who work alongside healthcare teams, coaching them to understand the Framework. My experience working alongside healthcare teams has shown that local leaders (ie, ward sisters, lead clinicians and lead allied healthcare professionals, team leaders, etc), middle managers, (eg, divisional or service managers) and senior leaders (executive directors or the equivalent) all have an important role to play: Local leaders support staff to contextualise MMSF to the care setting where it is being introduced. When team members see how MMSF has challenged their leader’s mindset, they too become curious and engage in discussions. Middle managers have accountability for their division or service’s safety performance; we have learnt they are an important audience for MMSF implementation. When middle managers are not included in MMSF implementation, barriers to spread and sustainability of the Framework emerge. Senior leaders are also vital; both in terms of championing the Framework and also working through challenging conversations where another improvement approach (eg, LEAN or quality improvement methodology) is dominant in an organisation.

2. Recognise it takes time for healthcare teams to understand MMSF and to translate it into their clinical setting. In the Canadian collaborative, healthcare teams were given 2 months to familiarise themselves with and translate MMSF into their own context after first being introduced to it. During this time, participating teams were able to introduce MMSF to colleagues, and have inclusive discussions about what each dimension of the Framework meant for their care setting. One of the Canadian teams developed a Power Point slide describing staff falls with harm during the winter months in Canada as a way of educating their colleagues about each dimension of the Framework. In the United Arab Emirates, staff leading the implementation of the MMSF developed a nationally relevant story, presenting an analysis of multiple car pile-ups during the country’s foggy season through the five dimensions of MMSF. Innovations like these make MMSF relatable and accessible. In short, we have learnt that introducing healthcare teams to MMSF and then giving them time to reflect and innovate enables them to translate MMSF in ways that resonate with them and their teams.

3. Use patient and/or staff stories, presented through the five dimensions of MMSF to illustrate how each dimension of the framework contributes to patient safety. Healthcare teams who have implemented MMSF often describe their ‘lightbulb’ moment with MMSF as occurring when I share my family’s story of the cultural change and safety improvement we effected as carers for a loved one who had coexisting mental and physical health diagnoses. Feedback from both the UK and Canadian healthcare teams highlighted that sharing my family’s story translated MMSF from five abstract dimensions into what everyday care looks like through the dimensions of the Framework. For example, keeping our loved one safe meant tuning into what ‘the voices’ were telling her (sensitivity to operations), knowing the triggers and calmers, and horizon-scanning for these in new or different situations (anticipation and preparedness), sharing what worked well (ie, Safety II learning) across the broader family, as well as learning the lessons from past incidents and ensuring reliability in, for example, medication prescribing processes.

4. Empower staff and teams at all levels of a healthcare system to experiment with MMSF. Experience with implementing MMSF has shown that healthcare teams should be encouraged to test and work across the entire framework. The UK experience showed that teams who focused on one dimension of the Framework (eg, testing safety huddles to improve sensitivity to operations) did not benefit like those who took a more holistic approach. Why is this? The MMSF dimensions are interconnected and interdependent. Safety measurement and monitoring requires a holistic approach. Focusing on one dimension leads to limited gains. Thinking across MMSF in its entirety should be encouraged because this supports many different types of reflections including:

   • Questioning long-entrenched measures of past harm which we know do not add value, but which have almost become a comfort blanket for the organisation.

   • Considering the negative side effects of some approaches to measuring reliability; for example, compliance audits of safety checklists.

   • How to improve access and use soft intelligence so it is part of team and organisational sense-making.

   • What types of leading indicators signal emerging safety risks.

   • How teams and organisations translate soft intelligence and hard data from their work on the other four dimensions and use it to inform learning and improvement work.

Experimentation across the entire MMSF also supports embedding Safety II and resilience engineering. Healthcare teams have, for example, experimented with how to improve how they learn from good catches, adaptations and interventions which prevent dynamic changes and changing conditions spiralling into patient harm. For example, by reflecting on their focus in the ‘past harm’ dimension of the MMSF and/or looking at Safety II through the reliability dimension of MMSF. This has in turn led to the redesign of clinical systems and processes, and/or implementation of systems to enable anticipation and early intervention.

1. Consider the implementation context when using MMSF. For example, what safety initiatives are being implemented across the organisation? Is there a strong focus on measuring reductions in patient harm, and if so, what challenges does this create? Teams who have implemented MMSF have found it to be a positive, disruptive force where the organisation’s dominant safety improvement methodology focuses on measuring past harm. Healthcare organisations where past harm measures on falls, medication incidents, VTE, violence and aggression, and so on, presented on statistical process charts, showing trends of harm over time, have benefited from the MMSF’s ability to broaden their lens on safety measurement and monitoring, recognising that measuring levels of harm over time is one component of safety measurement and monitoring.

Nearly a decade on from when it was first published, MMSF has not been widely embedded. Healthcare organisations worldwide continue to focus on measuring harm, rather than developing a deeper understanding that improving safety requires healthcare organisations and regulators to do much more than measure safety through the rear-view mirror. The Canadian Learning Collaborative reminds us of the potential of the MMSF to change mindsets and approaches to safety measurement and monitoring. Further research is needed which demonstrates how MMSF changes healthcare professionals’ safety cognition and behaviours. This requires a shift by policy makers who need to understand that safety monitoring needs to be combined with safety measurement to achieve sustained improvements.