Article Text
PDF
Abstract
Introduction Performance data can be used to monitor and guide interventions aimed at improving the quality and safety of patient care. To use performance data effectively, nurses need to understand how to interpret and use data in meaningful ways to guide practice. Dashboards are interactive computerised tools that display performance data. In one large, urban teaching hospital in Toronto, Canada, unit-specific dashboards were implemented across the organisation.
Methods A qualitative study was undertaken to explore the perceptions and experiences of front-line nurses and managers associated with the implementation of a unit-level dashboard. Six units were selected to participate in the study. Data were analysed using a directed content analysis approach.
Results The sample included 56 study participants, including 51 front-line nurses and 5 unit managers. Three key themes emerged around nurses’ and unit managers’ perspectives on the implementation of unit-specific dashboards. Nurses and managers described that the Care Utilising Evidence dashboard was a visual tool that displayed data on the impact of the nursing care provided to patients. This tool also was used by the nurses and managers to keep track of processes of care and patient outcomes and experiences at a unit level. Further, nurses were able to use performance data to identify quality care improvements specific to their unit.
Conclusions The results highlight how unit-specific dashboards are being used to monitor performance and drive quality improvement efforts from the perspectives of nurses and unit managers. In practice, nurse leaders may consider investing in dashboards as a quality improvement strategy to optimise the use of performance data at their organisations.
- Audit and feedback
- Quality improvement
- Nurses
- Performance measures
- Qualitative research
Statistics from Altmetric.com
Introduction
Performance data can be used to monitor and guide interventions aimed at improving the quality and safety of patient care.1–21 Nurses play an important role in quality and performance improvement at the clinical level due to their interactions with patients and at management level where nurse leaders have a key role in informing resource allocation and patient safety monitoring.7 ,8 ,10–21 To use performance data effectively, nurses need to understand how to interpret and use data in meaningful ways to guide practice.15 ,18 Unless staff nurses have knowledge about performance improvement measures, exposure to data reporting mechanisms and shared accountability for quality, it may be difficult for them to participate in data-driven care.15 There is a growing literature base on using performance data in practice for quality monitoring and evaluating new and existing practices.7 ,10 ,12 ,15 ,18 ,19 Further, providing nursing staff with feedback on patient outcomes highlights the value of nurses’ contributions to quality care.18
One mechanism to provide feedback on patient outcomes, experiences and processes of care is the use of dashboards. The dashboard is an electronic tool that can be used to display outcomes; monitor progress; identify deficiencies and manage a variety of operational, clinical and professional practice activities in an organisation.15 ,19 The ultimate goal of the dashboard content and format is to optimise the strategic value of the information and to enable user interpretation by viewing meaningful descriptive comparisons, trends and significant variation.12 Dashboards provide a systematic way for healthcare leaders, managers and health care providers (nurses, physicians, health disciplines) to make rapid and more informed decisions as well as learn from past experiences and adjust plans accordingly.15 ,19 For example in nursing, dashboards have been used to provide information to guide staffing decisions for positive outcomes12 ,20 ,21; clinical activities12 ,19; evidence-based performance19 ,22; programme evaluation19 and nursing contributions to overall quality of care at organisation and unit levels.15
Until now, what is known about the use of dashboards as a mechanism for providing feedback on patient outcomes, experiences and processes of care is mainly drawn from descriptive articles. Further, there is a paucity of empirical evidence on engaging front-line nurses in using data from the dashboards to drive their care and quality improvement efforts. Understanding whether the implementation of dashboards is an effective strategy to engage front-line nurses to use data to drive their daily practice can provide valuable information to nurse leaders. In this context, a qualitative study was undertaken to explore the perceptions and experiences of front-line nurses and managers associated with the implementation of a unit-level dashboard in one teaching hospital.
Methods
Care Utilising Evidence (CUE) dashboard implementation
The Care Utilising Evidence (CUE) dashboard initiative was implemented in a large, urban 500-bed teaching hospital in Toronto, Ontario, Canada. CUE was a nurse-led audit and feedback initiative that was initiated as a sustainability strategy aligned with the implementation of the Registered Nurses Association of Ontario's best practice guidelines (RNAO BPGs) across the hospital. This initiative included the following two essential components: (1) a flexible, interactive computerised dashboard tool for displaying performance data for each unit's selected RNAO BPGs on smart board and/or computer workstation and (2) an organisation-wide engagement strategy targeting nurses, nurse managers and leaders across all levels of the organisation. Details on the development of the CUE dashboard and pilot phase are provided elsewhere.22
Until now, CUE dashboards have been rolled out to 22 inpatient and outpatient clinical units within the organisation. Implementation varied with participating clinical unit, including units displaying their dashboards on a smart board accessible in a staff room; on individual computers that nurses use for patient documentation or hard copy printouts on bulletin boards. In some units, the CUE dashboards are used as a monitoring function for their RNAO BPGs and discussed at monthly staff meetings. For others, the onus is on the staff nurses to access the CUE dashboard individually during their clinical shift. The project coordinator met with unit managers and staff nurses to discuss the data displayed on the CUE dashboard at the request of the unit manager. For some units, this was every second month or when new data were available; for others, there were only two meetings of the project coordinator with the clinical units during the last 2 years. Since the initial implementation in 2011, dashboards have been updated every 2–3 months, or more frequently at the request of the unit manager. Further, front-line nurses and nurse managers have been asking for data related to other corporate nursing initiatives such as falls prevention, delirium prevention, Health Outcomes for Better Information and Care (HOBIC) and nurse-to-nurse bedside shift reporting.
Recruitment of participants
Ethics approval was obtained from the institutional research ethics board. Eligibility criteria included staff nurses and managers from six selected units (outpatient mobility, emergency department, general internal medicine, general surgery, coronary care unit and respirology). These units were selected as they represent a diversity of patient populations and were at various stages of implementation with launch months ranging from September 2011 to March 2012 (see table 1). Staff nurses were sent an email invitation by their unit clerical assistant outlining the study purpose and data collection requirements. Managers were invited to participate in an interview through email from a research assistant. Those who agreed to participate contacted the research assistant and were scheduled for an interview at a time and location convenient for study participants. Written informed consent was obtained prior to conducting the interview.
Staff nurses sample description
Data collection
Key questions that were asked during the interviews that yielded data for analysis included: What are your impressions of the overall initiative? Do you think the CUE dashboard is valuable? How would you like to see the data used? How would you envision using data from the CUE dashboard to improve your practice (nurse sample) and improve nursing practice (manager sample)? Interviews with staff nurses were conducted by two nursing research students in June through August 2013. Prior to conducting the interviews, the two students were trained by a team of experienced qualitative research personnel at the hospital to ensure consistency. Both students conducted mock interviews supervised by a member of the research team who provided feedback. Subsequently, both students were observed and coached while interviewing nurses until the research team was confident in their ability to conduct an interview independently. An interview guide was used to explore staff nurses’ and unit managers’ perceptions associated with the value and benefits of having performance data displayed on the CUE dashboard and enablers to using the dashboard to drive improvement efforts in their daily practice in their unit. Interviews were conducted until saturation of themes was achieved and no new insights were derived from the interviews.23
Data analysis
Interviews were audio-taped and transcribed for analysis by the research assistants. Data were analysed using a directed content analysis approach.24 ,25 Specifically, the analytical process involved three investigators and two research students independently reviewing all transcripts line by line to identify sections of text that serve as codes. The most frequently reported codes were then grouped into broader categories. Data collection and analysis were iterative in nature where themes, subthemes and subcategories were added to reflect variations in data. From this analysis, a coding schema was developed and refined. The principal investigator developed an audit trail that included the triangulation of responses from the staff nurses and managers from the different clinical units to the open-ended questions and the summative content analysis. This step was done to ensure methodological rigour and trustworthiness of the data analysis.24
Results
Sample description
A total of 56 interviews were conducted with 51 staff nurses and 5 unit managers. Table 1 provides further details on the sample of staff nurses. Interviews with staff nurses lasted an average of 23 min with a range of 10–45 min. Four unit managers were master's prepared and one was baccalaureate prepared. These participants managed the general internal medicine, general surgery, respirology, outpatient mobility, emergency and the coronary care units, respectively. Interviews lasted an average of 22.42 min with a range of 10.18–45.32 min.
Themes
There was consistency among the perceptions of the staff nurses and the managers from the six participating units, of which the themes were derived. Each cohort's perspective contributed to a more comprehensive view of the value of unit-specific nursing performance dashboards (CUE dashboards). The following three key themes emerged from the qualitative interview data set: (1) seeing the impact; (2) keeping on track and (3) identifying care improvements.
Seeing the impact
Nurses reported that the CUE dashboard was a visual cue and tool displaying data that enabled them to see the impact of their care on patient outcomes and experiences associated with implementation of best practice guidelines. Twenty-two study participants described the value of seeing the data as it provided nurses with a visualisation of how they were doing. Prior to the implementation of the CUE dashboard, nurses reported that they were often unaware of the outcomes of the care they were providing to patients. As noted by a nurse working in the outpatient mobility unit:
I think it's a great tool to have because we do all this work and we may not be acknowledged for it or may have any idea how we are doing. When you see that data you can say it really is impacting our practice and patient experience. (Outpatient Mobility Nurse 4)
Participants also reported that the CUE dashboard acknowledged and highlighted the work that nurses do to ensure they provide the best care possible and maintain the standards of practice. For example, one nurse described feeling validated that the data were being looked at and provided impetus for doing patient care activities (assessments and follow-up care) and documenting in patients’ charts. Knowing how they are doing also evoked a sense of pride for doing a good job and impacting patient care and outcomes as well as gave encouragement for continuing to provide quality nursing care.
You can look it up and see what we're doing well, they discuss the data and go over everything and give us updates on what's going on. The encouragement is it tells you that you're doing well and you'll feel encouraged that yes I'm doing a good job and you're doing a good job as a unit and we need to keep on doing this. (General Internal Medicine Nurse 4)
The staff likes to see how their work is being implemented and the results of their work and I thought what a great idea to get their work highlighted and embedded on the unit that they can have rich discussions about it. Nurses on a whole are competitive and like to see their numbers look great and so it gives us a visual cue how well we're actually doing with some of our measurement and so I think it's rewarding. (Manager 5)
Keeping on track
Study participants described the utility of the CUE dashboard as a monitoring device. The CUE dashboard was used as a tool to ensure that they were achieving the intended outcomes over time associated with each unit's selected RNAO BPGs. Participants (both nurses and unit managers) reported the importance of being able to see and trend the data over time and know that the units are keeping on track and whether collective goals are achieved for the selected BPGs. Further, nurses described that they were able to pick out patterns with the trended data displayed on the CUE dashboard that evoked conversations among the nurses and managers. This was most frequent when the trended data from the CUE dashboard indicated that the outcomes were not being achieved or declining. In this context, the conversations were critical and reflective in nature whereby nurses and managers would discuss why they were not achieving the outcomes they wanted for patient care.
In terms of keeping that up, where were we lacking, why did we drop one month? It's good to keep on top of it so it gives me an idea of how we're doing. So I think that when we keep an eye on how we're doing. (Outpatient Mobility 3)
The staff likes to see how their work is being implemented and the results of their work and I thought what a great idea to get their work highlighted and embedded on the unit that they can have rich discussions about it. Nurses on a whole are competitive and like to see their numbers look great and so it gives us a visual cue how well we're actually doing with some of our measurement and so I think it's rewarding. (Manager 5)
It's a good tracking device you see the charts and see how you're doing and how you were maybe last month. Maybe you slid this month and hopefully next month you'll go back up. So I think it's a good visualization to keep you on track and not kind of veer off. (Emergency Room Nurse 4)
If you're looking at graphs and it shows that patients feel that the nurses are addressing their fears and anxiety, you know that what we have implemented, because that was one of our best practice guidelines that we looked at, you know that its working and we're being effective. (Respirology Nurse)
Identifying care improvements
Study participants also reported that the data displayed on the CUE dashboard were used to make improvements and advance nursing practice. Twenty-seven study participants reported the view that the data displayed on the CUE dashboard were useful to guide improvement efforts. Participants described using the data to identify areas for improvement and remind them of what to focus their practice on. For example, a respirology nurse noted: “It makes you focus more on the things you need to focus on with the patient in terms of you may be doing that all the time, but then you’re more conscious and you’re actually documenting it.”
Units were able to use these data to prompt a discussion on areas of strengths and what to be nurtured or weaknesses to be improved. For example, gaps in providing a self-care management instruction form to outpatients; discussing fears and anxieties with patients and pain assessment and management were identified from the data displayed on the CUE dashboards in units. As a surgical nurse noted: “I think it brings home the message that you know where your strengths and weaknesses are as a team and reminds you of what's important and what you should be focusing on and how it actually impacts patients. It gives you the big picture.” From this interpretation and trending, units were able to focus their improvement efforts to address care gaps in their daily practice. Participants also linked making improvements to better patient care and safety outcomes and as one nurse described as being able ‘to maximise patient potential’ as noted below.
When you look at all the data you can see exactly where you are and what you need to work on so that can improve your practice, prevent these barriers or make things easier for practice then patients could be more satisfied. It's [CUE dashboard] a good tool being implemented to let us have the kind of awareness of how we're doing and what can we do to increase patients’ safety outcomes and to maximize patient potential. It [CUE dashboard] could really prompt you to say ‘okay, well let me see what I can do about pain management. You just make a special effort and see if any at all when the next report comes if it reflects the effort that you have been making. (General Internal Medicine 3)
I think the CUE dashboard is set up in a way to promote PDSA [Plan Do Study Act]. You look at okay so we seem to have a dip here, we might have a little problem here, what can we do about it, let's focus on it, let's try and fix it. It is also the opportunity to identify areas that we can make improvements, sort of trend it and circle back on it. (Manager 3)
Although a dashboard has been developed and available in the coronary care unit, the nurses interviewed in this unit were not aware of the CUE dashboard. However, they described the value of having performance data displayed on a dashboard at the unit level. Specifically, participants from the coronary care unit expressed that there is potential for the CUE dashboard to be valuable to monitor and guide practice improvements if used more effectively. As one nurse noted: “I think it could be more valuable than it is at the moment.”
Discussion
Our study findings elucidate how nurses and unit managers perceive the value of unit-specific dashboards as a key strategy to monitor performance and drive quality improvement efforts. Study participants perceived the CUE dashboard to be or have the potential for being a valuable tool to inform nurses of the impact of their care on patient outcomes and experiences. In our study, the display of data on the CUE dashboard enabled nurses to keep track of processes of care (eg, assessment form completion) and patient outcomes and experiences at a unit level. Our findings are consistent with what was reported in descriptive literature.8 ,18 ,19 For example, one article described how to use dashboards to showcase whether a programme is effective in meeting its goals and objectives.19 Another article highlighted that staff nurses reported anecdotally that they became more aware of their unit's impact on patient outcomes through report cards, which in turn provided them with a sense of control over and satisfaction with their work.18 Our study is one of the first empirical studies on the use of dashboards and provides a description of how front-line nurses used the data displayed on their unit dashboards to validate the work processes that they were engaging in as part of their daily work and evoke a sense of pride for providing quality care to patients.
Our results also provide insight into how nurses can leverage the use of performance data to drive quality care. This theme is consistent with what others have outlined as key benefits of displaying and using data, including to: identify strengths and vulnerabilities, establish areas for quality improvement and employ targeted strategies to improve care.8 ,12 ,19 For example, a study that explored feedback of pain scores to advanced practice nurses found that these nurses used the data provided to improve their practice related to pain management.8 In our study, nurses also described the benefit of using the data displayed on the dashboard to advance nursing practice and drive quality care. In terms of advancing nursing practice, the data displayed on the CUE dashboard served as a reminder for nurses on what to focus on, what to continue with and what to improve in their daily practice.
Collectively, our results elucidate how front-line nurses were able to visualise, interpret and act on performance data on care processes, patient outcomes and patient experience specific to their unit. Our study findings support the value and utility of using dashboards at the patient care unit level. On one hand, dashboards may be used as a tool to promote a sense of pride for when process, outcomes and experiences are positive, and quality nursing care is to be sustained for patients. On the other hand, its utilisation also promotes inquiry, discussion and ongoing improvement efforts when targets are not being reached or when there is a negative trend in process, outcome and experience measures. Given the importance of front-line nurses being able to identify gaps with the data displayed on the CUE dashboard, there is a need for nurse leaders to invest in creating and sustaining the use of performance dashboards as part of ongoing quality improvement efforts. Key to these efforts being successful and sustainable is making data available, providing support to interpret data and investing in resources to make improvements for front-line nursing staff.7 ,15 ,18 Further research is called for to examine the effect of using performance dashboards as part of quality improvement efforts that employ mixed methods approaches.
Study limitations
Study findings need to be interpreted with the following limitations. First, the study was conducted at one organisation which may limit the transferability of findings to other healthcare settings. Second, given that the source of data was self-report in nature, there is a possibility that social desirability bias of participants influenced their responses. Third, the study population involved nurses only and thus study findings may not be transferrable to other healthcare disciplines or in an interprofessional context. These limitations are common in qualitative research and evaluating quality improvement initiatives. The research team employed a sampling strategy that drew from six different clinical units. Fourth, the interview questions were framed in a positive manner and may have influenced the study participants to focus on the value and benefits of unit-specific nursing performance dashboards.
Conclusion
Our study findings have important implications for nurse leaders in their efforts to leverage the use of unit-specific dashboards to improve performance. In our study, nurses viewed the CUE dashboard as a valuable tool to seeing the impact; keeping track of process, outcome and experiential indicators and making improvements to advance quality nursing care. Nurse leaders can use study findings as rationale for investing in using dashboards as part of ongoing quality improvement activities. Efforts are required by leaders to build organisation and unit-level capacity for front-line nursing staff who are best positioned to use and improve patient outcomes. These efforts include providing (1) feedback on patient outcomes and experiences and (2) opportunities for front-line nurses to interpret and use data in meaningful ways to make improvements.7 ,8 ,18 Further, mixed methods studies to examine the impact of using performance dashboards as part of quality improvement strategies are recommended.
References
Footnotes
-
Contributors LJ: made substantial contributions to conception and design, acquisition of data or analysis and interpretation of data; drafted the article and revised it critically for important intellectual content and provided final approval of the version to be published. SB, JL, YL, AC and HC: also contributed to the design and analysis and interpretation of data; revision of the article critically for important intellectual content and provided final approval of the version to be published.
-
Competing interests None.
-
Ethics approval St. Michael's Hospital Research Ethics Board.
-
Provenance and peer review Not commissioned; externally peer reviewed.