Sexton et al describe an observed association between leadership WalkRounds (WR) with feedback and improved levels of safety culture within healthcare settings.1 This work builds on previous data from this group evaluating WR in building a safety culture.2 These encouraging findings spur the need for understanding the robustness of evidence that the WR concept is built on in order to evaluate if continuation and expansion of the WR concept should be promulgated.
The research data that WR are based on are largely observational sets or pre and post studies without control groups or objective outcome measures.3 This is fertile ground for bias and confounding that will undermine the probity of the findings. Sources of bias relate to institutional incentives around WR programs succeeding and the want held by individuals to be seen to be implementing initiatives that improve quality. In the present study described by Sexton et al, the cross sectional observational nature of the data collection is exposed to confounding with clinicians involved in a WR program potentially working in an environment with a superior safety culture regardless of the presence of regular WR (with or without feedback).1 Self-selection bias will also be at play as it is with any voluntary survey method as will be recall bias with culturally high achieving environments and poorly functioning settings over and under estimating their performance respectively, and perhaps ascribi...
Sexton et al describe an observed association between leadership WalkRounds (WR) with feedback and improved levels of safety culture within healthcare settings.1 This work builds on previous data from this group evaluating WR in building a safety culture.2 These encouraging findings spur the need for understanding the robustness of evidence that the WR concept is built on in order to evaluate if continuation and expansion of the WR concept should be promulgated.
The research data that WR are based on are largely observational sets or pre and post studies without control groups or objective outcome measures.3 This is fertile ground for bias and confounding that will undermine the probity of the findings. Sources of bias relate to institutional incentives around WR programs succeeding and the want held by individuals to be seen to be implementing initiatives that improve quality. In the present study described by Sexton et al, the cross sectional observational nature of the data collection is exposed to confounding with clinicians involved in a WR program potentially working in an environment with a superior safety culture regardless of the presence of regular WR (with or without feedback).1 Self-selection bias will also be at play as it is with any voluntary survey method as will be recall bias with culturally high achieving environments and poorly functioning settings over and under estimating their performance respectively, and perhaps ascribing this to the presence or absence of WR (with or without feedback). The limited data in this field using a pure WR intervention (outside a bundled intervention) with control groups have reported mixed findings and possibly a detrimental effect of WR in some settings.3,4
Another potential concern may be that the outcome data point most assessed and focused on in the WR area is safety culture or other relatively subjective outcomes that are used as a surrogate for clinical outcomes relying on linked evidence outside of the WR studies.5 Data linking an intervention to improve safety culture with a corresponding recorded enhancement in clinical outcome measures appear to be lacking. As is the case in much of the quality and safety space, studies that employ an empirical randomised controlled trial (RCT) approach with objective clinical endpoints are absent from the literature.
Overall, it might be argued that within the quality and safety field RCTs are not only needed but are perhaps mandatory if the area is to move forward meaningfully. It may seem to some that explanations positioning the area as somehow different or unique when compared with other health spheres is likely to be met with growing cynicism and perhaps the time to turn the corner has arrived.
References:
1. Sexton JB, Adair KC, Leonard MW, Frankel TC, Proulx J, Watson SR, Magnus B, Bogan B, Jamal M, Schwendimann R, Frankel AS. Providing feedback following Leadership WalkRounds is associated with better patient safety culture, higher employee engagement and lower burnout. BMJ Quality & Safety. 2018 Apr;27(4)261-270.
2. Frankel A, Graydon-Baker E, Neppl C, Simmonds T, Gustafson M, Gandhi TK. Patient safety leadership WalkRounds™. Joint Commission Journal on Quality and Patient Safety. 2003 Jan;29(1):16-26.
3. Singer SJ, Tucker AL. The evolving literature on safety WalkRounds: emerging themes and practical messages. BMJ Quality & Safety. 2014 Oct;23(10):789.
4. Singer SJ, Rivard PE, Hayes JE, Shokeen P, Gaba D, Rosen A. Improving patient care through leadership engagement with frontline staff: A Department of Veterans Affairs case study. Joint Commission Journal on Quality and Patient Safety. 2013 Aug;39(8):349-60.
5. DiCuccio MH. The Relationship Between Patient Safety Culture and Patient Outcomes: A Systematic Review. Journal of Patient Safety. 2015 Sep;11(3):135-42.
Authors do not acknowledge some of the most common criticisms of these studies:
(1) Physician health program (PHP) data may be suspect because PHPs benefit from presenting a rosy picture of their effectiveness.
(2) Self-reports from those being evaluated by PHPs, which have much to lose from responding to surveys in ways that criticize these programs, may not be reliable.
(3) There are considerable reasons to doubt that "programme completion," "return to practice," and "no relapse/recurrence" reflect treatment efficacy. Unwarranted referrals may also result in coerced treatment for physicians who do not have a substance use disorder or problematic performance, making "graduation" not meaningful for the purposes of drawing conclusions about PHP treatment effectiveness.
Other concerns with this research will be addressed in forthcoming publications by the commentator.
Thanks to the authors for this insight. I wondered if they had seen this content http://qualitysafety.bmj.com/content/26/1/61 from Schmidtke et al. which deals with how boards are presented with data, including the consideration of chance (common cause variation). The material seems highly compatible.
Thank you very much for your letter. We agree that the Schmidtke et al paper is highly relevant. In our discussion we note that 'recent research has emphasised the importance of meaningful representation and interpretation of data by boards', citing the accompanying editorial by Mountford and Wakefield which provides an overview both of the Schmidtke et al paper and another paper from the same issue by Anhøj et al on 'Red Amber Green' stoplight reports.
Thank you for this article it summarises the situation well but omits to mention compassion fatigue in any detail . This is an important concept in organisations who need to change and recognise individual coping skills and support people to make positive changes in their own lives. Without self compassion we cannot be compassionate towards others. So whatever changes are made to the organisation it will not make any difference if people are not supported to change themselves. See this article - When Caring Stops Staffing Does Not Really Matter - https://www.nursingeconomics.net/necfiles/staffingUnleashed/su_ND10.pdf Or see my blog for more discussion on self compassion - http://drmarjorieghisoni.edublogs.org/
Badawy et al describe, using statistical analysis, potential inaccuracy in the recording of respiratory rates (RR) in a large cohort of inpatients across a range of inpatient settings and add to the body of data suggesting widespread inaccuracy in the measurement of RR.1 The accurate recording of RR is an important safety and quality issue and the data provided by Badawy et al further underlines the challenges with measurement of this parameter in the inpatient setting.2 Having elegantly demonstrated the problem, the extension of this finding is a need to explore what methods can be potentially employed to improve the accuracy and recording of RR measurement.
Several potential validated solutions may be adduced to address the deficiency in accurate RR measurement and recording. First, consideration could be given to introduction of a system of audit whereby healthcare workers are observed recording RR measurements during their routine practice. Despite a likely Hawthorne effect, the results of this can be collated then non-punitively and anonymously presented to organizational governance structures and health care workers. This concept has been successfully applied into staff hand hygiene quality improvement implementation with this approach having been shown to improve staff performance in this domain with an attendant systematic reduction in adverse event rates.3
Second, the provision of technological solutions, such as a touch pad ba...
Badawy et al describe, using statistical analysis, potential inaccuracy in the recording of respiratory rates (RR) in a large cohort of inpatients across a range of inpatient settings and add to the body of data suggesting widespread inaccuracy in the measurement of RR.1 The accurate recording of RR is an important safety and quality issue and the data provided by Badawy et al further underlines the challenges with measurement of this parameter in the inpatient setting.2 Having elegantly demonstrated the problem, the extension of this finding is a need to explore what methods can be potentially employed to improve the accuracy and recording of RR measurement.
Several potential validated solutions may be adduced to address the deficiency in accurate RR measurement and recording. First, consideration could be given to introduction of a system of audit whereby healthcare workers are observed recording RR measurements during their routine practice. Despite a likely Hawthorne effect, the results of this can be collated then non-punitively and anonymously presented to organizational governance structures and health care workers. This concept has been successfully applied into staff hand hygiene quality improvement implementation with this approach having been shown to improve staff performance in this domain with an attendant systematic reduction in adverse event rates.3
Second, the provision of technological solutions, such as a touch pad based application to record respiratory rates using finger tapping may also have a role in improving accuracy and has been demonstrated in paediatric settings to be potentially effective.4 This technology employs an algorithm whereby the interval between taps (each tap corresponding to a breath observed) is used to calculate a RR. This provides a real-time self-refining measurement of respiratory rate, with more taps generating greater accuracy. To further improve accuracy, and data utility, results could be directly fed into a real-time electronic medical record system.
Finally, complementing the introduction of data collection on performance (with audit of that data) and the potential integration of technological assistive structures would also be the promulgation of education measures. Education measures could focus staff on the data around the historical inaccuracy of RR recording, the assistive technology initiatives being put into place and the importance of accurate measurement for safety and quality. In addition, ongoing feedback to healthcare staff of observed accuracy, as done for hand hygiene measures, would also be important. Multifaceted education of this nature has been shown to be effective for other quality change initiatives.5
In conclusion, a combination of integrated observation and audit, technological implementation and integration and staff education could be used to address the important challenges in measurement of respiratory rate identified by Badawy et al.
References:
1. Badawy J, Nguyen OK, Clark C, et al. Is everyone really breathing 20 times a minute? Assessing epidemiology and variation in recorded respiratory rate in hospitalised adults. BMJ Qual Saf 2017:bmjqs-2017.
2. Fieselmann JF, Hendryx MS, Helms CM, et al. Respiratory rate predicts cardiopulmonary arrest for internal medicine inpatients. J Gen Intern Med 1993;8:354–60.
3. Pittet D, Hugonnet S, Harbarth S et al. Effectiveness of a hospital-wide programme to improve compliance with hand hygiene. The Lancet 2000;356(9238):1307-12.
4. Karlen W, Gan H, Chiu M, et al. Improving the accuracy and efficiency of respiratory rate measurements in children using mobile devices. PLoS One 2014;9(6):e99266.
5. Naikoba S, Hayward A. The effectiveness of interventions aimed at increasing handwashing in healthcare workers-a systematic review. Journal of Hospital Infection 2001;47(3):173-80.
In this paper, Professor Sutton's team attribute higher hospital
death rates at the weekend to the patients being sicker. Sutton is joining
very erudite company (Prof Hawking, Prof Winston and the BMA). This group
is rapidly becoming the 'climate change deniers' of healthcare. Not
including this study, there have been 50 very large studies (>100,000
patients) published so far in this area (supplied on request). 44 show...
In this paper, Professor Sutton's team attribute higher hospital
death rates at the weekend to the patients being sicker. Sutton is joining
very erudite company (Prof Hawking, Prof Winston and the BMA). This group
is rapidly becoming the 'climate change deniers' of healthcare. Not
including this study, there have been 50 very large studies (>100,000
patients) published so far in this area (supplied on request). 44 showed a
weekend effect. These studies used multivariate analysis (to take out
confounding variables, like sickness).
This effect has been shown in emergency and elective admissions, all
over the developed world. It is nothing to do with the UK. Even the degree
of increased risk (approximately 10%) is the same, in almost all of the
studies. It is even more strange that Sutton, in this paper, concluded
"Sunday daytime was .. associated with a higher mortality risk .. compared
with Wednesday daytime" (relative risk 6%) - but this was not emphasised.
In other words, whether you have an emergency admission, or a have a
planned operation, you have an approximately 10% greater chance of dying
if you are admitted at the weekend. Perhaps these Professors (and the
BMA) should read the literature before they continue to confuse the
public. I presume they would trust the NHS to look after their own health
(or that of their family) at the weekend.
Yours faithfully
Dr Andrew Stein
Consultant Physician
Conflict of Interest:
I was one of the authors of NHSE's 7DS 10 Clinical Standards in Dec 2013
With great interest we read the article of Flott et. al. (1), describing the challenges of using patient-reported feedback. We recognize the challenges described and performed a bachelorproject in the intensive care unit (ICU) in the University Medical Center Groningen (UMCG). We think the results from our project provide a potential promising practical solution to make feedback more useful.
In 2013 the UMCG participated in an independent multi-center study conducted among relatives of ICU patients (2). In the open questions of the questionnaire more dissatisfaction than expected was found, which fueled the quest for an alternative, simple and continuous feedback system. In this study we compared the quality and amount of feedback gathered by an oral survey during the first two weeks and an app during the consecutive two weeks.
Between February 20th and March 18th 2017, patients above sixteen years old, listed for discharge from the ICU that day and their relatives were approached to participate in this study. The oral survey consisted of two simple questions: “How satisfied are you with your stay in the ICU? (grade 1-10)” and ”Do you have specific suggestions of improvement for the ICU?”. The RateIt app (Rate It Limited®, Hong Kong) was used consisting of the same two questions as in the oral survey.
A total of 208 responses (133 patients and 75 relatives) were included. The median satisfaction score was 8. Despite this high score many suggestions for...
With great interest we read the article of Flott et. al. (1), describing the challenges of using patient-reported feedback. We recognize the challenges described and performed a bachelorproject in the intensive care unit (ICU) in the University Medical Center Groningen (UMCG). We think the results from our project provide a potential promising practical solution to make feedback more useful.
In 2013 the UMCG participated in an independent multi-center study conducted among relatives of ICU patients (2). In the open questions of the questionnaire more dissatisfaction than expected was found, which fueled the quest for an alternative, simple and continuous feedback system. In this study we compared the quality and amount of feedback gathered by an oral survey during the first two weeks and an app during the consecutive two weeks.
Between February 20th and March 18th 2017, patients above sixteen years old, listed for discharge from the ICU that day and their relatives were approached to participate in this study. The oral survey consisted of two simple questions: “How satisfied are you with your stay in the ICU? (grade 1-10)” and ”Do you have specific suggestions of improvement for the ICU?”. The RateIt app (Rate It Limited®, Hong Kong) was used consisting of the same two questions as in the oral survey.
A total of 208 responses (133 patients and 75 relatives) were included. The median satisfaction score was 8. Despite this high score many suggestions for improvement (n=95 suggestions given by 68 respondents) were given. The oral survey provided more often suggestions for improvement compared with the app (50 vs. 18 respondents). Suggestions for improvement were more frequently made by relatives compared with patients (57 suggestions given by 37 relatives vs. 38 suggestions given by 31 patients). All improvement suggestions were classified to one of six categories: ‘Surroundings’ 48/95 (51%), ‘Information, communication and education’ 23/95 (24%), ‘Patient care’ 15/95 (16%), ‘Attitude, handling and relation caregiver with patient/relatives 7/95 (7%), ‘Emotional support’ 1/95 (1%) and ‘Care for relatives’ 1/95 (1%).
This simple study showed that an oral survey results in more suggestions for improvement than an app. The lack of complexity of the survey resulted in very specific, useful and practical suggestions for improvement, which were easily transformed into clear recommendations, such as: “respect sufficient rest of our patients” or “don’t forget to provide food to the patients who are able to eat”. The survey can easily be repeated in the course of time. These results may give a new perspective on how to conduct feedback studies.
The key suggestions for improvement found in this study were presented to the department in the form of a coat rack, which was an improvement option frequently mentioned by relatives (A coat rack was missing in one of our family rooms). This coat rack will be hung in central places in our department. On this coatrack recommendations based on the most important improvement suggestions will be hung. We think this is one example of a simple, but practical solution to make feedback more useful: every month the recommendations will be replaced by new ones, reminding all caregivers in our department of the feedback given by our patients and their relatives and thereby striving to improve our care.
We are well aware of the fact that the surveys used in the studies described in the article of Flott et al1 are much larger and more complex than the one we used in our study. We just wanted to show that a learning point could be: don’t overcomplicate.
References
1. Flott KM, Graham C, Darzi A, Mayer E. Can we use patient-reported feedback to drive change? The challenges of using patient-reported feedback and how they might be addressed. BMJ Qual Saf 2017;26:502-507.
2. Jensen HI, Gerritsen RT, Koopmans M, Zijlstra JG, Randall Curtis J, Ording H. Families’ experiences of intensive care unit quality of care: Development and validation of a European questionnaire (euroQ2). Journal of Critical Care 2015;30(5):884-890.
This study uses rigorous analysis to obtain important insights about the realtime information that our patients are handed at discharge. It is puzzling that the EMRs used were not named. One can infer from a look through the MSU website that they have both Cerner and Epic, but why is that necessary? The heart of quality/safety work is one of transparency balanced by humility, i.e. we shouldn't expect our IT systems to be any more perfect than we are, but they won't improve if we don't have more openness. The lack of scientific foundations and published post-marketing surveillance for our EHRs, especially the ascendant ones, was initially surprising. However, as they achieve complete market dominance, with less overt scientific review and public guidance and commentary, the silence is deafening. Is the BMJQS's failure to simply identify the names (or maybe I missed the citations) an oversight, or part of nondisclosure agreements with the vendors at the MSU institutions or at BMJQS?
As you point out Root Cause Analysis will often fail with hospital adverse event (AE) data because it was not designed to deal with data arising in a complex system.1 The same can be said for Pareto analysis. Statistical process control (SPC) methods are often used to summarise AE data, particularly hospital infection data such as surgical site infections (SSIs) and bacteraemias.2 Standard SPC also frequently fails to summarise these complex data correctly.
With binary SSI data an approximate expected rate is frequently available so cumulative observed minus expected and CUSUM analysis are appropriate.2 However, the changing observed rate is not seen unless the numbers of procedures is large enough for them to be grouped by months or quarters. This is often infrequent. Even when such aggregation is possible difficulties arise as the number of procedures in each month may differ markedly. This problem can be dealt with, at least approximately, by employing a generalised additive model (GAM) analysis to the binary data that predicts the observed AE rate at various places in the time series.
Count and rate data such as bacteraemias or new isolates of an antibiotic-resistant organism will usually not have an expected rate available. These data are often grouped by months and a Shewhat chart used for their display. This chart requires a stable centre-line about which reliable control limits can be drawn. Often the mean value is used as the expected rate even though...
As you point out Root Cause Analysis will often fail with hospital adverse event (AE) data because it was not designed to deal with data arising in a complex system.1 The same can be said for Pareto analysis. Statistical process control (SPC) methods are often used to summarise AE data, particularly hospital infection data such as surgical site infections (SSIs) and bacteraemias.2 Standard SPC also frequently fails to summarise these complex data correctly.
With binary SSI data an approximate expected rate is frequently available so cumulative observed minus expected and CUSUM analysis are appropriate.2 However, the changing observed rate is not seen unless the numbers of procedures is large enough for them to be grouped by months or quarters. This is often infrequent. Even when such aggregation is possible difficulties arise as the number of procedures in each month may differ markedly. This problem can be dealt with, at least approximately, by employing a generalised additive model (GAM) analysis to the binary data that predicts the observed AE rate at various places in the time series.
Count and rate data such as bacteraemias or new isolates of an antibiotic-resistant organism will usually not have an expected rate available. These data are often grouped by months and a Shewhat chart used for their display. This chart requires a stable centre-line about which reliable control limits can be drawn. Often the mean value is used as the expected rate even though it may be representative of few or none of the monthly data values. This makes the control limits meaningless. A probable way round this is to employ confidence limits for the monthly rates. Viewed as a likelihood supported range this enables the extent of each of the monthly counts or rates to be assessed. If a GAM analysis is added to this the predicted rate and its confidence limits can also be obtained throughout the time series.2
This approach is more in keeping with the complexity of the processes responsible for the AE than is standard SPC that was not designed to deal with complex systems.
As an aside, it is worth noting that some swamps may be valuable ecosystems. This popular analogy is thus a poor one. Like root-cause analysis it belongs to the area of simple/complicated systems, not complex ones.
1. Morton, A., Whitby, M., Tierney, N., Sibanda, N. and Mengersen, K. 2016. Statistical Methods for Hospital Monitoring. Wiley StatsRef: Statistics Reference Online. 1–8.
2. Morton A, Mengersen K, Whitby M. and Playford G. Statistical Methods for Hospital Monitoring with R. Chichester John Wiley and Sons 2013.
To the editor,
Sexton et al describe an observed association between leadership WalkRounds (WR) with feedback and improved levels of safety culture within healthcare settings.1 This work builds on previous data from this group evaluating WR in building a safety culture.2 These encouraging findings spur the need for understanding the robustness of evidence that the WR concept is built on in order to evaluate if continuation and expansion of the WR concept should be promulgated.
The research data that WR are based on are largely observational sets or pre and post studies without control groups or objective outcome measures.3 This is fertile ground for bias and confounding that will undermine the probity of the findings. Sources of bias relate to institutional incentives around WR programs succeeding and the want held by individuals to be seen to be implementing initiatives that improve quality. In the present study described by Sexton et al, the cross sectional observational nature of the data collection is exposed to confounding with clinicians involved in a WR program potentially working in an environment with a superior safety culture regardless of the presence of regular WR (with or without feedback).1 Self-selection bias will also be at play as it is with any voluntary survey method as will be recall bias with culturally high achieving environments and poorly functioning settings over and under estimating their performance respectively, and perhaps ascribi...
Show MoreAuthors do not acknowledge some of the most common criticisms of these studies:
(1) Physician health program (PHP) data may be suspect because PHPs benefit from presenting a rosy picture of their effectiveness.
(2) Self-reports from those being evaluated by PHPs, which have much to lose from responding to surveys in ways that criticize these programs, may not be reliable.
(3) There are considerable reasons to doubt that "programme completion," "return to practice," and "no relapse/recurrence" reflect treatment efficacy. Unwarranted referrals may also result in coerced treatment for physicians who do not have a substance use disorder or problematic performance, making "graduation" not meaningful for the purposes of drawing conclusions about PHP treatment effectiveness.
Other concerns with this research will be addressed in forthcoming publications by the commentator.
Thanks to the authors for this insight. I wondered if they had seen this content http://qualitysafety.bmj.com/content/26/1/61 from Schmidtke et al. which deals with how boards are presented with data, including the consideration of chance (common cause variation). The material seems highly compatible.
Thank you very much for your letter. We agree that the Schmidtke et al paper is highly relevant. In our discussion we note that 'recent research has emphasised the importance of meaningful representation and interpretation of data by boards', citing the accompanying editorial by Mountford and Wakefield which provides an overview both of the Schmidtke et al paper and another paper from the same issue by Anhøj et al on 'Red Amber Green' stoplight reports.
Thank you for this article it summarises the situation well but omits to mention compassion fatigue in any detail . This is an important concept in organisations who need to change and recognise individual coping skills and support people to make positive changes in their own lives. Without self compassion we cannot be compassionate towards others. So whatever changes are made to the organisation it will not make any difference if people are not supported to change themselves. See this article - When Caring Stops Staffing Does Not Really Matter - https://www.nursingeconomics.net/necfiles/staffingUnleashed/su_ND10.pdf Or see my blog for more discussion on self compassion - http://drmarjorieghisoni.edublogs.org/
To the Editor,
Badawy et al describe, using statistical analysis, potential inaccuracy in the recording of respiratory rates (RR) in a large cohort of inpatients across a range of inpatient settings and add to the body of data suggesting widespread inaccuracy in the measurement of RR.1 The accurate recording of RR is an important safety and quality issue and the data provided by Badawy et al further underlines the challenges with measurement of this parameter in the inpatient setting.2 Having elegantly demonstrated the problem, the extension of this finding is a need to explore what methods can be potentially employed to improve the accuracy and recording of RR measurement.
Several potential validated solutions may be adduced to address the deficiency in accurate RR measurement and recording. First, consideration could be given to introduction of a system of audit whereby healthcare workers are observed recording RR measurements during their routine practice. Despite a likely Hawthorne effect, the results of this can be collated then non-punitively and anonymously presented to organizational governance structures and health care workers. This concept has been successfully applied into staff hand hygiene quality improvement implementation with this approach having been shown to improve staff performance in this domain with an attendant systematic reduction in adverse event rates.3
Second, the provision of technological solutions, such as a touch pad ba...
Show MoreIn this paper, Professor Sutton's team attribute higher hospital death rates at the weekend to the patients being sicker. Sutton is joining very erudite company (Prof Hawking, Prof Winston and the BMA). This group is rapidly becoming the 'climate change deniers' of healthcare. Not including this study, there have been 50 very large studies (>100,000 patients) published so far in this area (supplied on request). 44 show...
With great interest we read the article of Flott et. al. (1), describing the challenges of using patient-reported feedback. We recognize the challenges described and performed a bachelorproject in the intensive care unit (ICU) in the University Medical Center Groningen (UMCG). We think the results from our project provide a potential promising practical solution to make feedback more useful.
Show MoreIn 2013 the UMCG participated in an independent multi-center study conducted among relatives of ICU patients (2). In the open questions of the questionnaire more dissatisfaction than expected was found, which fueled the quest for an alternative, simple and continuous feedback system. In this study we compared the quality and amount of feedback gathered by an oral survey during the first two weeks and an app during the consecutive two weeks.
Between February 20th and March 18th 2017, patients above sixteen years old, listed for discharge from the ICU that day and their relatives were approached to participate in this study. The oral survey consisted of two simple questions: “How satisfied are you with your stay in the ICU? (grade 1-10)” and ”Do you have specific suggestions of improvement for the ICU?”. The RateIt app (Rate It Limited®, Hong Kong) was used consisting of the same two questions as in the oral survey.
A total of 208 responses (133 patients and 75 relatives) were included. The median satisfaction score was 8. Despite this high score many suggestions for...
This study uses rigorous analysis to obtain important insights about the realtime information that our patients are handed at discharge. It is puzzling that the EMRs used were not named. One can infer from a look through the MSU website that they have both Cerner and Epic, but why is that necessary? The heart of quality/safety work is one of transparency balanced by humility, i.e. we shouldn't expect our IT systems to be any more perfect than we are, but they won't improve if we don't have more openness. The lack of scientific foundations and published post-marketing surveillance for our EHRs, especially the ascendant ones, was initially surprising. However, as they achieve complete market dominance, with less overt scientific review and public guidance and commentary, the silence is deafening. Is the BMJQS's failure to simply identify the names (or maybe I missed the citations) an oversight, or part of nondisclosure agreements with the vendors at the MSU institutions or at BMJQS?
As you point out Root Cause Analysis will often fail with hospital adverse event (AE) data because it was not designed to deal with data arising in a complex system.1 The same can be said for Pareto analysis. Statistical process control (SPC) methods are often used to summarise AE data, particularly hospital infection data such as surgical site infections (SSIs) and bacteraemias.2 Standard SPC also frequently fails to summarise these complex data correctly.
Show MoreWith binary SSI data an approximate expected rate is frequently available so cumulative observed minus expected and CUSUM analysis are appropriate.2 However, the changing observed rate is not seen unless the numbers of procedures is large enough for them to be grouped by months or quarters. This is often infrequent. Even when such aggregation is possible difficulties arise as the number of procedures in each month may differ markedly. This problem can be dealt with, at least approximately, by employing a generalised additive model (GAM) analysis to the binary data that predicts the observed AE rate at various places in the time series.
Count and rate data such as bacteraemias or new isolates of an antibiotic-resistant organism will usually not have an expected rate available. These data are often grouped by months and a Shewhat chart used for their display. This chart requires a stable centre-line about which reliable control limits can be drawn. Often the mean value is used as the expected rate even though...
Pages