I read this paper1 first published on 5 March 2018 in your journal with great interest.
The great pace of health information technology (health IT) advancement in recent decades held promise in improving patient safety and quality of care, but unfortunately there has since been inadvertent consequences and carry-over effects of technology-related safety concerns in its use and implementation.2 This paper has further fuelled the boon or bane debate of health IT.3
Ironically, the implementation of a national, multifaceted, quality improvement (QI) programme of ‘de-implementing’ electronic health record (EHR) notifications to primary care physicians (PCPs) has shown some benefit.1 This has, in some way, proven that being too reliant on technology in healthcare may yet rear its ugly head.
The paper has shown that high volume of EHR notifications can overwhelm PCPs;1 the proposed measure of breaking these down into “low-value” and “high-value”, and enforcing certain mandatory ones, may merely be an intermediate stopgap technique. Determining which is which, by its nature, is difficult to do and standardise.
Further, implementing a nationwide programme such as this poses certain challenges that the authors have not considered – will there be a difference between urban and rural healthcare facilities in its implementation, given resource limitations?4 Will technology have improved or changed by the time this programme is fully implemented nationwide?...
I read this paper1 first published on 5 March 2018 in your journal with great interest.
The great pace of health information technology (health IT) advancement in recent decades held promise in improving patient safety and quality of care, but unfortunately there has since been inadvertent consequences and carry-over effects of technology-related safety concerns in its use and implementation.2 This paper has further fuelled the boon or bane debate of health IT.3
Ironically, the implementation of a national, multifaceted, quality improvement (QI) programme of ‘de-implementing’ electronic health record (EHR) notifications to primary care physicians (PCPs) has shown some benefit.1 This has, in some way, proven that being too reliant on technology in healthcare may yet rear its ugly head.
The paper has shown that high volume of EHR notifications can overwhelm PCPs;1 the proposed measure of breaking these down into “low-value” and “high-value”, and enforcing certain mandatory ones, may merely be an intermediate stopgap technique. Determining which is which, by its nature, is difficult to do and standardise.
Further, implementing a nationwide programme such as this poses certain challenges that the authors have not considered – will there be a difference between urban and rural healthcare facilities in its implementation, given resource limitations?4 Will technology have improved or changed by the time this programme is fully implemented nationwide? If so, the costs of constantly chasing our technological tails will be great, especially if applied to larger countries, or ones with sprawling rural/regional areas like in Australia.
National programmes pose other barriers as well: would measurements be commensurate with other driving factors like local differences in practice and accessibility? The authors assumed that reduced notifications is a positive outcome, but if a regional facility has limited access to certain diagnostic tools or treatment capacity, the EHR notifications for these things will inevitably be less. Does that truly mean that the medicine practised in these areas are optimal and evidence-based, as espoused by the article?
But the paper has also shown that technological de-implementation of a health IT system is a double-edged sword. The results have indicated that there are PCPs and facilities enabling higher notifications post-intervention.1 Perhaps the real issue here isn’t using one form of technology to counter another’s pitfalls: we’re so focussed on technology alone where we’ve become so used to ordering diagnostic tests and monitoring because of its availability, at the expense of proper clinical practice and reasons, that we need to be constantly reminded of what we’ve ordered. Perhaps we should aim to integrate technology into a healthcare culture where principles of safety and quality are emphasised instead.5
(430 words)
References:
1. Shah T, Patel-Teague S, Kroupa L, et al. Impact of a national QI programme on reducing electronic health record notifications to clinicians. BMJ quality & safety 2018 doi: 10.1136/bmjqs-2017-007447
2. Magrabi F, Aarts J, Nohr C, et al. A comparative review of patient safety initiatives for national health information technology. International journal of medical informatics 2013;82(5):e139-48. doi: 10.1016/j.ijmedinf.2012.11.014 [published Online First: 2012/12/26]
3. Jacobson J. Health information technology: bane or boon? The American journal of nursing 2014;114(12):18-9. doi: 10.1097/01.Naj.0000457401.30598.69 [published Online First: 2014/11/26]
4. Chipp CL, Johnson ME, Brems C, et al. Adaptations to Health Care Barriers as Reported by Rural and Urban Providers. Journal of health care for the poor and underserved 2008;19(2):532-49. doi: 10.1353/hpu.0.0002
5. Thimbleby H. Technology and the Future of Healthcare. Journal of Public Health Research 2013;2(3):e28. doi: 10.4081/jphr.2013.e28
Sexton et al describe an observed association between leadership WalkRounds (WR) with feedback and improved levels of safety culture within healthcare settings.1 This work builds on previous data from this group evaluating WR in building a safety culture.2 These encouraging findings spur the need for understanding the robustness of evidence that the WR concept is built on in order to evaluate if continuation and expansion of the WR concept should be promulgated.
The research data that WR are based on are largely observational sets or pre and post studies without control groups or objective outcome measures.3 This is fertile ground for bias and confounding that will undermine the probity of the findings. Sources of bias relate to institutional incentives around WR programs succeeding and the want held by individuals to be seen to be implementing initiatives that improve quality. In the present study described by Sexton et al, the cross sectional observational nature of the data collection is exposed to confounding with clinicians involved in a WR program potentially working in an environment with a superior safety culture regardless of the presence of regular WR (with or without feedback).1 Self-selection bias will also be at play as it is with any voluntary survey method as will be recall bias with culturally high achieving environments and poorly functioning settings over and under estimating their performance respectively, and perhaps ascribi...
Sexton et al describe an observed association between leadership WalkRounds (WR) with feedback and improved levels of safety culture within healthcare settings.1 This work builds on previous data from this group evaluating WR in building a safety culture.2 These encouraging findings spur the need for understanding the robustness of evidence that the WR concept is built on in order to evaluate if continuation and expansion of the WR concept should be promulgated.
The research data that WR are based on are largely observational sets or pre and post studies without control groups or objective outcome measures.3 This is fertile ground for bias and confounding that will undermine the probity of the findings. Sources of bias relate to institutional incentives around WR programs succeeding and the want held by individuals to be seen to be implementing initiatives that improve quality. In the present study described by Sexton et al, the cross sectional observational nature of the data collection is exposed to confounding with clinicians involved in a WR program potentially working in an environment with a superior safety culture regardless of the presence of regular WR (with or without feedback).1 Self-selection bias will also be at play as it is with any voluntary survey method as will be recall bias with culturally high achieving environments and poorly functioning settings over and under estimating their performance respectively, and perhaps ascribing this to the presence or absence of WR (with or without feedback). The limited data in this field using a pure WR intervention (outside a bundled intervention) with control groups have reported mixed findings and possibly a detrimental effect of WR in some settings.3,4
Another potential concern may be that the outcome data point most assessed and focused on in the WR area is safety culture or other relatively subjective outcomes that are used as a surrogate for clinical outcomes relying on linked evidence outside of the WR studies.5 Data linking an intervention to improve safety culture with a corresponding recorded enhancement in clinical outcome measures appear to be lacking. As is the case in much of the quality and safety space, studies that employ an empirical randomised controlled trial (RCT) approach with objective clinical endpoints are absent from the literature.
Overall, it might be argued that within the quality and safety field RCTs are not only needed but are perhaps mandatory if the area is to move forward meaningfully. It may seem to some that explanations positioning the area as somehow different or unique when compared with other health spheres is likely to be met with growing cynicism and perhaps the time to turn the corner has arrived.
References:
1. Sexton JB, Adair KC, Leonard MW, Frankel TC, Proulx J, Watson SR, Magnus B, Bogan B, Jamal M, Schwendimann R, Frankel AS. Providing feedback following Leadership WalkRounds is associated with better patient safety culture, higher employee engagement and lower burnout. BMJ Quality & Safety. 2018 Apr;27(4)261-270.
2. Frankel A, Graydon-Baker E, Neppl C, Simmonds T, Gustafson M, Gandhi TK. Patient safety leadership WalkRounds™. Joint Commission Journal on Quality and Patient Safety. 2003 Jan;29(1):16-26.
3. Singer SJ, Tucker AL. The evolving literature on safety WalkRounds: emerging themes and practical messages. BMJ Quality & Safety. 2014 Oct;23(10):789.
4. Singer SJ, Rivard PE, Hayes JE, Shokeen P, Gaba D, Rosen A. Improving patient care through leadership engagement with frontline staff: A Department of Veterans Affairs case study. Joint Commission Journal on Quality and Patient Safety. 2013 Aug;39(8):349-60.
5. DiCuccio MH. The Relationship Between Patient Safety Culture and Patient Outcomes: A Systematic Review. Journal of Patient Safety. 2015 Sep;11(3):135-42.
The population with intellectual disabilities have multiple morbidities and greater health needs compared with the general population. This population experiences health and healthcare inequities and inequalities. To reduce the health inequality gap people with intellectual disabilities should be involved as partners in their healthcare. This will require access to relevant information and the development of tools that support collaboration, such as tailored patient decision aids (PDA) (1).
The population with intellectual disabilities is rarely considered or involved (2)at the guideline development stage. The consequent failure of clinical guidelines to adequately address the health needs of people with intellectual disabilities exacerbates already poor access to health and healthcare. An examination of clinical guidelines from seven countries(3) found that most clinical guidelines failed to address people with intellectual disabilities as being at high risk for particular conditions when appropriate.
Guidelines and PDAs developed with the general population in mind may not reflect the complexity and multi-morbidity of individual patients with intellectual disabilities and their ‘real world’ lives. Many people with intellectual disabilities have visual, hearing, mobility, memory and dexterity difficulties. Clinicians and guidelines developers may not be aware of the complexity of the task their ask their patients with intellectual disabilities and their...
The population with intellectual disabilities have multiple morbidities and greater health needs compared with the general population. This population experiences health and healthcare inequities and inequalities. To reduce the health inequality gap people with intellectual disabilities should be involved as partners in their healthcare. This will require access to relevant information and the development of tools that support collaboration, such as tailored patient decision aids (PDA) (1).
The population with intellectual disabilities is rarely considered or involved (2)at the guideline development stage. The consequent failure of clinical guidelines to adequately address the health needs of people with intellectual disabilities exacerbates already poor access to health and healthcare. An examination of clinical guidelines from seven countries(3) found that most clinical guidelines failed to address people with intellectual disabilities as being at high risk for particular conditions when appropriate.
Guidelines and PDAs developed with the general population in mind may not reflect the complexity and multi-morbidity of individual patients with intellectual disabilities and their ‘real world’ lives. Many people with intellectual disabilities have visual, hearing, mobility, memory and dexterity difficulties. Clinicians and guidelines developers may not be aware of the complexity of the task their ask their patients with intellectual disabilities and their carers to undertake e.g. blood glucose monitoring, sliding scale insulin administration, inhaler use, administration of epilepsy rescue medication etc..
Significant gaps in the reporting of evaluations of PDAs have been identified (4). The SUNDAE Checklist (Standards for UNiversal reporting of patient Decision Aid Evaluations) should facilitate evaluation of any PDA, in ‘hard to reach’ population groups, such as the population with intellectual disabilities.
Before applying a guideline or using PDAs with a person with an intellectual disability the following questions could be asked
• Does this person with an intellectual disability understand the connection between the illness and some consequent intervention e.g. diet/exercise, prescribed medication, blood testing?
• What level of health literacy has the person with intellectual disability and/or their carer?
• Do the outcomes measured matter to this person with intellectual disability?
• Is this person with intellectual disability very different from the people studied?
• Are the treatments practical in the ‘real world’ living environment of this person with intellectual disability?
• Do treatment comparisons reflect the circumstances of the person with intellectual disability?
• How certain can the evidence be applied to a person with intellectual disability?
• Do the advantages outweigh the disadvantages for this person with intellectual disability?
1. BMJ 2017; editorial Shehan, Strydom, Hassiotis 358 doi: https://doi.org/10.1136/bmj.j3896
2. Mizen LAM, Maclie ML, Cooper S-A, Melville CA. Clinical guidelines contribute to the health inequities experienced by individuals with intellectual disabilities. Implementation Science. 2012;7(42):1–9.
3. Mizen LAM, Maclie ML, Cooper S-A, Melville CA. Clinical guidelines contribute to the health inequities experienced by individuals with intellectual disabilities. Implementation Science. 2012;7(42):1–9
4. Sepucha KR, Abhyankar P, Hoffman AS, et al. Standards for UNiversal reporting of patient Decision Aid Evaluation studies: the development of SUNDAE Checklist. BMJ Qual Saf Published Online First: 21 December 2017. doi: 10.1136/bmjqs-2017-006986
Authors do not acknowledge some of the most common criticisms of these studies:
(1) Physician health program (PHP) data may be suspect because PHPs benefit from presenting a rosy picture of their effectiveness.
(2) Self-reports from those being evaluated by PHPs, which have much to lose from responding to surveys in ways that criticize these programs, may not be reliable.
(3) There are considerable reasons to doubt that "programme completion," "return to practice," and "no relapse/recurrence" reflect treatment efficacy. Unwarranted referrals may also result in coerced treatment for physicians who do not have a substance use disorder or problematic performance, making "graduation" not meaningful for the purposes of drawing conclusions about PHP treatment effectiveness.
Other concerns with this research will be addressed in forthcoming publications by the commentator.
Thank you for this article it summarises the situation well but omits to mention compassion fatigue in any detail . This is an important concept in organisations who need to change and recognise individual coping skills and support people to make positive changes in their own lives. Without self compassion we cannot be compassionate towards others. So whatever changes are made to the organisation it will not make any difference if people are not supported to change themselves. See this article - When Caring Stops Staffing Does Not Really Matter - https://www.nursingeconomics.net/necfiles/staffingUnleashed/su_ND10.pdf Or see my blog for more discussion on self compassion - http://drmarjorieghisoni.edublogs.org/
Thank you very much for your letter. We agree that the Schmidtke et al paper is highly relevant. In our discussion we note that 'recent research has emphasised the importance of meaningful representation and interpretation of data by boards', citing the accompanying editorial by Mountford and Wakefield which provides an overview both of the Schmidtke et al paper and another paper from the same issue by Anhøj et al on 'Red Amber Green' stoplight reports.
Thanks to the authors for this insight. I wondered if they had seen this content http://qualitysafety.bmj.com/content/26/1/61 from Schmidtke et al. which deals with how boards are presented with data, including the consideration of chance (common cause variation). The material seems highly compatible.
Badawy et al describe, using statistical analysis, potential inaccuracy in the recording of respiratory rates (RR) in a large cohort of inpatients across a range of inpatient settings and add to the body of data suggesting widespread inaccuracy in the measurement of RR.1 The accurate recording of RR is an important safety and quality issue and the data provided by Badawy et al further underlines the challenges with measurement of this parameter in the inpatient setting.2 Having elegantly demonstrated the problem, the extension of this finding is a need to explore what methods can be potentially employed to improve the accuracy and recording of RR measurement.
Several potential validated solutions may be adduced to address the deficiency in accurate RR measurement and recording. First, consideration could be given to introduction of a system of audit whereby healthcare workers are observed recording RR measurements during their routine practice. Despite a likely Hawthorne effect, the results of this can be collated then non-punitively and anonymously presented to organizational governance structures and health care workers. This concept has been successfully applied into staff hand hygiene quality improvement implementation with this approach having been shown to improve staff performance in this domain with an attendant systematic reduction in adverse event rates.3
Second, the provision of technological solutions, such as a touch pad ba...
Badawy et al describe, using statistical analysis, potential inaccuracy in the recording of respiratory rates (RR) in a large cohort of inpatients across a range of inpatient settings and add to the body of data suggesting widespread inaccuracy in the measurement of RR.1 The accurate recording of RR is an important safety and quality issue and the data provided by Badawy et al further underlines the challenges with measurement of this parameter in the inpatient setting.2 Having elegantly demonstrated the problem, the extension of this finding is a need to explore what methods can be potentially employed to improve the accuracy and recording of RR measurement.
Several potential validated solutions may be adduced to address the deficiency in accurate RR measurement and recording. First, consideration could be given to introduction of a system of audit whereby healthcare workers are observed recording RR measurements during their routine practice. Despite a likely Hawthorne effect, the results of this can be collated then non-punitively and anonymously presented to organizational governance structures and health care workers. This concept has been successfully applied into staff hand hygiene quality improvement implementation with this approach having been shown to improve staff performance in this domain with an attendant systematic reduction in adverse event rates.3
Second, the provision of technological solutions, such as a touch pad based application to record respiratory rates using finger tapping may also have a role in improving accuracy and has been demonstrated in paediatric settings to be potentially effective.4 This technology employs an algorithm whereby the interval between taps (each tap corresponding to a breath observed) is used to calculate a RR. This provides a real-time self-refining measurement of respiratory rate, with more taps generating greater accuracy. To further improve accuracy, and data utility, results could be directly fed into a real-time electronic medical record system.
Finally, complementing the introduction of data collection on performance (with audit of that data) and the potential integration of technological assistive structures would also be the promulgation of education measures. Education measures could focus staff on the data around the historical inaccuracy of RR recording, the assistive technology initiatives being put into place and the importance of accurate measurement for safety and quality. In addition, ongoing feedback to healthcare staff of observed accuracy, as done for hand hygiene measures, would also be important. Multifaceted education of this nature has been shown to be effective for other quality change initiatives.5
In conclusion, a combination of integrated observation and audit, technological implementation and integration and staff education could be used to address the important challenges in measurement of respiratory rate identified by Badawy et al.
References:
1. Badawy J, Nguyen OK, Clark C, et al. Is everyone really breathing 20 times a minute? Assessing epidemiology and variation in recorded respiratory rate in hospitalised adults. BMJ Qual Saf 2017:bmjqs-2017.
2. Fieselmann JF, Hendryx MS, Helms CM, et al. Respiratory rate predicts cardiopulmonary arrest for internal medicine inpatients. J Gen Intern Med 1993;8:354–60.
3. Pittet D, Hugonnet S, Harbarth S et al. Effectiveness of a hospital-wide programme to improve compliance with hand hygiene. The Lancet 2000;356(9238):1307-12.
4. Karlen W, Gan H, Chiu M, et al. Improving the accuracy and efficiency of respiratory rate measurements in children using mobile devices. PLoS One 2014;9(6):e99266.
5. Naikoba S, Hayward A. The effectiveness of interventions aimed at increasing handwashing in healthcare workers-a systematic review. Journal of Hospital Infection 2001;47(3):173-80.
With great interest we read the article of Flott et. al. (1), describing the challenges of using patient-reported feedback. We recognize the challenges described and performed a bachelorproject in the intensive care unit (ICU) in the University Medical Center Groningen (UMCG). We think the results from our project provide a potential promising practical solution to make feedback more useful.
In 2013 the UMCG participated in an independent multi-center study conducted among relatives of ICU patients (2). In the open questions of the questionnaire more dissatisfaction than expected was found, which fueled the quest for an alternative, simple and continuous feedback system. In this study we compared the quality and amount of feedback gathered by an oral survey during the first two weeks and an app during the consecutive two weeks.
Between February 20th and March 18th 2017, patients above sixteen years old, listed for discharge from the ICU that day and their relatives were approached to participate in this study. The oral survey consisted of two simple questions: “How satisfied are you with your stay in the ICU? (grade 1-10)” and ”Do you have specific suggestions of improvement for the ICU?”. The RateIt app (Rate It Limited®, Hong Kong) was used consisting of the same two questions as in the oral survey.
A total of 208 responses (133 patients and 75 relatives) were included. The median satisfaction score was 8. Despite this high score many suggestions for...
With great interest we read the article of Flott et. al. (1), describing the challenges of using patient-reported feedback. We recognize the challenges described and performed a bachelorproject in the intensive care unit (ICU) in the University Medical Center Groningen (UMCG). We think the results from our project provide a potential promising practical solution to make feedback more useful.
In 2013 the UMCG participated in an independent multi-center study conducted among relatives of ICU patients (2). In the open questions of the questionnaire more dissatisfaction than expected was found, which fueled the quest for an alternative, simple and continuous feedback system. In this study we compared the quality and amount of feedback gathered by an oral survey during the first two weeks and an app during the consecutive two weeks.
Between February 20th and March 18th 2017, patients above sixteen years old, listed for discharge from the ICU that day and their relatives were approached to participate in this study. The oral survey consisted of two simple questions: “How satisfied are you with your stay in the ICU? (grade 1-10)” and ”Do you have specific suggestions of improvement for the ICU?”. The RateIt app (Rate It Limited®, Hong Kong) was used consisting of the same two questions as in the oral survey.
A total of 208 responses (133 patients and 75 relatives) were included. The median satisfaction score was 8. Despite this high score many suggestions for improvement (n=95 suggestions given by 68 respondents) were given. The oral survey provided more often suggestions for improvement compared with the app (50 vs. 18 respondents). Suggestions for improvement were more frequently made by relatives compared with patients (57 suggestions given by 37 relatives vs. 38 suggestions given by 31 patients). All improvement suggestions were classified to one of six categories: ‘Surroundings’ 48/95 (51%), ‘Information, communication and education’ 23/95 (24%), ‘Patient care’ 15/95 (16%), ‘Attitude, handling and relation caregiver with patient/relatives 7/95 (7%), ‘Emotional support’ 1/95 (1%) and ‘Care for relatives’ 1/95 (1%).
This simple study showed that an oral survey results in more suggestions for improvement than an app. The lack of complexity of the survey resulted in very specific, useful and practical suggestions for improvement, which were easily transformed into clear recommendations, such as: “respect sufficient rest of our patients” or “don’t forget to provide food to the patients who are able to eat”. The survey can easily be repeated in the course of time. These results may give a new perspective on how to conduct feedback studies.
The key suggestions for improvement found in this study were presented to the department in the form of a coat rack, which was an improvement option frequently mentioned by relatives (A coat rack was missing in one of our family rooms). This coat rack will be hung in central places in our department. On this coatrack recommendations based on the most important improvement suggestions will be hung. We think this is one example of a simple, but practical solution to make feedback more useful: every month the recommendations will be replaced by new ones, reminding all caregivers in our department of the feedback given by our patients and their relatives and thereby striving to improve our care.
We are well aware of the fact that the surveys used in the studies described in the article of Flott et al1 are much larger and more complex than the one we used in our study. We just wanted to show that a learning point could be: don’t overcomplicate.
References
1. Flott KM, Graham C, Darzi A, Mayer E. Can we use patient-reported feedback to drive change? The challenges of using patient-reported feedback and how they might be addressed. BMJ Qual Saf 2017;26:502-507.
2. Jensen HI, Gerritsen RT, Koopmans M, Zijlstra JG, Randall Curtis J, Ording H. Families’ experiences of intensive care unit quality of care: Development and validation of a European questionnaire (euroQ2). Journal of Critical Care 2015;30(5):884-890.
As you point out Root Cause Analysis will often fail with hospital adverse event (AE) data because it was not designed to deal with data arising in a complex system.1 The same can be said for Pareto analysis. Statistical process control (SPC) methods are often used to summarise AE data, particularly hospital infection data such as surgical site infections (SSIs) and bacteraemias.2 Standard SPC also frequently fails to summarise these complex data correctly.
With binary SSI data an approximate expected rate is frequently available so cumulative observed minus expected and CUSUM analysis are appropriate.2 However, the changing observed rate is not seen unless the numbers of procedures is large enough for them to be grouped by months or quarters. This is often infrequent. Even when such aggregation is possible difficulties arise as the number of procedures in each month may differ markedly. This problem can be dealt with, at least approximately, by employing a generalised additive model (GAM) analysis to the binary data that predicts the observed AE rate at various places in the time series.
Count and rate data such as bacteraemias or new isolates of an antibiotic-resistant organism will usually not have an expected rate available. These data are often grouped by months and a Shewhat chart used for their display. This chart requires a stable centre-line about which reliable control limits can be drawn. Often the mean value is used as the expected rate even though...
As you point out Root Cause Analysis will often fail with hospital adverse event (AE) data because it was not designed to deal with data arising in a complex system.1 The same can be said for Pareto analysis. Statistical process control (SPC) methods are often used to summarise AE data, particularly hospital infection data such as surgical site infections (SSIs) and bacteraemias.2 Standard SPC also frequently fails to summarise these complex data correctly.
With binary SSI data an approximate expected rate is frequently available so cumulative observed minus expected and CUSUM analysis are appropriate.2 However, the changing observed rate is not seen unless the numbers of procedures is large enough for them to be grouped by months or quarters. This is often infrequent. Even when such aggregation is possible difficulties arise as the number of procedures in each month may differ markedly. This problem can be dealt with, at least approximately, by employing a generalised additive model (GAM) analysis to the binary data that predicts the observed AE rate at various places in the time series.
Count and rate data such as bacteraemias or new isolates of an antibiotic-resistant organism will usually not have an expected rate available. These data are often grouped by months and a Shewhat chart used for their display. This chart requires a stable centre-line about which reliable control limits can be drawn. Often the mean value is used as the expected rate even though it may be representative of few or none of the monthly data values. This makes the control limits meaningless. A probable way round this is to employ confidence limits for the monthly rates. Viewed as a likelihood supported range this enables the extent of each of the monthly counts or rates to be assessed. If a GAM analysis is added to this the predicted rate and its confidence limits can also be obtained throughout the time series.2
This approach is more in keeping with the complexity of the processes responsible for the AE than is standard SPC that was not designed to deal with complex systems.
As an aside, it is worth noting that some swamps may be valuable ecosystems. This popular analogy is thus a poor one. Like root-cause analysis it belongs to the area of simple/complicated systems, not complex ones.
1. Morton, A., Whitby, M., Tierney, N., Sibanda, N. and Mengersen, K. 2016. Statistical Methods for Hospital Monitoring. Wiley StatsRef: Statistics Reference Online. 1–8.
2. Morton A, Mengersen K, Whitby M. and Playford G. Statistical Methods for Hospital Monitoring with R. Chichester John Wiley and Sons 2013.
I read this paper1 first published on 5 March 2018 in your journal with great interest.
The great pace of health information technology (health IT) advancement in recent decades held promise in improving patient safety and quality of care, but unfortunately there has since been inadvertent consequences and carry-over effects of technology-related safety concerns in its use and implementation.2 This paper has further fuelled the boon or bane debate of health IT.3
Ironically, the implementation of a national, multifaceted, quality improvement (QI) programme of ‘de-implementing’ electronic health record (EHR) notifications to primary care physicians (PCPs) has shown some benefit.1 This has, in some way, proven that being too reliant on technology in healthcare may yet rear its ugly head.
The paper has shown that high volume of EHR notifications can overwhelm PCPs;1 the proposed measure of breaking these down into “low-value” and “high-value”, and enforcing certain mandatory ones, may merely be an intermediate stopgap technique. Determining which is which, by its nature, is difficult to do and standardise.
Further, implementing a nationwide programme such as this poses certain challenges that the authors have not considered – will there be a difference between urban and rural healthcare facilities in its implementation, given resource limitations?4 Will technology have improved or changed by the time this programme is fully implemented nationwide?...
Show MoreTo the editor,
Sexton et al describe an observed association between leadership WalkRounds (WR) with feedback and improved levels of safety culture within healthcare settings.1 This work builds on previous data from this group evaluating WR in building a safety culture.2 These encouraging findings spur the need for understanding the robustness of evidence that the WR concept is built on in order to evaluate if continuation and expansion of the WR concept should be promulgated.
The research data that WR are based on are largely observational sets or pre and post studies without control groups or objective outcome measures.3 This is fertile ground for bias and confounding that will undermine the probity of the findings. Sources of bias relate to institutional incentives around WR programs succeeding and the want held by individuals to be seen to be implementing initiatives that improve quality. In the present study described by Sexton et al, the cross sectional observational nature of the data collection is exposed to confounding with clinicians involved in a WR program potentially working in an environment with a superior safety culture regardless of the presence of regular WR (with or without feedback).1 Self-selection bias will also be at play as it is with any voluntary survey method as will be recall bias with culturally high achieving environments and poorly functioning settings over and under estimating their performance respectively, and perhaps ascribi...
Show MoreThe population with intellectual disabilities have multiple morbidities and greater health needs compared with the general population. This population experiences health and healthcare inequities and inequalities. To reduce the health inequality gap people with intellectual disabilities should be involved as partners in their healthcare. This will require access to relevant information and the development of tools that support collaboration, such as tailored patient decision aids (PDA) (1).
Show MoreThe population with intellectual disabilities is rarely considered or involved (2)at the guideline development stage. The consequent failure of clinical guidelines to adequately address the health needs of people with intellectual disabilities exacerbates already poor access to health and healthcare. An examination of clinical guidelines from seven countries(3) found that most clinical guidelines failed to address people with intellectual disabilities as being at high risk for particular conditions when appropriate.
Guidelines and PDAs developed with the general population in mind may not reflect the complexity and multi-morbidity of individual patients with intellectual disabilities and their ‘real world’ lives. Many people with intellectual disabilities have visual, hearing, mobility, memory and dexterity difficulties. Clinicians and guidelines developers may not be aware of the complexity of the task their ask their patients with intellectual disabilities and their...
Authors do not acknowledge some of the most common criticisms of these studies:
(1) Physician health program (PHP) data may be suspect because PHPs benefit from presenting a rosy picture of their effectiveness.
(2) Self-reports from those being evaluated by PHPs, which have much to lose from responding to surveys in ways that criticize these programs, may not be reliable.
(3) There are considerable reasons to doubt that "programme completion," "return to practice," and "no relapse/recurrence" reflect treatment efficacy. Unwarranted referrals may also result in coerced treatment for physicians who do not have a substance use disorder or problematic performance, making "graduation" not meaningful for the purposes of drawing conclusions about PHP treatment effectiveness.
Other concerns with this research will be addressed in forthcoming publications by the commentator.
Thank you for this article it summarises the situation well but omits to mention compassion fatigue in any detail . This is an important concept in organisations who need to change and recognise individual coping skills and support people to make positive changes in their own lives. Without self compassion we cannot be compassionate towards others. So whatever changes are made to the organisation it will not make any difference if people are not supported to change themselves. See this article - When Caring Stops Staffing Does Not Really Matter - https://www.nursingeconomics.net/necfiles/staffingUnleashed/su_ND10.pdf Or see my blog for more discussion on self compassion - http://drmarjorieghisoni.edublogs.org/
Thank you very much for your letter. We agree that the Schmidtke et al paper is highly relevant. In our discussion we note that 'recent research has emphasised the importance of meaningful representation and interpretation of data by boards', citing the accompanying editorial by Mountford and Wakefield which provides an overview both of the Schmidtke et al paper and another paper from the same issue by Anhøj et al on 'Red Amber Green' stoplight reports.
Thanks to the authors for this insight. I wondered if they had seen this content http://qualitysafety.bmj.com/content/26/1/61 from Schmidtke et al. which deals with how boards are presented with data, including the consideration of chance (common cause variation). The material seems highly compatible.
To the Editor,
Badawy et al describe, using statistical analysis, potential inaccuracy in the recording of respiratory rates (RR) in a large cohort of inpatients across a range of inpatient settings and add to the body of data suggesting widespread inaccuracy in the measurement of RR.1 The accurate recording of RR is an important safety and quality issue and the data provided by Badawy et al further underlines the challenges with measurement of this parameter in the inpatient setting.2 Having elegantly demonstrated the problem, the extension of this finding is a need to explore what methods can be potentially employed to improve the accuracy and recording of RR measurement.
Several potential validated solutions may be adduced to address the deficiency in accurate RR measurement and recording. First, consideration could be given to introduction of a system of audit whereby healthcare workers are observed recording RR measurements during their routine practice. Despite a likely Hawthorne effect, the results of this can be collated then non-punitively and anonymously presented to organizational governance structures and health care workers. This concept has been successfully applied into staff hand hygiene quality improvement implementation with this approach having been shown to improve staff performance in this domain with an attendant systematic reduction in adverse event rates.3
Second, the provision of technological solutions, such as a touch pad ba...
Show MoreWith great interest we read the article of Flott et. al. (1), describing the challenges of using patient-reported feedback. We recognize the challenges described and performed a bachelorproject in the intensive care unit (ICU) in the University Medical Center Groningen (UMCG). We think the results from our project provide a potential promising practical solution to make feedback more useful.
Show MoreIn 2013 the UMCG participated in an independent multi-center study conducted among relatives of ICU patients (2). In the open questions of the questionnaire more dissatisfaction than expected was found, which fueled the quest for an alternative, simple and continuous feedback system. In this study we compared the quality and amount of feedback gathered by an oral survey during the first two weeks and an app during the consecutive two weeks.
Between February 20th and March 18th 2017, patients above sixteen years old, listed for discharge from the ICU that day and their relatives were approached to participate in this study. The oral survey consisted of two simple questions: “How satisfied are you with your stay in the ICU? (grade 1-10)” and ”Do you have specific suggestions of improvement for the ICU?”. The RateIt app (Rate It Limited®, Hong Kong) was used consisting of the same two questions as in the oral survey.
A total of 208 responses (133 patients and 75 relatives) were included. The median satisfaction score was 8. Despite this high score many suggestions for...
As you point out Root Cause Analysis will often fail with hospital adverse event (AE) data because it was not designed to deal with data arising in a complex system.1 The same can be said for Pareto analysis. Statistical process control (SPC) methods are often used to summarise AE data, particularly hospital infection data such as surgical site infections (SSIs) and bacteraemias.2 Standard SPC also frequently fails to summarise these complex data correctly.
Show MoreWith binary SSI data an approximate expected rate is frequently available so cumulative observed minus expected and CUSUM analysis are appropriate.2 However, the changing observed rate is not seen unless the numbers of procedures is large enough for them to be grouped by months or quarters. This is often infrequent. Even when such aggregation is possible difficulties arise as the number of procedures in each month may differ markedly. This problem can be dealt with, at least approximately, by employing a generalised additive model (GAM) analysis to the binary data that predicts the observed AE rate at various places in the time series.
Count and rate data such as bacteraemias or new isolates of an antibiotic-resistant organism will usually not have an expected rate available. These data are often grouped by months and a Shewhat chart used for their display. This chart requires a stable centre-line about which reliable control limits can be drawn. Often the mean value is used as the expected rate even though...
Pages