Sexton et al describe an observed association between leadership WalkRounds (WR) with feedback and improved levels of safety culture within healthcare settings.1 This work builds on previous data from this group evaluating WR in building a safety culture.2 These encouraging findings spur the need for understanding the robustness of evidence that the WR concept is built on in order to evaluate if continuation and expansion of the WR concept should be promulgated.
The research data that WR are based on are largely observational sets or pre and post studies without control groups or objective outcome measures.3 This is fertile ground for bias and confounding that will undermine the probity of the findings. Sources of bias relate to institutional incentives around WR programs succeeding and the want held by individuals to be seen to be implementing initiatives that improve quality. In the present study described by Sexton et al, the cross sectional observational nature of the data collection is exposed to confounding with clinicians involved in a WR program potentially working in an environment with a superior safety culture regardless of the presence of regular WR (with or without feedback).1 Self-selection bias will also be at play as it is with any voluntary survey method as will be recall bias with culturally high achieving environments and poorly functioning settings over and under estimating their performance respectively, and perhaps ascribi...
Sexton et al describe an observed association between leadership WalkRounds (WR) with feedback and improved levels of safety culture within healthcare settings.1 This work builds on previous data from this group evaluating WR in building a safety culture.2 These encouraging findings spur the need for understanding the robustness of evidence that the WR concept is built on in order to evaluate if continuation and expansion of the WR concept should be promulgated.
The research data that WR are based on are largely observational sets or pre and post studies without control groups or objective outcome measures.3 This is fertile ground for bias and confounding that will undermine the probity of the findings. Sources of bias relate to institutional incentives around WR programs succeeding and the want held by individuals to be seen to be implementing initiatives that improve quality. In the present study described by Sexton et al, the cross sectional observational nature of the data collection is exposed to confounding with clinicians involved in a WR program potentially working in an environment with a superior safety culture regardless of the presence of regular WR (with or without feedback).1 Self-selection bias will also be at play as it is with any voluntary survey method as will be recall bias with culturally high achieving environments and poorly functioning settings over and under estimating their performance respectively, and perhaps ascribing this to the presence or absence of WR (with or without feedback). The limited data in this field using a pure WR intervention (outside a bundled intervention) with control groups have reported mixed findings and possibly a detrimental effect of WR in some settings.3,4
Another potential concern may be that the outcome data point most assessed and focused on in the WR area is safety culture or other relatively subjective outcomes that are used as a surrogate for clinical outcomes relying on linked evidence outside of the WR studies.5 Data linking an intervention to improve safety culture with a corresponding recorded enhancement in clinical outcome measures appear to be lacking. As is the case in much of the quality and safety space, studies that employ an empirical randomised controlled trial (RCT) approach with objective clinical endpoints are absent from the literature.
Overall, it might be argued that within the quality and safety field RCTs are not only needed but are perhaps mandatory if the area is to move forward meaningfully. It may seem to some that explanations positioning the area as somehow different or unique when compared with other health spheres is likely to be met with growing cynicism and perhaps the time to turn the corner has arrived.
References:
1. Sexton JB, Adair KC, Leonard MW, Frankel TC, Proulx J, Watson SR, Magnus B, Bogan B, Jamal M, Schwendimann R, Frankel AS. Providing feedback following Leadership WalkRounds is associated with better patient safety culture, higher employee engagement and lower burnout. BMJ Quality & Safety. 2018 Apr;27(4)261-270.
2. Frankel A, Graydon-Baker E, Neppl C, Simmonds T, Gustafson M, Gandhi TK. Patient safety leadership WalkRounds™. Joint Commission Journal on Quality and Patient Safety. 2003 Jan;29(1):16-26.
3. Singer SJ, Tucker AL. The evolving literature on safety WalkRounds: emerging themes and practical messages. BMJ Quality & Safety. 2014 Oct;23(10):789.
4. Singer SJ, Rivard PE, Hayes JE, Shokeen P, Gaba D, Rosen A. Improving patient care through leadership engagement with frontline staff: A Department of Veterans Affairs case study. Joint Commission Journal on Quality and Patient Safety. 2013 Aug;39(8):349-60.
5. DiCuccio MH. The Relationship Between Patient Safety Culture and Patient Outcomes: A Systematic Review. Journal of Patient Safety. 2015 Sep;11(3):135-42.
The population with intellectual disabilities have multiple morbidities and greater health needs compared with the general population. This population experiences health and healthcare inequities and inequalities. To reduce the health inequality gap people with intellectual disabilities should be involved as partners in their healthcare. This will require access to relevant information and the development of tools that support collaboration, such as tailored patient decision aids (PDA) (1).
The population with intellectual disabilities is rarely considered or involved (2)at the guideline development stage. The consequent failure of clinical guidelines to adequately address the health needs of people with intellectual disabilities exacerbates already poor access to health and healthcare. An examination of clinical guidelines from seven countries(3) found that most clinical guidelines failed to address people with intellectual disabilities as being at high risk for particular conditions when appropriate.
Guidelines and PDAs developed with the general population in mind may not reflect the complexity and multi-morbidity of individual patients with intellectual disabilities and their ‘real world’ lives. Many people with intellectual disabilities have visual, hearing, mobility, memory and dexterity difficulties. Clinicians and guidelines developers may not be aware of the complexity of the task their ask their patients with intellectual disabilities and their...
The population with intellectual disabilities have multiple morbidities and greater health needs compared with the general population. This population experiences health and healthcare inequities and inequalities. To reduce the health inequality gap people with intellectual disabilities should be involved as partners in their healthcare. This will require access to relevant information and the development of tools that support collaboration, such as tailored patient decision aids (PDA) (1).
The population with intellectual disabilities is rarely considered or involved (2)at the guideline development stage. The consequent failure of clinical guidelines to adequately address the health needs of people with intellectual disabilities exacerbates already poor access to health and healthcare. An examination of clinical guidelines from seven countries(3) found that most clinical guidelines failed to address people with intellectual disabilities as being at high risk for particular conditions when appropriate.
Guidelines and PDAs developed with the general population in mind may not reflect the complexity and multi-morbidity of individual patients with intellectual disabilities and their ‘real world’ lives. Many people with intellectual disabilities have visual, hearing, mobility, memory and dexterity difficulties. Clinicians and guidelines developers may not be aware of the complexity of the task their ask their patients with intellectual disabilities and their carers to undertake e.g. blood glucose monitoring, sliding scale insulin administration, inhaler use, administration of epilepsy rescue medication etc..
Significant gaps in the reporting of evaluations of PDAs have been identified (4). The SUNDAE Checklist (Standards for UNiversal reporting of patient Decision Aid Evaluations) should facilitate evaluation of any PDA, in ‘hard to reach’ population groups, such as the population with intellectual disabilities.
Before applying a guideline or using PDAs with a person with an intellectual disability the following questions could be asked
• Does this person with an intellectual disability understand the connection between the illness and some consequent intervention e.g. diet/exercise, prescribed medication, blood testing?
• What level of health literacy has the person with intellectual disability and/or their carer?
• Do the outcomes measured matter to this person with intellectual disability?
• Is this person with intellectual disability very different from the people studied?
• Are the treatments practical in the ‘real world’ living environment of this person with intellectual disability?
• Do treatment comparisons reflect the circumstances of the person with intellectual disability?
• How certain can the evidence be applied to a person with intellectual disability?
• Do the advantages outweigh the disadvantages for this person with intellectual disability?
1. BMJ 2017; editorial Shehan, Strydom, Hassiotis 358 doi: https://doi.org/10.1136/bmj.j3896
2. Mizen LAM, Maclie ML, Cooper S-A, Melville CA. Clinical guidelines contribute to the health inequities experienced by individuals with intellectual disabilities. Implementation Science. 2012;7(42):1–9.
3. Mizen LAM, Maclie ML, Cooper S-A, Melville CA. Clinical guidelines contribute to the health inequities experienced by individuals with intellectual disabilities. Implementation Science. 2012;7(42):1–9
4. Sepucha KR, Abhyankar P, Hoffman AS, et al. Standards for UNiversal reporting of patient Decision Aid Evaluation studies: the development of SUNDAE Checklist. BMJ Qual Saf Published Online First: 21 December 2017. doi: 10.1136/bmjqs-2017-006986
Authors do not acknowledge some of the most common criticisms of these studies:
(1) Physician health program (PHP) data may be suspect because PHPs benefit from presenting a rosy picture of their effectiveness.
(2) Self-reports from those being evaluated by PHPs, which have much to lose from responding to surveys in ways that criticize these programs, may not be reliable.
(3) There are considerable reasons to doubt that "programme completion," "return to practice," and "no relapse/recurrence" reflect treatment efficacy. Unwarranted referrals may also result in coerced treatment for physicians who do not have a substance use disorder or problematic performance, making "graduation" not meaningful for the purposes of drawing conclusions about PHP treatment effectiveness.
Other concerns with this research will be addressed in forthcoming publications by the commentator.
Thank you for this article it summarises the situation well but omits to mention compassion fatigue in any detail . This is an important concept in organisations who need to change and recognise individual coping skills and support people to make positive changes in their own lives. Without self compassion we cannot be compassionate towards others. So whatever changes are made to the organisation it will not make any difference if people are not supported to change themselves. See this article - When Caring Stops Staffing Does Not Really Matter - https://www.nursingeconomics.net/necfiles/staffingUnleashed/su_ND10.pdf Or see my blog for more discussion on self compassion - http://drmarjorieghisoni.edublogs.org/
Thank you very much for your letter. We agree that the Schmidtke et al paper is highly relevant. In our discussion we note that 'recent research has emphasised the importance of meaningful representation and interpretation of data by boards', citing the accompanying editorial by Mountford and Wakefield which provides an overview both of the Schmidtke et al paper and another paper from the same issue by Anhøj et al on 'Red Amber Green' stoplight reports.
Thanks to the authors for this insight. I wondered if they had seen this content http://qualitysafety.bmj.com/content/26/1/61 from Schmidtke et al. which deals with how boards are presented with data, including the consideration of chance (common cause variation). The material seems highly compatible.
Badawy et al describe, using statistical analysis, potential inaccuracy in the recording of respiratory rates (RR) in a large cohort of inpatients across a range of inpatient settings and add to the body of data suggesting widespread inaccuracy in the measurement of RR.1 The accurate recording of RR is an important safety and quality issue and the data provided by Badawy et al further underlines the challenges with measurement of this parameter in the inpatient setting.2 Having elegantly demonstrated the problem, the extension of this finding is a need to explore what methods can be potentially employed to improve the accuracy and recording of RR measurement.
Several potential validated solutions may be adduced to address the deficiency in accurate RR measurement and recording. First, consideration could be given to introduction of a system of audit whereby healthcare workers are observed recording RR measurements during their routine practice. Despite a likely Hawthorne effect, the results of this can be collated then non-punitively and anonymously presented to organizational governance structures and health care workers. This concept has been successfully applied into staff hand hygiene quality improvement implementation with this approach having been shown to improve staff performance in this domain with an attendant systematic reduction in adverse event rates.3
Second, the provision of technological solutions, such as a touch pad ba...
Badawy et al describe, using statistical analysis, potential inaccuracy in the recording of respiratory rates (RR) in a large cohort of inpatients across a range of inpatient settings and add to the body of data suggesting widespread inaccuracy in the measurement of RR.1 The accurate recording of RR is an important safety and quality issue and the data provided by Badawy et al further underlines the challenges with measurement of this parameter in the inpatient setting.2 Having elegantly demonstrated the problem, the extension of this finding is a need to explore what methods can be potentially employed to improve the accuracy and recording of RR measurement.
Several potential validated solutions may be adduced to address the deficiency in accurate RR measurement and recording. First, consideration could be given to introduction of a system of audit whereby healthcare workers are observed recording RR measurements during their routine practice. Despite a likely Hawthorne effect, the results of this can be collated then non-punitively and anonymously presented to organizational governance structures and health care workers. This concept has been successfully applied into staff hand hygiene quality improvement implementation with this approach having been shown to improve staff performance in this domain with an attendant systematic reduction in adverse event rates.3
Second, the provision of technological solutions, such as a touch pad based application to record respiratory rates using finger tapping may also have a role in improving accuracy and has been demonstrated in paediatric settings to be potentially effective.4 This technology employs an algorithm whereby the interval between taps (each tap corresponding to a breath observed) is used to calculate a RR. This provides a real-time self-refining measurement of respiratory rate, with more taps generating greater accuracy. To further improve accuracy, and data utility, results could be directly fed into a real-time electronic medical record system.
Finally, complementing the introduction of data collection on performance (with audit of that data) and the potential integration of technological assistive structures would also be the promulgation of education measures. Education measures could focus staff on the data around the historical inaccuracy of RR recording, the assistive technology initiatives being put into place and the importance of accurate measurement for safety and quality. In addition, ongoing feedback to healthcare staff of observed accuracy, as done for hand hygiene measures, would also be important. Multifaceted education of this nature has been shown to be effective for other quality change initiatives.5
In conclusion, a combination of integrated observation and audit, technological implementation and integration and staff education could be used to address the important challenges in measurement of respiratory rate identified by Badawy et al.
References:
1. Badawy J, Nguyen OK, Clark C, et al. Is everyone really breathing 20 times a minute? Assessing epidemiology and variation in recorded respiratory rate in hospitalised adults. BMJ Qual Saf 2017:bmjqs-2017.
2. Fieselmann JF, Hendryx MS, Helms CM, et al. Respiratory rate predicts cardiopulmonary arrest for internal medicine inpatients. J Gen Intern Med 1993;8:354–60.
3. Pittet D, Hugonnet S, Harbarth S et al. Effectiveness of a hospital-wide programme to improve compliance with hand hygiene. The Lancet 2000;356(9238):1307-12.
4. Karlen W, Gan H, Chiu M, et al. Improving the accuracy and efficiency of respiratory rate measurements in children using mobile devices. PLoS One 2014;9(6):e99266.
5. Naikoba S, Hayward A. The effectiveness of interventions aimed at increasing handwashing in healthcare workers-a systematic review. Journal of Hospital Infection 2001;47(3):173-80.
With great interest we read the article of Flott et. al. (1), describing the challenges of using patient-reported feedback. We recognize the challenges described and performed a bachelorproject in the intensive care unit (ICU) in the University Medical Center Groningen (UMCG). We think the results from our project provide a potential promising practical solution to make feedback more useful.
In 2013 the UMCG participated in an independent multi-center study conducted among relatives of ICU patients (2). In the open questions of the questionnaire more dissatisfaction than expected was found, which fueled the quest for an alternative, simple and continuous feedback system. In this study we compared the quality and amount of feedback gathered by an oral survey during the first two weeks and an app during the consecutive two weeks.
Between February 20th and March 18th 2017, patients above sixteen years old, listed for discharge from the ICU that day and their relatives were approached to participate in this study. The oral survey consisted of two simple questions: “How satisfied are you with your stay in the ICU? (grade 1-10)” and ”Do you have specific suggestions of improvement for the ICU?”. The RateIt app (Rate It Limited®, Hong Kong) was used consisting of the same two questions as in the oral survey.
A total of 208 responses (133 patients and 75 relatives) were included. The median satisfaction score was 8. Despite this high score many suggestions for...
With great interest we read the article of Flott et. al. (1), describing the challenges of using patient-reported feedback. We recognize the challenges described and performed a bachelorproject in the intensive care unit (ICU) in the University Medical Center Groningen (UMCG). We think the results from our project provide a potential promising practical solution to make feedback more useful.
In 2013 the UMCG participated in an independent multi-center study conducted among relatives of ICU patients (2). In the open questions of the questionnaire more dissatisfaction than expected was found, which fueled the quest for an alternative, simple and continuous feedback system. In this study we compared the quality and amount of feedback gathered by an oral survey during the first two weeks and an app during the consecutive two weeks.
Between February 20th and March 18th 2017, patients above sixteen years old, listed for discharge from the ICU that day and their relatives were approached to participate in this study. The oral survey consisted of two simple questions: “How satisfied are you with your stay in the ICU? (grade 1-10)” and ”Do you have specific suggestions of improvement for the ICU?”. The RateIt app (Rate It Limited®, Hong Kong) was used consisting of the same two questions as in the oral survey.
A total of 208 responses (133 patients and 75 relatives) were included. The median satisfaction score was 8. Despite this high score many suggestions for improvement (n=95 suggestions given by 68 respondents) were given. The oral survey provided more often suggestions for improvement compared with the app (50 vs. 18 respondents). Suggestions for improvement were more frequently made by relatives compared with patients (57 suggestions given by 37 relatives vs. 38 suggestions given by 31 patients). All improvement suggestions were classified to one of six categories: ‘Surroundings’ 48/95 (51%), ‘Information, communication and education’ 23/95 (24%), ‘Patient care’ 15/95 (16%), ‘Attitude, handling and relation caregiver with patient/relatives 7/95 (7%), ‘Emotional support’ 1/95 (1%) and ‘Care for relatives’ 1/95 (1%).
This simple study showed that an oral survey results in more suggestions for improvement than an app. The lack of complexity of the survey resulted in very specific, useful and practical suggestions for improvement, which were easily transformed into clear recommendations, such as: “respect sufficient rest of our patients” or “don’t forget to provide food to the patients who are able to eat”. The survey can easily be repeated in the course of time. These results may give a new perspective on how to conduct feedback studies.
The key suggestions for improvement found in this study were presented to the department in the form of a coat rack, which was an improvement option frequently mentioned by relatives (A coat rack was missing in one of our family rooms). This coat rack will be hung in central places in our department. On this coatrack recommendations based on the most important improvement suggestions will be hung. We think this is one example of a simple, but practical solution to make feedback more useful: every month the recommendations will be replaced by new ones, reminding all caregivers in our department of the feedback given by our patients and their relatives and thereby striving to improve our care.
We are well aware of the fact that the surveys used in the studies described in the article of Flott et al1 are much larger and more complex than the one we used in our study. We just wanted to show that a learning point could be: don’t overcomplicate.
References
1. Flott KM, Graham C, Darzi A, Mayer E. Can we use patient-reported feedback to drive change? The challenges of using patient-reported feedback and how they might be addressed. BMJ Qual Saf 2017;26:502-507.
2. Jensen HI, Gerritsen RT, Koopmans M, Zijlstra JG, Randall Curtis J, Ording H. Families’ experiences of intensive care unit quality of care: Development and validation of a European questionnaire (euroQ2). Journal of Critical Care 2015;30(5):884-890.
As you point out Root Cause Analysis will often fail with hospital adverse event (AE) data because it was not designed to deal with data arising in a complex system.1 The same can be said for Pareto analysis. Statistical process control (SPC) methods are often used to summarise AE data, particularly hospital infection data such as surgical site infections (SSIs) and bacteraemias.2 Standard SPC also frequently fails to summarise these complex data correctly.
With binary SSI data an approximate expected rate is frequently available so cumulative observed minus expected and CUSUM analysis are appropriate.2 However, the changing observed rate is not seen unless the numbers of procedures is large enough for them to be grouped by months or quarters. This is often infrequent. Even when such aggregation is possible difficulties arise as the number of procedures in each month may differ markedly. This problem can be dealt with, at least approximately, by employing a generalised additive model (GAM) analysis to the binary data that predicts the observed AE rate at various places in the time series.
Count and rate data such as bacteraemias or new isolates of an antibiotic-resistant organism will usually not have an expected rate available. These data are often grouped by months and a Shewhat chart used for their display. This chart requires a stable centre-line about which reliable control limits can be drawn. Often the mean value is used as the expected rate even though...
As you point out Root Cause Analysis will often fail with hospital adverse event (AE) data because it was not designed to deal with data arising in a complex system.1 The same can be said for Pareto analysis. Statistical process control (SPC) methods are often used to summarise AE data, particularly hospital infection data such as surgical site infections (SSIs) and bacteraemias.2 Standard SPC also frequently fails to summarise these complex data correctly.
With binary SSI data an approximate expected rate is frequently available so cumulative observed minus expected and CUSUM analysis are appropriate.2 However, the changing observed rate is not seen unless the numbers of procedures is large enough for them to be grouped by months or quarters. This is often infrequent. Even when such aggregation is possible difficulties arise as the number of procedures in each month may differ markedly. This problem can be dealt with, at least approximately, by employing a generalised additive model (GAM) analysis to the binary data that predicts the observed AE rate at various places in the time series.
Count and rate data such as bacteraemias or new isolates of an antibiotic-resistant organism will usually not have an expected rate available. These data are often grouped by months and a Shewhat chart used for their display. This chart requires a stable centre-line about which reliable control limits can be drawn. Often the mean value is used as the expected rate even though it may be representative of few or none of the monthly data values. This makes the control limits meaningless. A probable way round this is to employ confidence limits for the monthly rates. Viewed as a likelihood supported range this enables the extent of each of the monthly counts or rates to be assessed. If a GAM analysis is added to this the predicted rate and its confidence limits can also be obtained throughout the time series.2
This approach is more in keeping with the complexity of the processes responsible for the AE than is standard SPC that was not designed to deal with complex systems.
As an aside, it is worth noting that some swamps may be valuable ecosystems. This popular analogy is thus a poor one. Like root-cause analysis it belongs to the area of simple/complicated systems, not complex ones.
1. Morton, A., Whitby, M., Tierney, N., Sibanda, N. and Mengersen, K. 2016. Statistical Methods for Hospital Monitoring. Wiley StatsRef: Statistics Reference Online. 1–8.
2. Morton A, Mengersen K, Whitby M. and Playford G. Statistical Methods for Hospital Monitoring with R. Chichester John Wiley and Sons 2013.
Vindrola-Padros and colleagues provide a helpful examination of co-production of quality improvement knowledge by university-based researchers in cooperation with members of service organizations. Another important type of embedded researcher consists of “fully embedded,” researchers, who are academically trained but employed by large care delivery systems. These individuals typically work in research units in the delivery systems. Their work is funded both by the systems themselves and by external, private and public organizations, such as the Agency for Healthcare Research and Quality (AHRQ). These fully embedded researchers contribute actively to national professional forums and journals and sometimes collaborate with embedded researchers in other systems.
AHRQ leverages relationships with fully embedded researchers because of their deep and nuanced knowledge of internal system data and operations. Health systems-based researchers’ ready access to care sites within which to test new approaches, and to data sources that permit rapid analysis of results of those tests, are of great value to AHRQ as we seek to find solutions to real-world problems in areas of national importance. AHRQ-supported work of this kind demonstrates the value of health delivery organizations becoming “learning health systems”(1) – using their own internal data and resources to drive quality improvement and sharing their findings with other organizations.
Vindrola-Padros and colleagues provide a helpful examination of co-production of quality improvement knowledge by university-based researchers in cooperation with members of service organizations. Another important type of embedded researcher consists of “fully embedded,” researchers, who are academically trained but employed by large care delivery systems. These individuals typically work in research units in the delivery systems. Their work is funded both by the systems themselves and by external, private and public organizations, such as the Agency for Healthcare Research and Quality (AHRQ). These fully embedded researchers contribute actively to national professional forums and journals and sometimes collaborate with embedded researchers in other systems.
AHRQ leverages relationships with fully embedded researchers because of their deep and nuanced knowledge of internal system data and operations. Health systems-based researchers’ ready access to care sites within which to test new approaches, and to data sources that permit rapid analysis of results of those tests, are of great value to AHRQ as we seek to find solutions to real-world problems in areas of national importance. AHRQ-supported work of this kind demonstrates the value of health delivery organizations becoming “learning health systems”(1) – using their own internal data and resources to drive quality improvement and sharing their findings with other organizations.
AHRQ’s collaboration with researchers in the Palo Alto Medical Foundation (PAMF) Research Institute provides a powerful example of how partnership between fully embedded researchers and external funding agencies contributes to health system learning. AHRQ partnered with Kaiser Permanente and PAMF researchers to study implementation of a Lean-based redesign to improve care delivery efficiency in PAMF’s primary care clinics. (2) Applying Lean analysis techniques, PAMF discovered inefficiencies in a pilot primary care clinic and redesigned work roles and work flow to enhance coordination among physicians and to better support them. Key changes included:
• New roles for medical assistants as a “flow managers,” facilitating physician’s work and performing administrative tasks like handling email that previously burdened physicians
• New workflows – including daily huddles for scheduling; agenda setting during patient visits
• Co-location of physician-medical assistant teams in a shared workspace.
PAMF then tested these new roles and processes in three additional clinics, assessed the improvements’ effects, and rolled the changes out to 13 additional clinics.
PAMF researchers interviewed staff to uncover factors influencing successful implementation of these changes and system requirements for successful redesign of care. (3-4) To assess changes in efficiency, they analyzed rich and timely internal data sources such as:
• Physician efficiency metrics derived from PAMF’s time-stamped EHR data and other operational sources
• PAMF’s routine patient and personnel surveys
• Standardized quality metrics that PAMF reports.
Their research showed that PAMF’s primary care redesigns boosted efficiency without sacrificing quality and satisfaction. (5) AHRQ and PAMF disseminated these valuable findings widely through practice –oriented briefs, conference presentations, and webinars, as well as in peer-reviewed papers.
PAMF’s fully embedded researchers promoted internal learning by tracking progress and outcomes of the Lean improvement efforts and providing feedback to their system’s leaders and staff. AHRQ and the PAMF researchers promoted system-wide learning about Lean-based primary care redesign by broadly disseminating the study’s findings and implementation lessons.
3. Hung D, Gray C, Martinez M, Schmittdiel J, Harrison, MI. Acceptance of Lean redesigns in primary care: a contextual analysis. Health Care Manage Rev 2017; 42:203-212.
4. Gray C, Harrison MI, Hung D. Medical assistants as flow managers in primary care: challenges and recommendations. J. Healthc Manag 2016; 61:181-191.
5. Hung D, Harrison MI, Martinez M, Luft H. Scaling Lean in primary care: impacts on system performance. Am J Manag Care 2017; 23(3):161-168.
To the editor,
Sexton et al describe an observed association between leadership WalkRounds (WR) with feedback and improved levels of safety culture within healthcare settings.1 This work builds on previous data from this group evaluating WR in building a safety culture.2 These encouraging findings spur the need for understanding the robustness of evidence that the WR concept is built on in order to evaluate if continuation and expansion of the WR concept should be promulgated.
The research data that WR are based on are largely observational sets or pre and post studies without control groups or objective outcome measures.3 This is fertile ground for bias and confounding that will undermine the probity of the findings. Sources of bias relate to institutional incentives around WR programs succeeding and the want held by individuals to be seen to be implementing initiatives that improve quality. In the present study described by Sexton et al, the cross sectional observational nature of the data collection is exposed to confounding with clinicians involved in a WR program potentially working in an environment with a superior safety culture regardless of the presence of regular WR (with or without feedback).1 Self-selection bias will also be at play as it is with any voluntary survey method as will be recall bias with culturally high achieving environments and poorly functioning settings over and under estimating their performance respectively, and perhaps ascribi...
Show MoreThe population with intellectual disabilities have multiple morbidities and greater health needs compared with the general population. This population experiences health and healthcare inequities and inequalities. To reduce the health inequality gap people with intellectual disabilities should be involved as partners in their healthcare. This will require access to relevant information and the development of tools that support collaboration, such as tailored patient decision aids (PDA) (1).
Show MoreThe population with intellectual disabilities is rarely considered or involved (2)at the guideline development stage. The consequent failure of clinical guidelines to adequately address the health needs of people with intellectual disabilities exacerbates already poor access to health and healthcare. An examination of clinical guidelines from seven countries(3) found that most clinical guidelines failed to address people with intellectual disabilities as being at high risk for particular conditions when appropriate.
Guidelines and PDAs developed with the general population in mind may not reflect the complexity and multi-morbidity of individual patients with intellectual disabilities and their ‘real world’ lives. Many people with intellectual disabilities have visual, hearing, mobility, memory and dexterity difficulties. Clinicians and guidelines developers may not be aware of the complexity of the task their ask their patients with intellectual disabilities and their...
Authors do not acknowledge some of the most common criticisms of these studies:
(1) Physician health program (PHP) data may be suspect because PHPs benefit from presenting a rosy picture of their effectiveness.
(2) Self-reports from those being evaluated by PHPs, which have much to lose from responding to surveys in ways that criticize these programs, may not be reliable.
(3) There are considerable reasons to doubt that "programme completion," "return to practice," and "no relapse/recurrence" reflect treatment efficacy. Unwarranted referrals may also result in coerced treatment for physicians who do not have a substance use disorder or problematic performance, making "graduation" not meaningful for the purposes of drawing conclusions about PHP treatment effectiveness.
Other concerns with this research will be addressed in forthcoming publications by the commentator.
Thank you for this article it summarises the situation well but omits to mention compassion fatigue in any detail . This is an important concept in organisations who need to change and recognise individual coping skills and support people to make positive changes in their own lives. Without self compassion we cannot be compassionate towards others. So whatever changes are made to the organisation it will not make any difference if people are not supported to change themselves. See this article - When Caring Stops Staffing Does Not Really Matter - https://www.nursingeconomics.net/necfiles/staffingUnleashed/su_ND10.pdf Or see my blog for more discussion on self compassion - http://drmarjorieghisoni.edublogs.org/
Thank you very much for your letter. We agree that the Schmidtke et al paper is highly relevant. In our discussion we note that 'recent research has emphasised the importance of meaningful representation and interpretation of data by boards', citing the accompanying editorial by Mountford and Wakefield which provides an overview both of the Schmidtke et al paper and another paper from the same issue by Anhøj et al on 'Red Amber Green' stoplight reports.
Thanks to the authors for this insight. I wondered if they had seen this content http://qualitysafety.bmj.com/content/26/1/61 from Schmidtke et al. which deals with how boards are presented with data, including the consideration of chance (common cause variation). The material seems highly compatible.
To the Editor,
Badawy et al describe, using statistical analysis, potential inaccuracy in the recording of respiratory rates (RR) in a large cohort of inpatients across a range of inpatient settings and add to the body of data suggesting widespread inaccuracy in the measurement of RR.1 The accurate recording of RR is an important safety and quality issue and the data provided by Badawy et al further underlines the challenges with measurement of this parameter in the inpatient setting.2 Having elegantly demonstrated the problem, the extension of this finding is a need to explore what methods can be potentially employed to improve the accuracy and recording of RR measurement.
Several potential validated solutions may be adduced to address the deficiency in accurate RR measurement and recording. First, consideration could be given to introduction of a system of audit whereby healthcare workers are observed recording RR measurements during their routine practice. Despite a likely Hawthorne effect, the results of this can be collated then non-punitively and anonymously presented to organizational governance structures and health care workers. This concept has been successfully applied into staff hand hygiene quality improvement implementation with this approach having been shown to improve staff performance in this domain with an attendant systematic reduction in adverse event rates.3
Second, the provision of technological solutions, such as a touch pad ba...
Show MoreWith great interest we read the article of Flott et. al. (1), describing the challenges of using patient-reported feedback. We recognize the challenges described and performed a bachelorproject in the intensive care unit (ICU) in the University Medical Center Groningen (UMCG). We think the results from our project provide a potential promising practical solution to make feedback more useful.
Show MoreIn 2013 the UMCG participated in an independent multi-center study conducted among relatives of ICU patients (2). In the open questions of the questionnaire more dissatisfaction than expected was found, which fueled the quest for an alternative, simple and continuous feedback system. In this study we compared the quality and amount of feedback gathered by an oral survey during the first two weeks and an app during the consecutive two weeks.
Between February 20th and March 18th 2017, patients above sixteen years old, listed for discharge from the ICU that day and their relatives were approached to participate in this study. The oral survey consisted of two simple questions: “How satisfied are you with your stay in the ICU? (grade 1-10)” and ”Do you have specific suggestions of improvement for the ICU?”. The RateIt app (Rate It Limited®, Hong Kong) was used consisting of the same two questions as in the oral survey.
A total of 208 responses (133 patients and 75 relatives) were included. The median satisfaction score was 8. Despite this high score many suggestions for...
As you point out Root Cause Analysis will often fail with hospital adverse event (AE) data because it was not designed to deal with data arising in a complex system.1 The same can be said for Pareto analysis. Statistical process control (SPC) methods are often used to summarise AE data, particularly hospital infection data such as surgical site infections (SSIs) and bacteraemias.2 Standard SPC also frequently fails to summarise these complex data correctly.
Show MoreWith binary SSI data an approximate expected rate is frequently available so cumulative observed minus expected and CUSUM analysis are appropriate.2 However, the changing observed rate is not seen unless the numbers of procedures is large enough for them to be grouped by months or quarters. This is often infrequent. Even when such aggregation is possible difficulties arise as the number of procedures in each month may differ markedly. This problem can be dealt with, at least approximately, by employing a generalised additive model (GAM) analysis to the binary data that predicts the observed AE rate at various places in the time series.
Count and rate data such as bacteraemias or new isolates of an antibiotic-resistant organism will usually not have an expected rate available. These data are often grouped by months and a Shewhat chart used for their display. This chart requires a stable centre-line about which reliable control limits can be drawn. Often the mean value is used as the expected rate even though...
Vindrola-Padros and colleagues provide a helpful examination of co-production of quality improvement knowledge by university-based researchers in cooperation with members of service organizations. Another important type of embedded researcher consists of “fully embedded,” researchers, who are academically trained but employed by large care delivery systems. These individuals typically work in research units in the delivery systems. Their work is funded both by the systems themselves and by external, private and public organizations, such as the Agency for Healthcare Research and Quality (AHRQ). These fully embedded researchers contribute actively to national professional forums and journals and sometimes collaborate with embedded researchers in other systems.
AHRQ leverages relationships with fully embedded researchers because of their deep and nuanced knowledge of internal system data and operations. Health systems-based researchers’ ready access to care sites within which to test new approaches, and to data sources that permit rapid analysis of results of those tests, are of great value to AHRQ as we seek to find solutions to real-world problems in areas of national importance. AHRQ-supported work of this kind demonstrates the value of health delivery organizations becoming “learning health systems”(1) – using their own internal data and resources to drive quality improvement and sharing their findings with other organizations.
AHRQ’s collaboration w...
Show MorePages