We use cookies to improve our service and to tailor our content and advertising to you.More infoYou can manage your cookie settings via your browser at any time. To learn more about how we use cookies, please see our cookies policy.
As you point out Root Cause Analysis will often fail with hospital adverse event (AE) data because it was not designed to deal with data arising in a complex system.1 The same can be said for Pareto analysis. Statistical process control (SPC) methods are often used to summarise AE data, particularly hospital infection data such as surgical site infections (SSIs) and bacteraemias.2 Standard SPC also frequently fails to summarise these complex data correctly.
With binary SSI data an approximate expected rate is frequently available so cumulative observed minus expected and CUSUM analysis are appropriate.2 However, the changing observed rate is not seen unless the numbers of procedures is large enough for them to be grouped by months or quarters. This is often infrequent. Even when such aggregation is possible difficulties arise as the number of procedures in each month may differ markedly. This problem can be dealt with, at least approximately, by employing a generalised additive model (GAM) analysis to the binary data that predicts the observed AE rate at various places in the time series.
Count and rate data such as bacteraemias or new isolates of an antibiotic-resistant organism will usually not have an expected rate available. These data are often grouped by months and a Shewhat chart used for their display. This chart requires a stable centre-line about which reliable control limits can be drawn. Often the mean value is used as the expected rate even though...
As you point out Root Cause Analysis will often fail with hospital adverse event (AE) data because it was not designed to deal with data arising in a complex system.1 The same can be said for Pareto analysis. Statistical process control (SPC) methods are often used to summarise AE data, particularly hospital infection data such as surgical site infections (SSIs) and bacteraemias.2 Standard SPC also frequently fails to summarise these complex data correctly.
With binary SSI data an approximate expected rate is frequently available so cumulative observed minus expected and CUSUM analysis are appropriate.2 However, the changing observed rate is not seen unless the numbers of procedures is large enough for them to be grouped by months or quarters. This is often infrequent. Even when such aggregation is possible difficulties arise as the number of procedures in each month may differ markedly. This problem can be dealt with, at least approximately, by employing a generalised additive model (GAM) analysis to the binary data that predicts the observed AE rate at various places in the time series.
Count and rate data such as bacteraemias or new isolates of an antibiotic-resistant organism will usually not have an expected rate available. These data are often grouped by months and a Shewhat chart used for their display. This chart requires a stable centre-line about which reliable control limits can be drawn. Often the mean value is used as the expected rate even though it may be representative of few or none of the monthly data values. This makes the control limits meaningless. A probable way round this is to employ confidence limits for the monthly rates. Viewed as a likelihood supported range this enables the extent of each of the monthly counts or rates to be assessed. If a GAM analysis is added to this the predicted rate and its confidence limits can also be obtained throughout the time series.2
This approach is more in keeping with the complexity of the processes responsible for the AE than is standard SPC that was not designed to deal with complex systems.
As an aside, it is worth noting that some swamps may be valuable ecosystems. This popular analogy is thus a poor one. Like root-cause analysis it belongs to the area of simple/complicated systems, not complex ones.
1. Morton, A., Whitby, M., Tierney, N., Sibanda, N. and Mengersen, K. 2016. Statistical Methods for Hospital Monitoring. Wiley StatsRef: Statistics Reference Online. 1–8.
2. Morton A, Mengersen K, Whitby M. and Playford G. Statistical Methods for Hospital Monitoring with R. Chichester John Wiley and Sons 2013.
Vindrola-Padros and colleagues provide a helpful examination of co-production of quality improvement knowledge by university-based researchers in cooperation with members of service organizations. Another important type of embedded researcher consists of “fully embedded,” researchers, who are academically trained but employed by large care delivery systems. These individuals typically work in research units in the delivery systems. Their work is funded both by the systems themselves and by external, private and public organizations, such as the Agency for Healthcare Research and Quality (AHRQ). These fully embedded researchers contribute actively to national professional forums and journals and sometimes collaborate with embedded researchers in other systems.
AHRQ leverages relationships with fully embedded researchers because of their deep and nuanced knowledge of internal system data and operations. Health systems-based researchers’ ready access to care sites within which to test new approaches, and to data sources that permit rapid analysis of results of those tests, are of great value to AHRQ as we seek to find solutions to real-world problems in areas of national importance. AHRQ-supported work of this kind demonstrates the value of health delivery organizations becoming “learning health systems”(1) – using their own internal data and resources to drive quality improvement and sharing their findings with other organizations.
Vindrola-Padros and colleagues provide a helpful examination of co-production of quality improvement knowledge by university-based researchers in cooperation with members of service organizations. Another important type of embedded researcher consists of “fully embedded,” researchers, who are academically trained but employed by large care delivery systems. These individuals typically work in research units in the delivery systems. Their work is funded both by the systems themselves and by external, private and public organizations, such as the Agency for Healthcare Research and Quality (AHRQ). These fully embedded researchers contribute actively to national professional forums and journals and sometimes collaborate with embedded researchers in other systems.
AHRQ leverages relationships with fully embedded researchers because of their deep and nuanced knowledge of internal system data and operations. Health systems-based researchers’ ready access to care sites within which to test new approaches, and to data sources that permit rapid analysis of results of those tests, are of great value to AHRQ as we seek to find solutions to real-world problems in areas of national importance. AHRQ-supported work of this kind demonstrates the value of health delivery organizations becoming “learning health systems”(1) – using their own internal data and resources to drive quality improvement and sharing their findings with other organizations.
AHRQ’s collaboration with researchers in the Palo Alto Medical Foundation (PAMF) Research Institute provides a powerful example of how partnership between fully embedded researchers and external funding agencies contributes to health system learning. AHRQ partnered with Kaiser Permanente and PAMF researchers to study implementation of a Lean-based redesign to improve care delivery efficiency in PAMF’s primary care clinics. (2) Applying Lean analysis techniques, PAMF discovered inefficiencies in a pilot primary care clinic and redesigned work roles and work flow to enhance coordination among physicians and to better support them. Key changes included:
• New roles for medical assistants as a “flow managers,” facilitating physician’s work and performing administrative tasks like handling email that previously burdened physicians
• New workflows – including daily huddles for scheduling; agenda setting during patient visits
• Co-location of physician-medical assistant teams in a shared workspace.
PAMF then tested these new roles and processes in three additional clinics, assessed the improvements’ effects, and rolled the changes out to 13 additional clinics.
PAMF researchers interviewed staff to uncover factors influencing successful implementation of these changes and system requirements for successful redesign of care. (3-4) To assess changes in efficiency, they analyzed rich and timely internal data sources such as:
• Physician efficiency metrics derived from PAMF’s time-stamped EHR data and other operational sources
• PAMF’s routine patient and personnel surveys
• Standardized quality metrics that PAMF reports.
Their research showed that PAMF’s primary care redesigns boosted efficiency without sacrificing quality and satisfaction. (5) AHRQ and PAMF disseminated these valuable findings widely through practice –oriented briefs, conference presentations, and webinars, as well as in peer-reviewed papers.
PAMF’s fully embedded researchers promoted internal learning by tracking progress and outcomes of the Lean improvement efforts and providing feedback to their system’s leaders and staff. AHRQ and the PAMF researchers promoted system-wide learning about Lean-based primary care redesign by broadly disseminating the study’s findings and implementation lessons.
3. Hung D, Gray C, Martinez M, Schmittdiel J, Harrison, MI. Acceptance of Lean redesigns in primary care: a contextual analysis. Health Care Manage Rev 2017; 42:203-212.
4. Gray C, Harrison MI, Hung D. Medical assistants as flow managers in primary care: challenges and recommendations. J. Healthc Manag 2016; 61:181-191.
5. Hung D, Harrison MI, Martinez M, Luft H. Scaling Lean in primary care: impacts on system performance. Am J Manag Care 2017; 23(3):161-168.
This study uses rigorous analysis to obtain important insights about the realtime information that our patients are handed at discharge. It is puzzling that the EMRs used were not named. One can infer from a look through the MSU website that they have both Cerner and Epic, but why is that necessary? The heart of quality/safety work is one of transparency balanced by humility, i.e. we shouldn't expect our IT systems to be any more perfect than we are, but they won't improve if we don't have more openness. The lack of scientific foundations and published post-marketing surveillance for our EHRs, especially the ascendant ones, was initially surprising. However, as they achieve complete market dominance, with less overt scientific review and public guidance and commentary, the silence is deafening. Is the BMJQS's failure to simply identify the names (or maybe I missed the citations) an oversight, or part of nondisclosure agreements with the vendors at the MSU institutions or at BMJQS?
In this article, the authors propose that little evidence exists in
healthcare to show that application of Highly Reliable Organization (HRO)
principles has resulted in significant or sustained improvement in
performance. Further, they attribute the problem partially to under-
recognizing the role of habit in the process. While we fully agree that
forming habitual behavior is essential to creating...
In this article, the authors propose that little evidence exists in
healthcare to show that application of Highly Reliable Organization (HRO)
principles has resulted in significant or sustained improvement in
performance. Further, they attribute the problem partially to under-
recognizing the role of habit in the process. While we fully agree that
forming habitual behavior is essential to creating an HRO, we are far less
pessimistic about the state of affairs in the healthcare industry,
particularly in pediatrics.
Vogus and Hillgoss primarily drew their many references to suboptimal
implementation of HRO principles from the adult patient care arena. In
pediatrics, multiple hospital systems have used HRO principles to achieve
significant and sustained improvement in safety performance related to
specific metrics such as reducing Adverse Drug Events(1), Hand Hygiene
compliance(2), Serious Safety Events(3), as well as reducing all forms of
preventable harm across entire systems(4). We are limited by the number
of references we are allowed to cite, however we could list at least
another dozen papers that show the use of high reliability principles,
effectively implemented and linked to sustained and robust improvement in
clinical outcomes. Further, beginning over a decade ago, pediatric
hospitals joined together to form collaboratives (currently over 100
participants), agreed NOT to compete on quality, to share data, and
discover best practices. The collaboratives implement high reliability
principles and measure compliance with clinical practice expectations in
order to improve outcomes on multiple patient safety related measures.
Compliance to various care practice bundles by participating hospitals is
at the core of these efforts, and the results are significant(5).
We applaud the Virginia Mason Production System (VMPS) and the
Comprehensive Unit-based Safety Program (CUSP) and the results they have
achieved. The cultivation of mindfulness and shared habits through the
various mechanisms described are all variations of similar tactics and
strategies we have been using in pediatrics. However, we do not agree with
the statement that these programs "operate independently of any specific
leader". While specific leaders can and do change, we believe that
vigorous and steady leadership from the top is necessary to maintain the
gains. Therefore, we agree the process can be fragile in the face of
leadership change.
The authors did not discuss an element that we believe is essential
to achieve optimal quality and safety outcomes: transparency. Early in the
Ohio Children's Hospitals' collaborative, the declaration by hospital CEOs
to put inter-hospital rivalries and competition aside for the sake of
safety was made. Their stated goal was to develop the kind of trust which
allowed open data and best practice sharing, so that a culture of rapid
information sharing (all teach, all learn) could develop. Real and
sustained progress requires internal and external transparency and we
suggest the Ohio Solutions for Patient Safety Collaborative results
support this belief(5).
To be clear, in pediatrics we are not yet where we want to be
regarding quality and safety. Most systems have declared elimination of
preventable harm as the ultimate goal - recognizing that, in many ways, it
is an un-ending journey, but the only truly acceptable aspirational goal.
Yet pediatrics is making measurable and significant progress using HRO
principles, and we fully embrace the importance of developing mindful
habits as an important part of that process.
References
1. McClead RE, Catt C, Davis JT, Morvay S, Merandi J, Lewe D, Stewart B,
Brilli RJ. An Internal Quality Improvement Collaborative Significantly
Reduces Hospital-wide Medication Error Related Adverse Drug Events. J
Pediatr 2014;165(6):1222-9.
2. Toltziz P, O'Riordan M, Cunningham DJ, Ryckman FC, Bracke TM, Olivea
J, Lyren A. A Statewide Collaborative to Reduce Pediatric Surgical Site
Infections. Pediatrics 2014;134(4):e1174-e1180.
3. Muething SE, Goudie A, Schoettker PJ, Donnelly LF, Goodfriend MA,
Bracke TM, Brady PW, Wheeler DS, Anderson JM, Kotagal UR. Quality
Improvement Initiative to Reduce Serious Safety Events and Improve Patient
Safety Culture. Pediatrics 2012;130:e423-e431.
4. Brilli RJ, McClead RE Jr, Crandall WV, Stoverock L, Berry JC, Wheeler
TA, Davis JT. A comprehensive patient safety program can significantly
reduce preventable harm, associated costs, and hospital mortality. J
Pediatr. 2013 Dec;163(6):1638-45. Epub 2013 Jul 30.
5. Lyren A, Brilli R, Bird M, Lashutka N, Muething S. Ohio Children's
Hospitals' Solutions for Patient Safety: A Framework for Pediatric Patient
Safety Improvement. J Healthc Qual 2015;38(4):213-222.
I read with interest the paper by Gillespie and Reader presenting the
Healthcare Complaints Analysis Tool (HCAT) (1). The authors suggest that
the HCAT could be used "as an alternative metric of success in meeting
standards" and as a way "to benchmark units or regions". However, this
makes the assumption that the volume and strength of complaints received
is an accurate reflection of the standard of care being delivered....
I read with interest the paper by Gillespie and Reader presenting the
Healthcare Complaints Analysis Tool (HCAT) (1). The authors suggest that
the HCAT could be used "as an alternative metric of success in meeting
standards" and as a way "to benchmark units or regions". However, this
makes the assumption that the volume and strength of complaints received
is an accurate reflection of the standard of care being delivered. In
fact, it may be more heavily influenced by the ability and willingness of
patients (or their relatives) to make a complaint. A hospital or unit
could have a poor standard of care but receive few complaints, especially
if it has a high proportion of patients from demographic groups that are
less likely to complain. For example, a recent report from the
Parliamentary and Health Service Ombudsman found far fewer complaints from
the elderly than would be expected based upon their service usage (2).
Patients from certain ethnic minorities and less affluent social grades
have also been identified as groups less likely to complain (3). Moreover,
many complaints may be verbalised but not formally articulated in a
written statement (4). The HCAT may have a valuable role in organising
complaints, but using it to benchmark quality as the authors suggest could
be misleading and give a false sense of reassurance. We must have a
mechanism to systematically assess poor quality care and whilst written
patient complaints can be part of this, it should not be regarded as an
independent metric of quality.
1. Gillespie A and Reader TW. The Healthcare Complaints Analysis
Tool: development and reliability testing of a method for service
monitoring and organisational learning. BMJ Qual. Saf. 2016 25:937-946
2. Breaking down the barriers: older people and complaints about
health care. Parliamentary and Health Service Ombudsman December 2015.
Available at: http://www.ombudsman.org.uk/about-us/news-centre/press-
releases/2015/frail-older-people-too-afraid-to-complain-about-poor-care
[accessed 11/12/16]
3. Fear of raising concerns about care. A research report for the
Care Quality Commission. April 2013. Available at:
https://www.cqc.org.uk/sites/default/files/documents/201304_fear_of_raising_complaints_icm_care_research_report_final.pdf
[accessed 11/12/16]
4. Cornwell J, Levenson R, Sonola L, Poteliakhoff E. Continuity of
care for older hospital patients. A call for action. The King's Fund,
March 2012. Available at:
https://www.kingsfund.org.uk/sites/files/kf/field/field_publication_file/continuity
-of-care-for-older-hospital-patients-mar-2012.pdf [accessed 11/12/16]
We have read with great interest the article by Schiff G D et al.,1
in which 6.1% of errors reported to the United States Pharmacopeia MEDMARX
reporting system were classified as being related to the computerized
prescription order entry (CPOE) system, representing the third most
frequently reported errors in this notification system.
Similarly, in a study conducted in our hospital, appro...
We have read with great interest the article by Schiff G D et al.,1
in which 6.1% of errors reported to the United States Pharmacopeia MEDMARX
reporting system were classified as being related to the computerized
prescription order entry (CPOE) system, representing the third most
frequently reported errors in this notification system.
Similarly, in a study conducted in our hospital, approximately 24% of
drug-related problems were due to the use of the CPOE.2 This type of error
was more frequently detected after a team of clinical pharmacists reviewed
the drug treatment of hospital inpatients.
One of the major limitations of current classifications of drug-
related problems is that they do not include the various types of CPOE-
related errors.3,4 Consequently, Schiff et al. developed a new taxonomy
for this type of error, which is essential for epidemiological
surveillance and for the continual improvement of the safety of CPOE
systems. These authors identified the 25 most frequent CPOE-related
errors. Similarly, the 8 most frequent types of CPOE-related error in our
study were the following: 1) Drugs included in the hospital formulary but
prescribed as "not available" in the CPOE (for example, spelling mistakes
or the use of the brand rather than the generic name lead to a failure to
find the drug in the application). 2) Duplicate orders (exact same drug
and dosage). 3) Incorrect entry of a prescribed dose resulting in a higher
or lower recommended dosage. 4) Inappropriate frequency of administration
(frequencies are often specified in a free text comment). For example,
digoxin, 1 tablet per day; the free text comments field may state "except
Saturday and Sunday". Since the free text comment bypasses the computer
circuit designed for discontinuous regimens, the medication chart will
state that this drug should be administered on Saturdays and Sundays. 5)
Inappropriate route of administration. 6) Inappropriate treatment duration
(due to failure to use of the end-date field or days of duration).7)
Unintended discrepancies in dosage (prescribed dosage different from
patients' existing dosage). 8) Designation of a clinical trial drug as
"not included in the hospital formulary" instead of the use of a specific
clinical trials application for the CPOE.
Unlike the study by Schiff et al., one of the most frequently
encountered CPOE-related errors in our experience was prescription of a
drug included in the hospital formulary using an option in the CPOE
designed for those drugs not available in the formulary, which can lead to
a delay in administering the drug to the patient, because the prescribed
drug requires pharmaceutical validation as if it were not included in the
formulary and nursing staff do not visualize it as included in the
medication chart to be administered. Unlike our study, one of the codes
identified by Schiff et al. was nursing administration issues. The lack of
this type of error in our study was due to the specific implantation,
parallel to that of the CPOE, of a computerized application that includes
information on the mode of drug administration for nursing staff with the
aim of unifying this process in the hospital.5 Together with this
information, the application allows the time of administration of each
drug to be specified, such that the time of administration will appear
automatically after its prescription, as well as the compatible diluent(s)
for those drugs requiring dilution.
Our study may have identified a lower number of types of CPOE-related
errors because the sample was drawn from a single hospital and because we
included only those errors due to the use of the CPOE.
In addition to analysing CPOE-related errors, Schiff et al. also
evaluated the causes of these errors and identified codes for their
prevention. Similarly, in our hospital, several strategies were
progressively adopted to reduce this type of error. Thus, administration
units were adapted to paediatric patients, numerous computerized protocols
were designed to standardise drugs associated with specific processes, and
the CPOE was modified to allow visualization, in the lower part of the
admission order, of the drug and dosage previously taken by the patient
before admission. However, several safety aspects related to the CPOE
remain to be resolved.
One of the limitations of our study is that it is difficult to extrapolate
the CPOE system to other hospital settings, given that the system was
designed and developed specifically for the characteristics of our
hospital and is not commercially available. One of the strengths of our
study is that the data are drawn from a prospective review of all the drug
treatments by a team of clinical pharmacists, while data from other
studies have been drawn from voluntary notification systems, which could
lead to underdetection of errors as well as a lack of data on their
registration.
In our opinion, the study by Schiff et al. is a highly valuable
contribution, because, in addition to providing a new classification of
CPOE-related errors, it also describes strategies for their prevention.
Given the strong impact of this type of errors, a common classification
system for CPOE-related errors is essential. Such a system would allow
benchmarking between different hospitals independently of the CPOE system
used, which in turn would allow the development of error prevention
systems and/or new CPOE systems to avoid them.
Olatz Urbina Bengoa1, Olivia Ferrandez Quirante1, Marta De Antonio
Cusco1, Nuria Carballo Martinez1
,Santiago Grau Cerrato1
1Pharmacy Department, Hospital del Mar
Pg Maritim, 25-29, CP 08003-Barcelona
Tel: 932483704
Fax: 932483256
References
1. Schiff GD, Amato MG, Eguale T, et al. Computerised physician order
entry-related medication errors: analysis of reported errors and
vulnerability testing of current systems. BMJ Qual Saf 2015;24(4):264-71.
2. Urbina O, Ferrandez O, Grau S, et al. Design of a score to
identify hospitalized patients at risk of drug-related problems.
Pharmacoepidemiol Drug Saf 2014;23(9):923-32.
3. Pharmaceutical Care Network Europe. The PCNE classification for
drug-related problems V 6.2 [Internet]. 2010; http://www.pcne.org.
(Accesed 22 Nov 2016)
4. van Mil JW, Westerlund LO, Hersberger KE, Schaefer MA. Drug-
related problem classification systems. Ann Pharmacother 2004;38(5):859-
67.
5. Salas E, Bastida M, Grau S, et al. Quality project to improve drug
administration in the hospital trust of thecitycouncil of Barcelona.
International Forum on Quality and Safety in Health Care. British Medical
Journal Group. Barcelona, April 2007. (Oral communication). (Data not
published).
In this paper, Professor Sutton's team attribute higher hospital
death rates at the weekend to the patients being sicker. Sutton is joining
very erudite company (Prof Hawking, Prof Winston and the BMA). This group
is rapidly becoming the 'climate change deniers' of healthcare. Not
including this study, there have been 50 very large studies (>100,000
patients) published so far in this area (supplied on request). 44 show...
In this paper, Professor Sutton's team attribute higher hospital
death rates at the weekend to the patients being sicker. Sutton is joining
very erudite company (Prof Hawking, Prof Winston and the BMA). This group
is rapidly becoming the 'climate change deniers' of healthcare. Not
including this study, there have been 50 very large studies (>100,000
patients) published so far in this area (supplied on request). 44 showed a
weekend effect. These studies used multivariate analysis (to take out
confounding variables, like sickness).
This effect has been shown in emergency and elective admissions, all
over the developed world. It is nothing to do with the UK. Even the degree
of increased risk (approximately 10%) is the same, in almost all of the
studies. It is even more strange that Sutton, in this paper, concluded
"Sunday daytime was .. associated with a higher mortality risk .. compared
with Wednesday daytime" (relative risk 6%) - but this was not emphasised.
In other words, whether you have an emergency admission, or a have a
planned operation, you have an approximately 10% greater chance of dying
if you are admitted at the weekend. Perhaps these Professors (and the
BMA) should read the literature before they continue to confuse the
public. I presume they would trust the NHS to look after their own health
(or that of their family) at the weekend.
Yours faithfully
Dr Andrew Stein
Consultant Physician
Conflict of Interest:
I was one of the authors of NHSE's 7DS 10 Clinical Standards in Dec 2013
Statistical process control works well when there is independence and
linearity. Complex systems produce data that are often not independent,
often nonlinear and display self-organisation and emergent behaviour. To
say that statistical process control works when behaviour is emergent may
make little sense. Increasingly adverse events like colonisation with
antibiotic-resistant organisms arise in a complex system. Although...
Statistical process control works well when there is independence and
linearity. Complex systems produce data that are often not independent,
often nonlinear and display self-organisation and emergent behaviour. To
say that statistical process control works when behaviour is emergent may
make little sense. Increasingly adverse events like colonisation with
antibiotic-resistant organisms arise in a complex system. Although the
latter may be influenced favourably by using bundles, checklists and
morbidity and mortality analysis, this is not the same as statistical
process control. Each recent year and especially the most recent, we have
seen the apparent superiority of generalised additive models in preparing
summaries of the hospital's infection management annual report. The idea
that generalised additive models work best when there is a complex system
could be important if substantiated.
Reynolds et al1 reported the impact of providing prescriber feedback
in reducing prescribing errors. The authors have concluded that reducing
prescribing errors needs a multifaceted approach and feedback alone is not
sufficient. Medication errors are often preventable and inappropriate
prescribing is identified as an important contributing factor to
medication errors.2 It is interesting to note that despite regular
feedb...
Reynolds et al1 reported the impact of providing prescriber feedback
in reducing prescribing errors. The authors have concluded that reducing
prescribing errors needs a multifaceted approach and feedback alone is not
sufficient. Medication errors are often preventable and inappropriate
prescribing is identified as an important contributing factor to
medication errors.2 It is interesting to note that despite regular
feedback, prescribing errors are not improved. It shows a failure in
improving underlying prescribing culture.
We need a shift in how we consider safety issues in an organisation.
It is important to assess underlying safety climate in an organisation.
Medication safety issues should be discussed as part of interdisciplinary
rounding or daily safety huddles. If medication errors were considered as
a safety concern during huddles or interdisciplinary rounding then
prescribers could see how errors can impact individual patients and it may
result in practice changes. Interventions that focus on improving safety
culture has proven to be effective in reducing adverse events in hospitals
such as catheter associated blood stream infections.3
It is also well recognised that a committed leadership and supportive
organisational culture is important in bringing practice changes.4 Senior
doctors play an important role in developing the prescribing culture of
junior doctors. Junior doctors have reported that early in their career,
their senior colleagues primarily influence prescribing practices.5 While
the authors decision to target junior doctors prescribing is based on
valid reasons, the lack of involvement of senior doctors in the feedback
process may have limited its effectiveness in bringing practice changes.
It may have been worthwhile considering the prescribing practices of a
team including the consultants and not just the prescribing of junior
doctors.
In conclusion, it is evident that feedback alone does not change
prescribing practices and we need a shift in our approach towards safety
and build an organisational culture that consider safety as a key
priority.
References:
1. Reynolds M, Jheeta S, Benn J, et al. Improving feedback on junior
doctors' prescribing errors: mixed-methods evaluation of a quality
improvement project. BMJ Qual Saf 2016:bmjqs-2015-004717.
2. Kohn LT, Corrigan JM, Donaldson MS. To err is human: building a safer
health system. Washington D.C: National Academies Press, 2000.
3. Pronovost PJ, Berenholtz SM, Goeschel CA, et al. Creating high
reliability in health care organizations. Health Serv Res
2006;41(4p2):1599-617.
4. Kaplan GS, Patterson SH, Ching JM, et al. Why Lean doesn't work for
everyone. BMJ Qual Saf 2014:bmjqs-2014-003248.
5. De Souza V, MacFarlane A, Murphy AW, et al. A qualitative study of
factors influencing antimicrobial prescribing by non-consultant hospital
doctors. J Antimicrob Chemother 2006;58(4):840-43.
I read with interest the article by Peerally et al (1) on 'The
problem with root cause analysis'. I reflected on the recent cases that
happened at Royal North Shore Hospital and Sydney Hospital (2,3,4) which
led me to consider which investigative tool is best applied to different
incidences and identified risks.
The use of appropriate tools and involvement of key stakeholders are
crucial elements to a successful investig...
I read with interest the article by Peerally et al (1) on 'The
problem with root cause analysis'. I reflected on the recent cases that
happened at Royal North Shore Hospital and Sydney Hospital (2,3,4) which
led me to consider which investigative tool is best applied to different
incidences and identified risks.
The use of appropriate tools and involvement of key stakeholders are
crucial elements to a successful investigative process and outcomes,
however, we cannot ignore the reality of the process cost versus event
severity and risk.
Use of tools by subject matter expert
Root cause analysis (RCA) is a tool used in many investigative incidences
(5,6). Often as a result recommendations are made yet similar errors still
happen. As correctly mentioned by Peerally et al, most investigations of
incidences are done by the local team involved with RCA tools but with a
lack of expert accident investigator involvement to ensure regular
feedback loops and ongoing corrective actions.
I do agree that hospitals should move toward proactively preventing
adverse incidences for high probability, high severity risks. Preventing
adverse incidences can eliminate harm to patients, reduce liability for
organisations and reduce both operating costs and the need for resources.
A proactive approach often uses Failure Mode Effect Analysis (FMEA) tools.
FMEA often requires a higher level of investigative expertise and as such
often costs more so it may be optimal to assess risks on a probability
severity matrix to identify which tools are optimal.
The proposal of engaging an independent professional body, while
preferable, can be time-consuming and expensive. I propose for most cases
(with exception for cases with significant legal liability) this level of
expertise and independence could be developed within the organisation. The
body i.e. quality or risk management department, should comprise of people
with qualifications such as system thinking, sound interviewing
techniques, able to involve staff, human factor analysis, current clinical
practice, health management and have the ability to analyse data (7). This
department could then act as a quasi-independent body to avoid situational
bias and provide a platform for disseminating the results to intra-
hospitals, inter-hospitals and governmental bodies as shared learning to
help prevent occurrence or recurrence. As a largely independent department
within the organisation, they can for most cases facilitate the
investigative processes objectively thus eliminating tendency to blame
(8,9,10).
Key stake holders' involvement
The involvement of key stakeholders is very crucial in any investigative
process; leaders, managers, clinicians.
The leaders provide governance, leadership and support to the managers.
They are involved in the investigative process to gain their input,
consensus and to commit resources for any recommendations that might be
made. It is critical leaders set departmental performance indicators with
due acknowledgment for the resources needed to achieve them as too often
the burden of performance and blame is levied on departments, middle
management and individuals where identified risk avoidance is under-
resourced.
The managers (department managers, quality and risk managers) are required
to provide a safe environment for practice. They are to ensure that the
protocols and standards of care are adhered to and patients are managed in
a consistent manner. The role of the manager also includes identifying
risks and establish processes to prevent the risks from reaching the
patient with the support from the leader.
The clinicians are required to conduct the procedures/practices in
compliance with their scope of practice, organisational and regulatory
boards.
Conclusion
The usage of an appropriate tool by a qualified person with the right
expertise makes a difference. It would be economically unrealistic to
apply full FMEA processes for every incident or identified risk profile,
so the establishment of an organisational risk severity/probability matrix
needs to be developed so the most appropriate tool is used.
The involvement of key people ensures that a holistic approach is applied
and outcomes of the investigations are implemented with feedback checks
and balances and shared across intra-departments, inter-hospitals and at
national level (11).
References:
1)Peerally MF, Carr S, Waring J, Dixon-Woods M. The problem with root
cause analysis. BMJ Qual Saf. 2016 Aug;1:1-6
2)Bodies swapped: Dead baby mistakenly cremated and daughter finds
mother's body mislabelled at Royal North Shore Hospital [television
broadcast]. Sydney: The Sydney Morning Herald; 2016 Aug 31. Available
from: www.smh.com.au/nsw/daughter-finds-mothers-body-mislabelled-in-morgue
-mixup-at-royal-north-shore-hospital-20160830-gr4g3n.html
3)Joseph AP, Hunyor SN. The Royal North Shore Hospital inquiry: a analysis
of the recommendations and the implications for quality and safety in
Australian public hospitals. Med J Aust. 2008 April ;188(8):469-72
4)Family want justice for fatal gas mix up [television broadcast]. Sydney:
Skynews; 2016 Jul 26. Available from: http://www.skynews.com.au/news/top-
stories/2016/07/26/incorrect-gas-fitting-behind-nsw-baby-death.html
5)Clifford SP, Mick PB, Derhake, BM. A Case of Transfusion Error in a
Trauma Patient with Subsequent Root Cause Analysis Leading to
Instituitional Change. J Investig High Impact Case Rep. 2016 May; 4(2):1-4
6)Van-Galen LS, Struik PW, Driesen BEJM, Merten H, Ludikhuize J, Van der
Spoel JI, Kramer MHH, Nanayakkara PWB. Delayed Recognition of
Deterioration of Patients in General Wards Is Mostly Caused by Human
Related Monitoring Failures: A Root Cause Analysis of Unplanned ICU
Admissions. 2016 Aug; 11(8):1-14
7) Ibrahim JE. What is the quality of our quality managers? Is it time for
quality managers in Australia to be certified? J.Qual Clin Practice.
2000;20(1):32
8)Smetzer JL, Cohen MR. Lessons from the Denver medication error/criminal
negligence case: look beyond blaming individuals. Hosp Pharm. 1998;33:640-
57.
9)Leape L. Error in medicine. JAMA. 1994;272:1851-7
10)Runciman W, Merry A, Smith AM. Improving patients' safety by gathering
information. Anonymous reporting has an important role. BMJ. 2001;323:7308
11)Leape LL. Why should we report adverse incidents? J Eval Clin Pract.
1999;5:1-4
As you point out Root Cause Analysis will often fail with hospital adverse event (AE) data because it was not designed to deal with data arising in a complex system.1 The same can be said for Pareto analysis. Statistical process control (SPC) methods are often used to summarise AE data, particularly hospital infection data such as surgical site infections (SSIs) and bacteraemias.2 Standard SPC also frequently fails to summarise these complex data correctly.
Show MoreWith binary SSI data an approximate expected rate is frequently available so cumulative observed minus expected and CUSUM analysis are appropriate.2 However, the changing observed rate is not seen unless the numbers of procedures is large enough for them to be grouped by months or quarters. This is often infrequent. Even when such aggregation is possible difficulties arise as the number of procedures in each month may differ markedly. This problem can be dealt with, at least approximately, by employing a generalised additive model (GAM) analysis to the binary data that predicts the observed AE rate at various places in the time series.
Count and rate data such as bacteraemias or new isolates of an antibiotic-resistant organism will usually not have an expected rate available. These data are often grouped by months and a Shewhat chart used for their display. This chart requires a stable centre-line about which reliable control limits can be drawn. Often the mean value is used as the expected rate even though...
Vindrola-Padros and colleagues provide a helpful examination of co-production of quality improvement knowledge by university-based researchers in cooperation with members of service organizations. Another important type of embedded researcher consists of “fully embedded,” researchers, who are academically trained but employed by large care delivery systems. These individuals typically work in research units in the delivery systems. Their work is funded both by the systems themselves and by external, private and public organizations, such as the Agency for Healthcare Research and Quality (AHRQ). These fully embedded researchers contribute actively to national professional forums and journals and sometimes collaborate with embedded researchers in other systems.
AHRQ leverages relationships with fully embedded researchers because of their deep and nuanced knowledge of internal system data and operations. Health systems-based researchers’ ready access to care sites within which to test new approaches, and to data sources that permit rapid analysis of results of those tests, are of great value to AHRQ as we seek to find solutions to real-world problems in areas of national importance. AHRQ-supported work of this kind demonstrates the value of health delivery organizations becoming “learning health systems”(1) – using their own internal data and resources to drive quality improvement and sharing their findings with other organizations.
AHRQ’s collaboration w...
Show MoreThis study uses rigorous analysis to obtain important insights about the realtime information that our patients are handed at discharge. It is puzzling that the EMRs used were not named. One can infer from a look through the MSU website that they have both Cerner and Epic, but why is that necessary? The heart of quality/safety work is one of transparency balanced by humility, i.e. we shouldn't expect our IT systems to be any more perfect than we are, but they won't improve if we don't have more openness. The lack of scientific foundations and published post-marketing surveillance for our EHRs, especially the ascendant ones, was initially surprising. However, as they achieve complete market dominance, with less overt scientific review and public guidance and commentary, the silence is deafening. Is the BMJQS's failure to simply identify the names (or maybe I missed the citations) an oversight, or part of nondisclosure agreements with the vendors at the MSU institutions or at BMJQS?
To the Editor:
In this article, the authors propose that little evidence exists in healthcare to show that application of Highly Reliable Organization (HRO) principles has resulted in significant or sustained improvement in performance. Further, they attribute the problem partially to under- recognizing the role of habit in the process. While we fully agree that forming habitual behavior is essential to creating...
I read with interest the paper by Gillespie and Reader presenting the Healthcare Complaints Analysis Tool (HCAT) (1). The authors suggest that the HCAT could be used "as an alternative metric of success in meeting standards" and as a way "to benchmark units or regions". However, this makes the assumption that the volume and strength of complaints received is an accurate reflection of the standard of care being delivered....
To the Editor,
We have read with great interest the article by Schiff G D et al.,1 in which 6.1% of errors reported to the United States Pharmacopeia MEDMARX reporting system were classified as being related to the computerized prescription order entry (CPOE) system, representing the third most frequently reported errors in this notification system.
Similarly, in a study conducted in our hospital, appro...
In this paper, Professor Sutton's team attribute higher hospital death rates at the weekend to the patients being sicker. Sutton is joining very erudite company (Prof Hawking, Prof Winston and the BMA). This group is rapidly becoming the 'climate change deniers' of healthcare. Not including this study, there have been 50 very large studies (>100,000 patients) published so far in this area (supplied on request). 44 show...
Statistical process control works well when there is independence and linearity. Complex systems produce data that are often not independent, often nonlinear and display self-organisation and emergent behaviour. To say that statistical process control works when behaviour is emergent may make little sense. Increasingly adverse events like colonisation with antibiotic-resistant organisms arise in a complex system. Although...
Reynolds et al1 reported the impact of providing prescriber feedback in reducing prescribing errors. The authors have concluded that reducing prescribing errors needs a multifaceted approach and feedback alone is not sufficient. Medication errors are often preventable and inappropriate prescribing is identified as an important contributing factor to medication errors.2 It is interesting to note that despite regular feedb...
I read with interest the article by Peerally et al (1) on 'The problem with root cause analysis'. I reflected on the recent cases that happened at Royal North Shore Hospital and Sydney Hospital (2,3,4) which led me to consider which investigative tool is best applied to different incidences and identified risks. The use of appropriate tools and involvement of key stakeholders are crucial elements to a successful investig...
Pages