How safe is the safety paradigm?
- 1Netherlands Institute for Health Sciences, Erasmus MC, University Medical Center Rotterdam, Rotterdam, The Netherlands
- 2Department of Social Medicine, Division of Clinical Methods and Public Health, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands
- Correspondence to: Dr O A Arah Department of Social Medicine, Division of Clinical Methods and Public Health, Academic Medical Center, University of Amsterdam, P O Box 22700, Amsterdam 1100 DE, The Netherlands;
- Accepted 12 March 2004
This paper reviews safety initiatives in the health systems of the UK, Canada, Australia, and the US. Initiatives to tackle safety shortcomings involve public-private collaborations. Patient safety agencies (to institute learning, action and safety culture), adverse event reporting and, to a lesser extent, safety related performance indicators are currently used to design safer health systems. Their benefits are mixed, but there is little debate as to their possible side effects. Foreseeable adverse effects of multiple safety organisations stem from them being too many, too vague, too narrowly focused, threatened by the medical practice environment, and too optimistic. Safety related performance indicators are most developed in the US but suffer from inadequacies of administrative data, underreporting, variable indicator definitions, “extended” use, and low sensitivity of the diagnosis coding system, and arguable preventability of the prescribed conditions. A critical appraisal of the implications of these deficiencies is important to assure the safety of current health system safety initiatives and to establish evidence based safety. It is necessary to embed health system safety (as well as patient safety) in the societal culture, structures, and policies which promote effective, user centred, high performance care while allowing for healthy innovation.
Health system safety matters because lapses in safety harm patients, their families and, ultimately, society. Media coverage of many cases of serious injury or death as the result of unsafe health care has heightened public concern about health care safety. In response, policymakers have, in many countries, issued reports or set up agencies to try to tackle a problem that is clearly longstanding and endemic. However, like the wider agenda of healthcare quality,1 patient safety has only recently become a priority on the policy agenda. This recent attention focuses on the risks of harm from health care (for instance, dangerous drug overdose), the risks in the care environment (for all patients and providers), the flawed system designs which enable injury occurrence,2–6 and better patient involvement in harm management.7
Following these considerable public, professional, and political interests in patient safety, several industrialized countries—for example, UK,8,9 Canada,10,11 Australia,12,13 Denmark,14,15 and USA5—have now established safety initiatives. Notably, these interests have resulted in large public-private collaborations, safety organizations, adverse event reporting systems,16 research into errors and adverse events,17,18 and calls for system redesign.5,19,20 Also, patient safety has become a core dimension of performance measurement and management frameworks.
However, these safety initiatives raise several questions. What are the key components of the efforts and what are they actually aimed at? Can other countries really adopt similar initiatives? If so, are they sufficiently safe and sound for cross-national learning? Will such initiatives solve the problems of health system safety? Although safety is the topic of a growing number of scientific papers, few articles have tried to critique the actual reactions to safety in health care.
In this paper we briefly examine the national agenda on and measurement of patient safety in four advanced health systems—namely, the UK, Canada, Australia, and the USA—and reflect on the potential shortcomings of these efforts. Although it is not our intention to give an exhaustive overview of national safety efforts in these four countries, we want to start a reflective discussion on their nature, direction, and potential shortcomings.
Our approach consisted of (1) exploring the relevant literature, documents, and websites on safety in the UK, Canada, Australia, and the USA; (2) seeking information from the revealed safety agencies to summarize their goals and current safety activities; (3) reviewing their national health system performance frameworks and indicators; and (4) where possible, searching the scientific literature for supporting evidence for safety related performance indicators seen in national indicator frameworks. We then reflected on all findings (summarized in the accompanying tables) from a systems perspective. This exploratory policy analysis entailed reviewing the key issues on the safety agenda and examining the “whats” and “whys” of these issues. The key findings are presented in two sections on (1) national patient safety agencies and reporting systems and (2) safety related performance indicators.
NATIONAL PATIENT SAFETY AGENCIES AND REPORTING SYSTEMS
Table 1 gives an overview of the national patient safety agencies found in the four countries.
The British take quality and safety in the NHS quite seriously.8,9,21–26 The Bristol case27,28 and other high profile affairs29–31 probably served to place the safety debate in the public domain. Following these events and two major safety publications,8,9 the government set up the National Patient Safety Agency (NPSA).25,26 The NPSA collates, analyzes, and feeds back information on lapses in patient safety within the NHS, and works with NHS staff and organizations to promote a fair and open culture (table 2).25 It also recently introduced a national reporting and learning system across the NHS to complement local level vital reporting, learning and action. Some examples of other stakeholders who are nationally or locally involved in patient safety across the UK are shown in the footnotes to table 1.
Health Canada, the Canadian Institutes of Health Research (CIHR), and the Canadian Institute for Health Information (CIHI) are sponsoring the investigation of the national hospital adverse events and errors to be published in late 2004.10,11,32 Also, Health Canada is currently funding an exploration of the possibility of a national incident reporting system. The public-private roundtable on patient safety organized by the Royal College of Physicians and Surgeons in September 2001 led to the formation of the National Steering Committee on Patient Safety (NSCPS) which has made 19 recommendations for a national integrated strategy along five themes (see table 2).33 These recommendations are similar to those made by Baker and Norton in their systematic review and report to Health Canada.34 In February 2003 the Canadian Federal Budget earmarked $50 million over 5 years for the creation of a Canadian Patient Safety Agency. The Institute for Safe Medication Practices (ISMP Canada) is an independent not-for-profit body that promotes safe medication practices, collects and analyzes medication errors, and recommends improvements.35
The Commonwealth Department of Health funded, in 1994, the Quality in Australian Health Care Study (QAHCS) to detail the degree of adverse events in Australian hospitals. Modeled on the Harvard Medical Practice Study in the US,36,37 the QAHCS38 in June 1995 showed that 16.6% of admissions had adverse events of which 51% could be preventable.38 This publication attracted much attention, immediately resulting in the National Taskforce on Quality in Australian Health Care which recommended increased funding and establishment of a national organization for safety and quality of care. The subsequent formation in October 1996 of a National Expert Advisory Group on Safety and Quality in Australian Healthcare led to the establishment of the Australian Council for Safety and Quality in Health Care (ACSQHC) in 2000.12 It coordinates the national safety action and has so far made four reports to the Health Ministers.39–42 Other national safety efforts include the Australian Patient Safety Foundation (APSF) which developed the Australian Incident Monitoring System13,43,44 and a classification system for coding and reporting.45 The Australian initiatives served as the template upon which the US and UK built similar safety agencies.
The Institute of Medicine’s (IOM) report “To Err is Human”5 is probably the most strategic publication on patient safety in the US. This report put deaths from medical error in the US at about 44 000–98 000 per annum, with 7000 of these resulting from medication errors alone. Established within 3 months of the IOM report, the Quality Interagency Coordination Task Force (QuIC) recommended some 100 actions to improve safety.46 The Agency for Health Care Research and Quality (AHRQ) oversees the federal government’s interests in safety47 (tables 1 and 2). AHRQ sponsored the Stanford University Evidenced Practice Center and the University of California to produce a critical appraisal of evidence on safety practices48 and the AHRQ patient safety indicators (as part of the three-part AHRQ Quality Indicators project).49 The Joint Commission on Accreditation of Healthcare Organizations (JCAHO), as an independent accrediting organization, has patient safety requirements for hospitals which it accredits.50 The National Patient Safety Foundation (NPSF) aims to lead the transition from a culture of blame to that of safety.51 MedMARX is a medication error reporting program of the US Pharmacopoeia to share experiences and best quality practices.52 Other notable national or widespread initiatives are from the Institute for Safe Medication Practice53 (ISMP, a not-for-profit agency that educates professionals or institutions on adverse events and runs a national Medication Errors Reporting Program with the US Pharmacopoeia), the Food and Drug Administration (with its voluntary MedWatch reporting system),54 the voluntary National Nosocomial Infection Survey of the Centers for the Disease Control and Prevention,55 and the National Quality Forum.56,57
SAFETY RELATED PERFORMANCE INDICATORS
The development and the use of indicators within conceptual frameworks are major ways through which national governments drive performance improvement in their health systems.58,59 The Canadian60 and Australian61 health system performance frameworks have one indicator each for patient safety—admission to hospital for hip fracture and hospital separations with an adverse event, respectively. Table 3 summarizes these two indicators. However, the ACSQHC also has sentinel events indicators which were approved by the Australian Health Ministers and which represent binomial catastrophic events of system failure deemed suitable for national aggregation. The US (AHRQ) probably has the most comprehensive set of indicators on patient safety.49 The AHRQ patient safety indicators (PSIs) represent a major undertaking to screen, at provider or system level, patient safety issues using hospital administrative data. These indicators—with empirical averages ranging from four transfusion reactions per million discharges at risk to 244.08 obstetric traumas in 1000 vaginal deliveries with instrumentation—cover surgical, obstetric and medical conditions, risk adjusted for age, sex, diagnosis related group, and comorbidity categories49 (see table 4 for an overview). The Canadian Institute for Health Information has also adopted the AHRQ safety indicators. Other indicator systems in the USA include the JCAHO Indicator Measurement System for infection control and the JCAHO sentinel events.
This paper has explored briefly the national safety initiatives in four advanced health systems. The use of patient safety agencies, incident reporting and learning systems, and indicators to address patient safety concerns is emerging. There are far more initiatives in use than can be highlighted in this paper, but this review gives a snapshot of the national level safety agenda. The benefits of such programmatic efforts are assumed, so we will not discuss them here. The focus here is on their disadvantages. In rushing to do something about errors in medicine, countries must concurrently critique what they are doing—given the wide range of ideas and tools being deployed—and the potential for these to become part of accountability mechanisms. Surely societies cannot afford to wait for evidence of failure of these efforts before addressing them. We first discuss the safety agencies and reporting systems together, and then the safety indicators.
It is important to note that the different institutional approaches to safety are related to the differences in the structure and control of the various health systems. For instance, the pluralistic US health system is, as would be expected, characterized by more safety bodies than is the case in a more uniform system such as the UK NHS (see table 1).
Safety agencies and adverse event reporting systems
Foreseeable disadvantages of existing safety agencies and adverse event reporting systems seem to stem from them being too many, too vague, too narrowly focused, threatened by the medical practice environment, and too optimistic.
The safety organizations and reporting systems appear too many and may impose confusion and excessive regulatory burden on health actors. This overcrowding of the “performance environment”58 has been discussed in the case of NHS regulation.62 Ineffective extraneous regulation and bureaucracy slow innovation. Professionals may become risk averse, innovate less, and potentially decrease overall clinical effectiveness (because when patients are not treated, they avoid harm but they also lose out in gaining any benefits). The practice of medicine, as we know it, is an art and can be a risky venture for all involved. No two patients are identical—nor are any two errors. Even in this era of clinical guidelines, medical intricacies imply that professional providers will be innovative. Innovation is the backbone of learning and growth, offering relative advantage over the status quo.63
There are many definitions51 and concepts of patient safety which risk making the core notions vague. The specifications, nature, and usefulness of tools used for reporting and learning from incidents are largely dependent on these still poorly aligned terminology and subjective conceptualizations. This makes room for unfortunate mix ups and wrong analysis. There are debates on whether safety should focus on medical injuries64 or errors.65 Furthermore, safety and effectiveness as dimensions of performance appear to overlap when we include errors of omission within safety. For emphasis and action, errors of omission deserve to be separated out as safety issues, especially since not doing the right thing is not only ineffective, but also harmful when the wrong thing is done instead.
Too narrowly focused
Although this paper examines patient safety initiatives, it is evident that current efforts focus too narrowly on the patient with little explicit linkage to wider aspects of health system safety. Risk management entails reducing harm not just to patients but also to staff, visitors,3 and the environment (including the society). Many national safety initiatives try to address all these, albeit disjointedly. Furthermore, as shown in tables 3 and 4, staff and environmental safety measures are not currently captured in performance frameworks. Obviously, health systems cannot treat these other aspects as independent concerns.
Threatened by the medical practice environment
The medical practice environment remains prohibitively litigious. The current safety paradigm does not seem to be winning against the tort system and apportioning of blame. Espousing a no-fault rationale in the public domain, while the blame culture rages, has done little to stem the litigation tide. Nor has the current approach to “safety culture” (restricted to the health and legal systems) helped. As in the wider quality movement, voluntary disclosure and partial openness further undermine these efforts.66–68
Current thinking on patient safety reflects a zero tolerance of harm. With abundant referencing to the aviation and other industries, and from quality management cycles to cognitive meta-systems,69 research has enabled health systems to adopt systems engineering and causal analytical rationales and tools from the non-medical world. Where these allied and industrial sectors have established safety nets, expected minimums, and unavoidable maximums, medicine has no acceptable levels of adverse events and errors, nor should it. A practical approach so far has been to fight the frequency and severity of adverse events to the barest levels possible. An obvious disadvantage is that we become too hopeful in the short to medium term but disenchanted in the long term when we get mixed results. In the absence of exhaustive data on the lapses and of evidence of what works and what does not, our zero tolerance may remain naively optimistic.
Even at a national headline level, just reporting on hip fracture as seen in the Canadian health indicators framework will not give a balanced view of health care safety (table 3). The US AHRQ safety indicators are more extensive but suffer from the shortcomings related to administrative data, underreporting, indicator definitions, and preventability of the prescribed conditions (see table 4).70 Their use of hospital administrative data imports the problems of incomplete data and inaccurate and variable coding of ICD-9-CM (International Classification of Diseases, 9th edition, Clinical Modification) diagnosis fields.71 Also, the ICD-9-CM coding system was not created with safety issues in mind and is therefore inadequate for identifying errors and injuries.72 Basing safety tools on administrative data increases the likelihood of low sensitivity of these indicators in flagging incidents.73 Although much has been done to increase the validity and reliability of the AHRQ performance safety indicators, there is still a long way to go. Combining anesthetic complications together into one measure is clearly counterproductive as the total count may stabilize while the constituent conditions vary tremendously. In addition, the AHRQ performance safety indicators specify medical conditions less than they do surgical problems. Consequently, these safety indicators will be more problematic than other quality measures among end users. Given the litigious medical practice environment, such ambiguities are unhelpful, if not unsafe.
What can be done?
Countries can and should strive towards evidence based safety. The issues raised above will need to be addressed to create re-prioritized, coherent, deeper, more focused, and realistic safety initiatives with an encouraging medicolegal environment. This re-prioritization should learn from established approaches used in evidence based medicine and rational decision making. Safety policies should invest more in safety issues with the largest impact and conduct a better search for more rigorous evidence of effectiveness.
By investing more, both financially and otherwise, countries must critically:
avoid useless and potentially harmful duplication of initiatives or systems;
clarify and unify concepts and definitions;
expand and integrate the scope of safety within and beyond the health system;
address the policies and societal environment which hinder the inception of a culture of safety, fairness. and openness with the wider society;
redesign the training and working conditions of providers;
engage patients and their families as partners in health; and
search for better data sets and coding system for safety indicators as well as re-evaluate the preventability of measured conditions.
Health system safety has recently become an urgent issue in many industrialized nations, notably, UK, Canada, Australia and USA.
These countries have all engaged in safety initiatives such as patient safety agencies, adverse event reporting and learning systems, and the use of safety performance indicators.
The benefits of such programmatic efforts are assumed, but it is still unclear how effective these multiple initiatives are. Furthermore, little attention has been paid to their potential side effects.
These shortcomings which can exacerbate the initial safety and health problems should be anticipated and guarded against from the outset, especially as these initiatives can become accountability tools.
Both effects and side effects of current initiatives need careful rigorous evaluation to achieve evidence based safety in health systems.
Ensuring safety of health systems has led to the creation of agencies, networks, and policies that may well become new bureaucracies and regulatory mechanisms, and may discourage learning and growth when they overburden professionals with new jargons, new protocols, and new responsibilities. Providers, who must work with patients as partners, are the key to any system redesign to confront patient safety.20,74 “[I]gnoring the occasional need for innovation can lead to robot-like execution of procedures in an unthinking, unreflecting manner which is surely not in the best interest of the patient.”75
Furthermore, as other countries adopt these patient safety models which become part of national accountability and performance frameworks, it is imperative to explore their transferability and to find out how such efforts actually affect the safety culture, structures, practices, and outcomes. Contextual analysis is indispensable for successful application, acceptance, and appraisal of safety tools.76 It is not enough to investigate whether these systems work; it is imperative to find out how and why they reduce errors and adverse events.19,77
We argue that, when it comes to routine surveillance or performance measurement using indicators or both, performers become what they measure or are expected to deliver. How effective these initiatives become depends on how well they are embedded in the medical culture, structures, and policies which promote effective patient centred care. As avoiding harm has been a fundamental tenet of medicine since Hippocrates, we should remember to balance the “doing no harm” (safety) with “doing good” (effectiveness or quality health care) at the level of both the individual patient and the health system.
The views in this article are entirely those of the authors. No official endorsements by the various institutions or affiliated persons are intended or should be inferred. The authors are grateful to the following for information on their respective organizations or countries: Lucy Bolton (National Patient Safety Agency, UK), Pierrete Leonard (Royal College of Physicians and Surgeons of Canada), Helen McElroy and Michekke Budak (Health Canada), Mark Bisby (Canadian Institutes for Health Research), Barbara Levings (Australian Council for Safety and Quality in Health Care), Deborah Queenan (Agency for Healthcare Research and Quality, USA), and Richard J. Croteau (Joint Commisison on Accreditation of Healthcare Organizations, USA).
Funding: Ministry of Health, Welfare and Sport, The Hague, the Netherlands.