Article Text
PDF
Statistics from Altmetric.com
The coming of age of “safety” indicators
Can safer practices—especially pertaining to medication use—be measured and analyzed within the existing paradigm of performance assessment in health care? More than 20 years of experience in performance measurement and evaluation, primarily through indicators, have built a path upon which the more controversial, challenging, and yet similar concepts of safe practices can indeed proceed. Whether descriptive and qualitative (organizational culture towards punishment when errors are identified1–3) or quantitative (use of successive measurement methods to build a performance profile4–6), safer practice is affected by the same latent confounders of performance measurement.
The latency of these confounders complicates identification of the explanatory factors regarding the safety of processes. Consequently, and with the recognition that many explanatory factors remain to be uncovered, the evaluation of safety faces at least the following challenges:
-
design of the measurement tools;
-
methods of analysis;
-
identification of linkages between what is done and what happens.
These challenges are also considered by Paton and Lelliott in their paper on the quality of prescribing for psychiatric inpatients in this issue of QSHC.7 They present critical first steps in addressing medication use and safety issues. Based on the findings of this paper, a few additional comments on the above three challenges are appropriate.
DESIGN OF THE MEASUREMENT TOOLS
While the use of indicators (numerator and denominator based constructs) is well accepted in the measurement of healthcare performance, consensus does not exist on how to design robust measures to quantify safety of care. Although there has been much discussion about why the quantification of medication use or medical decision making in general is different from, say, mortality analysis or readmissions pattern evaluation, the core of the discrepancy is probably more philosophical in nature than methodological. Indeed, for most performance measurement indicators currently in use in health care there is an acceptance of relativity and perhaps even determinism—specifically, relativity encompasses the need for adjustment of a performance rates interpretation to the context and the environment. Determinism pertains to a collective belief that, as long as there are patients and healthcare organizations, certain consequences of performance such as infections, readmissions, mortality, and even dissatisfaction are to be expected. In contrast to both relativity and determinism, today’s paradigm of safety seems to recommend an uncompromising pursuit of zero tolerance with little consideration for context or historical beliefs and expectations. It is perhaps not a coincidence that, while risk managers have always dealt with the consequences of errors, it is rare to find rate based indicators in risk management.8 What one finds is zero tolerance whereby even one mishap that is translated into a lawsuit or a legal conflict is too many. For the risk manager, comparisons between hospitals of frequency of complaints, lawsuits, and legal fees seem irrelevant. All that matters is for them as an organization to minimize or eradicate errors and their consequences.
METHODS OF ANALYSIS
Measurement methods are derived from the philosophy, clarified at the outset, of how knowledge gleaned about practices and their impact on patients will be used. If zero tolerance is the only way communities, policy makers, and even direct care providers are willing to look at the issue of safety, our challenge will be furthered since the now “traditional” means of measurement through comparison analysis will be of little value. On the other hand, it may be possible that the initial categorical attitude of zero tolerance may prove to be too stringent and often unrealistic. In that case, the path—no matter how serpentine—prepared by previous research on performance measurement can be passable and even conducive to the accumulation of new knowledge in the area of safer practices and their outcome.
IDENTIFICATION OF LINKAGES BETWEEN WHAT IS DONE AND WHAT HAPPENS
The most challenging exercise in this framework of understanding safer practices is perhaps that of interpretation. Finding causal relationships between processes and outputs or, in general, what has been done and what has resulted remains probabilistic even when replicated across settings. The web of factors that could have influenced an observed outcome is fundamental to identify and classify by the nature of its influence. Methodologically, understanding the relationship between safer practices and better outcomes immediately brings the unit of analysis to the patient level and the clinical and demographic information necessary to incorporate into the web of factors (often latent) and the consequent correlations and analyses. The paper by Paton and Lelliott7 attempts to tackle the issue of “true” indicator development in medication use, especially in psychiatry. The authors recognize the difficulties involved in establishing causal relationships between dispensing medication and appropriateness. As this paper deals with medication dispensing in psychiatry, the analysis of prescribing, dispensing, administering, and monitoring of medications represents a formidable challenge. Paton and Lelliott do not discuss “patient safety” but rather they describe dispensing practices only. How variability in these practices would impact on the patient remains hypothetical and requires a separate research focus and design. For now, it is sufficient to recognize that true indicators of medication use can be designed and that indicators can significantly help in performance measurement and evaluation. Furthermore, the authors report their observations from a project bound in time and place. The true test of safety indicators will require an ongoing initiative from a project to a program.9 Only then will the impact of indicators, changes in practice, and patient outcomes be better addressed. To paraphrase and adapt one of Buddha’s sayings: “Where you are today depends on where you were yesterday. Where you will be tomorrow depends on your goals today.” And today we still seem to be searching for our common goals regarding safety in health care.
The coming of age of “safety” indicators
Footnotes
-
Dr Vahé A Kazandjian is the President of the Center for Performance Sciences, a global outcomes research organization, and Adjunct Professor, the Johns Hopkins University Bloomberg School of Public Health, Baltimore, Maryland. He is the original architect of and is responsible for the Maryland Quality Indicator Project® (QIP), the continuous performance improvement program used worldwide over the last 19 years by more than 1800 healthcare organizations. In the UK alone, over 125 hospitals from the NHS and the private sector have participated in the international component of the QIP since 1992. Dr Kazandjian is an advisor to the WHO, Europe Office, for the development of indicator projects in European countries.