Recently Neuhauser and Dias raised an important question [1]: are randomised clinical trials (RCT) necessary in quality improvement?

They conclude that “RCTs need to be embedded in generalized replicatable theory. Otherwise it is a scientific house without foundation” and argue that RCTs comparing drugs cannot be replicated for two reasons: (1) after a decade or two the control arm has changed and (2) replication may be considered unethical if the original trial showed a difference. Thus it is tempting to draw the conclusion that the authors suggest that RCT should not be used when the effect of a new drug is to be tested.

Why is it so important that an experiment can be replicated? Say drug B is shown to be superior to drug A, and later drug C is shown to be superior to drug B. Then, who really cares whether the experiment comparing drug A and B can be replicated or not?

The authors’ claim that RCTs cannot be replicated may be challenged. If the conditions for the control arm has changed then the intervention arm conditions are likely to have changed in parallel over time. Then we expect to be able to replicate the findings. But we have no way of knowing this. In fact, given that the same drug is used over the years, it is unethical not to replicate the experiments regularly, according to paragraph 6 in the introduction to the Helsinki Declaration (www.wma.net/e/policy/b3.htm).

We agree with the authors that statistical process control (SPC) has an important role to play in medicine [2]. But we feel that the use of SPC for the comparison of drugs is quite risky due to the potential for the introduction of all kinds of bias [2,3]. Another reason why SPC may be inferior to RCTs is that the application of SPC requires the processes studied to be brought in statistical control prior to the intervention [2]. This may not be possible to achieve because the patient mix may vary over time. It may be argued, then, that statistical risk adjustment may take care of this problem. However, it is well known that this approach is fraud with problems [2,3]. By contrast, the attainment of statistical control is not an issue in the case of RCTs, precisely due to the random assignment of patients to the intervention groups.

References 1. Neuhauser D, Dias M. Quality improvement research: are randomized trials necessary? Qual Saf Health Care 2007; 16:77-80. 2. Winkel P, Zhang NF. Statistical Development of Quality in Medicine. John Wiley and Sons Inc, 2007; 1 – 263. 3. Deeks JJ, Dinnes J, D’Amico R, Sowdon AJ, Sakarovitch C, Song F, Petticrew M, Altman DG. Evaluating non-randomised intervention studies. Health Technol Assess 2003;7:1-173.

Dear Editor,

As a retired USAF pilot-physician, I commend Singh et al. for their excellent use of aviation Situational Awareness as an analysis tool. I wish only to add a subtle dimension to their illustration of situational awareness: "LSA" - loss of situational awareness – began as a universally recognized NATO acronym. But…one can not lose what one never had. SA in military and air carrier aviation universally begins at a maximum, and may deteriorate backwards from Level 3 of Endsley’s model. Maximal SA at the outset of a mission is achieved by all team members studying all environmental factors that might affect the outcome; mentally rehearsing the mission timeline, actions, and threats; and planning for contingencies during the preflight briefing. The team begins with a high level, mental model of what is to come. Sadly, pre-event reviews are vanishingly rare in medicine, and physicians begin with very limited, or absent SA, as illustrated by the outpatient case in the paper. Further, SA in outpatient medicine, if it exists at all, is compromised by the fragmentation and time displacement of cues and communications. Ironically, is easier to discern SA, good and bad, in the confines of the high risk areas of inpatient care: the OR, Labor and Delivery, and ED. Medical team training courses emphasize the value of briefings in setting the stage for good SA, and train high risk team leaders to conduct them.

Thus far team training is not widely deployed or accepted. Nor are briefings cited in such patient safety resources as the AHRQ Web M&M.1 Another fundamental, but subtle, difference between aviation and medicine is decision making. The hapless primary care physician was "flying solo" and making independent judgments. Despite the Hollywood images, fighter pilots rarely make solo decisions. Flying in multiples for "mutual support," air combat teams operate with strong, visible, designated leadership, but simultaneously practice collaborative, consensus decision making. Similarly, "cockpit resource management," the progenitor of medical team training, reversed decades of left-seat, hierarchical, autocratic decisions that placed passengers at the same risk level as the described patient in favor of collaborative decision making after inputs by all, even passengers.2 Lastly, the authors omit any discussion of a post-event debriefing of this adverse outcome. Thus, learning was not captured, nor system improvements made. Debriefings – as short as 30s or lasting for hours – are mandatory in aviation and result in real-time, actual, lasting CQI. Early efforts to use traditional M&M conferences offer some promise in debriefings.3 Medicine has much to adopt and adapt from other high risk professions, aviation, nuclear power, and even mining. The authors have advanced that journey significantly.

References

1. http://webmm.ahrq.gov/

2. Foushee HC. Dyads and triads at 25 000 feet: factors affecting group process and aircrew performance. American Psychologist 1984; 39: 885-993.

3. http://www.patientsafety.gov.

No conflicting interests: Government employee only.

Dear Editor,

According to MM Bismark et Al (1) complaints that are brought to a Commissioner in NZ offer a potentially valuable “window” on serious threats to patient safety. In Italy, the consultative and conciliatory commissions (“Commissioni miste conciliative”) and the ombudsmen (second level organisations) supply unsatisfactory results or are not even established (2). Furthermore, epidemiological data like those of the NZQHS are very scarce. We had only a possibility in this field: to carry out a survey to explore if an independent patient’s agency could facilitate local learning and action to improve the quality of health services.

Two hundred forty complaints and accounts of presumed untoward events presented by people to our voluntary Patients Agency in Milan (Cittadinanzattiva) were analysed. Every complaint enclosed one or more clinical records. Our aim was to compare allegations contained in the written complaints and the results of a retrospective case records review. An expert physician examined all the clinical records produced and identified when an “avoidable adverse event” was present (3). A medicolegal expert independently reviewed 89 complaints. According to the patients the cause of the complaint was: *A delay in diagnosis and treatment (89 cases). *The failure or a complication in the technical performance of an indicated operation (82 cases), or of an indicated invasive procedure (24 cases ). *Lack of care or attention, failure to attend, lack in monitoring of a patient (19 cases). *Delay or failure in treatment (9 cases). *Others (9 cases). In the physician record review, 87 out of 240 complaints were associated with a preventable adverse event (mainly occurring in hospital); 22 were serious and preventable, 12 of them from a failure or a complication of an indicated operation.

We know that there are a number of potential shortcomings in our study. Nevertheless, the survey confirm what kind of useful information about substandard medical care we can gather also from this type of documentation. With no patient lawyer or representative (4) present in the hospital, and with claim management firmly controlled by the medical experts, who wishes to complain will be more motivated if the management of the claims was supported by an independent Agency to ensure objectivity before the disputes, taking on responsibility for reporting the incident to the interested organization and for feed-back to the citizen. But, in my opinion, the Italian voluntary Patients Agencies have a very weak voice.

Roberto Natangelo
Retired Physician
Cittadinanzattiva (Active Citizenship. Via Mecenate n. 25. 20138 Milan. Italy)
e-mail: roberto.natangelo @ libero.it

References

1. – Bismark MM, Brennan TA, Paterson RJ, Davis PB, Studdert DM. Relationship between complaints and quality of care in New Zealand: a descriptive analysis of complainants and non-complainants following adverse events. Qual Saf Health Care 2006;15:17-22.

2. – Convegno: “La comunicazione pubblica in sanità” (Meeting: Public communication in health service) Associazione Stampa Medica Italiana. Atti interventi. Roma 6-10 maggio 2002.

3. - Wilson RM , Runciman WB, Gibberd RW, Harrison BT, Newby L, Hamilton JD. The quality in Australian Health Care Study. Med J Aust 1995;153:458- 471.

4. - Entwistle VA, Andrew JE, Emslie MJ, Walker KA, Dorrian C, Angus VC, Conniff AO. Public opinion on systems for feeding back views to the National Health Service. Qual Saf Health Care 2003;12:435-442.

Dear Editor,

In their recent study, Shojana et al highlight the importance of necropsy to clinical care by demonstrating how diagnostic sensitivity for three conditions is overestimated without necropsy results.[1] This study prompted an editorial by Guly calling for more research to demonstrate that increasing necropsy rates can improve patient care.[2]

Clearly, the evidence establishing the value of necropsy for identifying diagnostic and management issues relevant to patient care[3] is not preventing the international decline in the number of hospital necropsies. Thus, we support Guly’s petition for more evidence and describe our efforts to improve communication between pathologists and clinicians to facilitate such research.

At the Victorian Institute of Forensic Medicine, forensic necropsies are conducted on approximately 80% of hospital deaths investigated by the Coroner’s Office in Victoria, Australia.[4] A significant barrier to using the lessons of forensic necropsy for the improvement of clinical care is the lack of communication channels between Coroners and clinicians. The Clinical Liaison Service, which is the medical investigation unit assisting the State Coroners Office in Victoria, attempts to bridge the gap between Coroners and clinicians.

Established in 2002, the Clinical Liaison Service reviews the hospital care of the deaths reported to the Coroner. This unit developed a standardized review process that integrates the necropsy results with the review of medical records to identify potential system failures in clinical practice. The review process includes a multi-disciplinary discussion with a Coroner, forensic pathologist, clinicians and coronial staff to determine which issues, if any, should be investigated further for the goal of system improvement and death prevention. Approximately 2,000 hospital deaths have been reviewed by the Clinical Liaison Service and twenty-five percent of these have undergone review at the multi- disciplinary discussion.

At the conclusion of the investigation the Coroner makes a formal legal finding, that includes the issues of concern and recommendations to improve healthcare practice. As the Coroner’s recommendations are not always widely distributed,[5] the Clinical Liaison Service provides feedback to hospital staff to improve health professionals understanding of cases with patient safety implications. This feedback includes face to face presentations and a synopsis of noteworthy cases in the unit’s quarterly publication the Coronial Communiqué.[6]

As the work by Shojana et al. shows, necropsy results have the capacity to impact clinical practice far more broadly than at the individual case level alone. In Victoria, a national database, the National Coroners Information System (NCIS), has been established to provide a national repository of information about each Coroner’s case, including the forensic necropsy report.

It is vital that health researchers and clinicians consider the lessons from necropsy results in individual cases as well as in an aggregated form. Furthermore, their resulting information must be communicated widely or many valuable lessons may be overlooked.

References

1. Shojania KG, Burton EC, McDonald KM, et al.Overestimation of clinical diagnostic performance rates caused by low necropsy rates. Qual Saf Health Care 2005;14:408–13.

2. Guly H. More necropsies will improve patient care: has the case been made? Qual Saf Health Care. 2005 Dec;14(6):397.

3. Darok M, Gatternig R, Mannweiler S. Late complications after medical treatment--malpractice or fate? Med Law. 2004;23(3):489-94.

4. Emmett SL, Ibrahim JE, Charles A, Ranson DL. Coronial autopsies: a rising tide of objections. Med J Aust. 2004 Aug 2;181(3):173.

5. Bugeja L, Ranson D. Coroners' recommendations: a lost opportunity. J Law Med. 2005 Nov;13(2):173-5.

6. Coronial Communique (http://www.vifm.org/communique.html)