Dear Editor

I share much of Dr Morris' enthusiasm for decision support. While involved in an evaluation of a decision support targeting vancomycin ordering practice,[1] I had the opportunity to observe the potential impact of this approach. However, as a clinician using the same computerized order entry system in daily practice, I also recognized the major limitation of this approach: users will not tolerate many such interventions at any given time.

Too many alarms can contribute to a sense of general noise, so that they lose their value.[2] Similarly, too many triggers for guidelines, or even alerts such as potential drug-drug interactions, will result in users clicking past all such screens, so that in the end, none of the screens will work. Forcing users to follow a given guideline or respond to a particular alert runs the risk of adding an intolerable time burden to frequent users of the system – e.g., interns and residents who write multiple orders a day.

Consider the admission orders for even a routine medical admission. Possible guidelines might relate to many medications (e.g., choices of antibiotics), diagnostic orders ('does this patient really need a KUB – the yield of plain abdominal radiographs is known to be low in most clinical situations'[3]), various prophylactic strategies (“do you want to order DVT prophylaxis?' '...stress ulcer prophylaxis?' etc.)

Elderly patients will trigger even more guidelines - does the patient need fall precautions? How about a soft matteress or other decubitus ulcer precautions? Pneumococcal vaccination prior to discharge?[4] Vlu vaccine? [4] Does the patient have an advanced directive?[5]

The list goes on, and this does not even include guidelines triggered by specific admitting diagnoses e.g., guidelines for treatment of community acquired pneumonia, acute coronary syndrome, hip fracture, gastrointestinal bleeding, stroke, etc. etc, not to mention important secondary diagnoses – 'This patient has diabetes: do you want to add an angiotensin converting enzyme inhibitor.' 'This patient has a diagnosis of congestive heart failure, but there is no record of an echocardiogram or other assessment of ejection fraction.' 'This patient is on prednisone; would you like to add a bisphosphonate to protect against osteoporosis?'

Thus, computerized systems offer a greater chance of success for implementation of a single guideline, it is unlikely that this benefit will generalize to more than a handful of such protocols at any given time. Further research will need determine optimal strategies for harnessing the potential of computerized decision support. Currently, though, it is unrealistic to think that an institution acquiring an order entry system could expect to impact practice in more than a few areas using this approach.

References

(1) Shojania KG, Yokoe D, Platt R, Fiskio J, Ma'luf N, Bates DW. Reducing vancomycin use utilizing a computer guideline: results of a randomized controlled trial. J Am Med Inform Assoc 1998;5:554-562.

(2) Cropp AJ, Woods LA, Raney D, Bredle DL. Name that tone. The proliferation of alarms in the intensive care unit. Chest 1994;105:1217- 1220.

(3) Harpole LH, Khorasani R, Fiskio J, Kuperman GJ, Bates DW. Automated evidence-based critiquing of orders for abdominal radiographs: impact on utilization and appropriateness. J Am Med Inform Assoc 1997;4:511-521.

(4) Dexter PR, Perkins S, Overhage JM, Maharry K, Kohler RB, McDonald CJ. A computerized reminder system to increase the use of preventive care for hospitalized patients. N Engl J Med 2001;345:965-970.

(5) Heffner JE, Barbieri C, Fracica P, Brown LK. Communicating do-not- resuscitate orders with a computer-based system. Arch Intern Med 1998;158:1090-1095.

Dear Editor

Dr Shojania raises important issues that must be solved before widespread implementation of many decision-support tools is possible. I appreciate his letter. Dr Shojania and I, however, have been addressing different kinds of decision-support. The tools my colleagues and I have implemented, both locally and at external sites, are explicit tools that generate specific instructions, in contrast to the suggestions to the clinician that are generated by most guidelines. Our tools are computerized protocols that could function as closed-loop instruments. (We use them in the open-loop mode with the bedside clinician always reading the instruction(s) before executing the change in therapy.)

In my opinion, the important issues raised by Dr Shojania will be most productively addressed when clear distinctions are made between different strategies of decision-support and the different tools that could be used to achieve those strategies. One important distinction is that between diagnostic and therapeutic decision-support tools.

We have limited our work since 1985 to therapeutic protocols. Within the therapeutic protocol domain, explicit computerized protocols have been successfully implemented and exported by us, for clinical trial purposes, with a clinician compliance of 95 %. We do not yet know if similar success will be found with application of these tools within clinical practice. A number of concerns with clinical practice use are apparent to us. They include the issues raised by Dr Shojania.

The Editor, BMJ Quality and Safety,

Mitchell, Schuster, Smith et al (1) present the results of semi-structured interviews with 11 international patient safety experts, 15 years after publication of the US Institute of Medicine's landmark report 'To Err Is Human'.(2) One of the Institute's recommendations was the introduction of healthcare incident reporting.

Qualitative analysis of the interviews by Mitchell et al identified 5 problems - sheer volume of reports received and inadequate report processing, lack of involvement by medical practitioners, insufficient response and feedback to reports, inadequate funding and organizational backing, and failure to use electronic records. The article stated that the interviewees noted little progress in utilizing adverse event information to positively influence patient outcome - a sobering finding.

In drawing on the opinion and experience of these national and international experts, maybe we are missing critical elements essential for utilization of incident reports. Their perspective and experience of incident reporting may have diluted or lost the value of this activity at the local healthcare facility. Taking anaesthesia as my example, reporting of adverse events and 'near misses' functions most robustly in the local department setting, where regular morbidity and mortality meetings allow free discussion of events in a 'no blame' culture. Knowledge of working conditions, staffing levels, rosters and surgical capabilities allows useful input regarding system factors involved in the incident, human factor analysis, and provision of feedback to people and the institution, in a timely fashion. Discussion of adverse events is a learning experience for everyone involved. The costs incurred by a department in the process of incident submission, analysis, discussion and feedback is minimal, as it is part of the contracted and expected duties of anaesthetic consultants and trainees. Trainees have evidence of reporting leading directly to tangible action, as closing the loop with feedback occurs soon after the incident. Anaesthesia incident reporting has become embedded in the functioning of my specialty at a departmental level, because it has addressed the identified problems - thus doctors do report, the data collected is processed, and feedback is provided in a timely fashion, the heads of department support and facilitate morbidity and mortality reporting, and minimal costs are incurred. Electronic record usage is utilised in some departments and will enhance audit and review when more readily available. Trainees are shown that safety and quality, with incident reporting, are integral parts of everyday practice, not an added extra, or afterthought.

It is important that statewide and national reporting systems [Australian examples are Victorian Consultative Council on Anaesthetic Mortality and Morbidity (3), web-based reporting systems such as webAIRS (4)] are established, to identify patterns of failures or worrying trends in care, and recommend changes in practice. However, the considerable time-lag between adverse event occurrence, and their de-identified analysis and recommendations may reduce the impact of findings. The 5 problems identified by the interviewees are seen with these larger investigative bodies.

Incident reporting is an essential part of health safety culture and climate (5), but to reap the benefits of the essential analysis and feedback components, I advocate decentralizing the process to craft groups and hospital departments. The manageable volume of reports, increased medical involvement, and feedback loops possible with smaller units would enhance learning and quality improvement.

There is much to learn from incident reporting - we must not be buried under the pile of reports. We need to act locally, in our departments, our wards, our units, to promote and demand the collection of good data. We must provide the analysis and facilitate discussion of reports, and have safer and better quality care result from the feedback given.

References

1. Mitchell I, Schuster A, Smith K et al. Patient safety incident reporting: a qualitative study of thoughts and perceptions of experts 15 years after 'To Err is Human'. BMJ Quality and Safety Published Online First: 12th August 2015 doi:10.1136/bmjqs-2015-004405 Accessed 22nd August 2015

2. Kohn L, Corrigan J, Donaldson M. To err is human: building a safer health system. Washington, DC. National Academy Press 2000.

3. http://www.health.vic.gov.au/vccamm

4. webAIRS: https://www.anztadc.net [Internet]: Accessed 7th September 2015

5. Benn J, Koutantji M, Wallace L et al. Feedback from incident reporting: information and action to improve patient safety. Qual Saf Health Care 2009;18(1):11-21

Conflict of Interest:

None declared

I was interested to read the thoughtful article by Imogen Mitchell and colleagues, echoing longstanding criticisms by Kaveh Shojania, Charles Vincent and others on the low value of high volume incident reporting. There is little learning from categorical data, repeated many times, around falls or minor medication errors. However, there was a time at the National Patient Safety Agency, where we experimented with a multidisciplinary meeting to make sense of reports, screened by skilled clinical reviewers. We concentrated on the freetext - the `story' of the incident. These weekly meetings became a dynamic and illuminating discussion of problem and solutions, with fiercely contested accounts from differing perspectives (surgeon, theatre nurse, GP, human factors or design experts). This space for reflective learning (Lamont 2011)reached places that desk-based data analysis couldn't go. The key was in the quality of the clinical reviewers, the range of clincal and safety expertise at the weekly meetings and a national function with ability to act on the risks, once articulated in this forum. It would be a shame to lose this potential for learning from a few carefully selected incidents, with the right analytic and sense-making skills and resources, in justified criticism of the limited safety gains to date from large incident reporting systems.

Tara Lamont Deputy Director NIHR Dissemination Centre

Lamont Tara. "Re-cognising" risks: from space shuttles to chest drains BMJ 2011; 343 :d4393

Conflict of Interest:

I worked at the National Patient Safety Agency 2004-2011.