Article Text
PDF
Statistics from Altmetric.com
Introduction
To improve their diagnosis and management skills, clinicians need consistent, timely and accurate feedback. Feedback helps clinicians become better calibrated, leading to more appropriate clinical decisions. Miscalibration—when clinicians’ confidence in the accuracy of their decisions does not align with their actual accuracy—may lead to overconfidence and diagnostic error.1 Consistent structured feedback leads to improved outcomes such as accurate diagnosis of acute chest pain,2 improved prehospital emergency care3 and lower costs of hospitalisation.4
Despite its benefits, significant gaps exist in delivering feedback to clinicians. In particular, clinicians do not consistently receive patient outcome feedback, that is, information on the subsequent clinical course and outcomes of patients that they have diagnosed and treated. For example, emergency department (ED) physicians in Canada reported receiving outcome feedback on only 15% of cases they encountered.5 Among US internal medicine residents, 58% reported almost never or only sometimes ultimately learning about their patients’ outcomes.6 Scientific knowledge on how to effectively provide clinicians with patient outcome feedback is underdeveloped. In contrast with traditional audit and feedback where clinicians receive aggregated metrics of clinical performance compared against explicit standards,7 patient outcome feedback provides clinicians with objective clinical information on their patient’s eventual diagnoses, treatment and clinical course. Rather than conferring black-and-white external judgements, patient outcome feedback provides clinicians with narratives regarding what happened to the patients they treated, which are crucial data for self-evaluation of clinical performance.
Developing effective feedback pathways is difficult. Fragmented healthcare systems with organisational and regulatory barriers, such as seen in the USA, make feedback-related information flow challenging. Even integrated healthcare networks such as the US Veterans Affairs health system8 and those in other countries, like the UK, Australia and Canada, only have sporadic disease-specific feedback programmes.9 To our knowledge, none of these systems have created universal fail–safe patient outcome feedback pathways to account for the numerous patient transitions between primary and subspecialty care, outpatient and inpatient care, general wards and intensive care units, medical and surgical care, and community hospitals and tertiary referral centres. Creating these feedback pathways presents significant logistical hurdles.
This paper discusses challenges to the development of systems for effective patient outcome feedback and proposes the application of a sociotechnical approach using health information technology (IT) to support the implementation of such systems. The concepts discussed herein are applicable not only to fragmented systems of care but also integrated health systems that plan to leverage the benefits of integration for providing effective clinician feedback.
Barriers to effective clinician feedback
To account for patients moving through a space-time continuum of care, we designate past care as ‘upstream’ and future care as ‘downstream’. Feedback on patient outcomes flows from a downstream source back upstream to a recipient. While we focus on clinicians as feedback recipients, feedback can be generated by any member of the downstream multidisciplinary team who cares for the patient (eg, specialist consultant, inpatient physician, practice colleague, care coordinator, social worker) and may even include the patient10 (figure 1). This conceptual approach is broad and is an extension of traditional handoffs during common patient transitions.
Communication pathways for health information technology (IT)-supported patient outcome feedback. A patient initially diagnosed and/or treated by a clinician (upstream) subsequently undergoes care by other clinicians/care teams (downstream) who now have new knowledge of the patient’s clinical course and outcomes. This information is fed back to upstream clinician from a member of the downstream care team and/or the patient. A health IT-supported system facilitates this feedback and additional two-way communication that may result.
Feedback should be accurate, specific and concise, but to be most effective, it should be delivered and displayed in a manner that is timely, consistent, reduces cognitive load and encourages further dialogue.11 Structural, psychological and interpersonal barriers to effective feedback communication exist between clinicians across healthcare settings.6 12–15 Some barriers are more common in fragmented healthcare systems while others are universal (table 1).
Barriers to effective clinician–patient outcome feedback that can be addressed by health information technology (IT)
Decentralised systems of care present structural challenges to providing routine patient outcome feedback to clinicians, mostly because patient care and data are organised in silos lacking the necessary processes and infrastructure for sharing information. Data transfer may be tightly controlled not only by state/institutional regulators but also by developers of health IT systems. Information blocking constitutes practices that interfere with access, exchange or use of electronic health information16 and can be a significant barrier to clinician feedback. In recent years, efforts have been made in the USA and other developed nations such as countries in the European Union, the UK, Canada, Australia, New Zealand and South Korea to make information systems interoperable across organisations, resulting in the creation of health information exchanges. However, usage and usability of these exchanges remain low internationally.17 In the USA, electronic health record (EHR) vendors have created networks where different hospital systems using the same EHR can share patient information (eg, Epic Care Everywhere), however these networks have not been specifically used to support systematic patient outcome feedback.
Certain barriers to feedback are common to all healthcare systems including integrated systems,8 simply because feedback involves human communication. Obtaining and providing feedback requires time and effort from both upstream clinicians and downstream sources. Most downstream clinicians lack the necessary administrative resources and time to track patients whose outcomes need to be communicated to upstream clinicians, much less to compose and deliver the feedback.6 Many clinicians may also fear professional judgement and conflict, be reluctant to be viewed as assigning blame and worried about the discoverability of diagnostic discrepancies which may have medicolegal repercussions. Other clinicians may also perceive that feedback to colleagues is ineffective or irrelevant.15 Meanwhile, upstream clinicians often feel that the feedback they do receive is suboptimal. Feedback may be delivered to the wrong clinician or may be delivered late, making it difficult to remember clinical decisions.12 There are also no clear standards for the content, format and manner of feedback delivery appropriate for specific clinical settings, thus feedback may be less effective.18
A sociotechnical approach using health IT to support effective clinician feedback
The current health IT infrastructure supporting care delivery in many countries can be harnessed to develop and sustain systematic processes to bridge feedback-related gaps between clinicians. In this section, we discuss how health IT, specifically the EHR, can be used to establish a feedback process that can both overcome barriers to feedback and be used to evaluate the feedback system itself. However, harnessing the benefits of health IT in the delivery of consistent patient outcome feedback to clinicians requires accounting for the complex adaptive sociotechnical healthcare system composed of clinician-users, their tasks/workflow, environment, organisations and technology as well the interactions between them. We thus also propose a set of essential sociotechnical considerations that need to be addressed for effective implementation of a health IT-supported feedback process.
Overcoming barriers to feedback using health IT
Health IT facilitates communication among clinicians and has the potential to break down practice silos within and between healthcare organisations. Table 1 shows the many ways in which health IT can address barriers to feedback.
Structural barriers, such as challenges in keeping track of upstream clinicians and their contact information, being reminded when feedback is due, having a standard format and method of communication and maintaining data security, can be addressed directly by health IT, facilitating timely, consistent, clear, accurate and secure feedback communication. Psychological and interpersonal barriers are more difficult to overcome and may not be addressed directly by health IT. However, having a consistent and reliable feedback system will generate learning and may itself be useful in changing the culture around feedback.19 Receiving feedback in a standard health IT-enabled format containing objective outcome information can help clinicians evaluate their own clinical decisions, promoting self-awareness and learning. Asynchronous health IT-supported feedback can enable clinicians to review patient outcomes privately, reducing the need for direct encounters that may be perceived as confrontational. Among US resident physicians, structured written feedback including diagnosis comparisons outlining reasons for any changes in diagnosis after care transitions was viewed positively and noted to contribute to improving clinical practice and identifying system vulnerabilities.20 Among German primary care physicians, patient outcome feedback resulted in self-reflection and resolve to improve analytical reasoning and change clinical practice.21
Consistently relaying patient outcomes to upstream clinicians also eliminates reliance on individual motivations to seek feedback.13 If feedback becomes the norm across healthcare settings, that is, ‘this is just the way we do things around here,’ clinicians will communicate more freely in other ways (eg, phone calls and joint case conferences to discuss shared patients), eventually affecting clinician perceptions of feedback and deepening mutual trust. This would reduce fears of professional conflict and medicolegal implications. In a US feedback programme between a tertiary referral paediatric ED and community EDs, the downstream ED worked together with upstream EDs to modify feedback delivery processes to allay fears about medicolegal risks (eg, using secure electronic messaging).22
Evaluating the feedback process and responding to feedback using health IT
Health IT can also support efforts to evaluate the feedback process itself by tracking and aggregating data on feedback reports created, delivered, received and viewed. These data will help clinicians and administrators determine whether the feedback process is functioning well and whether improvements need to be made. For instance, direct communication between clinicians prompted by feedback reports can be tracked and provide some evidence that feedback reports are being responded to by recipients.
To translate feedback into action and improvements, health IT-supported feedback systems require oversight and collaboration from clinical and administrative leaders on both sides of the feedback loop. A recent international systematic review showed that leaders need to gather and interpret data generated by the feedback process, encourage discussion of feedback information among their multidisciplinary staff, identify data trends that require system-wide improvements, and ensure that any medical errors uncovered are routed through the organisation’s usual safety/quality improvement process.23 Leaders also need to harness organisational resources to support individual and group efforts to respond to feedback.
Using the EHR to create feedback loops
Although standalone health IT systems for feedback can be developed, a more immediate option is to use existing EHR systems to support feedback. One study used an EHR-generated templated feedback form delivered via the EHR’s messaging function to determine the effect of structured feedback to resident trainees who admitted patients overnight.20 After the intervention, residents indicated improved comfort with sharing feedback and self-efficacy in identifying cognitive biases.
Commonly used EHR platforms can already automate several steps in the feedback process. For example, EHR platforms can: (1) generate patient lists based on prespecified criteria, (2) compile a database of upstream clinicians and their contact information, (3) create templated forms/letters, (4) insert data into forms/letters from patient records, (5) support secure electronic messaging between clinicians, and (6) track delivery/receipt of messages and access of patient records.24 These functions can be readily adapted to implement and evaluate a formal feedback process within a healthcare organisation.
However, orchestrating an effective and sustainable feedback process cannot be accomplished merely by stringing EHR functions together. Downstream and upstream clinicians’ feedback goals and their competing tasks and responsibilities will drive implementation.11 Feedback processes must minimise additional burden by being well integrated into clinicians’ workflow.23 Healthcare organisations’ EHR systems, their interoperability, and the legal/policy environments in which they operate will also affect implementation and must be carefully considered.
Implementing a feedback process within a sociotechnical health system
To illustrate essential considerations for successful implementation, we use an eight-dimension sociotechnical model for studying health IT developed by authors Sittig and Singh. The eight dimensions include: hardware and software; clinical content, human–computer interface; people; workflow and communication; organisational policies and procedures; external rules, regulations and pressures; and system measurement and monitoring.25 Prior to implementing a health IT-supported feedback system, this model can comprehensively identify barriers that need to be overcome and potential factors to be leveraged for success in each dimension (table 2). This requires partnerships between downstream clinicians and their upstream colleagues early in development to ensure that the design and implementation of the feedback system will fulfil common feedback goals, be well integrated into their workflow, allow for system measurement and monitoring, and comply with internal and external regulations. Collaboration at the outset can also help allay fears of professional judgement and interpersonal conflict by setting shared goals and expectations for a constructive feedback culture.
Considerations in implementing a health information technology-supported feedback system across eight sociotechnical dimensions
For successful implementation, it is essential to consider each dimension. Basic hardware and software requirements are available in existing EHR platforms but additional innovations could facilitate more design features. Feedback content needs to be jointly decided by downstream and upstream clinicians by considering the balance between relevant information and resources required to extract and integrate data into a concise and coherent report. Highly specific feedback (eg, details of the clinical course) may be desirable but may be beyond the capabilities of the current IT platform to automatically populate reports and will be labor-intensive for clinicians to generate.
The design of the human–computer interface should be user-friendly; usability will enable clinicians to create, deliver and review feedback in a manner that eases their workload while enhancing the system’s sustainability over time. However, depending on the platform used, not all feedback system functions can be performed automatically by health IT, and thus humans will still be needed to fill gaps in the process. Designers of the feedback system will need to consider details of clinicians’ workflow so that the feedback process can be integrated seamlessly without undue burden to clinicians. Although participating in a systematic feedback process will require clinicians’ time, this is preferable to current inefficient methods and may yield improvements in practice and patient outcomes. Aside from clinicians, buy-in is needed from personnel integral to the smooth functioning of the system such as health IT experts necessary for system maintenance and troubleshooting.
Organisational policies (eg, data-sharing procedures) and external regulations (eg, patient privacy laws) will affect the feedback process; thus, support from administrators and institutional leadership, including legal counsel, will be necessary to establish institutional norms regarding feedback, ensure adherence to all applicable rules, and help assess and mitigate potential legal concerns. Implementing an EHR-supported feedback process across healthcare organisations, especially in fragmented healthcare systems, will also pose additional challenges related to lack of interoperability. Ultimately, clinicians and administrators need to work with informatics experts, EHR vendors and policymakers to solve this problem. In the short term, work-arounds may be possible through established public or private health information exchanges. Finally, strategies are needed to measure how well the system is functioning and its impact on outcomes. Certain metrics can be tracked such as reductions in unnecessary transfers or by conducting periodical surveys of clinician-users to monitor how feedback is used in practice.
Conclusions and future directions
Clinicians require effective feedback on individual patient outcomes to improve their diagnosis and management skills, but systems to support this type of feedback are underdeveloped. Health IT has the potential to overcome barriers to effective feedback in both fragmented and integrated healthcare systems. For health IT-supported patient outcome feedback systems to be effective, they must be thoughtfully integrated into the sociotechnical aspects of the healthcare environment. This will require collaboration between healthcare organisations, informatics experts, EHR vendors and policymakers. Studies are needed to identify and mitigate the challenges to feedback, including constraints related to clinician time and organisational resources. Future studies should evaluate the impact of health IT-supported patient outcome feedback systems on clinical diagnosis and patient outcomes. If significant benefits are found, such systems could be prioritised for implementation through policy changes. The proposed strategies discussed herein can support clinician participation and advance the health IT structure and processes to enable feedback for improving patient outcomes.
Ethics statements
References
Footnotes
Twitter @TinaCifra, @DeanSittig, @HardeepSinghMD
Correction notice The article has been corrected since it was published online first. The authors would like to inform that an incorrect funding statement was published which has been amended now.
Contributors All authors named attest to have participated in the conception, design, execution, and writing of the manuscript and are accountable for all aspects of the work.
Funding This paper was funded under contract HSP233201500022I/75P00119F37006 from the Agency for Healthcare Research and Quality (AHRQ), US Department of Health and Human Services. Dr Cifra is also supported by the AHRQ through a K08 grant (HS026965) and an internal start-up grant from the University of Iowa Carver College of Medicine Department of Pediatrics. Dr Singh is funded in part by the Houston VA Health Services Research and Development (HSR&D) Center for Innovations in Quality, Effectiveness, and Safety (CIN13-413), the VA HSR&D Service (IIR17-127) and the Presidential Early Career Award for Scientists and Engineers (USA 14-274), the VA National Center for Patient Safety, AHRQ (R01HS27363), and the Gordon and Betty Moore Foundation. The content is solely the responsibility of the authors and does not necessarily represent the official views of the AHRQ or VA.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.