Diagnostic errors in medicine are common, harmful to patients, and costly.^1,2 Yet such errors remain understudied,^3–5 perhaps because their causes are subtle and solutions less obvious than for other types of errors. Making a correct diagnosis is critical to the practice of medicine. Case discussions have recently been used effectively to illustrate two important approaches to understanding and preventing diagnostic errors—namely, a systems approach⁶ and a cognitive approach.⁷ We present a real case and discuss how situational awareness, a framework used in aviation to understand causes of errors,⁸ can unify these two approaches to diagnostic errors and lead to a more parsimonious and useful model for understanding them.

Situational awareness (SA) is defined as a person’s perception of the elements in the environment within a volume of space and time, the comprehension of their meaning, and the projection of their status in the near future.⁹ In essence, SA is a shared understanding of “what’s going on” and “what is likely to happen next”. It is a critical concept in any field that involves complex and dynamic systems where safety is a priority, such as aviation.¹⁰ Indeed, problems with SA have been found to be a leading cause of aviation mishaps. In a study of accidents among major airlines, 88% of those involving human errors could be attributed to problems with SA.¹¹ In medicine we believe the construct of SA is especially useful because of the potential to combine theories from cognitive disciplines^5,7,12,13 and the systems approach,^6,14 and to derive innovative approaches to the prevention of diagnostic errors.

APPROACHES TO UNDERSTANDING DIAGNOSTIC ERRORS IN MEDICINE

The systems approach attributes diagnostic errors to breakdowns in delivery or function, such as the follow up of abnormal or critical laboratory and imaging results, productivity pressures, discontinuous care and handoffs, weakly standardized processes and policies, poor communicating systems within organizations, and lack of essential patient relevant information.^4,6,14–17 Even well trained individuals are at risk of making serious errors while working in poorly designed systems. The re-designing of systems has therefore been suggested to facilitate physicians’ clinical decision making processes and to provide them with a fail-safe work environment and a reliable organizational structure to minimize diagnostic errors.^4–6 A second approach comes from cognitive psychology—the study of how physicians reason, formulate judgments, and make diagnostic decisions.^7,12,18 The process of making a diagnosis consists of data gathering, data synthesis, and data verification, and each of these stages may be prone to error.^5,19 Diagnostic errors can hence arise from failures in perception, failed heuristics (shortcuts in reasoning or “rules of thumb”), biases, and other mental states (such as affect) that are called cognitive or affective dispositions to respond (CDRs or ADRs).¹³ This approach emphasizes that we can reduce diagnostic errors by designing strategies based on analysis of CDRs and ADRs.

In a recent study of 93 diagnostic errors by Graber et al,¹⁵ 548 different system related or cognitive factors (5.9 per case) were found. System related factors contributed to the diagnostic error in 65% of the cases and cognitive factors in 74%, highlighting the multifactorial origin of such errors.

AN AVIATION APPROACH TO DIAGNOSIS?

Similar to pilots in dynamic flight environments, clinicians make complicated clinical decisions with available information while facing demands and distractions from the environment. Endsley’s situational awareness model from aviation shown in fig 1 proposes three levels of situational awareness and the task/system and individual/cognitive factors that affect this.²⁰ These levels have been applied to clinical decision making in anesthesia by Wright et al,²¹ but can also be applied to medicine in general.

• Level 1 SA involves perception of information and cues from the environment. Although no interpretation or integration of data occurs at this stage, the chances of forming an incorrect picture of the situation increase without this basic perception.²⁰ Jones and Endsley found that 76% of SA errors in pilots could be traced to perceptions of needed information.⁸ Translating to medicine, this portrays “What are the current facts relevant to this case?”. In medicine this is primarily the process of getting information via history, physical examination, and diagnostic tests.

• Level 2 SA involves comprehension of the situation and the way an individual combines, interprets, stores and retains information. This level includes more than receiving or attending to information. It encompasses the integration of information and a determination of relevance to the person’s goals and objectives.²⁰ In medicine, comprehension is essential to understand the significance of information obtained from the history, physical examination, laboratory, and imaging findings. Comprehension leads to an accurate picture of “What is going on?” and to the ability to make judgments about the patient. Less skilled individuals (such as students or residents) may be able to collect relevant data and therefore have good level 1 SA, but often they cannot comprehend all the data and may have a lower level 2 SA.

• Level 3 SA is the ability to forecast future events and dynamics and is the highest level of understanding of the situation. The accuracy of this prediction depends on the precision gained in the above two levels of SA. This level is considered a characteristic of a skilled expert and translating to medicine would be a “What is most likely to happen if?” scenario.

McGuiness and Foy²² proposed a fourth level to this model—that of Resolution, a level which provides awareness of the best path to follow from several available paths to achieve the needed outcome in the situation and, translating to patient care, signifies “What exactly shall I do?”

SA incorporates the cognitive approach (individual perception, clinical decision making and information processing) and the systems approach (task/systems influences on individual including system properties such as complexity of data presented, automation, stress and workload issues, interface design). Such an approach gives an innovative opportunity to understand and address diagnostic errors and improve patient safety.

THE CASE

A 66 year old man with a history of prostate cancer treated with radiation presented to his primary care physician for a routine follow up appointment. He complained of neck pain and spasms for a few weeks. An antispasmodic was prescribed and no further history and examination were documented. Three months later he presented for a second clinic visit. He now complained of worsening pain in the lower part of the neck with radiation down both arms that was accompanied by arm weakness. The notes documented that he had difficulty carrying grocery bags and dated this to several months previously. A neurological examination showed weakness of both upper extremities. A diagnosis of cervical radiculopathy was made and cervical spine radiographs were ordered. The patient was advised to return for follow up in 3 weeks.

Complaints of neck pain are common in the outpatient setting and the majority are related to benign conditions such as muscular strain and arthritis. To arrive at a diagnosis, physicians often use heuristics—which are mental shortcuts in clinical decision making that often are correct.^7,23 In this case the physician initially used a pattern recognition heuristic and assumed a benign condition of the neck at the first visit. Given that the patient had a history of prostate cancer (a malignancy that often metastasizes to the spine), the physician should have documented additional information about the history and examination.²⁴ Our evaluation of this case is limited by having only the medical record to review, but applying the SA model to the first clinic visit illustrates a failure in level 1 SA. It appears that the physician did not perceive all the relevant information. This level 1 failure affects all subsequent levels of SA.

Ideally, the best time to assess a clinician’s decision making would be soon after the decision is made, thereby lessening recall or hindsight bias.²⁵ The tool that Endsley suggests as the best method to directly measure SA is called the Situation Awareness Global Assessment Technique (SAGAT).²⁶ SAGAT provides an objective measure of SA by directly comparing the subject’s self-reported SA with “reality”, as determined by the observer/investigator. This measurement is carried out in a simulated setting where a simulation is “frozen” at several points in time and subjects are asked specific questions to assess their level 1, level 2, and level 3 SA. This procedure overcomes the problems associated with hindsight bias and could potentially be applied to better understand the making of a diagnosis as it unfolds. Measures of SA have correlated with performance in aviation,²⁷ and similar association will need to be demonstrated in medicine before its applicability.

Returning to the case, at the second clinic visit level 1 SA appeared compromised as indicated by the clinician’s failure to record information about bowel/bladder symptoms and other neurological findings such as sensory level and reflexes. The clinician had a greater degree of level 1 SA than in the previous visit because the relevant clinical information about the prostate cancer history and some of the neurological symptoms and signs were documented in the progress note. However, the integration of this information led him to believe that this was a case of radiculopathy, a compression of a nerve root in the cervical spine, rather than to suspect spinal cord compression by a tumor. The clinician presumably did not fully comprehend and integrate all the available data and therefore did not consider cord compression due to metastatic prostate cancer. This is an example of a loss of level 2 SA.

Because of a loss of SA at level 2, forecasting future events and dynamics (consequences of a metastatic process leading to spinal cord compression syndrome in a patient with cancer who presents with back pain and neurological symptoms) was clearly lacking and thus SA at level 3 was also lost. This resulted in an incorrect resolution path which involved ordering radiographs and arranging for a future outpatient appointment rather than ordering an MRI scan, perhaps with admission to hospital.

A strength of the SA model is that it also includes system factors (see fig 1). For example, the loss of SA levels 1 and 2 may have been impacted by lack of time, excessive workload, interruptions, and information overload. The clinician in this case was working in an office that typically sees 9–10 patients during a half day session, and most of the patients have multiple medical problems.

The case continued …

A cervical radiograph was requested and the reason given by the primary care physician to the radiologist on the request form was “pain”. The radiologist, who was unaware of the other clinical findings of the patient, interpreted the radiograph as increased density in C6 (cervical vertebra) with a possibility of blastic bone metastases, and recommended a nuclear bone scan correlation. He personally communicated these findings to the primary care physician on the following day, who then called the patient to return for a scan.

Information given to the radiologist on the radiograph request form was only that of “pain”, which by itself conveys ambiguous information about a patient who has overt neurological findings. In order to provide optimal interpretations of radiological studies, radiologists must receive correct and pertinent clinical information. Although the test of choice in this case was an MRI scan, the clinician followed the radiologist’s interpretation (which was not based on the clinical findings) to obtain a bone scan, a test perhaps appropriate to confirm metastasis only if there was no ongoing neurological compromise. The radiologist’s effort to communicate the findings directly is laudable given the seriousness of the report, but he attained only partial level 1 SA. He never really understood the situation about ongoing neurological compromise. This could be due partly to poor radiology ordering systems leading to poor communication and information transfer, another example of the impact of systems factors on SA and diagnostic errors. Errors in communication (here due to inadequate communication regarding the past medical history and indication for the imaging procedure), which are well known as causes of mishaps in aviation, are also a very common reason for preventable adverse events and medical errors.²⁸

The case continued …

When the patient returned he was noted to have worsening leg weakness and unsteadiness. Concerned about metastatic prostate cancer, the primary care physician communicated with a consulting urologist. Although it is not clear what information was relayed to the consultant, the advice was to follow up in the urology clinic in 3 days. The patient was then sent for the bone scan. However, while having the scan, he developed worsening neck pain with left arm and leg numbness and paralysis while lying on the table with his neck extended. He was sent to the emergency room (ER) for immediate evaluation. Documentation by the ER physician revealed weakness, absence of bowel and bladder incontinence, and acknowledged the prostate cancer. After review of the case, an impression of “C spine radiculopathy possibly from blastic lesions” was made by the ER physician. He discussed the case with a neurosurgical consultant over the phone and a decision was made for an outpatient MRI scan and follow up in the neurosurgery clinic in 7 days. He was then discharged from the ER.

Using cognitive science to explain these chains of events illustrates another shortcut in reasoning called the “framing effect”. Using this effect, clinicians can arrive at different decisions depending on how the information was presented, or framed.⁷ For example, the primary care physician most likely presented the patient to other physicians as a case of “radiculopathy” rather than spinal cord compression due to metastasis. All the physicians framed their diagnosis into mental models (mental representations of external reality that play a major part in cognition) without considering alternative explanations. Once again, the aviation framework can be applied to this situation. Achieving SA depends on the use of mental models and pattern matching using cues from the environment. In addition to the primary care physician, three other physicians experienced problems with gaining SA because they were fixated on one cue or diagnosis (cervical radiculopathy) and ignored others (cord compression). Because they were unable to integrate adequately a correct comprehension of the situation (level 2), their prediction of possible outcomes and the resultant treatment path was incorrect (levels 3 and 4). In this instance, a lack of gaining SA (versus losing it) resulted in an inability to appreciate the severity of the patient’s problem and an alternative and incorrect path to approach the case. A lack of gaining SA could be due to workload issues in surgery and ER medicine, communication failures, time pressures, and other system issues. However, other cognitive and affective influences such as bias, conscious or unconscious complacency with a false sense of security, negative affective states from fatigue and burn out from being on call could all potentially cause failure to gain SA.

SA can also be applied to medical teams,²⁰ including those in ambulatory care where the team concept traverses several disciplines and locations and is hence more “distributed”. Team dynamics played a critical role in this case where the entire physician team lost their SA. Effective team SA depends on team members developing accurate expectations for team performance through “shared mental models” that allow for better team process and performance among them.^29,30 SA has been identified as an important team skill that can be used to decrease human error.³¹ To gain SA, each team member must adopt an attitude called “aggressive scepticism” in order to constantly evaluate the environment.³² In this case the use of aggressive scepticism would have led to more independent evaluations of the case by each consultant, or more information sharing and questioning among the consultants. Although none of the consultants personally evaluated the patient (which by itself was detrimental to their SA), enough information seemed available to them to interject with their own perspective. In fact, combining the perspectives of all team members may be the most reliable way to remain fully aware of the environment (and thus of a patient’s condition), leading to a high level of team SA. If one of the clinicians had raised a red flag for cord compression and alerted the others, both team and individual SA would have been recovered. Breakdown in team SA in this scenario could be partly responsible for the delayed diagnosis. This was probably due to both cognitive breakdowns and lack of good communication systems in the healthcare setting, in a background of other system inadequacies.³³

The case continued …

The next day the patient was brought in a wheelchair by the family with worsening weakness and was noted to have bladder incontinence. He was admitted to hospital and had an MRI scan which revealed that he had metastatic spinal cord compression at multiple cervical levels.

DISCUSSION

Recent literature highlights the choice of either a cognitive or systems framework to guide analysis and interpretation of diagnostic errors. This case discussion illustrates how the application of the SA model from human factors research in aviation combines the system and cognitive issues, thereby providing a richer understanding of diagnostic errors. This insight can then be used to identify ways to improve and maintain SA and offers a new foundation for interventions to prevent diagnostic errors in medicine.

The SA model has several strengths. Firstly, it recognizes the co-existing roles of systems and cognitive errors. Secondly, techniques such as Situational Awareness Global Assessment help to minimize hindsight bias in understanding clinical decision making. Using high fidelity simulators, the process of care can be suspended to allow investigators to interview diagnosticians and understand exactly why SA is being compromised. This has proved to be a powerful tool in air traffic control, but will certainly need modification for application to medicine. Application of such simulation techniques for research and training of anesthesiologists has already begun.^21,34–36

Thirdly, ongoing research in aviation and other industries can provide the groundwork to identify the types of interventions that can be translated to medicine. Research in this area can potentially identify cognitive constructs, mental models, and other individual determinants that facilitate or impede SA leading to strategies for training clinicians in acquiring, maintaining, and recovering SA. Once the reasons for loss of SA are identified, we can design interventions and changes in systems to lessen the likelihood of diagnostic errors. In aviation research, Endsley has suggested several system design characteristics that should try to support and enhance SA.³⁷

Two areas for intervention are likely to be fruitful in the future: improving the design of medical devices or instruments and improving teamwork and communication. The SAGAT can be used to measure how variations in the design of data displays²¹ or an instrument or a clinical decision support tool can influence SA. Physicians could be asked questions about perception, comprehension, and prediction of situations to assess their SA in different simulated settings. Given the increasing use of information technology to improve patient safety, such studies would be valuable to identify programs, processes, or tools with more optimal designs. Secondly, breakdowns in communication are probably a common cause of loss of SA in health care. As noted above, teams can lose SA and this can also be analyzed using SAGAT in a team simulator. Principles of crew resource management can then be applied to train practitioners to communicate better in teams.^38,39 Improvements in the commercial aviation safety record may be due in part to this training, which promotes staff assertion, briefings, and SA among team members to improve communication, efficiency, and effectiveness. The role for such team training is now being increasingly recognized in medicine.^38,40–42

Other approaches that combine systems and cognitive perspectives to understand medical errors are being recognized in the field of medical informatics. Proponents of the sociotechnical approach believe that understanding information technology requires a focus on the interrelation between technology and its social environment.⁴³ They have described and interpreted errors that result from the implementation of patient care information systems from different social science perspectives (information science, sociology, and cognitive science).⁴³ A comprehensive analysis method to study origins of health information technology related medical errors has also been developed by Horsky and colleagues.⁴⁴ While these approaches can help us to understand technology mediated decision making, they may be limited in understanding the cognitive processes active in diagnostic reasoning and decision making that interact not only with technology but also with patients, colleagues, and other components of a healthcare system.

The SA framework is unique in this regard and has several other distinctive features. A person’s SA is critical to the success of a decision process in any dynamic real world setting such as aviation or medicine. As illustrated above, the four levels of SA (perception, comprehension, projection, and resolution) also correspond closely to the steps in making a clinical diagnosis. In addition, there is no comprehensive theory of medical diagnostic problem solving that addresses both the system and cognitive origin of diagnostic errors to offer solutions. Research using this framework may therefore be a noteworthy step in this direction if some of the limitations are overcome.

Our assessment may have some limitations. In many fields, such as aviation, application of the SA approach usually involves a “gold standard” of what should be done in a certain circumstance. However, in many clinical situations a number of possible routes may be taken to a correct diagnosis and there are many uncertainties involved. For example, this framework may be better suited for use in anesthesiology where the essential variables needed for making decisions are continuously available from monitoring devices and the combination of values in a patient’s current state can be extrapolated to predict a future state. This may be a limitation of the application of SA to diagnostic decision making and will need further study. We believe that one of the next steps in investigating the usefulness of the SA approach is to apply it prospectively to a series of case reviews.