What is risk and how does it differ from uncertainty?

Roulette serves as a classic example for risk: unlike any clinical situation, all possible outcomes are known, as are their respective probabilities. You only have to decide how much risk you can tolerate. Whether you are an experienced emergency physician or not, the unfortunate gambling fellow from above poses a fundamentally different challenge—an uncertain situation. You can only observe the patient’s obvious cardiac risk factors obesity and smoking and apply some general knowledge of the several life-threatening conditions that may cause chest pain and collapse, including, for instance, not just the obvious possibility of a heart attack, but also a blood clot in the lungs or possibly a dissection of the aorta. In addition to these usual medical causes to consider, however, there is also the possibility he may just be acting to avoid paying his debt.

In contrast to roulette, in uncertain situations we do not know all possible outcomes and their respective probabilities. We also do not know how the likelihoods of different outcomes change when more information becomes available. In fact, likelihoods quickly become non-calculable in clinical practice. For instance, an experienced physician will be aware that chest pain in itself has a positive predictive value of only around 20% for a heart attack and may also be aware that obesity doubles the risk for coronary artery disease. However, calculating the three most likely causes of collapse for a given patient such as the elderly obese smoker with chest pain is already next to impossible. Uncertainty is thus characterised by incomplete knowledge about the potential outcomes (there could be diagnoses we did not think of) and complex relationships between cues and criterion.2 Your profession and professional experience may change the degree of uncertainty the collapsing gambler poses to you, for example, for a medical doctor versus somebody without medical knowledge, but even for medical doctors it is an uncertain situation because they encounter it without the accompanying information and diagnostic possibilities they would have in a hospital situation.

Emergency medicine (EM), where Lawton and colleagues place their study,1 is a particularly uncertain—or ill-defined—environment. Decisions in EM often have to be based on incomplete and unreliable information,3 4 are made under time pressure and with potentially devastating consequences.4–6 However, the degree of uncertainty any environment poses is by no means independent of one’s experience in it: humans are outstanding in identifying among the myriad of cues available in any given situation, those which best predict the criterion of interest and then combining the few important cues into simple decision rules (‘heuristics’).7 8

This mechanism allows people to ‘become an expert by making routine what to the novice requires creative problem-solving ability.’9 It does, however, require extensive exposure to the respective environment to develop. The more experience one has with a given situation, the better one will be able to foresee its future course.10 Thus, experience within a given field reduces uncertainty in that field. It is consequently of little surprise that physicians with limited EM experience tend to take seemingly safer decisions, whereas more experienced physicians seem to tolerate more uncertainty as found by Lawton and colleagues. Instead of explaining this phenomenon as a tolerance of uncertainty increasing with experience as Lawton and colleagues conclude, we would argue that the EM environment simply becomes less uncertain over time. Physicians who have been practising in EM for longer increasingly regard familiar problems as less uncertain—that is, it is not that they tolerate uncertainty better so much as they have less uncertainty to tolerate.

Making decisions while acknowledging uncertainty

A second question is how to define tolerance of uncertainty. The literature lacks a clear definition.11 Lawton et al operationalise tolerance of uncertainty simply as allowing for more risky decisions in a well-justified effort to reduce the number of diagnostic tests.1 And indeed, while junior physicians often aim for certainty as a prerequisite for action, more experienced physicians are more confident in the appropriateness of their actions even in situations where they do not (yet) fully understand a patient’s problem.12 13 Experts become more ‘comfortable with not knowing.’ Ilgen and colleagues recently reviewed the concepts of ‘comfort with uncertainty’ and suggested that skilful clinicians can be comfortable in their management plan and at the same time be not completely confident about their diagnosis.14 The reason for this is that skilful clinicians rely on their experience to be confident about having ruled out all conditions that require immediate action. They are able to be comfortable with their management plan while still being uncertain about the precise diagnosis. They then use an iterative process of pragmatic empiricism, forward planning, monitoring and ongoing reconceptualisation to find the exact diagnosis. This helps them to maintain the ability to act even in ill-defined environments that are characterised by an inherent uncertainty. Junior physicians by contrast first need to obtain experience and learn how to cope with uncertainty by identifying cues to monitor their actions.15 For example, if a patient reacts to a treatment as expected, this signals to a physician that (s)he is ‘on track’, but if the patient is not responding to treatment, physicians need to reconsider their diagnosis and ask for help because the conditions may be beyond their abilities to control or manage.14

When humans make decisions, we not only have access to the internal and external cues our decision is based on, we also monitor our cognitive operations. Such metacognition enables us to recognise how confident we are in a decision,16 thus, for instance, recognising when to slow down when we should17–19 and to over-rule first, intuitive impressions with more deliberate reasoning. The most prominent of these metacognitive impressions is fluency—the subjective impression of how easy or difficult a decision is.20 Several researchers have suggested that fluency is not an absolute judgement of the ease or difficulty of a given cognitive operation, but rather results from the difference between expected and experienced difficulty.20 Fluency does emerge from judgements and decisions, and from perception, retrieval or any other cognitive operation. Objects, for example, are fluent when we have seen them frequently or recently, memory is fluent when relied upon often. Thus, ‘the interpretation of a fluency experience relies on past experiences and the current context,’20 that is, the more exposure we have had or the more expert we are in a given context, the smaller the perceived difficulty of operating in it. We would suggest that this effect also results from learning to use the right cues to monitor whether we are on track.15 This hypothesis can and should be tested, because it could inform the ‘interventions that better support junior doctors to deal with uncertainty’ Lawton and colleagues call for.1 Interventions that help learners to identify the right cues to monitor themselves are defensible and unlikely to cause harm15 while for reasons of patient safety, it would obviously not be advisable to ask junior physicians to simply be more risk seeking when it comes to diagnostic testing. As pointed out earlier, junior physicians need more experience to be able to make good decisions.21 We argue here that junior physicians need to go through the phase where uncertainty is acknowledged, to facilitate the learning process needed to develop more confidence and to base that confidence judgement on the right cues. Ultimately, this could result in reducing the number of tests acquired on any patient. The question is whether such a learning process can be facilitated in a way that junior physicians reach the level of expert physicians faster.

Decision accuracy

This brings us to the third—but most crucial—question: what about accuracy? Regardless of how experienced we are within an uncertain environment, it is of little value if our decision is wrong. And even if a decision is right, we better be sure it is right for the right reasons, not just because it has gone well in the past.22 Otherwise, we will be wrong in a similar situation eventually. In previous research, physicians were found to be overconfident in their diagnoses,23 and experienced physicians in particular are less likely to seek advice, even when they were wrong.24 It has also been shown that confidence increases with seniority—but its calibration with accuracy not necessarily.25 Based on these results we would argue that looking only at confidence, and suggesting that higher is better, is in fact rather dangerous. A study by Friedman et al showed that medical students were more likely to get the diagnosis wrong than more experienced physicians, but were better calibrated than the more experienced physicians.26 In this study, the experienced physicians were more often not confident in their diagnosis rather than being overconfident,26 which is different from the study by Meyer et al who found a strong effect of overconfidence.24 This is only to show that more research is needed to unravel the complex relations between adequacy, accuracy and calibration in conjunction with confidence, risk management and uncertain situations even aside from experience.

Calibration is key

How do junior doctors get to the point where they can safely decide that no further tests are needed? Generally, experienced physicians are often considered to be overconfident and junior physicians, as is shown by Lawton and colleagues, are less confident. However, it could very well be the case that even experienced physicians are not confident in cases with which they have little experience, and that junior physicians are overconfident in certain situations too. In the end, what really matters is that the level of confidence is well calibrated with diagnostic accuracy. We want physicians (junior physicians and experienced physicians alike) to be more risk averse when encountering patients with symptoms they are unfamiliar with, while this might be an inappropriate course of action when they encounter a more familiar case. Good calibration is key for physicians of all levels of expertise and experience, because it will result in the right decisions. One way to improve calibration is to provide performance feedback (ie, informing physicians whether their diagnosis and/or treatment was correct or not).27 Several recent articles have pointed out the poor levels of calibration and the lack of feedback in diagnostic reasoning.28–30 Feedback is—besides experience—an important prerequisite for becoming an expert. Physicians, especially in the ED, often do not get feedback about their diagnostic performance, that is, they do not hear back about changes in the diagnosis after a patient has been admitted to the ward or was sent home. Such feedback is essential to enable physicians to recognise cues that a situation differs from the familiar situation they initially thought it was, and to decrease the potentially dangerous situation mentioned above of experienced physicians being confident in their diagnosis but still being wrong. In the field of medical education, interventions have been developed that have proven to be efficient in improving calibration of uncertainty and diagnostic performance.27 Mixed deliberate practice with feedback is what ultimately makes novices expert.31 32

In conclusion

The study of Lawton and colleagues showed that experienced physicians make less risk averse decisions than junior physicians.1 This reflects an important difference in clinical decision-making, but it only represents one piece of the puzzle. Physicians do not know when they are right or wrong, which sometimes results in overconfident decisions and other times in defensive medicine. Next steps should include studying tolerance of uncertainty in combination with overconfidence and diagnostic accuracy to determine where most miscalibration occurs. Furthermore, a focus on reducing miscalibration using strategies to enhance feedback and identifying the right cues to inform confidence in judgements are important ways to improve clinical reasoning.