Table 1

Clinical scenarios, cues and measures of evaluating appropriateness of physicians’ actions

Presented scenarioSituation cueExpected action*Background cueExpected action*
Fever case: A 43 y/o man admitted several days ago with a UTI, has a 101.5° fever and 2–3 loose stoolsPersistent fever in a hospitalised patient/patient under antibiotic treatmentEvaluation for healthcare associated or complicated infection. Asked about any of: recent hospitalisation, recent antibiotics use, other source of infection, presence of a complicated UTI or non-infectious cause of fever†Back surgery 1 month ago or prior hospitalisation until 4 days agoAny of: Broadened antibiotic coverage, or ordered imaging for epidural abscess or asked about evidence for surgical site infection
Glucose case: A 48 y/o man has a standing order for insulin but blood glucose is 90 mg/dLThere is also an order for glucose 50% (a standard Tx for high potassium levels)Evaluation for the cause of elevated potassium. Asked about any of: current medications, renal function or acid/base valuesPatient is treated with tacrolimus (may cause elevated potassium levels)Any of: Asked for tacrolimus level, reduced dose or requested a nephrological consultation
Behaviour (Confusion) case: A 19 y/o man admitted for sickle cell crisis is disoriented and pulled out his IV lineThe patient is confusedEvaluation for an acute change in mental status. Asked about any of recent opiate therapy, substance abuse or ordered any of: discontinue opiates, blood glucose, electrolytes or head CTPatient has a new low sodium level (119 mg/dL) and a persistently high WBC count (23 000)Any of: Treated low sodium, evaluated and treated for an infection
High Blood Pressure case: An 85 y/o woman has a high blood pressure of 180/90 mmHgNone†None†Home treatment with clonidine was discontinued (causes rebound high blood pressure); the patient has received large volumes of fluidsAny of: Reinstituted clonidine Tx., stopped fluid Tx. or ordered diuretic Tx
Medication case: A 31 y/o woman has difficulty sleeping and asks for a sleeping pillThe patient was admitted for acute liver injuryRefrained from benzodiazepines or asked about comorbidities (eg, respiratory compromise) or conflicting drugsThe patient is treated at home with a CPAP for obstructive sleep apnoeaRefrained from any sleep medication until CPAP treatment was reinstated
Chest Pain case: A 61 y/o woman presents with chest painThe patient has swelling of her legSuspicious for pulmonary embolism. Any of: CTA, D-dimer, V/Q scan, lower extremity venous US or dedicated examinationNone‡None‡
  • *A complete list of the evaluation criteria can be found in online supplementary appendix A.

  • †There were no relevant ‘appropriate responses’ for the situation cue of the High Blood Pressure case (the blood pressure in itself was not high enough to warrant a comprehensive evaluation).

  • ‡There were no relevant ‘appropriate responses’ for the background cue of the Chest Pain case (the patient had multiple comorbidities that could have been responsible for chest pain. Thus, we made the decision to present the background cue of leg-swelling to the nurse subjects and excluded it from the analysis).

  • CPAP, continuous positive airway pressure; CTA, CT angiogram; IV, intravenous; Tx, treatment; US, ultrasound; UTI, urinary tract infection; V/Q, ventilation/perfusion scan; WBC, white blood cell; y/o, year old.