Expertise | The level of participant expertise was classified as novice (ie, medical students), intermediate (ie, residents, fellows) or expert (ie, specialists, faculty). |
Intervention characteristics |
Intervention type | Interventions were placed in one of four categories: checklists, computerised decision support systems, instructions at test or guided reflection. Instructions at test were defined as interventions that instruct participants to use a certain reasoning strategy, where the instructions are provided together with the cases on which performance is measured, that is, the test cases. Additionally, interventions were classified based on the focus of their items. Interventions could be process focused (ie, the intervention was applicable to any task and supported participants’ general diagnostic process, such as a debiasing checklist) or content specific (ie, the intervention focused on the steps taken in a specific diagnostic task, such as ECG diagnosis). |
Intervention moment | Interventions were classified on whether the tool was used during initial diagnosis or to verify the initial diagnosis afterwards. |
Intervention items | Interventions were classified as either tools that only required the participant to read, but not respond to the items (ie, acknowledge) or tools that required the participant to respond to the items (ie, report). |
Study characteristics |
Case difficulty | Case difficulty was classified as either simple or complex. This characteristic was only recorded if the authors specifically reported the intended difficulty of their case sample. |
Diagnostic task | The diagnostic task in a study was classified as patient diagnosis (either standardised or real patients), visual diagnosis (eg, ECG, radiograph, dermatology diagnosis) or written case diagnosis. |
Same cases used with and without intervention | This variable was recorded for a study if participants had the opportunity to diagnose the same cases before and after the intervention was implemented, and the control group did not get this opportunity. Seeing the same cases could give participants more opportunities for considering the case and could therefore make it difficult to ascribe improvements in accuracy to the intervention tool, instead of to the intervention of simply revisiting a case. |
Study intention | Study intention was recorded as stated in the study aim. Studies were classified as either having the goal to evaluate the performance of a cognitive reasoning tool or to induce errors in participants and evaluate whether the tool could fix these induced errors. |