Author (year) | Study type and participants | Intervention | Outcome measures | Results | Conclusions | Outcomes Rating | Strength of Conclusions (1–5) |
Focused decision support using decision support tools on a specific condition | |||||||
Pozen et al36 (1984) | Non-randomised control trial with ER physicians | Incorporation of decision support tool that calculates the probability of patient having ischaemic heart disease. | Diagnostic accuracy, sensitivity, specificity, FP diagnosis rate, FN diagnosis rate, CCU admission rate, ER discharge rate. | Use of tool increased diagnostic accuracy and specificity; sensitivity remained unchanged. FPs and CCU admissions decreased. FNs remained unchanged. | Use of tool has potential to increase diagnostic accuracy and reduce admission rate. | 4b | 4 |
Selker et al37 (1998) | Non-randomised control trial with attending ER physician and residents | Computerised prediction tool that calculated probability of acute ischaemia that was printed on ECG. | Accuracy of triage decisions. | Tool reduced unnecessary CCU admissions and did not change appropriate admission rates. | Use of the tool has the potential to significantly reduce unnecessary hospitalisations. | 4b | 4 |
Bogusevicius et al38 (2002) | Randomised control trial with radiologists | Use of a computer-aided diagnosis tool to diagnose acute small bowel obstruction. | Specificity, sensitivity, FP predictive value, FN predictive value, time to diagnose, mortality and morbidity. | No significant advantage over contrast radiography for diagnostic accuracy. Significantly less time required to diagnosis. | Computer-aided diagnosis improved amount of time to diagnose, but no other indices. | 4b | 3 |
De Simone et al39 (2007) | Before/after with medical personnel | Use of computer-assisted diagnosis database for diagnosis and patient management of headaches compared with standard clinical method. | Diagnostic accuracy, mean visit duration, operators' subjective opinion of tool user friendliness, patients' subjective opinion of computer-assisted interview acceptability. | Slight increase in diagnostic accuracy. Duration of clinical visit comparable. Subjects felt tool was easy to use and patients felt the tool use was acceptable. | The tool improves diagnostic accuracy without increasing visit duration. | 4a | 3 |
Focused decision support using embedded decision support tools (Infobuttons) | |||||||
Cimino40 (2006) | Survey with nurses, attendings, residents, medical students | Use of online resources through an Infobutton Manager (IM) providing direct, context-specific information. | User satisfaction regarding information returned from online resources. | Satisfaction with the IM varied. Overall users felt IM had a positive effect on patient care decisions. | Context-specific access to health knowledge resources can be seen as useful. More research is needed on the impact on patient outcomes. | 1 | 3 |
General decision support using web-enabled differential diagnosis generators | |||||||
Ramnarayan et al41 (2006) | Before/after with paediatric interns and residents | Use of an internet diagnostic decision support tool (ISABEL) during diagnosis. | Change in proportion of unsafe diagnostic investigations and mean quality score of diagnosis following tool consultation. | Significant proportion of unsafe investigations reduced with tool use. Mean diagnostic quality score increased. | Use of stand-alone diagnostic system to improve diagnostic decision-making for junior physicians is beneficial. However, several barriers must be overcome in order for such tools to be most effective. | 3 | 3 |
Ramnarayan et al42 (2007) | Audit of system | Use of diagnostic decision support tool (ISABEL) when diagnosing patients with resuscitation issues in emergency room. | Percentage of time accurate diagnosis was in list provided by tool and proportion of time tool included must-not-miss diagnoses. | Tool contained correct diagnosis and must-not-miss diagnosis in nearly every case. | Diagnostic aid performs an acceptable degree of clinical accuracy in ED. Further research is needed to determine role of tool in clinical practice. | 2b | 3 |
Ramnarayan et al43 (2003) | Audit with clinicians of varying levels of expertise and system | Use of internet diagnostic decision support tool (ISABEL) as a reminder of possible diagnoses to consider. | Proportion of cases with expected diagnosis in results generated by tool. | Using hypothetical and real patient cases, tool returned the correct diagnosis in nearly every case. | Tool showed acceptable clinical accuracy by providing correct diagnosis for real and hypothetical cases. It is anticipated the use of tool is effective in assisting physicians to accurately diagnose children. | 2b | 4 |
Ramnarayan et al44 (2006) | Before/after with clinicians of varying levels of expertise | Use of a diagnostic system (ISABEL) to determine diagnosis after diagnosing case without system. | Rate of diagnostic errors, diagnostic quality score and time using system. | Decline in diagnostic errors. Increase in diagnostic quality score. No significant outcome by level of experience. Median time for system use was 1 min. | Study suggests promising role for reminder-based diagnostic decision support tool to reduce diagnostic errors. | 4b | 4 |
Graber and Matthew45 (2008) | Before/after with physicians and system | Use of diagnostic decision support tool (ISABEL) to determine correct diagnosis upon entry of patient key findings of complex medical cases. | Percentage of cases where tool returned the same diagnosis as listed in the NEJM. Amount of time using the tool. | When entering evidence manually tool returned correct diagnosis in nearly all cases. When pasting case text as listed in NEJM, tool contained correct diagnosis three-fourths of cases. Both entry approaches were fast. | Tool performed quickly and accurately in suggesting correct diagnoses and should be evaluated in natural environments to determine its potential to support clinical diagnosis and reduce the rate of diagnostic errors. | 2b | 3 |
Tang and Ng46 (2006) | Audit with physician and rheumatologist | Searching Google.com to determine correct diagnosis for case presented in NEJM. | Percentage of diagnoses from Google that corresponded with NEJM diagnosis. | Google searches revealed correct diagnosis in slightly more than half of cases. | As the internet becomes more available in clinical settings, use of web-based searching tools may help physicians diagnose difficult cases. | 2b | 2 |
Outcome Ratings reflect the level of impact for each intervention on reducing diagnostic errors.9 ,10 Strength of Conclusions was rated on a numerical scale (1–5) in accordance with Best Evidence in Medical Education guidelines (5=strongest).9 ,11 CCU, Coronary-care unit; ER, Emergency Room; FN, False Nagetive; FP, False Positive; NEJM, New England Journal of Medicine.