Dependent variables | MDCalc | PE-Dx CDS | Effect size (CI) | P value |
Effectiveness | ||||
% appropriate decision | 83.75% (134/160) | 94.38% (151/160) | 0.35* (0.32 to 0.37) | p<0.01 |
Confidence level† | 80.21 (18.94) | 82.71 (18.21) | 0.13‡ (−0.05 to 0.31) | p=0.14 |
Efficiency | ||||
Time per scenario (in seconds) | 117.37 (38.91) | 95.84 (25.42) | −0.55‡ (−0.71 to −0.38) | p<0.001 |
Number of clicks per scenario | 16.49 (4.82) | 17.90 (3.91) | 0.29‡ (0.09 to 0.48) | p<0.01 |
Number of scrolls per scenario | 7.47 (2.92) | 6.31 (2.50) | −0.39‡ (−0.57 to −0.21) | p<0.001 |
Number of navigation elements per scenario | 15.53 (5.01) | 10.23 (2.74) | −1.05‡ (−1.26 to −0.83) | p<0.001 |
Perceived workload¶ | 4.45 (1.53) | 3.64 (1.78) | −0.52‡ (−0.68 to −0.37) | p<0.001 |
Satisfaction§ | ||||
CUSQ: overall satisfaction | 5.62 (0.94) | 6.18 (0.66) | 0.58‡ (0.18 to 0.98) | p<0.001 |
CUSQ: system usefulness | 5.65 (0.95) | 6.26 (0.67) | 0.62‡ (0.19 to 1.05) | p<0.01 |
CUSQ: information quality | 5.73 (0.98) | 6.17 (0.73) | 0.43‡ (0.03 to 0.83) | p<0.05 |
CUSQ: interface quality | 5.48 (1.15) | 6.02 (0.86) | 0.45‡ (0.07 to 0.84) | p<0.05 |
*Cohen (h) effect size based on the inverse sine of the square root of the proportional values.
†One question: 0 (no confidence) to 100 (very high confidence).
‡Becker75 adjusted effect size for repeated measures.
§Response categories: 1 (strongly disagree) to 7 (strongly agree).55
¶Scale: 1 (low) to 10 (high).54
CUSQ, computer system usability questionnaire; PE-Dx CDS, clinical decision support for pulmonary embolism diagnosis.