Approach solution | Disjunctive (or) combinations | Optimal individual MA chart | |||
Optimal MA combination | Each MA chart individually | ||||
Parameters | Baseline data | N/A | Network | Local | Network |
Baseline size | N/A | 18 (3–24) | 3 (3) | 12 (3–24) | |
Baseline lag | N/A | 6 (0–24) | 3 (3–24) | 3 (0–12) | |
MA span | N/A | 12 (12) | 6 (6–12) | 12 (12) | |
CL width (σ) | N/A | 1 (1) | 1 (1–2) | 1 (1) | |
Chart performance | Sensitivity | 0.90 (0.88) | 0.72 (0.74) | 0.53 (0.45) | 0.73 (0.71) |
Specificity | 0.67 (0.75) | 0.79 (0.86) | 0.84 (0.86) | 0.77 (0.86) | |
PPV | 0.56(0.65) | 0.62 (0.73) | 0.61 (0.63) | 0.60 (0.72) | |
NPV | 0.94 (0.92) | 0.86 (0.87) | 0.79 (0.75) | 0.86 (0.85) | |
Positive LLR | 2.73 (3.55) | 3.45 (5.28) | 3.34 (3.25) | 3.23 (4.94) | |
Negative LLR | 0.15 (0.16) | 0.36 (0.30) | 0.56 (0.64) | 0.35 (0.34) | |
Youden’s index | 0.57 (0.63) | 0.51 (0.60) | 0.37 (0.31) | 0.50 (0.57) | |
Markedness | 0.50 (0.57) | 0.47 (0.60) | 0.40 (0.38) | 0.46 (0.57) | |
Diagnostic OR | 18.8 (22.3) | 9.70 (17.6) | 5.97 (5.07) | 9.20 (14.6) | |
Accuracy | 0.74 (0.80) | 0.77 (0.82) | 0.74 (0.72) | 0.76 (0.81) | |
F1 score | 0.69 (0.75) | 0.66 (0.74) | 0.54 (0.52) | 0.66 (0.72) | |
Signals/month | 0.24 (0.21) | 0.19 (0.17) | 0.08 (0.08) | 0.20 (0.18) | |
Rank | 7 (661) of 6 478 200 | 11 (1) of 3600 | 367 (748) of 3600 | 18 (36) of 3600 |
Chart parameters and performance for the optimal dual-chart approach, each of these two charts individually, and optimal individual charts (overall and relying on local data only). Parameter values are listed as median and, in parentheses, range over the best 20 solutions in each category. Performance measures and ranks (out of 3600 individual and 32.4 million dual charts) are based on the training dataset, with values in parentheses based on the validation dataset.
σ, estimated SD; CL, control limit; LLR, log likelihood ratio; MA, moving average;N/A, not applicable; NPV, negative predictive value; PPV, positive predictive value.