Level of agreement between reviewer pairs
| Inter-rater | Reviewer 1 versus 2 | Reviewer 2 versus 3 | Reviewer 1 versus 3 | ||||||
|---|---|---|---|---|---|---|---|---|---|
| κ (95% CI) | κ max | κ/κ max | κ (95% CI) | κ max | κ/κ max | κ (95% CI) | κ max | κ/κ max | |
| *Based on seven categories of seriousness; | |||||||||
| †Based on preventability scores (score 1–3 preventable, score 4–6 not preventable). | |||||||||
| Event type | 0.36 (0.32 to 0.40) | 0.98 | 0.37 | 0.35 (0.33 to 0.39) | 0.77 | 0.45 | 0.51 (0.47 to 0.55) | 0.88 | 0.57 |
| Seriousness* | 0.36 (0.32 to 0.40) | 0.63 | 0.59 | 0.37 (0.33 to 0.41) | 0.59 | 0.63 | 0.29 (0.25 to 0.33) | 0.78 | 0.38 |
| Serious v not serious | 0.53 (0.45 to 0.61) | 0.82 | 0.64 | 0.58 (0.50 to 0.66) | 0.80 | 0.73 | 0.37 (0.29 to 0.45) | 0.98 | 0.37 |
| Preventability decision (yes/no) | 0.31 (0.25 to 0.37) | 0.81 | 0.38 | 0.30 (0.24 to 0.36) | 0.81 | 0.37 | 0.50 (0.42 to 0.58) | 0.99 | 0.51 |
| Preventability score | 0.30 (0.24 to 0.36) | 0.81 | 0.37 | 0.24 (0.18 to 0.30) | 0.83 | 0.29 | 0.45 (0.39 to 0.51) | 0.90 | 0.50 |
| Preventability category† | 0.33 (0.27 to 0.39) | 0.80 | 0.42 | 0.31 (0.25 to 0.37) | 0.81 | 0.38 | 0.51 (0.45 to 0.57) | 0.96 | 0.53 |
| Intra-rater | Reviewer 1 | Reviewer 2 | Reviewer 3 | ||||||
| Event type | 0.64 (0.52 to 0.76) | 0.74 | 0.87 | 0.55 (0.45 to 0.65) | 0.87 | 0.63 | 0.69 (0.59 to 0.79) | 0.88 | 0.78 |