First 10 cases* | Last 10 cases | Overall | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
% Agree | κ | 95% | CI | % Agree | κ | 95% | CI | % Agree | κ | 95% | CI | p Value | |
Overall | 92 | 0.60 | 0.27 | 0.93 | 94 | 0.83 | 0.54 | 1.11 | 93 | 0.74 | 0.66 | 0.82 | <0.001 |
Case characteristics | 95 | 0.00 | −0.99 | 0.99 | 98 | 0.88 | 0.59 | 0.99 | 97 | 0.86 | 0.70 | 0.99 | <0.001 |
SCIP measures (Q1–3) | 90 | 0.78 | 0.57 | 0.99 | 77 | 0.55 | 0.42 | 0.67 | 83† | 0.63† | 0.52 | 0.74 | <0.001 |
Joint commission timeout (Q4) | 92 | 0.78 | 0.39 | 0.99 | 98 | 0.94 | 0.83 | 0.99 | 98 | 0.94 | 0.84 | 0.99 | <0.001 |
Briefing (Q5–11) | 90 | 0.79 | 0.63 | 0.94 | 90 | 0.78 | 0.62 | 0.94 | 88 | 0.73 | 0.64 | 0.83 | <0.001 |
Debrief (Q12–14) | 97 | 0.92 | 0.75 | 0.99 | 100 | 0.99 | 0.99 | 0.99 | 94 | 0.87 | 0.78 | 0.96 | <0.001 |
Active participation (Q15) | 100 | –‡ | – | – | 98 | –‡ | – | – | 99 | –‡ | – | – | – |
Surgical team member buy-in (Q16–19) | 80 | 0.63 | 0.19 | 0.99 | 98 | 0.88 | −0.62 | 0.99 | 83 | 0.44 | 0.09 | 0.78 | 0.012 |
Checklist outcomes (Q20–22) | 87 | 0.52 | 0.09 | 0.95 | 93 | 0.86 | 0.68 | 0.99 | 94 | 0.88 | 0.79 | 0.96 | <0.001 |
Surgical best practices (Q23–25) | 100 | –‡ | – | – | 100 | – | – | – | 100 | –‡ | – | – | – |
*Excluding the first case observed.
†The percent agreement and κ increased to 95% and 0.84 (95% CI 0.74 to 0.96), respectively, when the responses for ‘Yes w/o prompting’, and ‘Yes, prompted by the checklist’ were combined (ie, considered the same response).
‡The κ score and 95% CI could not be estimated for these items due to the high percent agreement.25
SCIP, Surgical Care Improvement Project.