Table 1

Inter-rater reliabilities

DiscriminationNumber of chartsNumber of categoriesκ95% CI
1. Broad content14886*
 Initial0.90 (p<0.01)0.88 to 0.92
 After discussion1.00 (p<0.01)0.99 to 1.00
2. Quality and safety content58914†
 Initial0.92 (p<0.01)0.89 to 0.94
 After discussion1.00 (p<0.01)1.00 to 1.00
3. Methods used to present quality and safety data
 3a. Appearance5898‡
  Initial0.88 (p<0.01)0.85 to 0.91
  After discussion0.98 (p<0.01)0.96 to 0.99
 3b. Comparison589
  Initial0.91 (p<0.01)0.87 to 0.94
  After discussion1.00 (p<0.01)1.00 to 1.00
 3c. Reference indicators’ depiction285
  Initial0.84 (p<0.01)0.76 to 0.91
  After discussion1.00 (p<0.01)1.00 to 1.00
  • *Quality and safety, financial, activity, patient surveys, staffing and other.

  • †Waiting/delays, healthcare acquired infections, incidents reports, mortality, pressure ulcers, falls, length of stay, readmissions, venous thromboembolism, cleanliness, catheter, medication errors, others not consistently enough appearing to warrant a specific category and one for graphs that were placed in multiple categories.

  • ‡Line, bar, both line and bar, line with reference, bar with reference, both line and bar with reference, pie, other.

  • §Time-series, between-groups, both time-series and between-groups.

  • ¶Reference indicators depicting either a standard or trend, or the role of chance.