Brief vignettes to illustrate three types of reviewer agreement
| No. | Vignette | Provider diagnosis | Reviewer 1 | Reviewer 2 | Reviewer 3 |
| 1 | 88 y/o male with h/o lymphoma presented with headache, cough, green sputum and fever. Provider did not order labs or x-ray to evaluate for pneumonia. | Acute bronchitis | Missed pneumonia | Missed pneumonia | NA |
| 2 | 61 y/o male with h/o hypertension and neck pain presented with intermittent numbness, weakness and tingling of both hands. | Peripheral neuropathy | Missed cervical myelopathy with cord compression | Missed cervical cord compression | NA |
| 3 | 45 y/o female with cough and fever; diagnosed with pharyngolaryngo-tracheitis. PCP read chest x-ray as normal; patient returned with continued symptoms and found to have lobar pneumonia on initial x-ray that had been read by the radiologist. | Tracheitis | Pneumonia | No error | Pneumonia |
| 4 | 87 y/o male with right hand swelling, pain and new onset bilateral decreased grip; diagnosed few weeks later with carpel tunnel syndrome. | Osteoarthritis | No error | Missed carpal tunnel syndrome | Missed carpal tunnel syndrome |
| 5 | 65 y/o male with left hand swelling and erythema after a small cut; treated for cellulitis at index visit. Returned 4 days later with ‘failure to respond’ and admitted for intravenous antibiotics with some improvement. Uric acid found elevated and thus additionally treated for gout; given prednisone. | Cellulitis | No error | Missed gout | No error |
| 6 | 42 y/o female with posthospitalisation follow-up for CHF exacerbation c/o 1 week shortness of breath and congestion. Treated for URI but Lasix increased to address CHF. Symptoms progressed so patient admitted for CHF exacerbation/URI. | CHF and bronchitis | Missed CHF exacerbation | No error | No error |
1–2 are case scenarios where both initial reviewers agreed on error; 3–4 are case scenarios when the independent third reviewer judged it to be an error after initial disagreement, and 5–6 are case scenarios when the independent third reviewer judged it not to be an error (after initial disagreement).
c/o, complaint of; CHF, chronic heart failure; h/o, history of; NA, not applicable; PCP, primary care provider; URI, upper respiratory infection; y/o, year old.