Table 3

Brief vignettes to illustrate three types of reviewer agreement

No.VignetteProvider diagnosisReviewer 1Reviewer 2Reviewer 3
188 y/o male with h/o lymphoma presented with headache, cough, green sputum and fever. Provider did not order labs or x-ray to evaluate for pneumonia.Acute bronchitisMissed pneumoniaMissed pneumoniaNA
261 y/o male with h/o hypertension and neck pain presented with intermittent numbness, weakness and tingling of both hands.Peripheral neuropathyMissed cervical myelopathy with cord compressionMissed cervical cord compressionNA
345 y/o female with cough and fever; diagnosed with pharyngolaryngo-tracheitis. PCP read chest x-ray as normal; patient returned with continued symptoms and found to have lobar pneumonia on initial x-ray that had been read by the radiologist.TracheitisPneumoniaNo errorPneumonia
487 y/o male with right hand swelling, pain and new onset bilateral decreased grip; diagnosed few weeks later with carpel tunnel syndrome.OsteoarthritisNo errorMissed carpal tunnel syndromeMissed carpal tunnel syndrome
565 y/o male with left hand swelling and erythema after a small cut; treated for cellulitis at index visit. Returned 4 days later with ‘failure to respond’ and admitted for intravenous antibiotics with some improvement. Uric acid found elevated and thus additionally treated for gout; given prednisone.CellulitisNo errorMissed goutNo error
642 y/o female with posthospitalisation follow-up for CHF exacerbation c/o 1 week shortness of breath and congestion. Treated for URI but Lasix increased to address CHF. Symptoms progressed so patient admitted for CHF exacerbation/URI.CHF and bronchitisMissed CHF exacerbationNo errorNo error
  • 1–2 are case scenarios where both initial reviewers agreed on error; 3–4 are case scenarios when the independent third reviewer judged it to be an error after initial disagreement, and 5–6 are case scenarios when the independent third reviewer judged it not to be an error (after initial disagreement).

  • c/o, complaint of; CHF, chronic heart failure; h/o, history of; NA, not applicable; PCP, primary care provider; URI, upper respiratory infection; y/o, year old.