Article Text

Download PDFPDF
Filling a gap in safety metrics: development of a patient-centred framework to identify and categorise patient-reported breakdowns related to the diagnostic process in ambulatory care
  1. Sigall K Bell1,
  2. Fabienne Bourgeois2,
  3. Catherine M DesRoches1,
  4. Joe Dong1,
  5. Kendall Harcourt1,
  6. Stephen K Liu3,
  7. Elizabeth Lowe4,
  8. Patricia McGaffigan5,
  9. Long H Ngo1,
  10. Sandy A Novack4,
  11. James D Ralston6,
  12. Liz Salmi1,
  13. Suz Schrandt7,
  14. Sue Sheridan7,
  15. Lauge Sokol-Hessner8,
  16. Glenda Thomas4,
  17. Eric J Thomas9,10
  1. 1 Department of Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA
  2. 2 Department of Pediatrics, Boston Children's Hospital, Harvard Medical School, Boston, Massachusetts, USA
  3. 3 Department of Medicine, Dartmouth College Geisel School of Medicine, Hanover, New Hampshire, USA
  4. 4 Patient and Family Advisory Council, Department of Social Work, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
  5. 5 Institute for Healthcare Improvement, Boston, Massachusetts, USA
  6. 6 Kaiser Permanente Washington Health Research Institute, Seattle, Washington, USA
  7. 7 Society to Improve Diagnosis in Medicine, Evanston, Illinois, USA
  8. 8 Department of Medicine and Department of Health Care Quality, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA
  9. 9 Department of Medicine, University of Texas McGovern Medical School, Houston, Texas, USA
  10. 10 Healthcare Quality and Safety, Memorial Hermann Texas Medical Center, Houston, Texas, USA
  1. Correspondence to Dr Sigall K Bell, Department of Medicine, Harvard Medical School, Boston, MA 02115, USA; sbell1{at}


Background Patients and families are important contributors to the diagnostic team, but their perspectives are not reflected in current diagnostic measures. Patients/families can identify some breakdowns in the diagnostic process beyond the clinician’s view. We aimed to develop a framework with patients/families to help organisations identify and categorise patient-reported diagnostic process-related breakdowns (PRDBs) to inform organisational learning.

Method A multi-stakeholder advisory group including patients, families, clinicians, and experts in diagnostic error, patient engagement and safety, and user-centred design, co-developed a framework for PRDBs in ambulatory care. We tested the framework using standard qualitative analysis methods with two physicians and one patient coder, analysing 2165 patient-reported ambulatory errors in two large surveys representing 25 425 US respondents. We tested intercoder reliability of breakdown categorisation using the Gwet’s AC1 and Cohen’s kappa statistic. We considered agreement coefficients 0.61–0.8=good agreement and 0.81–1.00=excellent agreement.

Results The framework describes 7 patient-reported breakdown categories (with 40 subcategories), 19 patient-identified contributing factors and 11 potential patient-reported impacts. Patients identified breakdowns in each step of the diagnostic process, including missing or inaccurate main concerns and symptoms; missing/outdated test results; and communication breakdowns such as not feeling heard or misalignment between patient and provider about symptoms, events, or their significance. The frequency of PRDBs was 6.4% in one dataset and 6.9% in the other. Intercoder reliability showed good-to-excellent reliability in each dataset: AC1 0.89 (95% CI 0.89 to 0.90) to 0.96 (95% CI 0.95 to 0.97); kappa 0.64 (95% CI 0.62, to 0.66) to 0.85 (95% CI 0.83 to 0.88).

Conclusions The PRDB framework, developed in partnership with patients/families, can help organisations identify and reliably categorise PRDBs, including some that are invisible to clinicians; guide interventions to engage patients and families as diagnostic partners; and inform whole organisational learning.

  • diagnostic errors
  • patient safety
  • communication

Data availability statement

Data may be obtained from a third party and are not publicly available.

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Data availability statement

Data may be obtained from a third party and are not publicly available.

View Full Text


  • Twitter @fcbourgeois, @cmd418, @TheLizArmy, @EJThomas_safety

  • Contributors SKB conceived the study, obtained funding, led the research, drafted the manuscript and is the guarantor of the submitted manuscript. EJT, FB, SL, LHN and CD contributed to the grant proposal for study funding. FB, LS, and SB conducted the qualitiative analysis. JD and LHN led the statistical analyses. KH participated as a project coordinator and research assistant. FB, CD, SL, EL, PM, LHN, SN, JR, LS, SSc, SSh, LS-H, GT, and FB-B (acknowledgement) participated in the Metrics Advisory Group. AN (acknowledgement) contributed to supplementary material figure design. All authors reviewed and approved the manuscript prior to submission. Each revision and final proofs were also shared with each author for review and feedback.

  • Funding Support for this work was generously provided by AHRQ (grant number: 5R01HS027367-02).

  • Competing interests None declared.

  • Patient and public involvement statement Patients and family members of patients participated in the PRDB framework development from project inception to publication (6 authors).

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Linked Articles

  • Editorial
    Laurien Kuhrij Perla J Marang-van de Mheen