Article Text
PDF
Abstract
Background Patients and families are important contributors to the diagnostic team, but their perspectives are not reflected in current diagnostic measures. Patients/families can identify some breakdowns in the diagnostic process beyond the clinician’s view. We aimed to develop a framework with patients/families to help organisations identify and categorise patient-reported diagnostic process-related breakdowns (PRDBs) to inform organisational learning.
Method A multi-stakeholder advisory group including patients, families, clinicians, and experts in diagnostic error, patient engagement and safety, and user-centred design, co-developed a framework for PRDBs in ambulatory care. We tested the framework using standard qualitative analysis methods with two physicians and one patient coder, analysing 2165 patient-reported ambulatory errors in two large surveys representing 25 425 US respondents. We tested intercoder reliability of breakdown categorisation using the Gwet’s AC1 and Cohen’s kappa statistic. We considered agreement coefficients 0.61–0.8=good agreement and 0.81–1.00=excellent agreement.
Results The framework describes 7 patient-reported breakdown categories (with 40 subcategories), 19 patient-identified contributing factors and 11 potential patient-reported impacts. Patients identified breakdowns in each step of the diagnostic process, including missing or inaccurate main concerns and symptoms; missing/outdated test results; and communication breakdowns such as not feeling heard or misalignment between patient and provider about symptoms, events, or their significance. The frequency of PRDBs was 6.4% in one dataset and 6.9% in the other. Intercoder reliability showed good-to-excellent reliability in each dataset: AC1 0.89 (95% CI 0.89 to 0.90) to 0.96 (95% CI 0.95 to 0.97); kappa 0.64 (95% CI 0.62, to 0.66) to 0.85 (95% CI 0.83 to 0.88).
Conclusions The PRDB framework, developed in partnership with patients/families, can help organisations identify and reliably categorise PRDBs, including some that are invisible to clinicians; guide interventions to engage patients and families as diagnostic partners; and inform whole organisational learning.
- diagnostic errors
- patient safety
- communication
Data availability statement
Data may be obtained from a third party and are not publicly available.
Statistics from Altmetric.com
Introduction
Diagnostic errors are common, harmful and expensive.1 2 Estimated to occur in 5% of ambulatory visits per year and contribute to 10% of deaths, these errors affect millions of patients annually, and represent the leading type of paid malpractice claims in the USA.1–4 Ambulatory diagnostic errors present a vexing challenge, at least in part because of limited measures to identify events that unfold over time and between various healthcare interactions. Studies measuring the incidence, causes and impact of diagnostic errors in ambulatory care rely on limited measurement methods including clinician surveys, malpractice claims files or electronic triggers that flag high-risk visit patterns for additional chart review.5 6 A long-standing tenet of patient safety emphasises the need for multiple measurement methods, particularly those representing different perspectives, to fully characterise the problem.7–9 However, studies measuring and characterising diagnostic errors often lack the patient and family perspective.10 11
The scarcity of patient-centred metrics is a missed opportunity. Patients and families have unique knowledge that informs the diagnostic process such as symptom changes, care at different centres and other events between encounters.1 12–15 Ambulatory diagnostic breakdowns occur not only during the office visit, but also in the space between visits where the breakdowns may only be seen by the patient and family. Mounting evidence demonstrates that patients/families can find clinically relevant errors and breakdowns,16–20 including those unapparent to clinicians or health systems,16 21–25 and that patient reports have the potential to improve organisational safety.12 16 22 24–28 In addition to identifying potential breakdowns, patients and families can also make important positive contributions to the diagnostic process. Empowering patients may better support co-production and shared decision-making in the diagnostic process, and a shared mental model of diagnosis between patients and clinicians.29–31 Shared mental models in healthcare can improve teamwork, and patients are a critical part of the diagnostic team.32 33
Landmark reports from the National Academies of Medicine (NAM), the National Quality Forum (NQF) and the Institute for Healthcare Improvement (IHI) urge patient engagement to improve diagnostic accuracy.1 34 35 As organisations seek to answer the call to engage patients and families in diagnosis, they will need a process and taxonomy that make sense to patients and families and align with their lived healthcare experiences. Recent studies suggest that patient reports of diagnostic breakdowns reflect missed opportunities for timely diagnosis36 and other themes that are under-represented in clinician-based diagnostic error frameworks.10 25 These authors urge development of ‘standardised categorisation mechanisms that capture how patients express diagnostic safety concerns’ in order to ‘highlight areas for improvement in the diagnostic process that might otherwise go undetected’.36 To date, however, there is no such classification system for patient-reported diagnostic process-related breakdowns (PRDBs) in ambulatory care.37
Published accounts of patient experiences with diagnostic errors to date are predominantly derived from consumer advocacy groups, inpatient errors or unsolicited patient complaints, which limit generalisability.10 36 38 Unsolicited patient complaints may be biased toward more empowered patients or more extreme events that reach a threshold for voluntary patient reporting. Patient fear of retribution or lack of awareness about reporting may also contribute to under-reporting, and may be more likely to impact underserved communities.
Filling these gaps, we developed a framework together with patients/families to help organisations identify and categorise PRDBs that may be beyond clinicians’ view, drawing on two large cross-sectional surveys identifying patient-reported errors in ambulatory care. We anticipated that framework for PRDBs could be used to: (1) more comprehensively describe the incidence, types, contributing factors and impacts of breakdowns related to the diagnostic process in ambulatory care; and (2) develop future strategies to systematically engage patients and families to improve diagnosis, using terms to describe experiences that are readily understood and observed by them. This paper focuses on the development of the PRDB framework. The detailed results of the qualitative analyses of the two datasets will be reported separately.
Methods
We developed the PRDB framework in ambulatory care through an iterative process involving a multi-stakeholder advisory group (MAG) and two broad population surveys, collectively representing 25 425 US respondents, and identifying 2165 patient-reported ambulatory errors (figure 1). We used this process to ground the framework in both expertise and evidence, combining the diverse experiences and perspectives of the advisory group with data derived from patient-reported ambulatory errors.
Development of the framework for patient-reported diagnostic process-related breakdowns. MAG, multi-stakeholder advisory group.
Patient-reported diagnostic process-related breakdowns
We defined a PRDB as a problem or delay reported by patients that could map to any part of the diagnostic process, as outlined in the NAM conceptual model.1 Examples include missing the patient’s main concern or inaccurately reporting symptoms, delayed or erroneous interpretation of diagnostic tests, planned but missing referrals or not communicating the diagnosis to the patient. We did not include medication or procedure breakdowns, as these are often considered their own type of error. We developed the term PRDB for this study and focused on breakdowns rather than diagnostic errors as the target measure for several reasons. First, patients may not feel as comfortable in their ability to identify a diagnostic error as in their ability to identify a breakdown in the process—a more patient-centred task.24 39 For example, in the 2017 national IHI study, about half of respondents did not fully understand the term ‘medical error’.40 Second, Reason’s model of errors and latent failures suggests greater yield in improving diagnostic safety if as many proximal failures as possible are measured (such as patient-identified process breakdowns), in addition to the resultant harmful event.41 Third, identifying these upstream breakdowns in the diagnostic process, particularly from the patient’s perspective, can inform more targeted patient-centred diagnostic error prevention strategies.
Multi-stakeholder advisory group
The MAG was comprised of 14 individuals including Caucasian, African American and Asian participants. Seven were patients or family members at three different hospital systems or representatives from the Society to Improve Diagnosis in Medicine, and included persons living with chronic illness and/or disabilities, and parents of children with complex diagnoses. The other seven participants were patient safety and engagement leaders including an IHI representative; ambulatory clinicians; and experts in diagnostic errors, user-centred design, electronic health records (EHRs), and health services research.
The MAG met two times over 6 months in a structured process, led by the research team. We started with a patient sharing a personal story of diagnostic delay to anchor the group’s work in a patient-centred clinical experience. Each session was 90 min and was followed by updates based on the group input and opportunities for additional feedback via email.
Datasets
To broaden the range of patient-reported events, we identified patient-reported errors from two cross-sectional surveys (online supplemental appendix 1).
Supplemental material
IHI dataset
The first data source was a nationally representative 2017 survey conducted by the National Opinion Research Center and commissioned by the National Patient Safety Foundation (now IHI) to assess the experiences of the American public with medical error.40 In the IHI survey, after defining ‘medical error’, participants were asked whether they were ever personally involved in a situation where a medical error was made in their own care or in the care of someone close to them, where they were very familiar with the care received. Those who responded ‘Yes’ were asked: ‘Thinking about the most recent time a medical error was made in your care/the care of someone close to you, in a few words, what happened?’ We identified ambulatory errors by selecting participant-reported errors that occurred in ‘doctor’s office, clinic, or health centre’, ‘urgent care centre or walk-in clinic’, or ‘surgicentre or outpatient surgical setting’. Among 2536 respondents to the IHI survey, 559 (22%) reported an error in their own care and an additional 475 (19%) reported an error in the care of a loved one. Among all errors, 416 of 1034 (40%) occurred in the ambulatory setting (IHI cohort) and all errors had narrative descriptions. The IHI cohort provided the opportunity to analyse patient and family-perceived ambulatory errors in a national survey representing the views of the American public that was oversampled for participants with low socioeconomic status.40 Data from this cohort enhanced generalisability of the PRDB framework.
Open notes dataset
The second data source was a survey at three US healthcare organisations assessing patient experience with accessing their visit notes through the patient portal (‘open notes’).42 All patients who used the portal and had at least one ambulatory visit note available in the preceding 12 months were eligible for the survey. In the open notes survey, participants were asked: ‘Have you ever found anything in your visit notes you thought was a mistake (not counting misspellings or typographical errors)?’ Among 22 889 patients who read ≥1 note and responded to error questions, 4831 (21%) reported an error. Of these, patients rated 2043 (42%) as somewhat or very serious and were asked to ‘Please describe the most serious mistake’. A total of 1749 (86%) patient-reported errors included descriptions that were available for qualitative analysis (open notes cohort).25
The open notes cohort enabled analysis of patient-reported breakdowns pertaining to office visits through review of ambulatory visit notes, a resource now available to all US patients through the 21st Century Cures Act, which mandates patient access to electronic health information.43 Data from this cohort broadened PRDB framework testing to a clinical context including access to electronic health information, and may be most relevant to future engagement of patients and families in diagnosis through the EHR. Further details of each dataset are available elsewhere.25 40
Development of the PRDB framework
Our framework development process is shown in figure 1. Because a framework for patient and family-reported breakdowns related to the ambulatory diagnostic process did not yet exist, we reviewed the literature for frameworks that focused on diagnostic error,1 10 44 45 ambulatory care,46–49 or patient and family reporting.20 50–52 We examined the Diagnostic Error Evaluation and Research classification system, the conceptual model in the NAM report on improving diagnosis, the Healthcare Complaints Analysis Tool (HCAT) and the WHO classification for patient safety incidents in primary care, among others.1 20 42 45 46 53 We also identified themes from previously published patient-reported errors related to diagnosis.1 10 38 54–58
First, the MAG discussed the summary of literature-based themes and a randomly selected subset of 20 patient-reported errors and provided additional types of breakdowns identified from patient reports and from personal perspective. We developed breakdown categories using a standard approach, determining where in the process of care the breakdown occurred, and then generating subcategories to describe what happened (details of what went wrong).45 Because the surveys informing the databases did not specifically ask patients to describe contributing factors or impacts, our primary focus was on the patient-reported breakdown categories, with additional information related to contributing factors and impacts described when it was available in patient reports and from MAG input. Together, the breakdown categories, contributing factors and impacts formed the preliminary framework.
Three researchers, including an internal medicine physician (SKB), a paediatrician (FB) and a patient living with cancer (LS), met regularly between MAG meetings, applying the preliminary framework to another randomly selected subset of 20 patient reports, adding labels for any unrepresented breakdowns, contributing factors and impacts, into a revised list with examples for group review.
Through iterative discussion among researchers and the MAG, we then applied the revised framework to additional randomly generated samples of patient reports, using 6 sets of 20 comments in total. We followed this procedure until review of patient reports did not generate any new information and we reached stakeholder consensus on categories. We developed definitions for each of the categories, with input from patient representatives. The MAG then reviewed the final categories and definitions and we integrated feedback.
Coding process
The three coders (SKB, FB, LS) each participated in at least three practice sessions during framework development, reviewing 20 randomly selected patient reports at a time, as described above. During these sessions, we rigorously examined coding disagreements to identify any categories requiring more discriminant definitions. Because we anticipated that the PRDB framework could inform future design of tools to help patients identify diagnostic breakdowns in their records, we also wanted to assess whether the framework could be used by non-clinicians. We included a patient coder to test patient use of the breakdown categories and ensure patient perspectives were incorporated into our final results. We used coding guidelines analogous to the HCAT,20 such as coding only empirically identifiable text (not inferences), and assigning as many breakdown categories as appropriate to each patient report. We calculated the frequency of PRDBs in each survey patient population. We then compared the PRDB framework to a common existing framework derived from clinician reports of diagnostic error and the NAM conceptual model of the diagnostic process to identify new categories and/or additional information within existing categories.1 45
Intercoder reliability
We focused intercoder reliability testing on breakdown categories, which also enabled a direct comparison of reliability with a commonly used clinician-centred diagnostic error framework that categorised the type of failure in the diagnostic process.45 We measured intercoder reliability by calculating Gwet’s AC1 statistic, a test used to examine reliability between two or more coders examining categorical data with a skewed distribution, as previously used in analysis of patient complaints (HCAT instrument).20 This was the most appropriate test for our data because like the HCAT, some categories in the PRDB framework were used at a much higher rate than others. However, we also calculated Cohen’s kappa statistic for each reliability measurement because this test is more commonly used and a more conservative measure.20 59 We evaluated complete matches; in other words, we counted as a disagreement any time one reviewer coded a category that the other did not. To interpret the agreement coefficients, we considered 0.01–0.2=poor/slight agreement; 0.21–0.40=fair agreement; 0.41–0.6=moderate agreement; 0.61–0.8=good agreement and 0.81–1.00=excellent agreement.
After coding the IHI dataset, the two physicians discussed disagreements, reaching consensus on a final adjudicated set of results. We examined reliability between the two physician raw coding results and between the patient coder and the adjudicated set. For the open notes dataset, we used a randomly selected sample of 200 (approximately 10%) of patient reports to compare physician coding. Because the two physician coders achieved good intercoder reliability in both datasets, one physician categorised the 1749 patient reports in the open notes dataset. The patient coder also categorised all 1749 reports, and we calculated kappa and AC1 scores accordingly.
Results
The framework for PRDBs
The final framework for PRDBs in ambulatory care has three domains: breakdown categories, contributing factors and impact (table 1). In total, 7 breakdown categories and 40 subcategories mapped directly to the diagnostic process, alongside 19 patient-identified contributing factors and 11 patient-reported impact subcategories. Analysis of patient reports with the framework yielded granular data related to PRDBs. For example, the framework enabled identification of breakdowns related to 10 different aspects of the patient history, 5 components of the test or referral process, and 6 types of communication breakdowns (table 1). Patients identified breakdowns related to every step of the diagnostic process. The most common categories in both datasets were Medical History, Explanation and Next Steps, Communication and Respect, and Tests and Referrals. The overall frequency of patient-reported diagnostic process-related breakdowns was 175 of 2536 (6.9%) in the IHI cohort, and 1466 of 22 889 (6.4%) in the open notes cohort.
The framework for patient-reported diagnostic process-related breakdowns in ambulatory care
Contributing factors to diagnostic breakdowns identified by patients and families in the data and in the MAG included clinician-related, patient-related and system-related factors. Participants described experiences of explicit or implicit bias related to their care, such as perceived effects of race, literacy or mental health on diagnostic considerations. For example, patients occasionally described clinicians who: did not take their symptoms seriously because of underlying illness, language or cultural gap; viewed all symptoms through the lens of a pre-existing diagnosis or anchored to diagnoses most common in a particular sociodemographic group. Participants highlighted social determinants of health and barriers to speaking up as important patient-related contributing factors to diagnostic breakdowns. Finally, they identified several system-related contributing factors including EHR problems, inadequate patient access to health information and insurance delays.
Patient-reported impacts included two categories: ‘Patient Activation or Mitigation by the healthcare system’ and ‘Negative Patient Impact.’ Patients felt that ‘Activation or Mitigation’— such as having to be extra vigilant, seek second opinions, or request records could be positive or negative. In some cases, patients and families described stronger relationships with providers after working through the problem. Patients and families also described several types of negative breakdown impacts, including physical, emotional, psychological, financial and relational dimensions. Occasionally, ‘Activation or Mitigation’ and ‘Negative Patient Impact’ coexisted, such as a patient speaking up about a diagnosis documentation error, but still experiencing a treatment delay (table 2). The detailed PRDB framework with definitions and examples is shown in online supplemental appendix 2, and examples of patient reports and how they were coded are shown in table 2.
Examples of patient reports using the framework for PRDBs
Comparison of the PRDB framework with existing diagnostic error frameworks
Compared with a commonly used taxonomy derived from clinician reports of diagnostic errors,45 patients provided additional information on existing categories (such as history, physical examination and tests/referrals), including missed main concerns, physical examination omissions, and missing test requisitions or delayed referrals. The PRDB framework also highlighted new breakdown categories such as ‘Communication and Respect’—a category present in about 30% of patient reports, and most frequently including not feeling heard or misalignment. ‘Access’ and ‘Documentation-related or Technology-related problems’ were less common categories but included new areas such as EHR access, outdated copied and pasted information and erroneous billing (diagnostic) codes.
Proposed adaptation of the NAM conceptual model of the diagnostic process,1 reflecting PRDB framework categories, is shown in figure 2. Patients viewed ‘Access’ as a gateway not only to care, but also to their health information and to the diagnostic process itself. They underscored crucial access to education about warning signs and how to escalate if symptoms persist or worsen. The PRDB framework enabled identification of issues that may be undetected by clinicians, most commonly including missing or inaccurate symptoms, problems or delays related to the explanation/plan, communication breakdowns, barriers to completing tests and referrals or missing/inaccurate follow-up about results; and occasionally adding insight into patient-reported barriers to care access, documentation on the wrong patient, or diagnoses that were not communicated or understood—a patient-reported measure advocated by safety experts.1 37 60
Adapted National Academies of Medicine (NAM) conceptual model of the diagnostic process1 reflecting patient-reported diagnostic process-related breakdown (PRDB) framework categories and examples of PRDBs that may not otherwise be detected by clinicians.
Reliability testing
Results of reliability testing between reviewer coding of patient-reported breakdowns are shown in online supplemental appendix 3. All comparisons between physician coders, and between physician and patient coders of patient-reported breakdowns in each dataset demonstrated good or excellent reliability AC1: 0.89 (95% CI 0.89 to 0.90) to 0.96 (95% CI 0.95 to 0.97); and kappa: 0.64 (95% CI 0.62 to 0.66) to 0.85 (95% CI 0.83 to 0.88).
Discussion
This study of >2000 patient reports is the first to establish and preliminarily test, together with patients and families, a patient-centred framework for breakdowns related to the ambulatory diagnostic process. This framework fills a gap in existing clinician-centred approaches for classifying diagnostic errors by bringing into focus breakdowns experienced by patients and families that may not be observed by clinicians, nor measured by healthcare systems. The frequency of PRDBs was 6.4% in one dataset and 6.9% in the other, demonstrating consistent outcomes in two different populations, and approximating the 5% incidence of ambulatory diagnostic errors.1 2 Our findings build on prior work underscoring the patient and family experience of diagnostic error10 36 38 by focusing on (1) ambulatory breakdowns, (2) more generalisable samples of safety events, and (3) a unique patient-centred framework that can be systematically applied to future research and quality improvement efforts. Use of the PRDB framework demonstrates intercoder reliability among breakdown categories that is similar to, or exceeds, previously published frameworks.20 45 We propose the PRDB framework to be further studied and refined. This would enable a more complete 360° picture of diagnostic safety, forged in partnership with patients and families—the greatest stakeholders in timely and accurate diagnosis.
Compared with commonly used clinician-based frameworks, the PRDB framework broadens the perspective on diagnostic breakdowns to include patient perspectives on access to care and health information, inaccurate patient main concerns or symptoms, important missing information (known by patients), physical examination omissions, test/referral delays or outdated results, patients/families not feeling heard, misalignment between patient and clinician perspectives on symptoms or visit events, and disrespect, among other breakdowns. In doing so, the PRDB framework touches on several cross-cutting themes prioritised by the NQF 2020 assessment of diagnostic measures and expert consensus research priorities, while adding insight into relational breakdowns in the diagnostic process.34 61 These findings can provide otherwise rare feedback to clinicians and systems and may help inform future development of patient engagement strategies and patient-reported measures related to the diagnostic process, as advocated by safety leaders.34 60 62
Consistent with prior research, patient and family insights also broaden contributing factors and breakdown impacts beyond those typically considered on incident review.11 40 63 Patients and families hold unique information about their own experience of explicit or implicit bias related to their care, and occasionally described perceived effects of race, literacy or mental health on diagnostic considerations. They also may bring important information regarding barriers to speaking up about diagnostic concerns that could inform new organisational approaches. Some participants recalled stories of missed or delayed diagnoses stemming from not knowing warning signs, or not knowing how to escalate if they sensed symptom progression but did not feel heard. Many patients and families also described a broader set of potential impacts of care breakdowns including emotional, financial and relational effects,64–67 as well as a few situations in which the breakdown led patients to change providers or leave care—a PRDB framework metric that may be of particular interest to organisations. Collectively, these insights resonate with current emphases on equity, social determinants of health, and measurement and prevention of emotional harm.68–71
Communication and Respect is an important new addition to clinician-based diagnostic error frameworks. The PRDB framework draws attention to the significance of patients ‘not feeling heard,’ and misalignment between patient and provider perceptions of concerns as a potential risk for delayed diagnosis, reinforcing prior patient studies.10 36 55 In addition, disrespect and ignoring patient knowledge are increasingly recognised as preventable breakdowns,10 66 and may carry broad and profound impact on the diagnostic process and safety (such as loss to follow-up or leaving care); they should therefore be considered in assessment of PRDBs.68 72 Finally, patients occasionally identified disagreements between clinicians or inadequate discussion about diagnostic uncertainty, sometimes taking it upon themselves to resolve next steps.73 Clinicians may welcome guidance as they grapple with their own uncertainty about how much to share with patients, and patients may benefit from education about inherent uncertainties in diagnosis.31 74 75 Collectively, our findings suggest a potential role for feeling heard and respected, alignment and discussion of uncertainty as organising principles in future diagnostic process-related communication practices and competencies.
How organisations can use the PRDB framework
The PRDB framework demonstrates how traditional clinician-centric measures may miss important events and safety and quality vulnerabilities, and how patient perspectives may give organisations new insights into how to improve care. However, these improvement opportunities can only be realised if linked to organisational safety processes.12 60 Organisations can use the PRDB framework in several specific ways (box 1). The framework may be a useful way to systematically categorise PRDBs in different clinical settings and among varied patient populations through research (‘Understand’), in order to ultimately guide targeted prevention efforts. Because PRDB categories were tested with a patient coder, it may also advance patient participatory research efforts.11 For cases that come to attention from clinician reporting or EHR triggers, consideration of the PRDB framework may prompt reviewers to ask different questions and seek patient/family input (‘Listen’). The framework may help guide patient and family interviews to further explore breakdowns, contributing factors and impacts.76–80
Improving diagnosis together with patients and families: four ways organisations can use the patient-reported diagnostic process-related breakdown (PRDB) framework
UNDERSTAND: analyse patient reports to better understand (and prevent) PRDBs
Apply the PRDB framework to unsolicited patient reports such as patient-reported safety events or patient ‘complaints’ involving diagnostic breakdowns.
Solicit and analyse patient reports of ambulatory diagnostic concerns,60 to characterise frequency and types of PRDBs for quality improvement (QI) purposes.
Use the framework as a research and improvement tool to study PRDBs in different populations or care settings such as vulnerable patient populations or emergency department care.
Advance patient participatory research11 91 in diagnostic error using an instrument tested with a patient coder.
LISTEN: guide patient/family debriefing about diagnostic breakdown events
Train patient relations/patient experience representatives and consider incorporating the PRDB framework into safety event management software to appreciate and categorise the range of problems patients may raise.
Use the PRDB framework to structure interviews with patients and families after harmful events, such as CRP cases of ambulatory diagnostic error, using concepts and terms derived from patient experiences.63 76
Complement clinician frameworks44 45 with the PRDB framework for incident review of clinician-generated cases to prompt new questions and/or seek patient/family input.77
ASK: inform new patient and family diagnostic engagement tools
Develop tools informed by the PRDB framework to prospectively engage patients, families or care partners in diagnosis, such as:
Use patient-reported diagnostic process measures60 62 derived from the PRDB framework in post(symptomatic) visit evaluations, such as: Was your main concern heard? Was the diagnosis/explanation communicated in a way you understood?
Assess the impact of new ambulatory engagement interventions by applying the PRDB framework as a pre/post-instrument to evaluate changes in PRDBs.
LEARN: enhance ‘whole organisational learning’
Make visible the value of patient/family perspectives by presenting cases analysed with the PRDB framework in educational settings such as Morbidity and Mortality conferences.
Provide feedback to QI departments about common themes from aggregated data elicited in organisational experience with use of the PRDB framework by department or clinic.
Share the PRDB framework with PFACs to guide patient–peer educational efforts and shape PFAC participation in improvement efforts.
Communicate lessons learnt, including what will be done differently, with patients and families such as via hospital newsletters, PFACs and patient advocacy groups.
The PRDB framework is derived from patient-reported errors, and therefore reflects things that went wrong. Translating these breakdowns into engagement strategies that help things go right requires further research. However, the PRDB framework can serve as a roadmap to translate measurement into actionable and meaningful steps.37 After ambulatory visits, organisations can proactively query patients about, and meaningfully respond to, the most common PRDBs at their own sites (‘Ask’).17 81 Both systems and cultures that support patient speaking up about PRDBs will be needed. Organisations can also promote strong diagnostic teams by preventing patient-reported pitfalls in the diagnostic process (online supplemental appendix 4).17 81–84 Our group is currently using the PRDB framework to develop and test a prospective tool to engage patients and families in the diagnostic process through sharing and co-producing visit notes.85
Finally, the PRDB framework can support more whole organisational learning (‘Learn’). High-reliability organisations may extend their collection of safety data to prevent harm to routinely include PRDBs.86 Presentation of cases analysed with the PRDB framework in educational settings like morbidity and mortality conferences strengthens the message that organisations value patient partnership, respect and transparency.14 80 87 The PRDB framework can also be shared with Patient and Family Advisory Councils (PFACs) to help shape patient–peer educational efforts and guide PFAC participation in improvement efforts.
Strengths, limitations and future research
The PRDB framework was derived from patient-reported errors in two large surveys, which are subject to response bias. The open notes dataset representing three sites may not be broadly generalisable to all patients, especially since portal registration was a prerequisite for reading notes, and known inequities related to portal use may bias participants.88 89 However, the IHI survey used a patient panel that was representative of the US population and intentionally oversampled for more vulnerable populations, including those who did not complete high school and whose annual income was <$50 000/year. The survey was administered online or by telephone (did not rely on portal use), and was offered in English or Spanish.40 About one-third of respondents met criteria for limited English proficiency based on confidence completing medical forms alone. Nonetheless, the views of patients and families from more diverse backgrounds may not be fully represented.
The main focus of this paper was to develop the PRDB framework and to test the reliability of categorising PRDBs. Data analysed included narratives of patient-reported errors but did not specifically ask respondents to comment on possible contributing factors or the impact of the error. Therefore, these elements of the PRDB framework require further testing. Future research can further expand and test the reliability of patient-reported contributing factors and impacts. Regarding existing categories, telemedicine-related, technology-related or documentation-related breakdowns currently categorised as ‘other’ may merit their own category in settings with substantial use of telemedicine or medical record-related analyses. Impacts may be more coherently organised and tested in the future at the patient or relationship level. Although we used a rigorous process to compare categories for discriminant definitions while applying the framework to the data, some overlap may persist between categories and may be further refined with future application of the framework to a greater number of patient reports. The PRDB framework provides a critical first step in bringing patient perspectives to diagnostic process categorisation. We anticipate that future studies focusing on additionally diverse patient populations and care settings will further refine the framework.
Conclusions
Patients and families are an important part of the diagnostic team, but their experiences with breakdowns are not reflected in most currently used diagnostic measures. As a result, organisations may miss important events that can lead to diagnostic delay, error and harm as well as unique insights held by patients and families about how to improve the diagnostic process. Designed with multi-stakeholder input and based on actual patient-reported ambulatory errors, the PRDB framework fills a gap in existing frameworks, can deepen organisational awareness of diagnostic process vulnerabilities, and establishes a foundation for more meaningful engagement of patients and families in the diagnostic process.
Data availability statement
Data may be obtained from a third party and are not publicly available.
Ethics statements
Patient consent for publication
Ethics approval
The study was reviewed by the IRB at Beth Israel Deaconess Medical Center (Protocol 2019P000970).
Acknowledgments
The authors thank Feleshia Battles-Byrdsong, Mark Graber, and the patients and families who participated in Institute for Healthcare Improvement (IHI) and open notes surveys for their contributions to the framework for patient-reported diagnostic process-related breakdowns. They thank Amanda Norris for her contributions to figure design. They also thank IHI and the Lucian Leape Institute for sharing the IHI survey results.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @fcbourgeois, @cmd418, @TheLizArmy, @EJThomas_safety
Contributors SKB conceived the study, obtained funding, led the research, drafted the manuscript and is the guarantor of the submitted manuscript. EJT, FB, SL, LHN and CD contributed to the grant proposal for study funding. FB, LS, and SB conducted the qualitiative analysis. JD and LHN led the statistical analyses. KH participated as a project coordinator and research assistant. FB, CD, SL, EL, PM, LHN, SN, JR, LS, SSc, SSh, LS-H, GT, and FB-B (acknowledgement) participated in the Metrics Advisory Group. AN (acknowledgement) contributed to supplementary material figure design. All authors reviewed and approved the manuscript prior to submission. Each revision and final proofs were also shared with each author for review and feedback.
Funding Support for this work was generously provided by AHRQ (grant number: 5R01HS027367-02).
Competing interests None declared.
Patient and public involvement statement Patients and family members of patients participated in the PRDB framework development from project inception to publication (6 authors).
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.