Article Text
Abstract
Background While physician notes are known to vary in organisation, content and quality, the relationship between note quality and clinical quality is uncertain.
Methods We performed a cross-sectional study of outpatient visit physician notes by adult patients with coronary artery disease or diabetes mellitus seen in 2010. We assessed physician note quality using the 9-item Physician Documentation Quality Instrument (PDQI-9) and compared this to disease-specific clinical quality scores constructed from data extracted from the electronic health record (EHR). We also assessed the presence of typical note subsections, and indicators of quality care in physician notes.
Results We evaluated 239 notes, written by 111 physicians; 110 notes were written by primary care physicians, 52 by cardiologists and 77 by endocrinologists. Reason for visit was absent in 10% of notes, medication list was not present in the note in 19.7% and timing for follow-up was absent in 18.0% of notes. Significant copy/pasted material was present in 10.5% of notes. Laboratory quality indicators were more often found in other EHR sections than in the physician note. Clinical quality scores for diabetes and coronary artery disease (CAD) showed no significant association with subjective note quality (diabetes: r −0.119, p=0.065, CAD: r −0.124, p=0.06).
Conclusions Notes varied in documentation method and length, and important note subsections were frequently missing. Key clinical data to support quality patient care were often not present in physician notes, but were often found elsewhere in the EHR. Subjective assessment of note quality did not correlate with clinical quality scores, suggesting that writing high-quality notes and meeting quality measures are not mutually reinforcing activities.
- Primary care
- Chronic disease management
- Chart review methodologies
Statistics from Altmetric.com
Introduction
While physician notes are essential to patient care, note content is variable,1–3 and note quality is difficult to define. Some structure is widely used and most often includes a narrative subjective account, past medical history, medications, physical exam findings test results and an assessment and plan, often by clinical problem, as suggested by Weed.4 Notes serve multiple roles, including as a reminder to an individual physician of what transpired during a given encounter, a means for communication among multiple providers, a teaching tool for students and residents, as documentation for reimbursement and assessment of quality, and as evidence in legal proceedings.5 The quality of physician notes has been variably defined by presence of specific note elements,6 by note completeness or accuracy,7–9 and by subjective assessment of form and content.10
Electronic health records (EHRs) are being rapidly adopted nationwide in the USA,11 and electronic documentation may improve diagnosis and quality of care.12 The nature of clinical documentation has changed under EHRs, with the physician note serving as only one part of an electronic clinical record that may also include automatically imported lab and test results, and structured data fields often filled in by non-clinicians. While EHRs are associated with more complete documentation,6 there is also substantial concern that the quality of physician notes may be deteriorating under EHRs, due to a lack of narrative, inappropriate use of copy-and-paste,13 increasing length and redundancy14 and poor formatting.15 However, there is little empiric data on the content of physician notes in EHRs,16 and very limited evidence is available regarding the relationship between physician note quality and the clinical quality of care provided.
Chronic diseases account for a great deal of overall resource usage.17 In this study, we describe the content and composition of typical physician notes for the chronic care of two common and costly chronic diseases, diabetes mellitus (DM) and coronary artery disease (CAD), in visits to general internists, cardiologists and endocrinologists. We describe the presence of quality indicators in physician notes versus elsewhere in the EHR and compare the subjective quality of notes to quality of care. We hypothesise that under EHRs, less data to support the quality of care will be reported in notes, as it is present elsewhere in the EHR, but notes will still share at least some common elements such as a reason for visit, and an assessment and plan. We further hypothesise that notes that physicians perceive as high quality, will be associated with high-quality care, suggesting that writing high-quality notes and delivering high-quality care are synergistic activities. We distinguish in this study between notes, which are typically textual and encounter-based, and other EHR documentation which is recorded in many other places including the medication list, laboratory results, and other locations. Our underlying aim was to explore the role that outpatient physician notes play in the care of patients with chronic disease.
Methods
Sample
We identified general internists, cardiologists and endocrinologists at clinics affiliated with Brigham and Women's Hospital or Massachusetts General Hospital, who used the Longitudinal Medical Record (an internally developed EHR) in 2010. Physician notes were created using free text, dictation, or note templates. Templates in the Longitudinal Medical Record are created by users, and can include data that is prepopulated from other parts of the record, such as medication lists, problem lists, or lab data. Physicians were categorised based on the documentation method (free form, dictation, template) they used most during 2010. We then identified moderately complex visits to these physicians for DM, CAD or both. Multiple visits by the same physician/patient pair and notes that were patient letters, or resident preceptor notes were excluded. We then selected a random sample of 239 notes from these visits.
Instrument development and data collection
We developed our data collection instrument after reviewing the literature on documentation quality, and with the assistance of clinical experts. Two physicians reviewed all notes in the sample. We initially pretested our instrument with 10 notes, and reviewed responses with the physician reviewers to ensure a common understanding of questions and definitions. Data collected included note length, presence of typical note sections, such as reason for visit, vital signs and medication list, and presence of copy/pasted data. Copy/pasted data were defined as content that appeared to the reviewer to be copied from elsewhere in the record and pasted into the note. Reviewers determined if a new symptom was present and, if so, whether the symptom was sufficiently described, whether a differential diagnosis was considered, and a final diagnosis documented. Reviewers also assessed if there was an adequate assessment and plan recorded. For each response, reviewers noted absence of documentation, partial documentation, or complete documentation of that note subsection. Examples of questions in our data collection instrument are included as online supplementary appendix figure 1.
Reviewers assessed the presence of data supporting quality care of diabetes and CAD in physician notes. For CAD, this included documentation of blood pressure and Body Mass Index (BMI), lipid panel within the last year, and current oral antiplatelet therapy and β-blocker use, if not contraindicated. For diabetes, this included documentation of blood pressure and BMI, lipid panel within the last year, HbA1c testing, urine microalbumin testing, diabetic eye and foot exam within the last year, current use of ACE inhibitors or angiotensin receptor blockers, and current use of any antihyperglycemic medication.
Reviewers assessed subjective note quality using a single-item general impression score, and the nine-item Physician Documentation Quality Instrument (PDQI-9, see online supplementary appendix table 1).18 The general impression score was a response to the question: ‘please rate the overall quality of this note’ with a 5-point Likert scale from 1 (‘very poor’) to 5 (‘excellent’). This question was intentionally open ended to explore what latent constructs drive physician impression of note quality. The subdomains of the PDQI-9 are up-to-date, accurate, thorough, useful, organised, comprehensible, succinct, synthesised and internally consistent. For each subdomain, reviewers responded using a 5-point Likert scale indicating how consistent the note was with that subdomain, from 1 (‘not at all’) to 5 (‘extremely’). The PDQI-9 was developed using admission notes, progress notes and discharge summaries from the internal medicine service at a single academic medical centre. The PDQI demonstrates strong discriminant validity and reliability of internal consistency. As Stetson et al18 demonstrated that the single-item general impression score shows strong correlation with the sum of the PDQI subscales, we used the single-item general impression score as our primary measure of subjective note quality. Correlation between quality scores of our two physician reviewers was good (r=0.5), and we report the mean of quality scores, and PDQI subscales of the two reviewers.
Measures of quality of care for CAD and DM were assessed at the patient level at the time of the index visit using data extracted from the EHR. CAD quality measures included BMI screening at the visit, blood pressure measurement at the visit, blood pressure control (<140/90) at the visit, cholesterol measurement within the year prior to the visit, cholesterol control (low density lipoprotein (LDL)<100) on the most recent lipid panel within 1 year, oral antiplatelet use at the visit and β-blocker use at the visit. Diabetes measures included BMI screening at the visit, blood pressure measurement at the visit, blood pressure control (<140/90) at the visit, cholesterol measurement within the year prior to the visit, cholesterol control (LDL<100) within the last year, HbA1c measurement at the visit, control (HbgA1c<9.0%) within the year prior to the visit, eye and foot exam within the year prior to the visit, ACE inhibitor or angiotensin receptor blocker use at the visit, and antihyperglycemic use at the visit. Clinical quality scores were created by assigning one point for each of the metrics met, and dividing by the total number of metrics.
Power calculation
We selected a random sample of notes from the identified visits, ensuring at least 90 primary care provider (PCP) notes and 90 specialist notes to ensure adequate power to account for the effect of specialty and documentation style (free text, dictation, template). We calculated that with a general impression score SD of 0.89, we could detect a difference of 0.66 points between documentation styles for a given specialty, with 80% power at a significance level (α) of 0.05.
Statistics
Descriptive statistics are presented as proportions and means with SDs. Comparisons between groups are made using χ2 tests and ANOVAs as appropriate. McNemar's test was used to compare the proportion of records where quality indicators were present in either the physician note or elsewhere in the EHR. We constructed multilevel linear regression models to examine the relationship between (1) general impression score and disease-specific quality indicators and (2) PDQI scales and disease-specific quality indicators. Models were adjusted for documentation style, note length and physician specialty, and clustered by individual physician. We also examined correlations between individual PDQI subscales, and the overall note quality general impression score.
Results
We evaluated 239 notes written by 111 physicians (table 1). Of these, 110 notes were written by primary care physicians, 52 by cardiologists and 77 by endocrinologists. For their documentation style, PCPs used templates and free text predominantly (42.3% and 40.9%, respectively). Cardiologists used free text predominantly (65.2%) followed by dictation (34.8%) and endocrinologists primarily used free text (58.8%) followed by templates (35.3%). Mean note length was 619 words for PCPs, 536 words for cardiologists and 659 words for endocrinologists (p=0.04). Evaluation of note subsections revealed many gaps (table 2). Reason for visit was absent in 10.8% of notes, medication list was absent in 19.7% of notes, and timing of follow-up was absent in 18.0% of notes. Copy/pasted material was present in 10.8% of notes overall and was more common in endocrinology notes (19.5%) and less common in cardiology notes (1.9%). In notes where a new symptom was present, the symptom was not described or only partially described in 38.8% of notes, a differential diagnosis for that symptom was not described or only partially described in 71.6% of notes, and a final diagnosis for that symptom was not documented, or only partially documented, in 50.8% of notes.
Data that supported quality care were often found in different parts of the medical record (table 3). Laboratory quality indicators in particular were more likely to be present in locations in the EHR other than the physician note. For example, a lipid panel in the prior year for CAD was present in 57.7% of physician notes while it was present elsewhere in the EHR in 84.6% of records (p<0.0001), while haemoglobin A1C within the last year for diabetes was present in 73.9% of physician notes, and was found elsewhere in the EHR in 92.5% (p<0.0001). Overall medication-based quality indicators were present in similar proportions in physician notes and in the EHR medication list. One exception was antihyperglycemic use in diabetics which was more commonly present in the note than in the EHR medication list (82.4% note vs 68.6% EHR medication list, p=0.0003). Data to support quality care was more often present in either the physician note or elsewhere in the EHR, than in the physician note or elsewhere in the EHR alone (eg, β-blocker for CAD, 67.3% present in the note, 59.1% present elsewhere in EHR and 79.1% present in either).
Between specialties, there were significant differences in documentation of quality metrics. Endocrinologist patient records had blood pressure and BMI more often elsewhere in the EHR than PCP charts (Blood pressure measurement 71.2% present elsewhere in EHR for PCPs, 89.0% present elsewhere in EHR for endocrinologists, p<0.0001). Endocrinologists also documented eye exams for diabetic patients more often in physician notes than PCPs (endocrinologists 64.4% vs PCPs 39.4%, p<0.01). PCP records contained ACE/ARB use in the EHR medication list more often than for endocrinologists (PCPs 63.6% vs 43.8% endocrinologists, p=0.02). Cardiologists less commonly documented a lipid panel in the last year than PCPs for patients with CAD in the physician note and elsewhere in the EHR (eg, present in note, 40.4% for cardiologists vs 68.1% for PCPs), and less commonly documented BMI in the physician note than PCPs (BMI in physician note 36.2% for cardiologists vs 61.7% for PCPs, p=0.01).
Subjective note general impression scores, PDQI subscales and PDQI total scores are presented in table 4. General impression scores showed no significant association with composite quality scores for diabetes (correlation coefficient −0.119, p=0.065) or CAD (correlation coefficient −0.124, p=0.06). The PDQI subscale ‘accurate’ decreased with increasing clinical quality (p=0.01) for patients with CAD. For patients with diabetes, the PDQI subscale ‘comprehensible’ decreased with increasing clinical quality (p=0.04) (table 4). Multivariate models controlling for documentation style, note length and physician specialty, and clustered for individual physician showed no significant association between the general impression score and disease-specific quality scores. The PDQI subscales that correlated most with the general impression score were ‘useful’ (r=0.75, p<0.001), ‘synthesised’ (r=0.70, p<0.001), and ‘organised’ (r=0.64, p<0.001).
Discussion
We analysed physician notes for the outpatient management of diabetes and CAD, two common and important conditions, and observed wide variation in documentation method, note length and note content. Note length was slightly shorter among cardiologists, than among PCPs or endocrinologists, but the variation of note length overall was large and overlapping. Expected subsections were missing, many from notes, and data related to quality measures, such as laboratory values and medications, were often not present in physician notes, but were present elsewhere in the EHR. There was no clear relationship between quality of notes and quality of care.
Our work is consistent with prior work and extends it in several ways. Efforts to assess the quality of clinic notes frequently have focused on the presence or absence of specific note elements. For example, Dawes observed wide variation in note style and deficiencies in note sections including medications, symptoms and physical exam findings in a sample of notes by general practitioners.19 Tang and colleagues measured the presence of problem lists, medication lists, allergies and appropriate assessment and plan in their assessment of note quality,6 while others have advocated assessment of completeness and correctness of notes as a measure of quality.7 ,8 Consistent with these studies, we observed that many notes are missing critical elements, including reason for visit, symptom description and assessment and plan, underscoring that there are not clear standards of what should be included in every progress note. In EHRs, where elements such as the medication list or vital signs can be input into dedicated fields separate from the note, it is unclear whether it is necessary or helpful to have these data repeated in the note, but it seems that all notes should include the reason for visit and planned follow-up. Thus, our study suggests physician notes continue to lack critical information.
We also observed that data to support high-quality care were present in multiple locations in the EHR, consistent with prior work.20 Laboratory quality markers were more often found elsewhere in the EHR rather than in the note, consistent with our hypothesis that quality data may be less likely to be present in physician notes under EHRs. Given the automated entry of lab data into the EHR, it makes sense that lab-based quality measures would more commonly be present, although a significant lab result recorded elsewhere and was never noted by the physician could also represent a quality issue. Conversely, use of antihyperglycemic medications in diabetic patients was more often present in the physician note than in the EHR medication list, and other markers, such as eye exam in diabetic patients and vital signs, were more often present in the physician note than elsewhere in the EHR. The lower rates of these quality data outside of physician notes in the EHR, suggest that the structured fields for these data are underused, including the medication list feature, and these data are still manually entered into the notes. Overall, the quality of patient records in which a given quality marker was present in either the note or elsewhere the EHR, was higher than in either the note or elsewhere in the EHR alone. This suggests that different physicians document these critical data to support quality care in different ways, and that automated assessment of quality of care using non-note EHR-based data may underestimate the quality of care delivered.
A significant challenge in comparing subjective physician note quality and quality of care as measured by automated data extraction, is that the concepts of note quality and clinical quality have historically been combined. We might think that physicians would perceive notes that document good care to be ‘good notes’, even if the notes had poor organisation, or missing data. Also, prior studies have used documentation of recommended care in physician notes as a marker of note quality,16 and often clinical quality is assessed through review of physician notes, mixing the concepts of physician note quality and clinical quality of care.21 In the EHR era, clinical quality is frequently measured through automated extraction of data from structured fields in the EHR20 ,22 and hence, current quality standards are often not dependent on the physician note.
In our work, subjective note quality did not correlate with EHR-extracted disease-specific quality indicators, either alone, or when controlled for physician specialty, note length and documentation style. We did observe that notes that scored more highly in the ‘accurate’ subscale for CAD, and for the ‘comprehensible’ subscale for DM, were associated with lower disease-specific quality of care, suggesting that for at least some note characteristics in some patient subsets, there may actually be a negative relationship between subjective note quality and clinical quality. Physician reviewers found notes that scored highly on the general impression score also scored higher on the PDQI subscales ‘useful’, ‘synthesised’ and ‘organised.’ Reviewers also observed that notes with clear narrative, and that appeared organised and easy to read, often lacked key quality data, while notes with automatically imported data and copy/pasted elements contained more quality indicators but were often long, less organised and more difficult to read, leading to lower subjective quality scores. Hence, it appeared that subjective note quality was driven less by exhaustive documentation of quality markers, and more by the synthesis, organisation and clinical usefulness of a note.
Ideally, EHR documentation systems would support the creation of subjectively high-quality notes and help ensure performance and documentation of delivery of high-quality care. The EHR could provide clinical guidelines and relevant results to clinicians as they write notes, supporting their clinical decision making and encouraging high-quality care. This would also free clinicians to focus on what they are best at, synthesising complex data, and writing high-quality notes that are easily understood by other clinicians. Our work suggests that current documentation practices are far from this ideal. The fact that clinical quality and subjective note quality are not closely correlated suggests that meeting quality measures and writing high-quality notes are distinct and unrelated activities.
As the EHR changes the ways in which medical data are collected and organised for patient care, the role of the physician note must likewise be carefully re-examined.12 With paper charts, physician notes often served as a common record where key data were manually compiled for easy retrieval. In current systems where many data elements are automatically imported and are easily accessed and trended over time, there may be less value of reproducing these data in a physician note. Similarly, in EHRs that have structured data entry for vital signs, medications, or presence of routine screening tests, there may be less value in repeating these elements in the note. However, there may be some value in adding medication lists to notes, as it allows for assessment of changes in medications over time, while EHR medication lists typically only report the current medication list. Our work suggests that use of structured data elements is suboptimal, and that data to support quality care are sometimes only in the text of the note. Again, there is still significant debate regarding what should actually be included in notes, especially as more information is recorded in other locations.23
This study has a number of limitations. First, it was conducted at practices at two centres, and only two conditions and two subspecialties were evaluated, thus, the results may not be generalisable to other settings, conditions or practice types. Second, the assessment of copy/pasted data was based on the reviewer's impression of obvious occurrences, not on a systematic or automated review of the clinical record. This likely led to a substantial underestimation of copy/pasted material. Third, note quality ratings by reviewers are inherently subjective, and the PDQI was initially developed for inpatient documentation. However, the PDQI shows excellent internal consistency and was tested on multiple types of notes.18 Finally, we did not perform an external validity check on the quality parameters in this study, although issues have been identified with quality measures extracted from EHRs.20 However, the measures used in this study have been in place for some time, and operational assessments done in prior years have suggested good validity.
In summary, notes for the management of CAD and DM vary substantially in length, documentation method and subjective quality, and data to support high-quality care are found in multiple sections of the EHR. Subjective judgments of note quality were not associated with measured quality of care. As EHRs continue to be adopted and evolve, evaluation is needed for examining what information should be included in notes, and to what extent notes should include repetition of data present elsewhere in the record. Further research is needed to understand how EHRs can simultaneously support subjectively high-quality physician documentation and high-quality care.
Acknowledgments
E John Orav provided assistance with sample size determination, and general statistical guidance. This study was presented in abstract form at the Society for General Internal Medicine Annual Meeting, Denver, CO 24–27 April 2013.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online supplement
- Data supplement 2 - Online figure
Footnotes
-
Contributors Study concept and design, analysis and interpretation of data, critical revision of the manuscript for important intellectual content: all authors. Acquisition of data: STE, PMN and LAV. Drafting of the manuscript: STE and DWB. Statistical analysis: STE. Obtained funding: DWB, GDS, LAV and PMN. Administrative, technical and material support: PMN and LAV. Study supervision: DWB and GDS.
-
Funding This work was supported by a grant from the Partners Siemens Research Council.
-
Competing interests None.
-
Ethics approval This study was approved by the Partners Human Research Committee.
-
Provenance and peer review Not commissioned; externally peer reviewed.
-
Data sharing statement The data is housed at the Clinical Quality and Informatics group at Partners Healthcare in Wellesley, Massachusetts.