INTRODUCTION

Comprehensive and timely communication between hospital-based consultants and community-based physicians providing post-discharge care is essential to ensure safe transitions from the hospital back to the community. Nonetheless, information transfer after discharge remains poor.1

The hospital discharge summary has been the tool of choice for the communication of information after hospitalization.2 A high-quality discharge summary provided to the right care provider at the right time has the potential to reduce adverse events after discharge, decrease healthcare costs, and promote positive outcomes for patients.3,4 Despite the proven benefit of timely and comprehensive discharge summaries, the availability of a discharge summary at the first post-discharge physician visit is as low as 12% to 34%.5 There may also be deficiencies in discharge summary content5–8 and accuracy.5,9 Community-based physicians have attributed more than one-third of post-discharge adverse events to poor transfer of information.10

Earlier work suggests that compared to conventional discharge summaries, electronically created discharge summaries have a greater likelihood of being generated, are more accurate, are delivered faster than dictated summaries, and are preferred by community physicians.11–15 In contrast, a recent study reveals that electronic discharge summaries often lack crucial information.16 There is also growing evidence that computer-generated clinical documentation as a whole is rated poorly by end users.17 We designed a randomized controlled trial to compare electronic discharge summaries with those generated by conventional dictation, using PCP satisfaction as the primary outcome.

METHODS

Setting

The study took place during May and June 2008, on the General Internal Medicine (GIM) service at St. Michael’s Hospital, a 513-bed tertiary care teaching hospital affiliated with the University of Toronto. At the time of the study, the hospital’s health information technology included online results viewing (Soarian; Siemens), but not clinical documentation or computerized physician order entry. The 64-bed GIM service consists of 4 clinical teaching teams composed of an attending hospitalist physician, and a group of housestaff (a second- or third-year internal medicine resident, 1 or 2 interns, and medical students in their penultimate or final year of undergraduate medical education). All housestaff are responsible for generating discharge summaries for their patients.

Inclusion/Exclusion Criteria

All patient discharges from the GIM service that occurred during the study period were eligible for inclusion. We excluded discharges that were transfers to another service, as well as those for patients who died during their hospital stay, or remained in hospital past the dates specified in the study protocol. We also excluded discharges from the ward that had been prepared by services other than GIM, discharges from the nonteaching GIM team, and the second discharge for patients who had been readmitted during the study period.

Dictated Summaries

Housestaff generated dictated discharge summaries by reciting their report into the hospital’s telephone-based dictation system. The housestaff had discretion over the information included and how this information was organized. Once dictated, the summaries were sent to an external firm to be transcribed, then returned to the hospital, uploaded to the hospital information system (HIS), and sent out to the PCPs. Dictated summaries did not require attending physician authentication before being posted to the HIS.

Electronic Summaries

The customized electronic discharge summary (EDS) program contains fields that have been shown to improve the quality of a discharge summary.2,4,5 Fields are grouped into 3 separate sections: 1) preadmission information, 2) hospital course, and 3) discharge and follow-up plans. Housestaff completed fields by a combination of free-text entry, cutting and pasting from the HIS patient record, and selection from pick lists (Fig. 1). After the summaries were finalized by the housestaff, they were electronically signed and authenticated by the attending physician, uploaded to the HIS, and sent out to the PCPs. The forms generated included a structured discharge summary report, as well as a computer-generated prescription, and patient letter.

Figure 1
figure 1

EDS web interface.

Full size image
Figure 2
figure 2

Discharge enrollment and outcomes.

Full size image

Trial Description

Our unit of randomization was the admitting team. Two teams were randomized to the dictated discharge summary arm and 2 teams were randomized to the EDS arm. Members of the teams assigned to the dictated summary arm were locked out of the EDS program to prevent crossover. Housestaff were informed of their team assignment at the beginning of the 2-month rotation. Each patient’s discharge summary was prepared either by dictation or by the EDS program, depending on the method assigned to the team to which the patient was admitted. All the teams attempted to contact each patient’s PCP at the time of their admission.

Outcome Assessment

The primary outcome of the study was overall PCP satisfaction. PCPs assessed summaries using a 100-mm visual analogue scale (VAS) ranging from 0 (worst) to 100 (best). A high-quality summary was defined as one that “efficiently communicates information necessary for continued patient care.” To provide more specific feedback, summary completeness (“all necessary information is included”), summary organization (“information is presented in a logical and clear fashion”), and summary timeliness (“time from patient discharge to summary receipt”) were also evaluated using a similar scale. This method of assessment has been previously shown to discriminate between apparent differences in summary quality.11 To confirm that summaries were being sent to the correct physician, we asked physicians receiving surveys if they were the patients’ PCP.

We sent surveys to the PCPs listed on the HIS for each patient discharged. Surveys were sent 1 week after the patient was discharged from hospital, and follow-up telephone calls made 2 weeks after the survey was sent.

At the end of the study period, housestaff completed surveys rating their overall satisfaction with the method of summary generation, its ease of use, and summary preparation time, using a 100-mm VAS.

Chart reviews identified the time from discharge to summary posting on the HIS. We reviewed the HIS 30 days after the patient’s discharge to assess completion of outpatient diagnostic tests ordered during the admission, attendance at specialist follow-up appointments, visits to our hospital’s emergency department (ED), and readmissions at our hospital. These measures have been used previously as indicators of unwanted outcomes after discharge.18 Readmission to the hospital, ED visits, and death, were grouped together to form a combined endpoint for adverse events after discharge.

Because the EDS system included a patient-specific discharge letter, we also assessed patient satisfaction with their care transition by means of telephone interviews, conducted 1 week after the time of discharge. If not initially reached, 1 further attempt was made within the second week after discharge. Patients were asked whether they received a discharge summary letter, whether their preferences for the transition had been taken into account, whether they understood their responsibilities for managing their health, and whether they knew the purpose for taking their medications. The latter 3 questions were used to generate a Care Transitions Model (CTM-3) score (ranging from 0 to 100), which has been previously validated to assess patient care transitions.19

Statistics

Statistical analyses were performed using the statistical software package SPSS version 16.0 (Chicago, IL). Sample size calculations were based on anticipated summary quality. A previous study using a 100-mm VAS identified a mean satisfaction of 75 for dictated summaries, with a standard deviation of 17.11 Our group determined that a 10-point absolute difference in summary quality would be clinically relevant for both PCPs and hospital administrators. With an alpha-error of 0.05 (2-sided) and a beta-error of 0.20 (1-sided), the unadjusted total sample needed to detect a 10-point difference in quality scores (a quality score for the EDS of 85) was 51 in each arm. Assuming a 50% PCP response rate, we calculated an adjusted sample size of 102 in each arm, or 204 total discharges. Given that over an average month, 170 patients are discharged from the GIM service at our hospital, we chose to conduct the study over a 2-month period, encompassing 1 housestaff rotation. We assumed the intracluster correlation coefficient (ICC) would be low (less than 0.1) and would therefore have little effect on sample size.20

Descriptive analyses included means, medians, and standard deviations for continuous variables, and frequencies for categorical variables. We compared the 2 arms using a 2-sided t-test or nonparametric Mann-Whitney test for continuous variables, and Chi-square or Fisher’s exact test for categorical variables.

When the study was completed we estimated the ICCs for each of the PCP survey outcomes using a random intercepts model. These were found to be low, such that accounting for clustering yielded slightly larger confidence intervals, but that the point estimates for the PCP survey results remained essentially unchanged. The results from the standard analysis are presented.

Ethics

Our institution’s Research Ethics Board reviewed and approved the study protocol. Housestaff participating in the study provided written consent after a 1-hour conference during which the study was described in detail. Those not wishing to participate were given the opportunity to opt out of the study, with no consequences to their rotation on the GIM service. The trial was registered with ClinicalTrials.gov (NCT00670865).

RESULTS

Sample Characteristics

The flow of discharges through the randomized trial is shown in Figure 2. Overall, there were 119 surveys returned out of 188 sent out to PCPs (63% response rate). There was no difference in response rates between the 2 arms (56 from the EDS arm [58%] vs. 63 from the dictation arm [68%]; P = 0.15). Of the 119 surveys returned, 24 showed that the responding physicians had not received a discharge summary by the time that they received our survey (9 from the EDS arm vs. 15 from the dictation arm; P = 0.29). One survey from the EDS arm was sent to a physician other than the PCP. Three summaries in the EDS arm were not uploaded to the HIS, representing 1.4% of the discharges eligible for inclusion. Discharge summary evaluation data were available for 46 surveys from the EDS arm, and 48 in the dictation arm. There was no significant difference between the groups in the mean days between patient discharge and the appearance of the discharge summary on the HIS (EDS 3.7 days vs. dictation 3.6 days; P = 0.84).

Baseline characteristics of the patient admissions are shown in Table 1. There were no statistically significant differences in the characteristics of the subgroup of patients for whom surveys were analyzed, as compared to those for whom surveys were excluded.

Table 1 Characteristics of Discharges
Full size table

Primary Care Physician Assessments

The results of the PCP assessments are shown in Table 2. There were no significant differences in PCP ratings of overall quality, summary completeness, organization, or timeliness between the 2 arms.

Table 2 Primary Care Physician Assessment of Discharge Summaries
Full size table

Patient Outcomes

The results of the 30-day patient outcomes are shown in Table 3. A total of 68 patients in the EDS arm and 61 patients in the dictation arm had postdischarge outpatient or specialist follow-up appointments arranged. There were no significant differences between the 2 groups in rates of attendance at outpatient follow-up tests and appointments (P = 0.42), or in the combined endpoint of readmissions, ED visits, or death (P = 0.89). For patient satisfaction, the telephone survey was completed by 50 patients discharged by EDS (48%), and 54 patients discharged with dictated summaries (52%). There was no difference in CTM-3 scores between the group discharged by EDS, and those discharged by conventional dictation (80.3 vs. 81.2 respectively, P = 0.81).

Table 3 Patient Outcomes
Full size table

Housestaff Satisfaction

All of the housestaff on service during the study period participated in the trial. The housestaff survey was completed by 12 of 13 residents on the GIM service during the study period. The overall satisfaction was in favor of the EDS, but was not statistically significant (mean rating 75.7 vs. 44.5; P = 0.10). Housestaff found the EDS easier to use than the dictation system (mean rating 86.5 vs. 49.2; P = 0.03). Time burden was rated as longer in the EDS arm than in the conventional arm, though the difference was not statistically significant (mean rating 36.8 vs. 55.2; P = 0.23, where 0=time consuming and 100=not time consuming).

DISCUSSION

We conducted a randomized controlled trial comparing traditional dictation with a novel web-based computer program, as methods of discharge summary generation. There was no difference in PCP overall satisfaction between these 2 methods. Patient satisfaction, completion of outpatient follow-up, and return to the hospital, were similarly unaffected. Housestaff appeared to prefer the EDS method overall.

Although an EDS program such as ours might seem intuitively to be superior to conventional dictation, existing evidence suggests otherwise. Electronic documentation systems can be overly inclusive, resulting in “information overload” and loss of focus on useful data.17 Copy and paste functions, used extensively in systems such as ours to transfer imaging and other reports, have been associated with inconsistent and outdated information.21 Electronic discharge summaries might also be deficient in information, compared to those produced by hand.16

Ours is the first randomized controlled trial to evaluate the use of a secure web-based computer program to generate individualized discharge summaries. In a previous study, discharges created from computer database information were compiled manually by 1 of the study authors.11 In our study, the medical housestaff used the EDS directly to prepare discharge summaries. Compared to similar previous studies, ours included more clinical endpoints (ED visits, hospital readmissions, completion of outpatient appointments and tests), as well as a previously validated, patient-centered measure of the adequacy of information communication during the transition out of hospital. Our PCP survey response rate was comparable to other similar studies.11,15 In contrast to previous studies of a similar design, our study assessed the quality of summaries in the immediate postdischarge period. By collecting data on community physician satisfaction earlier, our study provides a clinically relevant endpoint, and reduces the potential for recall bias.

The mean overall quality of dictated discharge summaries assessed in our study was higher than that reported for the summaries assessed by similar means in an earlier study.11 This high baseline quality of discharge summaries may have compromised our power to detect any difference from the use of the EDS. Nonetheless, the point estimates for the primary outcomes were similar. Though our results do not demonstrate a significant increase in discharge summary quality with the EDS system, they are reassuring in that no decrement was seen, and the confidence intervals do not include what we considered a clinically important difference. Moreover, the analysis of secondary endpoints suggests that EDS discharges are safe, user-friendly, and patient-centered.

The results of the housestaff survey suggest that the EDS method of preparing discharge summaries was slightly more time-consuming, but preferred overall. Our study was conducted within the first few months of the launch of the EDS, when it had yet to be incorporated into routine practice. We anticipate that the time required to complete discharge summaries by the EDS method (currently estimated at less than 30 minutes) will continue to decrease.

Three discharge summaries from the EDS arm were not posted on the HIS. Possible reasons include failure of the housestaff to generate the summary, and failure of the attending physician to finalize the report. This has been addressed by the addition of an email reminder function for attending physicians, which ensures that all discharge summaries are finalized as quickly and reliably as possible.

An electronic discharge summary program has several potential advantages compared to traditional dictation. First, the discharge summaries generated can be stored as a searchable database, providing a large repository of data that would otherwise be embedded in the paper record. Second, such a program can be modified to include reminders, forcing functions, and constraining functions, and to improve adherence to evidence-based therapies and screening tests for certain common conditions. Third, an electronic discharge summary program can be used to improve transitions during hospitalizations, such as when one resident signs over care of a patient to another at month end.

Further research in this area should focus on determining the effect of computer-assisted discharge summary generation on rates of medication errors after discharge. Qualitative data from focus groups of both PCPs and hospital-based physicians would assist in refining programs such as our EDS to make the discharge process more user-friendly, and to provide more meaningful information in a more efficient manner. A formal study of cost-effectiveness would be useful to inform decisions on the adoption of similar technological innovations.

Our study has a few weaknesses that must be considered. The study period was relatively short, with only 30 days of follow-up for patient outcomes. Only readmissions and ED visits at the study site were included. Our study did not include a run-in period to collect baseline values for PCP assessments of discharge summaries, and did not assess the accuracy of either of the discharge summary methods. Strict blinding of the PCPs to the method of discharge summary preparation may not have been achieved given the differences in output from each system. Nonetheless, we considered any bias from PCPs against or in favour of either method to be an outcome of interest, and therefore appropriately captured by the survey tool. We were not able to reach our prespecified number of discharges, and ultimately included only 45% of the original 209 eligible discharge summaries. This limited our ability to detect statistically significant differences. Finally, our single-center trial may not be fully generalizable to other settings.

We performed a randomized controlled trial of electronic vs. dictated discharge summary generation. The results of our study show that a web-based computer program can produce discharge summaries with overall quality not significantly different from those generated by the traditional method of dictation. Our program was well received by the medical housestaff, resulted in a low rate of summary nonreceipt, and had no discernable negative impact on clinical endpoints. As such, the EDS stands to play a key role in ongoing efforts to integrate computerized tools for patient care and enhance the quality of patient discharge.