A highly influential report from the Institute of Medicine, Washington DC, USA, “To err is human: building a safer health care system”, encouraged better documentation of errors and adverse events, so that causes for problems might be identified and remedied.¹ However, reporting of error and adverse events by health professionals depends on variables such as their awareness of the events, their judgment about events and their willingness to document the events. Incidence rates for adverse events vary 10-fold when reported by health professionals; differences are attributed to professional attitude and judgment rather than the underlying rate of errors and adverse events.^2,3 Since patients directly experience harm from adverse events, investigators have proposed patient-report to complement professional reporting.^4–8 Although the proposition that patient reporting of adverse events has merit, the feasibility of patient-report is not known.

This study investigates how an automated outpatient health assessment system can be used to identify adverse events.

METHOD

The health assessment survey and the study population

More than 20 years ago, the Dartmouth Cooperative Practice-Based Research Network identified important, wide gaps in the communication between patients and doctors.⁹ As an example, this research documented the need for and stimulated the development of patient-report measures of functional status.¹⁰ The Cooperative Practice-Based Research Network now widely disseminates a patient health assessment survey to improve patient–clinician communication.^11–16

Using computer-generated branching logic, up to 120 items of the survey inquire about health status, symptoms, concerns, chronic disease management, preventive interventions, specific healthcare experiences and confidence with self-care (http://www.howsyourhealth.org). Based on their own initiative or at the request of their doctor, people complete the health survey to obtain health and self-management information customised to their needs.

As no personal identifiable information is stored by any means, patients decide on how to use their information. For example, they can deliver a clinical summary of the information (electronically or in person) to their doctor so that they and the doctor can review and manage the important problems identified by the survey in a better way.

Item completion rates are >95%. Whenever a patient reports that he/she has completed the survey in the previous 6 months, the recent information is not stored. The data are exempt from institutional review board review as no personal identifiable data is stored.

This report is based on 44 860 cross-sectional responses by residents of the USA, aged 19–69 years during the 2-year time frame from April 2003 to April 2005.

A measure of adverse events

Near the end of the survey one item asks:

Part A: “Describe here any medical errors (mistakes) that you or your family have experienced. Errors include such things as mixed up medications or poor treatment that result in harm or additional problems. If possible, be sure to tell us the cause of the error and how it might have been avoided. Your response will help us to improve future care delivery.”

Part B: “If you wrote in an error or harm, please help us by choosing ANY of the following categories for this error. (Please mark all that apply.) It caused harm, hurt or injury. It happened within the last year. It happened to me.”

What is an adverse event?

Although an individual clinician receiving a patient report of an adverse event might be able to corroborate it with the information in the medical record, for this large study population of patients only their verbatim comments were available to us. To contend with the possibility that the patients’ reports might represent mere complaints about poor service, we used independent legal review to estimate how many, if any, patient-reports were possibly serious—that is, the percentage of reports that might, after additional review, cross the threshold from nuisance to potentially compensable injury.

A lawyer (MH) with extensive experience in medical malpractice litigation reviewed all patient-reported adverse events and selected those that seemed serious enough to represent compensable injury: when the costs of the legal remedy could be greater than the costs of litigation.

We derived a conservative estimate of potentially compensable injuries by subjecting the adverse events identified by the first legal review to another review. The cases that were agreed upon by two attorneys constitute our estimate of potentially serious adverse events. A verbatim list of these events is available at http://www.howsyourhealth.org/adverse.pdf.

ANALYSIS

We have limited the analysis to descriptive statistics as the data are cross-sectional and are based on a convenience sample of patients out of a much larger potential population of non-respondents in clinical practices or communities.

RESULTS

From the 2-year convenience sample of 44 860 survey respondents, 2979 indicated a harm, hurt or injury to themselves or a family member, and 610 of these adverse events had happened to the respondents during the previous year.

Association of possible adverse events with burden of illness

Table 1 lists the characteristics of patients who reported health-related adverse events and those who did not. All but four (age, hypertension, exercise and excessive alcohol consumption) of the many listed variables were statistically different between the two groups of patients. In the stepwise regression of the demographic and composite variables listed in table 1, burden of illness had by far the greatest association with a report of a possible adverse event.

Figure 1 depicts the strong association between possible adverse events and burden of illness. The annual percentage of health-related adverse events ranged eightfold, from 0.4% to 3.4% across quintiles of illness burden.

Severity of possible adverse events

In all, 574 of the 610 patients provided descriptive information about what went wrong. The majority of these events occurred in an outpatient setting; only 12% (66) of the possible adverse events transpired in a hospital.

Two expert malpractice attorneys agreed that 9% (52/574) of the adverse events seemed serious enough to represent a possible compensable injury. In absolute terms, twice as many serious events were experienced in the ambulatory setting (n = 35) than in a hospital (n = 17). However, in relative terms, 26% (17/66) of the hospital events were possibly compensable compared with 7% of those in an outpatient setting.

COMMENT

This study demonstrates that patients will use internet technology to report their perceptions of health-related adverse events. Patients with a greater burden of illness are more likely to report adverse events than those with little burden. The legal review adds credence to the notion that some patient reports will be more serious than mere complaints about poor service.

These results suggest an inexpensive mechanism by which a clinician can learn about a patient’s perceptions of previous poor care, acknowledge the problem, investigate its cause and initiate a remedy.¹⁷ By being patient-centered and giving the perceptions of possible adverse events a human face, such systems could be useful for influencing the way the outpatient clinicians think about safety.¹⁸ Similar systems are also available for inpatients (http://www.howsyourcare.org).

Despite the promise of such automated systems for assessing health and healthcare, the findings have at least three significant limitations.

First, since the patient denominator is a non-random sample from an unknown population, the adverse events reported in this study should not be interpreted as a benchmark rate based on a national probability sample of patients. A patient-reported adverse event will be biased by the background of the person who chose to respond to an online “health check-up” and their opportunities to interact with the healthcare system.

Second, in addition to the limitations intrinsic to our specific approach, both professional and patient-derived reports of “when things go wrong” have biases that reflect the attitudes and experiences of the reporters, their incentives or disincentives to report, and the ease of use and fidelity of the reporting system.^2–8,19

Finally, as yet there is no proof that patient-reported perceptions of health-related harms will improve the safety and quality of care.

In summary, we have shown how an internet-based, patient assessment survey can produce information about health-related adverse events. More research is needed to demonstrate the value of such automated systems in improving healthcare.

This research was supported by grants from The Commonwealth Fund and the Robert Wood Johnson Foundation.