METHODS

Data

Data on malpractice claims closed between 1 January 1990 and August 1997 were drawn from the databases of three states (Florida, Maryland, and Indiana) in 1998.^25–27 All three states require that any malpractice claim, regardless of payment amount, must be reported to the relevant state regulatory body when the claim is closed or settled. These databases are publicly available or are available on request for research purposes and contain detailed claim information including defendant name, nature of the injury, payment (if any), and dates of the lawsuit and closure. Malpractice claims are defined as any formal written demand for payment including—but not limited to—lawsuits filed in the state court.

Data on malpractice claims were merged with the American Medical Association’s (AMA) Physician Masterfile to obtain medical school and demographic information on physicians who did and did not have claims settled against them during our reporting period. Since numerical identifiers such as social security number or license number were not available, a matching algorithm was developed using last name, first name, date of birth, and current state of residence. This matched some 50% of the claims with the appropriate physician. Manual examination of the data was necessary for another 42% of the claims which contained errors in name spelling, changed state of residence, or multiple physicians with the same name. For manual merges, all identifying information—including address and specialty—were used to find matches. Although approximately 8% of the claims could not be merged with the Masterfile data, it is unlikely that these unmatched claims are consistently related to any particular school and therefore should not bias our results.

Claims rate measures

Measure 1

Our initial claims measure of interest was the percentage of graduates from a medical school for a given period of graduation† who had a claim closed against them between 1 January 1990 and 31 August 1997. Because we had claims information from just three states, we only looked at schools with at least 5% of their graduates living in these states during the time periods in question. A total of 47 schools (out of 122) were eliminated because of insufficient data. This approach avoided some of the problems associated with assigning outlier status to schools on which we had very little history on graduate claims.

Measure 2

Because there is some evidence that malpractice claims may be related to particular physician characteristics, we calculated a second claims measure that adjusted for some of these factors—that is, we constructed a “risk adjusted” measure of claims rates for each school’s graduates. Specifically, we calculated the difference between the observed rate of closed claims for each school/time period observation and the expected rate of closed claims by estimating the following physician specific equation using logistic regression:

where Y was equal to 1 if a particular graduate had any closed claim between 1 January 1990 and 31 August 1997 or otherwise equal to 0. Specialty comprised a series of dummy variables for the following (commonly sued) specialties: anesthesiology, obstetrics/gynecology, specialty surgery (orthopedic, neurosurgery, plastic, cardiothoracic), all other surgery, radiology, and emergency medicine. The omitted (comparison) category was all other specialties. Expected claims rates for each school/time period observation were calculated by entering the actual values of age and specialty of the graduates of a particular school/time period and deriving the predicted probability.‡

The difference between the observed (actual) and expected rate of claims for each school/time period was used as our second outlier measure. This variable is similar to those constructed in the health services research literature when comparing observed to expected mortality rates for a given hospital or provider.^28,29

Descriptive statistics for both malpractice claim rate measures are shown in table 1.

The two claims variables (measures 1 and 2) were then used to identify two sets of outlier medical schools for each time period. A medical school was classified as a measure 1 high (or low) outlier for a particular time period if the percentage of its graduates for that time period who were sued exceeded the 90th percentile (or was below the 10th percentile) of the time period specific distribution. From table 1, this definition implies that school A is a measure 1 high outlier for the period 1960–9 if more than 24.3% of its graduates from that time period had a claim closed against them in our claims dataset. Similarly, a medical school was classified as a measure 2 high (or low) outlier for a particular time period if the difference between the actual percentage of graduates sued and the predicted percentage of graduates sued exceeded the 90th percentile (or was less than the 10th percentile) of the time period specific distribution. From table 1, this definition implies that school B is a measure 2 low outlier for the period 1970–9 if the actual rate of claims per graduate for the time period was less than the predicted rate of claims by at least 0.064.

Given the available data, we restricted our study to US medical schools. In addition, allowing for the fact that schools and their curricula change over time and that recruiting foci also change, we divided a school’s malpractice claims record into four time periods (pre-1960, 1960–9, 1970–9, and 1980–9).

The fact that a physician’s own malpractice history contributes to his or her school’s outlier status raises an issue of endogeneity that must be addressed in the empirical estimation. In order to “break” this circular definition, we used lagged values of a school’s outlier status to predict physician probability of malpractice claim. Thus, if a physician graduated from medical school in the 1980s, his or her medical school’s outlier status for the 1970s was used to predict the physician’s probability of malpractice claim.

Analyses

The intention of our analyses was to examine the predictive power of measure 1 and measure 2 medical school outlier status (high or low). Our approach was twofold:

(1) We calculated the transition probabilities between the various “states” for both outlier variables—that is, we examined the probability that a high outlier in one period remained a high outlier in the subsequent period compared with changing to a mid-range school or to a low outlier. Our null hypothesis was that the outlier status of a medical school was a completely random event. In this case, the probability of moving from being a high or low outlier in the initial period to becoming a mid-range school in the subsequent period should be approximately the same as the probability of being a mid-range school in both time periods (fig 1; pr2 = pr5 = pr8). Conversely, schools that are classified as mid-range in the initial period should be just as likely as high or low outliers to become high or low outliers in the subsequent period (fig 1; pr1 = pr4 = pr7; pr3 = pr6 = pr9). Figure 1 outlines the probability tree(s) we sought to estimate.

(2) We estimated a logistic regression predicting any claim for an individual physician (based on our database between 1 January 1990 and 31 August 1997) using the observable characteristics derived from the AMA Masterfile (age, specialty), the percentage of the medical school’s graduates (for the decade) who were currently living in Florida, Maryland, or Indiana,§ and two dummy variables indicating whether the physician graduated from a school that was classified as a high or low outlier in the time period immediately preceding the physician’s time period of graduation. For example, if John Smith graduated from University Medical School in 1985, dummy variables indicating whether University Medical School was a measure 1 or measure 2 high or low outlier school in the 1970–9 period were included in the predictive equation. As discussed earlier, these lagged versions of the outlier variables were used to avoid any statistical bias due to endogeneity between the probability of a lawsuit and medical school outlier status.

RESULTS

Approximately 11% of physicians in our sample were sued at least once during the study time period. Their mean age was 45 years and one third were international (outside the US and Canada) medical school graduates (IMGs). Using our medical school outlier classification scheme, 3.7% of physicians in the sample graduated from a “low outlier” medical school and 10.3% graduated from a “high outlier” medical school. These descriptive data, together with a breakdown by major specialty, are shown in table 2.

Objective 1

The transition probabilities between the three “states” (below 10th percentile outlier, mid-range school, above 90th percentile outlier) do not support the null hypothesis that medical school outlier status is a random event (table 3). Specifically, medical schools that were classified as mid-range schools in the initial period were more likely to be mid-range schools in the subsequent period than high and low outliers were likely to become mid-range schools (measure 1: 81% v 68% (low outlier → mid-range) and 68% (high outlier → mid-range); measure 2: 79% v 68% and 68%). Conversely, medical schools that were classified as low outliers (below 10th percentile) in one period of graduates were more likely than mid-range schools to be low outliers in the subsequent period (measure 1: 32% v 9%; measure 2: 32% v 10%) and medical schools that were classified as high outliers (above 90th percentile) in one period were more likely than mid-range schools to be high outliers in the subsequent period (measure 1: 32% v 10%; measure 2: 32% v 11%). Most striking, perhaps, are the probabilities of low outlier schools becoming high outlier schools in the subsequent period and vice versa. In our analyses, none of the low outlier schools became high outlier schools and none of the high outlier schools became low outlier schools in the subsequent period. These results support the hypothesis that there is something consistent about the malpractice experience of graduates of high and low outlier schools over time.

To examine the statistical significance of lawsuit probabilities between medical school cohorts, we examined the correlations between the lawsuit rates of successive cohorts—for example, pre-1960 graduates with 1960–9 graduates; 1960–9 graduates with 1970–9 graduates. All correlations were relatively high and all were statistically significant at p<0.001. The correlation between the percentage of pre-1960 graduates and 1960–9 graduates sued was 0.49; between 1960–9 and 1970–9 graduates the correlation was 0.55; and between 1970–9 and 1980–9 graduates the correlation was 0.50.

Objective 2

Table 4 shows the results of our second set of analyses, predicting any claim for an individual physician (based on our database between 1 January 1990 and 31 August 1997). Explanatory variables included the observable characteristics derived from the AMA Masterfile (age, specialty), the percentage of a medical school’s graduates (for the decade) who live in the three states, and two dummy variables indicating whether the physician graduated from a school that was classified as a high or low outlier in the time period before the his/her time period of graduation. Since the logistic results using either measure 1 or measure 2 outliers were qualitatively identical, we only present results using measure 2 (“risk adjusted”) outliers. Both models had a reasonable goodness of fit (score statistics significant at p<0.0001; Hosmer Lemeshow tests failed to reject model (p=0.26; p=0.30)). The predictive ability was relatively low, as might be expected with our relatively sparse specification (c-statistic =0.70 for both).

The odds ratios on the physician characteristics are not surprising. Increasing age is positively associated with the probability of malpractice claims experience, but at a decreasing rate. Obstetricians/gynecologists, surgical specialists, general surgeons, radiologists, and emergency medicine specialists were more likely to be sued than the comparison category (all other physicians). Anesthesiologists were no more likely to be sued than the reference group. While this finding is inconsistent with some earlier results,^21,22 anecdotal evidence suggests that the aggressive efforts on the part of the American Society of Anesthesiologists such as the Closed Claims Project^30,31 may be having a significant impact on more recent claims experience of anesthesiologists.

The odds ratios on graduates of a medical school low outlier (<10th percentile) and high outlier (>90th percentile) support the hypothesis that there is something consistent about these medical school outliers. Specifically, physicians from low outlier medical schools—that is, medical schools whose graduates from earlier time periods were rarely sued—were less likely than physicians from mid-range schools to have a claim against them in the study time period. At the other end of the spectrum, physicians from high outlier medical schools were more likely to have a claim against them. This finding is not tautological because low and high outlier status is based on the malpractice claims experience of graduates in the decade before the graduation year of the physician in question.

Objective 3

We conducted one preliminary analysis to look for consistent differences between outlier and non-outlier schools. Using characteristics of individual medical schools published in the 1989 Medical Education issue of JAMA, we compared schools who were identified as being high outliers, low outliers, and mid-range schools, based on the malpractice claims experience of their 1980–9 graduates. The results are shown in table 5. Few consistent differences were seen across school types (high outlier, mid-range, low outlier) in any of the observable characteristics. This is partially due to the small sample sizes, although a number of potential differences that might be more pronounced in a larger dataset are highlighted. High outlier schools are more likely to be public institutions (75%) than low outlier schools (25%; p<0.05), and they also tended to be more recently established (mean age 67.6 years) than mid-range (110.5 years; p<0.05) and low outlier schools (94.8 years; p=NS). Furthermore, it appears that low outlier schools had significantly fewer residents and fellows (334.1) than mid-range (585.0; p<0.01) and high outlier schools (493.6; p=NS). This relationship may reflect better mentor/student ratios or perhaps differences in the pattern of cases treated at those institutions; however, without more detailed information our results are only suggestive.

DISCUSSION

We have shown a consistent difference in the malpractice claims history of physicians who graduate from certain schools. Because of the implications for medical school student selection and training, this finding warrants attention. Previous studies have shown strong links between physician characteristics and specialty and malpractice experience, but were unable to find any significant relationship between medical school attended and rate of claims or payments. By profiling medical schools by the claims experiences of their graduates, we were able to detect a relationship.

There are a number of possible explanations for these findings. One potential explanation is that some schools do, in fact, provide a lower quality of medical education than others, leaving a predictable proportion of their graduates not as well prepared for the practice of medicine. Perhaps somewhat less disturbing would be that we have simply discovered student heterogeneity with a non-random distribution of students between schools. Certain schools might attract students who are more likely to be sued, either as a result of their inherent characteristics or as a result of the specialties they enter. More detailed data on physician characteristics would provide better insight into this possible explanation and would help to understand whether schools that consistently produce graduates who are relatively prone to malpractice lawsuits fall below some threshold of adequate quality among the students they accept. Moreover, the institutional culture of schools may be a factor which is separate from the content of education in influencing student behavior and choices of specialty. Alternatively, some medical schools may be more effective than others in teaching their students to deal with the demands placed on them by patients and their families. The wealth of knowledge and skills to be imparted during a medical education, coupled with the array of tools we now have to communicate that information, present considerable challenges.^32,33

A rapidly changing healthcare environment with increasing financial pressures may increase malpractice claims as patients are under the care of more physicians, providing more technologically sophisticated care, in less personal environments. However complex the factors that lead to malpractice claims, examining the differences between high and low outlier medical schools may lead to methods for improving the education of physicians with the goal of improving the care that they provide. If we can identify what curricular elements or what teaching methods contribute to a medical school’s malpractice experience, we may be able to document “best practices” in medical education, improve that education, and eventually improve patient outcomes.

The strength of our findings is limited by the available data. Our data on malpractice claims come from only three states because most states either do not collect the data, do not make the data publicly available, or only allow inquiries regarding specific individual physicians. Several national databases exist but are not publicly available. We have tried to minimize the impact of this limitation by restricting our analysis to medical schools with at least 5% of their graduates in the three states studied. In addition, we explored alternative empirical specifications that included the physician’s probability of being in our sample (based on AMA Masterfile data) as a “selection correction factor” in the logistic regressions. Our results were unchanged. We were also limited in our ability to compare medical schools across a number of meaningful dimensions and over time. Because of the dearth of information, our research cannot address what underlying differences between medical schools might be responsible for the differences we have found.

Our research does highlight one important relationship—namely, that graduates of medical schools that have been previously identified as high or as low “malpractice claims risk” outliers are significantly more or less likely, respectively, to be sued. This finding is important in our quest to understand why certain physicians get sued. Further research exploring alternative explanations for these differences needs to be conducted. Are there (or have there been) significant differences in the curricula or admission policies between high and low outlier schools? Are there organizational characteristics or cultural differences that might explain the observed differences? This research could prove particularly challenging because of the difficulties associated with collecting information about curricula, culture, and characteristics of medical schools dating back to the 1960s and 1970s. The costliness of medical malpractice claims, combined with the implications of potential findings for quality of care, make further investigations in this area of critical importance.