Medical school attended as a predictor of medical malpractice claims
- 1Center for Health Services Research, The University of Tennessee Health Science Center, Memphis, TN 38163, USA
- 2Department of Medicine, Division of General Internal Medicine, The Feinberg School of Medicine of Northwestern University, Chicago, IL, USA
- 3Department of Health Administration and Policy, College of Public Health, University of Oklahoma Health Science Center, Oklahoma City, OK, USA.
- Correspondence to: Dr T M Waters, Center for Health Services Research, The University of Tennessee Health Science Center, 66 North Pauline, Suite 463, Memphis, TN 38163, USA;
- Accepted 11 April 2003
Objectives: Following earlier research which showed that certain types of physicians are more likely to be sued for malpractice, this study explored (1) whether graduates of certain medical schools have consistently higher rates of lawsuits against them, (2) if the rates of lawsuits against physicians are associated with their school of graduation, and (3) whether the characteristics of the medical school explain any differences found.
Design: Retrospective analysis of malpractice claims data from three states merged with physician data from the AMA Masterfile (n=30 288).
Study subjects: All US medical schools with at least 5% of graduates practising in three study states (n=89).
Main outcome measures: Proportion of graduates from a medical school for a particular decade sued for medical malpractice between 1990 and 1997 and odds ratio for lawsuits against physicians from high and low outlier schools; correlations between the lawsuit rates of successive cohorts of graduates of specific medical schools.
Results: Medical schools that are outliers for malpractice lawsuits against their graduates in one decade are likely to retain their outlier status in the subsequent decade. In addition, outlier status of a physician’s medical school in the decade before his or her graduation is predictive of that physician’s malpractice claims experience (p<0.01). All correlations of cohorts were relatively high and all were statistically significant at p<0.001. Comparison of outlier and non-outlier schools showed that some differences exist in school ownership (p<0.05), years since established (p<0.05), and mean number of residents and fellows (p<0.01).
Conclusions: Consistent differences in malpractice experience exist among medical schools. Further research exploring alternative explanations for these differences needs to be conducted.
Medical malpractice has been a major issue in the US for many years. In a development that has generated some international attention,1–3 both federal and state authorities in the US have made information on malpractice claims on physicians more accessible to hospitals, health plans, professional groups, and the public. The National Practitioner Data Bank (NPDB) requires any organisation making payments on behalf of a practitioner to report malpractice payments which it provides to hospitals, HMOs, group practices, licensing boards, professional societies, and other healthcare organisations (for example, clinics, nursing homes) who query the Data Bank.4 As many as 48 states also collect some malpractice claims data on claims opened or settled in their state (“State malpractice payment reporting requirements”, unpublished study, Division of Quality Assurance, Bureau of Health Professions, Health Resources and Services Administration, US Department of Health and Human Services, August 1995). These data are usually for use by state medical licensure boards but a few states make the data publicly available to differing degrees (box 1).5–9
Box 1 US states which make physician medical malpractice information available to the public (or plan to do so in the near future)
Underlying these developments is the belief that these data provide useful information about a physician, but there is a relative dearth of empirical studies indicating what the value of that information is. The Harvard Medical Practice Study has been interpreted as showing that a malpractice claim is not commonly related to medical negligence.10 Another study found that medical care was considered indefensible in about half the cases in which awards were paid, but awards were also made in about 21% of cases in which the care provided was considered standard.2 Other work suggests that a malpractice claim is a sign of poor communication between the patient or the patient’s family and the physician.11
Whether indicative of inadequate care, defective communication, or other factors, malpractice claims continue to be a matter of great concern for physicians in the US and are receiving increasing attention across a wide range of healthcare and dispute resolution systems in the EU and elsewhere.12–20 In an effort to minimize such litigation, a significant amount of research has been devoted to exploring physician related factors that may be predictive of malpractice claims. Sloan et al21 found that physicians who were board certified were not less likely to be sued and, in some cases, were more likely to be sued. They also found that two specialty categories were most at risk for being sued. Taragin et al22 reported interspecialty differences of up to 12-fold in the number of claims per year. They found that anesthesiology had the highest award rate (total number of awards/total claims filed), followed by obstetrics/gynecology and radiology. Sloan et al21 also examined the impact of the medical school location and “stature” on the malpractice experience of the physician and found only a modest influence of the medical school on the frequency or amount of claims.
We have assessed the possible role of specific medical schools in determining the likelihood that physicians will be sued. The presence of such an association would suggest that some alterations in medical education could ameliorate the rate of malpractice claims. Such a finding could be of interest to medical schools in countries other than the US. This study uses the historical malpractice claims experience of medical school graduates in previous decades and employs both probability theory and multivariate regression techniques to explore the relationship between the medical school and physician probability of a malpractice claim in more detail. In addition, a larger dataset than those previously employed, combining data from three states (Florida, Indiana, and Maryland), was used to test our hypotheses empirically.
We hypothesized that several characteristics of medical schools could result in differential experience of physician malpractice claims (box 2). Firstly, malpractice may be related to the medical school attended if the quality of medical education differs significantly between schools, leading to “high quality” and “low quality” physicians with respect to knowledge and technical prowess. It is also possible that the training in interpersonal skills may vary significantly between medical schools, as has already been shown for residency programs.23 This would be consistent with the finding that malpractice claims may be strongly related to the quality of the communication between the physician and patient (or patient family).11 Medical schools might also differ in the predisposition of their students and graduates to engage in behavior prone to lead to malpractice. For example, a study from India noted the relatively high use of alcohol, tranquilizers, and psychedelic drugs among medical students, interns and house physicians,24 a factor known to be associated with malpractice exposure that might vary substantially from one school to another. Finally, some schools might attract less qualified students, students with other characteristics that make them more likely to be sued, or disproportionate numbers of students who end up in specialties or geographical areas where a lawsuit is more likely.
Box 2 Possible hypotheses to explain why malpractice claims experience might be related to medical school attended
Quality of education
Training in interpersonal skills
Predisposition of the school’s graduates to engage in behaviour or enter specialties that might increase the frequency of malpractice lawsuits
Types of students that the school attracts (selection bias)
Institutional culture of medical schools
Irrespective of the outcome, malpractice lawsuits are extremely painful for all parties involved and are costly to litigate. In addition, based on the uncertainty and informational asymmetries present in the tort system, the presence or magnitude of settlements does not always reflect the merits of the claim. Medical schools, physicians, and policy makers should be interested in reducing all claims, whether or not they go to trial (verdict) or result in payment. For this reason, we have examined all claims against physicians, not just those that resulted in payments.
Given the variety of reasons for a potential relationship between malpractice claims experience and medical school attended, we set out to resolve the following specific objectives:
Objective 1: To determine whether some medical schools have a relatively high (or low) proportion of graduates who are sued for medical malpractice and, if so, whether schools identified as high (or low) outliers* for a particular period of time (years of graduation) are more likely to continue to be high (or low) outliers over time.
Objective 2: Controlling for the observable physician characteristics available (age and specialty), to determine whether a physician’s probability of having a malpractice claim is positively (or negatively) related to graduating from a high (or low) outlier school.
Objective 3: To explore whether the characteristics of the medical school explain any differences found.
Data on malpractice claims closed between 1 January 1990 and August 1997 were drawn from the databases of three states (Florida, Maryland, and Indiana) in 1998.25–27 All three states require that any malpractice claim, regardless of payment amount, must be reported to the relevant state regulatory body when the claim is closed or settled. These databases are publicly available or are available on request for research purposes and contain detailed claim information including defendant name, nature of the injury, payment (if any), and dates of the lawsuit and closure. Malpractice claims are defined as any formal written demand for payment including—but not limited to—lawsuits filed in the state court.
Data on malpractice claims were merged with the American Medical Association’s (AMA) Physician Masterfile to obtain medical school and demographic information on physicians who did and did not have claims settled against them during our reporting period. Since numerical identifiers such as social security number or license number were not available, a matching algorithm was developed using last name, first name, date of birth, and current state of residence. This matched some 50% of the claims with the appropriate physician. Manual examination of the data was necessary for another 42% of the claims which contained errors in name spelling, changed state of residence, or multiple physicians with the same name. For manual merges, all identifying information—including address and specialty—were used to find matches. Although approximately 8% of the claims could not be merged with the Masterfile data, it is unlikely that these unmatched claims are consistently related to any particular school and therefore should not bias our results.
Claims rate measures
Our initial claims measure of interest was the percentage of graduates from a medical school for a given period of graduation† who had a claim closed against them between 1 January 1990 and 31 August 1997. Because we had claims information from just three states, we only looked at schools with at least 5% of their graduates living in these states during the time periods in question. A total of 47 schools (out of 122) were eliminated because of insufficient data. This approach avoided some of the problems associated with assigning outlier status to schools on which we had very little history on graduate claims.
Because there is some evidence that malpractice claims may be related to particular physician characteristics, we calculated a second claims measure that adjusted for some of these factors—that is, we constructed a “risk adjusted” measure of claims rates for each school’s graduates. Specifically, we calculated the difference between the observed rate of closed claims for each school/time period observation and the expected rate of closed claims by estimating the following physician specific equation using logistic regression:
where Y was equal to 1 if a particular graduate had any closed claim between 1 January 1990 and 31 August 1997 or otherwise equal to 0. Specialty comprised a series of dummy variables for the following (commonly sued) specialties: anesthesiology, obstetrics/gynecology, specialty surgery (orthopedic, neurosurgery, plastic, cardiothoracic), all other surgery, radiology, and emergency medicine. The omitted (comparison) category was all other specialties. Expected claims rates for each school/time period observation were calculated by entering the actual values of age and specialty of the graduates of a particular school/time period and deriving the predicted probability.‡
The difference between the observed (actual) and expected rate of claims for each school/time period was used as our second outlier measure. This variable is similar to those constructed in the health services research literature when comparing observed to expected mortality rates for a given hospital or provider.28,29
Descriptive statistics for both malpractice claim rate measures are shown in table 1.
The two claims variables (measures 1 and 2) were then used to identify two sets of outlier medical schools for each time period. A medical school was classified as a measure 1 high (or low) outlier for a particular time period if the percentage of its graduates for that time period who were sued exceeded the 90th percentile (or was below the 10th percentile) of the time period specific distribution. From table 1, this definition implies that school A is a measure 1 high outlier for the period 1960–9 if more than 24.3% of its graduates from that time period had a claim closed against them in our claims dataset. Similarly, a medical school was classified as a measure 2 high (or low) outlier for a particular time period if the difference between the actual percentage of graduates sued and the predicted percentage of graduates sued exceeded the 90th percentile (or was less than the 10th percentile) of the time period specific distribution. From table 1, this definition implies that school B is a measure 2 low outlier for the period 1970–9 if the actual rate of claims per graduate for the time period was less than the predicted rate of claims by at least 0.064.
Given the available data, we restricted our study to US medical schools. In addition, allowing for the fact that schools and their curricula change over time and that recruiting foci also change, we divided a school’s malpractice claims record into four time periods (pre-1960, 1960–9, 1970–9, and 1980–9).
The fact that a physician’s own malpractice history contributes to his or her school’s outlier status raises an issue of endogeneity that must be addressed in the empirical estimation. In order to “break” this circular definition, we used lagged values of a school’s outlier status to predict physician probability of malpractice claim. Thus, if a physician graduated from medical school in the 1980s, his or her medical school’s outlier status for the 1970s was used to predict the physician’s probability of malpractice claim.
The intention of our analyses was to examine the predictive power of measure 1 and measure 2 medical school outlier status (high or low). Our approach was twofold:
(1) We calculated the transition probabilities between the various “states” for both outlier variables—that is, we examined the probability that a high outlier in one period remained a high outlier in the subsequent period compared with changing to a mid-range school or to a low outlier. Our null hypothesis was that the outlier status of a medical school was a completely random event. In this case, the probability of moving from being a high or low outlier in the initial period to becoming a mid-range school in the subsequent period should be approximately the same as the probability of being a mid-range school in both time periods (fig 1; pr2 = pr5 = pr8). Conversely, schools that are classified as mid-range in the initial period should be just as likely as high or low outliers to become high or low outliers in the subsequent period (fig 1; pr1 = pr4 = pr7; pr3 = pr6 = pr9). Figure 1 outlines the probability tree(s) we sought to estimate.
(2) We estimated a logistic regression predicting any claim for an individual physician (based on our database between 1 January 1990 and 31 August 1997) using the observable characteristics derived from the AMA Masterfile (age, specialty), the percentage of the medical school’s graduates (for the decade) who were currently living in Florida, Maryland, or Indiana,§ and two dummy variables indicating whether the physician graduated from a school that was classified as a high or low outlier in the time period immediately preceding the physician’s time period of graduation. For example, if John Smith graduated from University Medical School in 1985, dummy variables indicating whether University Medical School was a measure 1 or measure 2 high or low outlier school in the 1970–9 period were included in the predictive equation. As discussed earlier, these lagged versions of the outlier variables were used to avoid any statistical bias due to endogeneity between the probability of a lawsuit and medical school outlier status.
Approximately 11% of physicians in our sample were sued at least once during the study time period. Their mean age was 45 years and one third were international (outside the US and Canada) medical school graduates (IMGs). Using our medical school outlier classification scheme, 3.7% of physicians in the sample graduated from a “low outlier” medical school and 10.3% graduated from a “high outlier” medical school. These descriptive data, together with a breakdown by major specialty, are shown in table 2.
The transition probabilities between the three “states” (below 10th percentile outlier, mid-range school, above 90th percentile outlier) do not support the null hypothesis that medical school outlier status is a random event (table 3). Specifically, medical schools that were classified as mid-range schools in the initial period were more likely to be mid-range schools in the subsequent period than high and low outliers were likely to become mid-range schools (measure 1: 81% v 68% (low outlier → mid-range) and 68% (high outlier → mid-range); measure 2: 79% v 68% and 68%). Conversely, medical schools that were classified as low outliers (below 10th percentile) in one period of graduates were more likely than mid-range schools to be low outliers in the subsequent period (measure 1: 32% v 9%; measure 2: 32% v 10%) and medical schools that were classified as high outliers (above 90th percentile) in one period were more likely than mid-range schools to be high outliers in the subsequent period (measure 1: 32% v 10%; measure 2: 32% v 11%). Most striking, perhaps, are the probabilities of low outlier schools becoming high outlier schools in the subsequent period and vice versa. In our analyses, none of the low outlier schools became high outlier schools and none of the high outlier schools became low outlier schools in the subsequent period. These results support the hypothesis that there is something consistent about the malpractice experience of graduates of high and low outlier schools over time.
To examine the statistical significance of lawsuit probabilities between medical school cohorts, we examined the correlations between the lawsuit rates of successive cohorts—for example, pre-1960 graduates with 1960–9 graduates; 1960–9 graduates with 1970–9 graduates. All correlations were relatively high and all were statistically significant at p<0.001. The correlation between the percentage of pre-1960 graduates and 1960–9 graduates sued was 0.49; between 1960–9 and 1970–9 graduates the correlation was 0.55; and between 1970–9 and 1980–9 graduates the correlation was 0.50.
Table 4 shows the results of our second set of analyses, predicting any claim for an individual physician (based on our database between 1 January 1990 and 31 August 1997). Explanatory variables included the observable characteristics derived from the AMA Masterfile (age, specialty), the percentage of a medical school’s graduates (for the decade) who live in the three states, and two dummy variables indicating whether the physician graduated from a school that was classified as a high or low outlier in the time period before the his/her time period of graduation. Since the logistic results using either measure 1 or measure 2 outliers were qualitatively identical, we only present results using measure 2 (“risk adjusted”) outliers. Both models had a reasonable goodness of fit (score statistics significant at p<0.0001; Hosmer Lemeshow tests failed to reject model (p=0.26; p=0.30)). The predictive ability was relatively low, as might be expected with our relatively sparse specification (c-statistic =0.70 for both).
The odds ratios on the physician characteristics are not surprising. Increasing age is positively associated with the probability of malpractice claims experience, but at a decreasing rate. Obstetricians/gynecologists, surgical specialists, general surgeons, radiologists, and emergency medicine specialists were more likely to be sued than the comparison category (all other physicians). Anesthesiologists were no more likely to be sued than the reference group. While this finding is inconsistent with some earlier results,21,22 anecdotal evidence suggests that the aggressive efforts on the part of the American Society of Anesthesiologists such as the Closed Claims Project30,31 may be having a significant impact on more recent claims experience of anesthesiologists.
The odds ratios on graduates of a medical school low outlier (<10th percentile) and high outlier (>90th percentile) support the hypothesis that there is something consistent about these medical school outliers. Specifically, physicians from low outlier medical schools—that is, medical schools whose graduates from earlier time periods were rarely sued—were less likely than physicians from mid-range schools to have a claim against them in the study time period. At the other end of the spectrum, physicians from high outlier medical schools were more likely to have a claim against them. This finding is not tautological because low and high outlier status is based on the malpractice claims experience of graduates in the decade before the graduation year of the physician in question.
We conducted one preliminary analysis to look for consistent differences between outlier and non-outlier schools. Using characteristics of individual medical schools published in the 1989 Medical Education issue of JAMA, we compared schools who were identified as being high outliers, low outliers, and mid-range schools, based on the malpractice claims experience of their 1980–9 graduates. The results are shown in table 5. Few consistent differences were seen across school types (high outlier, mid-range, low outlier) in any of the observable characteristics. This is partially due to the small sample sizes, although a number of potential differences that might be more pronounced in a larger dataset are highlighted. High outlier schools are more likely to be public institutions (75%) than low outlier schools (25%; p<0.05), and they also tended to be more recently established (mean age 67.6 years) than mid-range (110.5 years; p<0.05) and low outlier schools (94.8 years; p=NS). Furthermore, it appears that low outlier schools had significantly fewer residents and fellows (334.1) than mid-range (585.0; p<0.01) and high outlier schools (493.6; p=NS). This relationship may reflect better mentor/student ratios or perhaps differences in the pattern of cases treated at those institutions; however, without more detailed information our results are only suggestive.
We have shown a consistent difference in the malpractice claims history of physicians who graduate from certain schools. Because of the implications for medical school student selection and training, this finding warrants attention. Previous studies have shown strong links between physician characteristics and specialty and malpractice experience, but were unable to find any significant relationship between medical school attended and rate of claims or payments. By profiling medical schools by the claims experiences of their graduates, we were able to detect a relationship.
There are a number of possible explanations for these findings. One potential explanation is that some schools do, in fact, provide a lower quality of medical education than others, leaving a predictable proportion of their graduates not as well prepared for the practice of medicine. Perhaps somewhat less disturbing would be that we have simply discovered student heterogeneity with a non-random distribution of students between schools. Certain schools might attract students who are more likely to be sued, either as a result of their inherent characteristics or as a result of the specialties they enter. More detailed data on physician characteristics would provide better insight into this possible explanation and would help to understand whether schools that consistently produce graduates who are relatively prone to malpractice lawsuits fall below some threshold of adequate quality among the students they accept. Moreover, the institutional culture of schools may be a factor which is separate from the content of education in influencing student behavior and choices of specialty. Alternatively, some medical schools may be more effective than others in teaching their students to deal with the demands placed on them by patients and their families. The wealth of knowledge and skills to be imparted during a medical education, coupled with the array of tools we now have to communicate that information, present considerable challenges.32,33
A rapidly changing healthcare environment with increasing financial pressures may increase malpractice claims as patients are under the care of more physicians, providing more technologically sophisticated care, in less personal environments. However complex the factors that lead to malpractice claims, examining the differences between high and low outlier medical schools may lead to methods for improving the education of physicians with the goal of improving the care that they provide. If we can identify what curricular elements or what teaching methods contribute to a medical school’s malpractice experience, we may be able to document “best practices” in medical education, improve that education, and eventually improve patient outcomes.
The strength of our findings is limited by the available data. Our data on malpractice claims come from only three states because most states either do not collect the data, do not make the data publicly available, or only allow inquiries regarding specific individual physicians. Several national databases exist but are not publicly available. We have tried to minimize the impact of this limitation by restricting our analysis to medical schools with at least 5% of their graduates in the three states studied. In addition, we explored alternative empirical specifications that included the physician’s probability of being in our sample (based on AMA Masterfile data) as a “selection correction factor” in the logistic regressions. Our results were unchanged. We were also limited in our ability to compare medical schools across a number of meaningful dimensions and over time. Because of the dearth of information, our research cannot address what underlying differences between medical schools might be responsible for the differences we have found.
The malpractice claims experiences of physicians who graduated from the same medical school are strongly related to each other.
Graduating from a medical school whose graduates are often sued significantly increases an individual physician’s likelihood of being sued.
Although consistent differences between high and low outlier medical schools were difficult to identify, we found that high outlier schools were more recently established and were more likely to be public institutions. Low outlier schools had fewer total residents and fellows.
Our research does highlight one important relationship—namely, that graduates of medical schools that have been previously identified as high or as low “malpractice claims risk” outliers are significantly more or less likely, respectively, to be sued. This finding is important in our quest to understand why certain physicians get sued. Further research exploring alternative explanations for these differences needs to be conducted. Are there (or have there been) significant differences in the curricula or admission policies between high and low outlier schools? Are there organizational characteristics or cultural differences that might explain the observed differences? This research could prove particularly challenging because of the difficulties associated with collecting information about curricula, culture, and characteristics of medical schools dating back to the 1960s and 1970s. The costliness of medical malpractice claims, combined with the implications of potential findings for quality of care, make further investigations in this area of critical importance.
The authors thank Joshua Pinkston and Ithai Lurie for their econometric and programming efforts and Dr Robert E Oshel for helpful comments and suggestions.
↵* ”High” and “low” outliers are defined as above the 90th percentile or below the 10th percentile of the distribution of the percentage of graduates sued. A second analysis adjusts the percentage of graduates sued for various “risk factors” such as (eventual) specialty of the graduate and age.
↵† Time periods of graduation were defined as: pre-1960, 1960–9, 1970–9, and 1980–9. Since physicians who had graduated from medical school after 1990 would hardly have had a chance to be sued, we omitted this time category from our analyses.
↵‡ Inserting the actual values of the independent variables provides the value x′α. The predicted probability is actually ex′α/(1 + ex′α).
↵§ This variable was included to control for any selection bias that might results from over-representation of certain schools in the dataset (e.g. more University of Florida graduates than University of Utah graduates in the three states).
See editorial commentary, pp 324–5
This research was supported by Division of Quality Assurance, Bureau of Health Professions, Health Resources and Services Administration, Department of Health and Human Services (Contract #240-97-0003). The design, conduct, interpretation, analysis, and manuscript development were the sole responsibility of the authors and were not subject to funding agency approval.