There has been an initiative to create public accountability for cardiac surgery results in the UK. In 2001, the public inquiry into the events at Bristol Royal Infirmary, Bristol, UK concluded that patients should have access to “the relative performance of the Trust … and the consultant units within the Trust”.¹ Also, in 2001, the Dr Foster organisation, Leeds, UK published “named cardiac surgical hospitals” mortality for coronary artery surgery throughout the UK (http://www.drfoster.co.uk, accessed 2 February 2007). In March 2005, named surgeon mortality data for coronary artery surgery for all UK surgeons were published by the Guardian newspaper, after a request under the newly introduced Freedom of Information Act (http://society.guardian.co.uk/nhsperformance/story/0,,1439210,00.html, accessed 2 February 2007). In April 2006 a website was launched by the Healthcare Commission and the Society of Cardiothoracic Surgeons of Great Britain and Ireland, which provided cardiac surgery mortality data for the public. (http://heartsurgery.healthcarecommission.org.uk, accessed 2 February 2007).

Supporters of public disclosure of surgical results argue that it will help drive improvements in quality.² Opponents believe that it promotes risk-averse behaviour by discouraging surgeons from accepting high risk patients who would otherwise benefit from surgery.^3–7 We have analysed a large UK regional cardiac surgery database to look for evidence of improvements in quality or risk-averse behaviour over the time period before and after publication of cardiac surgery results.

METHODS

The North West Quality Improvement Programme in Cardiac Interventions is a regional consortium involving all four National Health Service centres (Blackpool Victoria Hospital, Blackpool; The Cardiothoracic Centre, Liverpool, Manchester; Royal Infirmary, Manchester; South Manchester University Hospital, Manchester) performing adult cardiac surgery and percutaneous coronary interventions in the northwest of England (http://www.nwheartaudit.nhs.uk, accessed 2 February 2007).

Data were collected prospectively on a total of 25 730 consecutive patients undergoing adult cardiac surgery between 1 April 1997 and 31 March 2005 in the north west of England. For this study, only patients undergoing isolated coronary artery surgery were included. The dataset collected for each patient contained preoperative and operative variables to enable a predicted mortality to be calculated. Data were collected in each institution and communicated to a central source for analysis. Validation of data was conducted in each centre. Mortality was defined as any in-hospital death.

Design of the project

The specific questions we asked were:

• Have there been changes in observed, predicted and risk adjusted mortality since cardiac surgical results were published?

• Has there been a reduction in the number of high risk patients coming to surgery after the introduction of public disclosure?

Crude mortality was determined for each patient, and predicted mortality was calculated using the additive EuroSCORE.⁸ When patient factors needed to calculate the EuroSCORE were missing in the patient record, they were assumed to be absent (occurred in <2% of cases).

The Bristol public inquiry recommending that named surgeon outcomes should be in the public domain was published in 2001. Named hospital mortality was published later that year, and although data for individual surgeons were not published until April 2005 we have subdivided the time periods into prepublic disclosure (April 1997 to March 2001) and post-public disclosure’ (April 2001 to March 2005).

To investigate any changes in the risk spectrum of patients undergoing surgery during the study period we subdivided them into those at low risk (EuroSCORE <6) and high risk (EuroSCORE ⩾6) for further analysis.⁹ As we particularly wished to examine changes in the number of very high risk patients undergoing coronary artery surgery, we further subdivided the high risk patients into a very high risk group (EuroSCORE ⩾11). We also assessed the incidence of the various individual risk factors that are included in the EuroSCORE, before and since public accountability. Categorical data are shown as percentages, with χ² tests and χ² square tests for trend used accordingly. Data on age are shown as the median with 25th and 75th centiles, with a Wilcoxon rank sum test used to test for differences. All analyses were performed using SAS version 8.2.

RESULTS

Observed mortality fell over the study period from 2.4% in 1997–8 to 1.8% in 2004–5 (p = 0.014). There was a progressive increase in predicted mortality over the same period from 3.0 in 1997–8 to 3.5 in 2004–5 (p<0.001). The observed to expected mortality ratio fell from 0.8 in 1997–8 to 0.51 in 2004–5 (p<0.05). Table 1 shows the mortalities for each year. Both the observed and risk adjusted mortality rates were significantly lower in the period after public disclosure than they were before it, and table 2 shows that the predicted risk was higher.

The total number and percentage of patients who were at low risk, high risk and very high risk each year was 2694 (84.6%), 449 (14.1%) and 41 (1.3%) before public disclosure and 2654 (81.7%), 547 (16.8%) and 47 (1.4%) since public disclosure, respectively, demonstrating a significant increase in the number and proportion of high risk patients undergoing surgery (p<0.001), as shown in table 3. The proportion and number of very high risk patients remained unchanged (p = 0.35).

Table 4 shows the proportion of patients displaying the individual risk factors in the EuroSCORE. The median age of the patients increased from 63 to 65 (p<0.001), and there were significant increases in many other risk factors, including the proportion of patients aged >80 years, with renal dysfunction, recent myocardial infarction and peripheral vascular disease. Two risk factors were significantly less prevalent after public disclosure: prior coronary artery bypass graft (CABG) surgery and ejection fraction <30%.

DISCUSSION

Statement of principal findings

This study has shown that there has been a significant reduction in crude and risk adjusted mortality since the introduction of public disclosure of cardiac surgical results in the UK. The overall predicted risk of the patients increased over the period of study as did the total number of patients undergoing surgery who were at high risk. We found no evidence to suggest that public disclosure of outcomes has led to a significant number of patients at high risk being denied operations.

Strengths and weaknesses of the study

This study was conducted on a large regional database, and includes over 25 000 patients. The data have been subjected to local validation and have the confidence of clinicians. However, the data have not been validated externally, which is a weakness of our study. Cardiac surgery in northwest England makes up about one eighth of all the cardiac surgery in the UK, and we have shown previously that our mortality and predicted risk are in line with national trends.¹⁰

Revascularisation has changed over the past eight years; there has been an increase in the number of patients treated by percutaneous intervention (PCI) with the introduction of bare-metal stents and more recently, drug-eluting stents during the study period (http://www.bcis.org.uk, accessed 2 February 2007). Improved techniques of percutaneous coronary intervention have meant that many patients who would have been considered for surgery in 1997 would have undergone PCI in 2005, and this applies both to low risk patients with straightforward lesions, and high risk patients with complex coronary anatomy and significant comorbidity. To obtain a complete picture of patients undergoing revascularisation, it would be interesting to look at patients having both surgery and PCI, but this is not possible with the existing data and hence limits the findings of our study.

The requirement for public disclosure in the UK has gathered momentum. Named hospital mortality data were first published in 2001. Named surgeon data were not published until 2005, but it became clear to most members of the surgical community that publication of their outcomes was inevitable at some stage between 2001 and 2005. Because named hospital mortality data were published and the recommendations about public dissemination of named surgeon data from the Bristol Public Inquiry both occurred in 2001 (and risk-averse behaviour might potentially be created because surgeons were worried either about their own results, or those of their hospital) we believe that it is reasonable to use this date as a cut-off point for studying the effects of publication of results. We do, however, accept that there is no clearly defined date that demarcates the introduction of public accountability.

Our study has looked at all patients undergoing coronary artery surgery in a defined geographical area over a period of eight years and has detected no decrease in the number of high risk patients coming to surgery associated with publication of the results. Possibly, there may be an effect on individual surgeons or units, whereby high mortality results influence individual surgeons or hospitals to refuse to treat high risk patients either transiently or permanently. But any specific effect of this nature has not had an influence on the overall data and would not have been picked up by our methods.

Our study contains data only on patients who have undergone surgery. We were not able to look at patients who were referred for, and then did not undergo, surgery as this is outside the scope of the database. Possibly, more high risk patients were referred for surgery towards the end of the study period with improvements in overall care of patients with ischaemic heart disease, and although we found no decrease in the number of these patients undergoing surgery, those accepted may reflect a carefully selected subgroup, and we accept that an analysis containing all patients referred would give a more complete picture.

We have used the EuroSCORE to stratify patient into low risk and high risk groups.⁹ The EuroSCORE is a well-accepted risk adjustment tool and has, in general, been shown to differentiate well between patients with differing levels of operative risk. The score in both its additive and logistic versions has been criticised because it overpredicts risk for contemporary cardiac surgery. But we feel it is suitable for detecting increases in predicted operative mortality over time and for the stratification purposes we have applied.^10,11 For this study, we have also looked at changes in the incidence of various individual risk factors that make up the score and found significant increases in many of these since the introduction of public disclosure, which we feel make our findings robust.

Strengths and weaknesses of the study compared with other studies

There are a number of large cardiac surgery databases, including national voluntary databases such as those of the Society of Thoracic Surgeons and Veterans Administration in the USA and the Society of Cardiothoracic Surgeons in the UK.^12,13 There are also several databases based on regional quality improvement initiatives such as the north New England Cardiovascular Study Group and our own northwest England project (http://www.nwheartaudit.nhs.uk;http://www.nnecdsg.org, both accessed 2 February 2007). Evidence from these databases suggests that structured feedback of surgical outcomes to doctors leads to decreases in mortality without public disclosure,^12,14 and there is evidence of a similar effect on PCI.¹⁵ In New York and Pennsylvania, mortality data have been published for individual cardiac surgeons for 1990 and 1992, respectively. Supporters of public disclosure of outcomes have shown that publishing named surgeon outcomes for CABG in New York led to a 41% reduction in risk adjusted mortality over the first four years of the project.¹⁶ It has been suggested that this decrease in mortality was greater than the national trend in the same time period. But other workers have shown that similar reductions in mortality have been detected in states without routine data collection and reporting.^17,18 It has also been suggested that some of the improvements in quality seen in New York were due to “outmigration” to surrounding states where there was no policy of public disclosure, but this has been challenged.^2,19

Critics of public disclosure of outcomes argue that it discourages surgeons from accepting high risk patients for surgery. This is supported by several surveys of attitudes of doctors in states with public disclosure of results; in Pennsylvania more than half of the cardiologists surveyed thought they had more difficulty in finding surgeons willing to perform CABG on high risk patients, and in New York³ one-third of surgeons reported a significant alteration in their own professional practice and two-thirds said they had refused to treat at least one patient with high risk CABG in the previous year, primarily because of public reporting.⁴ Data from a survey of UK cardiac surgeons conducted independently by Newsnight (British TV news programme) in 2000 suggested that 80% of surgeons were in favour of public disclosure of data, but 90% thought that high risk cases were already being avoided in anticipation of the publication of results. When asked how well current algorithms adjusted for surgical risk, only 6% of surgeons felt that the available algorithms adjusted accurately for operative risk and 37% felt that the methods were inadequate.¹³ More recently, again from New York, 79% of interventional cardiologists agreed that publication of mortality statistics had influenced their decisions about whether to perform angioplasty on individual patients or not.²⁰

Although it is suggested that public disclosure of outcomes allows patients, referring doctors and purchasers preferentially to select units or surgeons with good results, and public reporting may motivate hospitals and surgeons to compete on quality and thereby improve overall outcomes, the benefits of public reporting have not been clearly shown and there are continuing concerns about unintended negative consequences.^6,7 Despite the reports of doctors’ attitudes and the importance of the subject, there are few previous studies presenting data on the effects of public accountability on promoting risk-averse behaviour, and existing data are conflicting.

In New York, it has been reported that there was a 73% increase in the number of high risk patients undergoing coronary artery surgery in the first three years of public reporting of surgeons’ outcomes. But it has subsequently been suggested that the average illness scores in all patients coming to surgical revascularisation actually decreased in both New York and Pennsylvania in the years immediately after introduction of public accountability.^2,20,21 It has also been shown that patients undergoing bypass surgery in states where public accountability existed had lower illness severity than those states that did not release that information and that the introduction of reporting has increased levels of racial and ethnic disparity.^21,22 Our study has shown both an increase in overall predicted operative mortality for all patients coming to coronary surgery and a significant increase in the number of high risk patients despite introducing a national programme of public reporting.

Meaning of the study

If publication of surgical mortality data had driven surgeons to turn down significant numbers of high risk patients we would expect to see that reflected in a decrease in the number of high risk cases coming to surgery. This study suggests that the effect may not be as large as is feared. The concern applies particularly to the highest risk cases, who have a high predicted mortality with surgery, but may well have higher predicted mortality if surgery is withheld. These patients may have much to gain in terms of symptomatic improvements and life expectancy if surgery is successful. Although our results show no decrease in the total number of high or very high risk patients coming to surgery, we still believe that it is possible that publication of results influences decision making in this group of patients. It may focus clinical minds on optimising treatment for a particular patient, resulting in more comprehensive multidisciplinary discussions around various alternative available treatments than previously, which may have contributed to the improvements in outcomes that we have seen. However, it may also be a further factor in what is often already a complex clinical decision that could result in some patients who might benefit not being offered surgery.

Overall we have seen a marked decrease in risk adjusted mortality for coronary artery surgery in the period spanning the introduction of public disclosure. We initiated a regional cardiac surgical database which included all patients and surgeons since as early as 1997, and have used that to provide structured feedback to doctors, which should help to improve outcomes.^12,14 It is not possible for us to separate incremental improvements in outcome due to public reporting from those obtained by simply collecting and using the data.

Cardiac surgery is unusual in the amount of clinical data collected and fed back to clinical teams. Most other areas of clinical medicine have not embraced the concepts of using routine data collection to improve quality. Public disclosure in cardiac surgery has helped to generate a data-rich environment and has probably helped units to attract appropriate resources for the associated information technology and personnel infrastructure. If public disclosure can drive data collection and analysis, but does not create significant risk-averse behaviour, its introduction may be beneficial in other areas of medicine.

Unanswered questions and future research

We have shown that despite clinical concerns, the introduction of public accountability has not led to a decrease in the number of high risk patients coming for coronary artery surgery. It would be informative to study all patients referred to surgery, rather than just those undergoing a procedure, to determine whether public accountability has led to an increase in surgical “turndowns”. It would also be interesting to study all patients undergoing revascularisation to examine trends in surgery (where UK results are currently subjected to public scrutiny) and PCI (where they are not). It would also be useful to monitor carefully case mix and outcomes if public accountability should be introduced into other disciplines of medicine.