Objectives: To estimate the proportion and characteristics of patients injured by medical care in New Zealand public hospitals who complain to an independent health ombudsman, the Health and Disability Commissioner (“the Commissioner”).
Design: The percentage of injured patients who lodge complaints was estimated by linking the Commissioner’s complaints database to records reviewed in the New Zealand Quality of Healthcare Study (NZQHS). Bivariate and multivariate analyses investigated sociodemographic and socioeconomic differences between complainants and non-complainants.
Setting: New Zealand public hospitals and the Office of the Commissioner in 1998.
Population: Patients who lodged claims with the Commissioner (n = 398) and patients identified by the NZQHS as having suffered an adverse event who did not lodge a complaint with the Commissioner (n = 847).
Main outcome measures: Adverse events, preventable adverse events, and complaints lodged with the Commissioner.
Results: Among adverse events identified by the NZQHS, 0.4% (3/850) resulted in complaints; among serious, preventable adverse events 4% (2/48) resulted in complaints. The propensity of injured patients to complain increased steeply with the severity of the injury: odds of complaint were 11 times greater after serious permanent injuries than after temporary injuries, and 18 times greater after deaths. Odds of complaining were significantly lower for patients who were elderly (odds ratio (OR) 0.2, 95% confidence interval (CI) 0.1 to 0.4), of Pacific ethnicity (OR 0.3, 95% CI 0.1 to 0.9), or lived in the most deprived areas (OR 0.3, 95% CI 0.2 to 0.6).
Conclusion: Most medical injuries never trigger a complaint to the Commissioner. Among complaints that are brought, severe and preventable injuries are common, offering a potentially valuable “window” on serious threats to patient safety. The relatively low propensity to complain among patients who are elderly, socioeconomically deprived, or of Pacific ethnicity suggests troubling disparities in access to and utilisation of complaints processes.
Statistics from Altmetric.com
There is growing international interest in harnessing patient dissatisfaction and complaints to address problems with quality in health care.1 The value of complaints as a marker of threats to patient safety depends on the answers to several questions. Do complaints track injuries, or are they prompted by more subjective concerns? Are complaints the “tip of the iceberg” in terms of quality of care problems and, if so, how representative are they of broader quality problems? The longstanding obstacle to addressing these questions is methodological in nature and concerns the elusiveness of an appropriate metric against which to measure the prevalence and reasonableness of complaints.
In New Zealand, injury compensation and complaints against healthcare professionals are dealt with in distinct settings. An innovative accident compensation system compensates injured patients on a “no fault” basis. An independent health ombudsman, the Health and Disability Commissioner (“the Commissioner”), has statutory responsibility for resolving patient complaints about quality of care, acts as a gatekeeper to disciplinary proceedings, and strives to use complaints as a catalyst for improving patient safety.2–4
In this study we linked information on quality of care complaints lodged with the Commissioner with adverse event data gathered in the New Zealand Quality of Healthcare Study (NZQHS). Together, these two datasets permit estimation of how frequently adverse events led to complaints, and a description of the characteristics of patients who did and did not complain to the Commissioner.
The Wellington Ethics Committee and the Harvard Institutional Review Board approved the study.
Baseline data on a random sample of patients who had experienced adverse events came from the NZQHS. As previously described,5 NZQHS used a two stage sampling process to develop a representative sample of 6579 medical records of patients discharged from publicly funded acute care hospitals in 1998, excluding psychiatric and same day discharges. Trained reviewers assessed each episode of care for the presence of an adverse event and, when an adverse event was detected, rendered a judgment on whether it was preventable. Following previous research in the United States,6,7 adverse events were defined as unintended injuries caused by healthcare management, rather than the underlying disease process, that resulted in disability. The study included all adverse events detected during or responsible for the index admission, as well as those occurring during the index admission that were detected on a subsequent admission. Serious adverse events were those which caused death or permanent disability.
The Commissioner provided data on all complaints received by his office that were associated with a public hospital admission in 1998 (n = 398). To determine which of these complaints involved adverse events and preventable adverse events, an investigator with medicolegal expertise (MMB) reviewed the relevant administrative information using the NZQHS process of structured implicit review. A second investigator (TAB), also a medicolegal expert, independently reviewed a random subsample of 98 complaints, a quarter of the sample. Inter-rater reliability for determination of whether an adverse event had occurred was high (κ = 0.84); reliability for preventability judgment was moderate (κ = 0.50), although slightly higher than previous estimates of reliability for preventability determinations.7,8
We matched the NZQHS records to complaints probabilistically using national hospital number, date of birth, sex, and date of injury. Clinical data from the medical record review was then compared with claims data to confirm that the NZQHS patient and complainant was the same person and that the complaint related to the same episode of care. Two investigators (MB, DS) reviewed potential matches and reached consensus on whether sufficient information existed to confirm a match with a high degree of certainty. Among 18 candidate matches, nine were determined not to be matches and two were excluded on the grounds that there was insufficient information to confirm a match.
The analyses are descriptive. We compared characteristics of patients who lodged complaints with the Commissioner after experiencing an adverse event (“complainants”) with characteristics of patients from the NZQHS who experienced an adverse event and did not file complaints (“non-complainants”). Matched cases—that is, patients from the NZQHS sample whose care involved an adverse event and who subsequently complained about that instance of care—were classified as complainants. Figure 1 shows the derivation of the two populations which were combined for the multivariate analysis.
Data were analysed using the SAS 9.0 statistical software package (Cary, NC, USA) Stata 8.0/SE (Stata Corp, College Station, Texas). We used t tests and χ2 tests to conduct bivariate comparisons of the characteristics of complainants and non-complainants. We investigated predictors of failure to complain despite having experienced an adverse event using multivariate logistic regression. The dependent variable in the regression analysis distinguished complainants from non-complainants. The independent variables were sex, ethnicity (Maori, Pacific, Non-Maori/non-Pacific), patient age (<1 year, 1–17 years, 18–44 years, 45–64 years, ⩾65 years), disability due to adverse event (temporary, permanent with <50% impairment, permanent with >50% impairment, death), and whether or not the event was preventable. An additional covariate provided a measure of the patient’s socioeconomic status using the New Zealand Index of Deprivation Score.9,10 This index, based on mesh blocks, combines nine census variables reflecting aspects of material and social deprivation; following previous studies,11 index scores were separated into quintiles for analysis.
To account for the stratified two stage cluster sampling design in the NZQHS,12 the bivariate and multivariate analyses were weighted. The weighting had negligible effects on our estimates.
The Commissioner received 398 complaints related to care delivered in public hospitals in 1998, 254 of which (64%) related to an episode of care in which the patient had experienced an adverse event. For 51% of the complaints the adverse event was judged to be preventable. The NZQHS review identified 850 adverse events, as previously reported,5 of which 315 were preventable, 124 were serious, and 48 were both serious and preventable.
Patients themselves instigated 105 (41%) of the 254 complaints involving adverse events (fig 2). Third party complaints were commonly laid by family members, primarily the patient’s child (17%), parent (16%), or spouse (13%). A total of 79% (313/398) of complaint letters and 75% (191/254) of the adverse event complaint letters expressed concern about a health professional’s attitude or communication either during the index admission or after the adverse event.
There were seven matches between the complaint sample and the full NZQHS sample (n = 6579). NZQHS reviewers judged three of these matches to involve adverse events (one preventable death, one preventable permanent disability, and one unpreventable temporary injury); the rest did not involve adverse events. (Besides injuries, the Commissioner also has jurisdiction to hear complaints relating to informed consent, discrimination, and a variety of other bases of dissatisfaction with care.) Hence, 0.4% (3/850) of the patients in the NZQHS sample who experienced adverse events complained. Among NZQHS patients judged to have experienced adverse events that were serious and preventable, 4% (2/48) complained.
Box 1 provides case studies of three patients. The first patient complained following an adverse event. The second patient suffered an adverse event and did not complain. The third patient complained but, because the poor outcome he experienced was attributable to the condition for which care was sought rather than the medical care itself, it was not an adverse event.
Box 1 Case studies of injured patients who did and did not complain
Adverse event with no subsequent complaint
Mrs A, an elderly woman with a history of hyperthyroidism, asthma, left ventricular failure, hypertension, and gastro-oesophageal reflux disorder was admitted to hospital with dizziness and vomiting. She was dehydrated with low sodium (113 mEq/l, normal range 135–145). Her list of 13 medications prescribed by her general practitioner included frusemide 80 mg a day and spironolactone 100 mg twice a day. She was diagnosed with hyponatraemia secondary to an excessive dose of diuretics. She was rehydrated and discharged 8 days later on a reduced frusemide dose of 40 mg/day. Mrs A did not complain.
Complaint following adverse event
Mr D, a middle aged farmer, sustained a penetrating injury to his right eye while cutting firewood. As a result of this injury Mr D suffered a detached retina. He was referred to an ophthalmologist who offered to reattach the retina using an operation he had recently learned in the United Kingdom. The scrub nurse was unfamiliar with the proposed operation which involved the use of diluted SF6 gas. The theatre supervisor was on a meal break because the operating schedule was running late. Due to a breakdown in communication between the nurse and the surgeon, the gas was not diluted and 100% gas was administered to Mr D’s eye, resulting in total blindness in that eye. He is no longer able to run his farm due to the loss of depth perception. Mr D’s complaint was upheld and his claim for no-fault compensation was accepted.
Complaint with no adverse event
Mr N, a young man, was admitted to hospital with a severe crush injury to his right middle finger. A senior orthopaedic registrar with extensive plastic surgery experience assessed Mr N and discussed the case with his consultant. They agreed that it would be appropriate for the registrar to attempt to preserve Mr N’s fingertip. The registrar stabilised the soft tissue with loose sutures and administered an antithrombotic prophylactic, dextran, to try to prevent thrombosis of the artery. Another orthopaedic registrar, who had not been involved with Mr N’s initial care, told Mr N that crush injuries should never be sutured, causing him considerable anxiety. Following discharge, the hospital tried to arrange a follow up appointment for Mr N but he insisted on going on holiday to a remote region of New Zealand and did not contact a general practitioner as had been agreed. His fingertip became infected and later required partial amputation. The Commissioner found that Mr N had received an appropriate standard of care.
Table 1 shows the characteristics of the patients in the full NZQHS sample, the subset of patients who experienced adverse events (except for the three who complained), and the complainants who experienced adverse events. The average age of the complainants was 47 years and 59% were female. Among complainants for whom ethnicity data were available, 14% were Maori and 3% were Pacific. In general, complainants’ injuries involved were quite severe, with 31% resulting in permanent injury and 28% in death. Most of the injuries (79%) were preventable. 44% (110/254) of complaints involved an injury that was both permanent and preventable.
Bivariate analyses showed several significant differences between injured complainants and injured non-complainants (table 1). Compared with complainants, non-complainants were significantly older (52 v 47 years, p = 0.003) and more likely to live in deprived socioeconomic areas. Complainants, on the other hand, were significantly more likely to have sustained injuries that led to permanent disability or death, and preventable adverse events were twice as common in this group (79% v 37%, p<0.001).
These differences persisted in multivariate comparisons of the complainants and non-complainants (table 2). Injury severity was a strong predictor of complaining, with odds of complaining increasing with injury severity. Compared to patients with temporary disability, the odds of complaining for patients with a permanent disability resulting in >50% impairment were 11.4 times greater (95% CI 5.9 to 22.1), and for patients who died they were 17.9 times greater (95% CI 9.3 to 34.2). There was also a strong independent correlation between preventability and odds of complaining (odds ratio (OR) 7.6, 95% confidence interval (CI) 5.0 to 11.6).
In addition, several sociodemographic factors were associated with propensity to complain after an adverse event. Odds of complaining for patients in the most deprived quintiles were one third those for patients in the most privileged quintile (OR 0.3, 95% CI 0.2 to 0.6). Elderly patients were significantly less likely than their younger counterparts to complain following an adverse event (OR 0.2, 95% CI 0.1 to 0.4). Odds of complaining for patients of Pacific ethnicity were lower than for non-Maori/non-Pacific patients (OR 0.3, 95% CI 0.1 to 0.9). The difference was not statistically significant for Maori patients, perhaps because of the relatively small numbers in this category.
This study is the first to match epidemiological data on medical injuries to complaints about quality of care lodged with a national health ombudsman. Three findings are noteworthy. Firstly, while the right to an appropriate standard of care is only one of 10 patient rights enforced by the Commissioner, most complaints involved an adverse event, often a serious one. Secondly, our results suggest that approximately one in 200 injured patients complain about their care to the Commissioner. Among patients who experience injuries that are both serious and preventable, one in 25 complain. Thirdly, the “under-complaining” phenomenon was not spread uniformly across the patient population: elderly patients and socioeconomically disadvantaged patients were especially unlikely to complain despite having suffered an injury, and propensity to complain increased steeply with the severity of the injury sustained.
Legitimacy of complaints
Although New Zealand doctors appear to support the use of complaints as a quality assurance tool, concerns abound in the medical community about the prevalence and reasonableness of many complaints brought before the Commissioner.13–15 The 2001 survey by Cunningham and colleagues13 of doctors’ attitudes about complaints found general support for a forum for hearing and investigating complaints, but considerable scepticism about the legitimacy of complaints actually lodged with the Commissioner. Only one in 10 doctors agreed with the statement that most complaints were warranted, and half disagreed with the statement that most complaints are about errors and actual wrongdoing.
The second of these opinions is partially correct as a matter of law. The Commissioner’s obligations extend beyond classic violations of quality. New Zealand law sets forth a variety of other rights, including rights to be treated with respect and to be free from discrimination or financial exploitation. Perceived breaches of all such rights are legitimate bases for complaint.16
Nonetheless, our analysis suggests that doctors’ attitudes about the reasonableness of complaints are at odds with reality, at least among the subset of complaints related to public hospital care. Nearly two thirds of the complainants in the study sample had experienced adverse events, of which 79% were preventable and 60% involved permanent injury or death; 93% of the adverse events were either preventable or serious injuries.
It is incorrect to interpret these results as evidence that complaints are usually triggered by doctors’ wrongdoing. The causes of adverse events in medicine are often multifactorial, involving a complex interplay between individual and system factors. On the other hand, the prevalence of adverse events—especially preventable and serious ones—refutes the notion that most complaints over quality of care are groundless.
Complaint rates were low. The finding that only 0.4% of adverse events and 4% of serious preventable adverse events triggered complaints is consistent with crude estimates of 0.3% (254/85 000) and 2.3% (110/4800) obtained using the number of complaints as the numerator and an extrapolation of adverse event rates to the national level as the denominator.
In an earlier analysis of the same survey data, Cunningham and colleagues17 found that one in three doctors had experienced a complaint at some time in their career; the study estimated a complaint rate of 5.7% per doctor per year and used these data to conclude that there was a “high incidence” of complaints. From the perspective of busy practitioners, this may be so. But from a health policy perspective, the prevalence of poor quality care rather than the number of physicians is the appropriate baseline against which to measure complaint frequency. Using this metric, we reach the opposite conclusion: complaints are rare in the sense that the vast majority of preventable adverse events never trigger one.
Why are complaint rates so low? Felstiner and colleagues’ conceptualisation of the evolution of disputes through a process of “naming, blaming, and claiming” helps organise the most likely explanations.18 Firstly, many patients may not be aware that they have sustained an injury from medical care. Disentangling medical injury from the progression of underlying illness is not straightforward, especially in the inpatient setting where that illness may be severe.
Secondly, patients may recognise their injury but be unaware of the Commissioner’s services, or unwilling to commit the time and energy needed to take action. In theory, the complaints process should pose few barriers—patients can lodge a complaint by writing a letter or by making a free phone call (0800 11 22 33); a lawyer’s assistance is not required. However, in perception or reality, it may not be straightforward for some aggrieved patients. The complaints process has been described as “confusing, cumbersome, difficult to access, and costly, both financially and emotionally.”19 Moreover, health professionals are provided with a copy of the letter of the complaint, including the patient’s name, so some patients may hold back out of concern that such action will bring tension into their relationship with their doctor.
Thirdly, despite both recognising their injury and understanding the complaint option, some patients may simply adopt the attitude that “what’s done can’t be undone”, put the event down to bad luck, and move on. Alternatively, they may elect to take action, but not with the Commissioner. Several other options are available to injured patients seeking redress or accountability following an adverse event. Monetary compensation is available through the national no-fault compensation scheme. Patients seeking an apology, an explanation, or system change to protect others from suffering a similar harm can have those interests met by bringing their concerns directly to the attention of their healthcare provider by using free independent patient advocacy services, or by lodging a complaint with the hospital.
Disparities in use of complaints
A study by Tapper and colleagues20 of complaints against surgeons found that they were more likely to be brought by women and patients in older age groups. Again, analyses of complaints that do not calibrate their frequency to the underlying rate and patterns of injury can be misleading. Women and the elderly are leading users of the healthcare system;21 they are also over-represented among injured patients.5 Calibrating complaints to baseline data on injury, we found no sex differences in complaint behaviour. Elderly patients, on the other hand, were one quarter as likely as their younger counterparts to complain following an adverse event.
Similarly, socioeconomic disparities in complaint behaviour are not readily apparent from a discrete analysis of complaint data. The incidence of complainants is fairly evenly distributed across the five deprivation quintiles, but multivariate regression analysis controlling for the presence and severity of injury showed that patients from the most socioeconomically deprived areas were significantly less likely to complain. These results echo studies of malpractice litigation in the United States in which both old age22,23 and lower socioeconomic status23,24 have been correlated with lower propensity to sue.
Severity of injury
The strong relationship we identified between severity of injury and propensity to take legal action is also consistent with findings from medicolegal research from the United States.23,24 Although the Commissioner’s complaint processes attract only a small proportion of adverse events, the odds that an injury will materialise as a complaint increase steeply with severity of the injury; the odds are also substantially greater if the injury is preventable. There is thus a clear “bias” in the severity and types of injuries that come before the Commissioner. Complaints data should not be construed as representative of general patterns of medical injury. On the other hand, the skew towards serious and preventable events is precisely what policymakers might hope for from a system whose goals are to protect consumers from the most serious safety hazards and identify opportunities for quality improvement.
Limitations of study
Our study has several limitations. Firstly, complaints relating to episodes of care in 1998 may have been (or might be) lodged later than 30 June 2004, the date our complaints sample was drawn, although this is unlikely because virtually all complaints are filed within 2 years of the date of the alleged injury (mean 10 months, median 5 months). The 5.5 year window for complaints that we allowed is therefore conservative.
Secondly, estimating adverse event rates through medical record review has recognised limitations.25,26 In our review of complaint files, inter-reviewer agreement on the preventability judgment was only fair. To the extent that complaints were judged not to involve preventable adverse events and they did, our regression analysis will underestimate the predictive value of preventability in complaining.
Thirdly, several of the other variables used in our analyses were suboptimal. Ethnicity data were missing for one quarter of complainants, and misclassification of ethnicity is a recognised problem.27 The use by the New Zealand Index of Deprivation of small area-based measures to assign socioeconomic characteristics at the individual level creates the potential for measurement error.10 The direction and magnitude of potential biases stemming from these data limitations are unknown, but we know of no reason why they would differ systematically between complainants and non-complainants and thus affect the results of our analyses.
Given the absence of tort remedies and the availability of a free independent complaints mechanism, it might be expected that patients in New Zealand would frequently lodge complaints following adverse events. Indeed, some physicians in New Zealand feel under siege by complaints processes and the medicolegal environment has been described as one of the “most hostile” in the world.28 Yet when complaints to the Commissioner are set against the underlying rate of injury, it becomes apparent that they represent only the tip of an iceberg of adverse events. That tip misrepresents what is beneath the surface in two important ways. Firstly, the relatively low propensity to complain among patients who are elderly, socioeconomically deprived, or of Pacific ethnicity suggests troubling disparities in access to and utilisation of complaints processes. Further research is required to better understand and address these disparities. Secondly, the probability of a complaint increases steeply with severity of injury, and preventable events are much more likely to lead to a complaint than unpreventable ones. In this regard, complaints offer a valuable portal for observing serious threats to patient safety and may facilitate efforts to improve quality.
The authors thank Jean-Marie Berthelot and Roy Lay-Yee for their expert statistical assistance. Elana Curtis offered valuable comments on the manuscript.
M M Bismark was a Harkness Fellow in Healthcare Policy in 2004–5.
Funding for this study was provided by the Commonwealth Fund, an independent non-profit organization based in New York City. The authors’ work was independent of the funders.
Competing interests: none.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.