Patient complaints are a growing concern for healthcare systems, not only because complaints reflect patient experiences and dissatisfaction but also because analysing complaints can assist in improving the quality of healthcare provision.1 This research cites that complaints systems are now an integral part of clinical governance toward improving patient care, along with other important indicators such as critical incident reporting, effective communication, clinical audit and risk management.

This mirrors the established importance of customer complaints in other industries as indicators of service quality. A clear relationship has been drawn between customer satisfaction and firm profitability; satisfied customers are more likely to be retained and give repeat business to the company that provides them with a satisfying experience.2 Many experts in this area conclude that complaints should be treated as research data and analysed regularly, such that they highlight opportunities for improvement through what constitutes an inexpensive means of research. Many other researchers echo the importance of customer complaints as both indicators of customer (dis)satisfaction and opportunities for service recovery and improvement,34 as well as linking complaints to the organisational bottom line.5

Research by Mayer and colleagues linking patient complaints and customer complaints literatures even further demonstrates that providing customer service training to emergency department medical staff dramatically improves patient satisfaction.6 These authors focused on any personnel with patient contact, including secretaries, social workers, respiratory therapists and physicians, who completed a training programme on basic customer service principles such as stress recognition and management, communication skills, services transitions and negotiation skills. Given the well-established link between customer complaints (and satisfaction) to important organisational outcomes, and the above example of the parallel between customer service and patient service, there is a convincing argument for examining patient complaints more fully as indicators of service quality and satisfaction.

Patients register many types of complaints.7^–12 While patients do make complaints, the degree to which they may be useful to hospitals is still in the speculative stage, though there is growing evidence that complaints correlate with other indicators of quality of care and safety. For example, it has been found that physician care providers with more patient complaints had more risk management files (files associated with adverse events or errors where a lawsuit may follow) than those with fewer patient complaints.13 In a study of surgical admissions, it was found that patient complaints were more likely to occur when the admission was associated with a complication.14 These authors argued that patient complaints might be useful as a marker for poor clinical outcomes. A positive association between consumer complaints in nursing homes and deficiencies in subsequent inspections resulted in the conclusion that complaints may signal quality concerns.15 It has even been suggested that patient complaints may serve as a signal that may reduce patient harm if heeded, and that it would be wise to integrate patient perspectives into quality improvement.16

There is also a growing body of literature that contends that a positive culture of patient safety is an important aspect of overall patient care.17^–19 To date, however, there have been no empirical investigations assessing the extent to which a positive patient safety culture influences the likelihood of patient complaints. This study attempted to address that knowledge gap. Specifically, we assessed a clinical factor that might be a determinant of patient complaints, and explored whether patient safety culture was a determinant of patient complaints.

The analyses for the first section of our study are based at the hospital unit level. We predicted that a more positive patient safety culture within a hospital unit would be related to a lower number of complaints against the unit.

The second type of analysis conducted used patient-level information as the unit of analysis. In this instance, the dependent variable was whether a given patient lodged a complaint or not. We proposed that patient complaints were more likely to occur when patients presented complex, resource-intensive cases, as these types of variables have been shown to be related to adverse events,20^–22 and thus may also be useful in predicting patient complaints.

METHODS

Data sources and study sample

The sample used in this study was obtained through three separate archival health data sources produced by the Calgary Health Region, a health organisation that provides regionalised healthcare services for over a million residents living in Alberta, Canada. The three data sources were linked by one or more common fields. There were two databases that housed patient-level information, including the patient complaint database and patient information while admitted to the hospital. The third database housed information at the patient care unit within each hospital level of analysis. This database provided the information required on patient safety culture for this study. Each of these will be described more fully in the Outcome and Predictors subsections of this paper.

We restricted the analyses to the calendar year 2005 and to patient care units across four general hospitals in a large metropolitan centre. We did this so we could obtain recent and complete data sets where the final set would be manageable in terms of linking the databases together. Complaints filed against long-term care facilities, or cross-site services such as parking and housekeeping, were not included in this study because they did not constitute an admitting unit in an acute-care facility and/or the databases associated with their services were housed in completely separate databases.

Outcome

The first database we used, called RESPOND, is a compilation of patient complaint reports. This database was started in 2002 as a formal mechanism for patients, patient advocates (eg, friends, family members) or members of the public to file a formal complaint with the health region by phone or in writing. The database allows for tracking such complaints in a systematic manner. In 2002 there were approximately 800 complaint reports filed, and in 2005 almost 1500 were filed (586 of which occurred in one of the patient care units in one of the four general hospitals, thus meeting our inclusion criteria). This reporting increase from 2002 to 2005 is not necessarily reflective of poorer care, but instead may be due to the fact that the RESPOND system has been widely advertised in the health region through the use of pamphlets, via advertising in the telephone book and the Region’s website, and referrals of patients with concerns to the RESPOND staff.

The complaints are entered by three full-time, professional staff members whose sole job is to deal with patient concerns and who have been working with this database since its inception. When the complaints are entered, they are categorised into a “Main Complaint” grouping. This categorisation was developed for use by the Region at the time RESPOND was implemented. The RESPOND staff decide on which category to assign a complaint when it is described to them. If there is more than one “Main Complaint,” then multiple report files are opened, one for each of them. Only one instance of a multiple complaint (two complaints) occurred in our data set. About 60% of the complaints are dealt with immediately by the staff members allocated to the data management of this system. The other complaints require formal review (ie, pulling of a patient chart, discussions with staff attending the patient, and a formal assessment and response by the relevant manager of the unit).

Predictors

The second database contained responses by the health region staff to a 40-item Patient Safety Culture 2005 scale used by the health region.23 This survey was developed specifically by the health region for internal use. Five of the items were used to create a mean score for each hospital unit on Patient Safety Culture. The other items did not directly ask about patient safety culture (items asked about communication channels, witnessing adverse events, work performance, patient care, etc). The five safety culture questionnaire items were: (1) My unit does a good job managing risks to ensure patient safety; (2) Senior management provides a climate that promotes patient safety; (3) I work in an environment where patient safety is a high priority; (4) Give your unit an overall grade on patient safety; (5) Give the Region an overall grade on patient safety. These items were selected because they included “patient safety” in the questions, thus focusing the item directly on patient safety, rather than communication or performance in general. The focus of each item was on the importance of patient safety in the workplace environment as it is perceived by the staff, rather than on patient safety attitudes or behaviours of the staff members themselves.

All items were rated on a Likert-type scale from 1 to 5, with higher numbers indicating a more positive response. The overall patient safety culture score was then derived as the mean of scores across items. The items were inter-correlated (ranging from 0.354 to 0.599), and the Cronbach alpha for the composite measure was 0.82. This survey was completed by a random sample of staff within each unit in 2005. Because we collapsed individual survey responses across units, we included only those units where three or more staff members had completed the survey.

The third database contained Admissions, wherein admission to any health facility in the region is entered as a separate case. There is extensive information regarding each case in this database including age, sex, primary diagnosis or procedure,24 patient care unit associated with primary treatment, and resource intensity of the case. Resource Intensity is a relative resource allocation methodology for estimating a facility’s costs for both acute inpatient and ambulatory care,25 and is a summary index incorporating the type of diagnosis/procedure, patient age, case complexity and length of stay. Higher numbers (greater than 1.0) on this measure indicate greater cost or more resources required than would be typically expected, whereas lower numbers (less than 1.0) indicate lesser cost or fewer resources required than would be typically expected.

An attempt was made to link the 586 Patient Complaint Reports (RESPOND) to the Admissions database so as to gather pertinent patient information for each complaint file. The links were made based on patient name and unit against which the complaint had been lodged. Specifically, we needed to secure the resource intensity value assigned to each patient who lodged a complaint. We confined our analysis of the association between resource intensity and complaints to inpatient units, because day-care unit data and emergency-room data for individual patient encounters do not include comparable data on patient resource intensity. Because of these issues, we were able to reliably link only 205 of the 586 cases from the RESPOND database to the Admissions database. A chi-square test revealed that the types of complaint reports for the 205 linked cases were not different from the types of complaints registered by the 381 non-linked cases (p = 0.199).

An additional 205 control patients were extracted from the Admissions database where there was no complaint associated with create a matched case-control design. The control patients were individually matched to cases on (1) the hospital unit at which the patient received care, (2) primary diagnosis or primary procedure as coded by the case-mix group (CMG) index,24 (3) age range (SD 10 years), and (4) sex of the patient who had lodged a complaint. Thus, we had a total of 410 individual-level data points (205 complaints and 205 non-complaints). None of the control cases was also a complaint case.

Analyses

Simple linear regression was used to assess the first hypothesis. We regressed the rate of complaint reports in each unit on the patient safety culture of each unit. Conditional logistic regression was used to test the hypothesis that case Resource Intensity would predict whether or not a given patient would lodge a complaint against a hospital unit. We used conditional logistic regression because of the matched case-control design of this aspect of the study.26

RESULTS

Descriptive and frequency analyses

There were a total of 586 complaint reports that met our criteria, arising from care on 45 different units, of which 205 were analysed in our matched case-control analysis. Although a patient might have contact with a number of units during their hospital stay, the unit against which the complaint is lodged is entered into the complaints database. As noted earlier, for the matched case-control analysis we removed more than half of the total number of complaints due to an inability to secure a Resource Intensity value for them because they were not admitted (ie, for those that had been lodged against outpatient facilities, emergency rooms, operating rooms and diagnostic imaging).

The number of complaint reports for each of the 45 units ranged from 3 to 85 (M = 13.02; SD = 19.25). We observed that as the number of patients treated through a unit increased, so too did the number of complaints against the units. We found that the correlation between these two variables was very high in our study, (r = 0.78; p<0.001), indicating that busier units were more likely to experience complaints. Thus, we created a rate of complaints variable that took into account the case volume of each unit (number of complaint reports/number of patients processed within each unit). The rate of complaints using this approach showed a range from a low of 1/10 000 to a high of 755/10 000 (M = 66; SD = 144). This rate of complaints variable then served as the criterion in the regression analysis.

A description of the 586 complaints is shown in table 1. Most of the complaints were due to the level or quality of care (28%), wait times (20%) or the attitude/behaviours of care providers (17%). These categories were developed by the health region and used by the staff when entering the complaint into the database. A description of the 45 acute care units from the four different sites as well as the rate of complaints appears in table 2.

The types of patient care units differ somewhat across hospitals by virtue of the site-specific regional specialisation of services that hospitals typically develop. For example, the first site was a children’s hospital and so did not have similar patient care units to the other three adult acute care hospitals. Some units were excluded because fewer than three staff rated patient safety culture. Some of the units were excluded because there were too few cases within the unit, since we were unable to match the complaint file reliably with the admission file for any one patient, or to match the complaint case with a control case.

From the Admissions database, we were able to determine the number of patients treated in the 45 units on which we had Patient Complaint data in 2005 (M = 11740; SD = 20 636, range 53 to 75 514). We also secured the age, gender, primary diagnosis, and resource intensity for the 205 cases where there was a clear link between the RESPOND and Admissions databases. The demographic distribution of this data set showed that almost twice as many complaints were lodged for female patients (65%) compared with male patients, and patient age ranged from 2 to 95 (M = 57; SD = 20.5). The individual-level predictor variable of Resource Intensity ranged from 0.18 to 40.5, and the mean of this variable in our sample was 1.80 (SD = 3.21).

The mean of the Patient Safety Culture measure was 3.75 (SD = 0.665). There were 578 surveys that were used in the analysis collapsed across 45 different units. The number of surveys for each of the units in the analyses ranged from 3 to 46 (M = 12.84; SD = 11.32). We assessed our justification for collapsing across the staff responses to the survey by calculating the r_wg interrater agreement index2728 for each of the 45 units. The mean r_wg for the Patient Safety Culture was 0.75.

Regression analyses

Using the sample of 45 units for which we had Patient Safety Culture aggregate scores as the predictor and probability of a complaint within hospital unit as the criterion, we found that Patient Safety Culture was not significant (R2(1, 42) = 0.016; F = 0.678; p = 0.415).

Using the 410 matched-case data set, we found that the addition of Resource Intensity over and above the constant significantly reduced the log-likelihood of the model (χ²(1) = 4.62; p = 0.032) in predicting complaint/no complaint. The coefficient (β = 0.145) resulted in an odds ratio for this coefficient of 1.16 (CI 0.994 to 1.344), indicating that for every one unit increase in Resource Intensity, the odds of making a complaint increased by 16%.

DISCUSSION

Our findings weakly supported one of our hypotheses, namely that the resource intensity of medical admissions cases was predictive of patient complaints. Patients who presented as complex cases, underwent many procedures and/or had long stays in hospital were more likely to lodge a complaint. However, the confidence interval around the odds ratio just included 1.0, indicating that this finding is not statistically significant at the conventional 95% confidence level. Thus, we present the following possible reasons for our finding with the caveat that it is not “significant” in a traditional framework of dichotomously judging statistical significance. The finding can be conceptualised as one of exposure and opportunity. That is, patients with a high resource intensity are often in the hospital for a long time, have multiple procedures and may be moved around to several different units. It may be useful for hospital staff to be aware that these types of patients have likely undergone significant hardship and duress. Special attention by care providers may be required in such circumstances. Resolving the concern as soon as possible—preferably before the patient leaves the hospital—is often termed “service recovery” and refers to patient-oriented complaint handling by the healthcare providers rather than shifting this responsibility to a patient advocate.29

We did not find support that patient safety culture had any effect on patient complaints. There are at least two possible reasons for this. First, patient safety culture may simply not matter—that is, patients may lodge complaints regardless of how safety-conscious the unit is. Alternatively, some units with high patient safety cultures may actively encourage patients to lodge a formal complaint if they are not satisfied with the level of care received. Yet another possibility is that there may be an association between safety culture and patient complaints, but only one of modest strength that is not readily exposed by the safety culture measure and patient sample size that were used in this study. It is not possible to tell whether one or more of these scenarios are at play in the health region studied, and further exploration of our study findings in relation to these possible explanations is something to consider pursuing in future research.

One intriguing finding from our study that was unanticipated was the high variability in rates of complaints. Specifically, the rate of complaints ranged from a low of 1/10 000 to a high of 755/10 000. It has been found in prior research that rates of complaint differ across units even after controlling for patient visits or bed days in hospital.30 Our data are consistent with this high variation. Perhaps hospital leaders should focus their initial energies and limited resources on improvement efforts designed to address the most common concerns expressed about the highest complaint units for all their patients, not just those with the most complex cases.

Limitations

The scope of the database searches and data compilation were limited to four acute care settings in a large metropolitan centre in western Canada over a limited time frame (ie, the calendar year of 2005). Because there were only four sites, and a limited number of comparator units per site, we did not have a large enough sample to determine whether site would be an important predictor or effect modifier, over and above our unit-level analysis. This question would be quite interesting for future research. Second, the patient complaints are lodged voluntarily; therefore, many poor experiences may have not been reported. In addition, because the complaint system was relatively new in 2005, it is likely that there is under-reporting of patient complaints, which is consistent with prior research even after an adverse event.31 With maturity, this issue should resolve itself to some degree but remains a limitation for this study.

Similarly, the safety climate survey was completed by only a sample of staff, and thus may not reflect the sentiments of all staff. In addition, this survey has not been fully assessed for its psychometric properties. It would have been more fortunate to have had safety culture data that were based on a standardised instrument (eg, AHRQ Hospital Survey on Patient Safety Culture32). However, this is the safety culture instrument that the Calgary Health Region implemented regionally as a tool, and its items did exhibit strong internal consistency. In addition, not all units from all the acute care settings were represented in the samples. For example, large units with potentially numerous complaints such as emergency rooms, operating theatres, and diagnostic imaging units had no admissions data associated with them and so were not included in the analyses. Finally, in the simple regression analysis using patient safety culture as a predictor, the power was lower than desired (.66), assuming a medium effect size (r = 0.30) and one-tailed alpha of 0.05.32 Thus, it would be premature to assume that safety culture is an irrelevant variable at this time. For the conditional regression analysis, the sample size was 410 and thus was adequately powered (.99), again assuming a medium effect size (r = 0.30) and one-tailed alpha of 0.05.33

Despite these limitations, our analyses are useful in highlighting that patient complaints may be more likely to occur when patients have a high clinical complexity. The lack of association with a unit’s safety culture implies that it may be non-modifiable clinical complexity factors that are the strongest determinants of patient complaints.