Article Text

Download PDFPDF

Identifying patient and practice characteristics associated with patient-reported experiences of safety problems and harm: a cross-sectional study using a multilevel modelling approach
  1. Ignacio Ricci-Cabello1,
  2. David Reeves2,
  3. Brian G Bell3,
  4. Jose M Valderas4
  1. 1 Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
  2. 2 NIHR School for Primary Care Research, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK
  3. 3 Primary Care, University of Nottingham, Nottingham, UK
  4. 4 NIHR PenCLAHRC and Institute for Health Services Research, University of Exeter Medical School, University of Exeter, Exeter, UK
  1. Correspondence to Dr Ignacio Ricci-Cabello, Nuffield Department of Primary Care Health Sciences, University of Oxford, Radcliffe Observatory Quarter, Woodstock Road, Oxford, OX2 6GG, UK; nacho.ricci.cabello{at}gmail.com

Abstract

Objective To identify patient and family practice characteristics associated with patient-reported experiences of safety problems and harm.

Design Cross-sectional study combining data from the individual postal administration of the validated Patient Reported Experiences and Outcomes of Safety in Primary Care (PREOS-PC) questionnaire to a random sample of patients in family practices (response rate=18.4%) and practice-level data for those practices obtained from NHS Digital. We built linear multilevel multivariate regression models to model the association between patient-level (clinical and sociodemographic) and practice-level (size and case-mix, human resources, indicators of quality and safety of care, and practice safety activation) characteristics, and outcome measures.

Setting Practices distributed across five regions in the North, Centre and South of England.

Participants 1190 patients registered in 45 practices purposefully sampled (maximal variation in practice size and levels of deprivation).

Main outcome measures Self-reported safety problems, harm and overall perception of safety.

Results Higher self-reported levels of safety problems were associated with younger age of patients (beta coefficient 0.15) and lower levels of practice safety activation (0.44). Higher self-reported levels of harm were associated with younger age (0.13) and worse self-reported health status (0.23). Lower self-reported healthcare safety was associated with lower levels of practice safety activation (0.40). The fully adjusted models explained 4.5% of the variance in experiences of safety problems, 8.6% of the variance in harm and 4.4% of the variance in perceptions of patient safety.

Conclusions Practices’ safety activation levels and patients’ age and health status are associated with patient-reported safety outcomes in English family practices. The development of interventions aimed at improving patient safety outcomes would benefit from focusing on the identified groups.

  • Adverse events, epidemiology and detection
  • General practice
  • Health services research
  • Medical error, measurement/epidemiology
  • Patient safety

Statistics from Altmetric.com

Introduction

The growing interest in patient safety in primary care is fully justified for a number of reasons.1 It has been estimated that the rate of patient safety incidents in primary care ranges from 2 to 3 incidents for every 100 consultations,2 and that around 4% of these incidents may be associated with severe harm (including long-term physical or psychological effects or death).2 In English general (family) practices (with around 750 000 general practitioner (GP) consultations each day), this would translate into 600–900 patients being severely harmed every day. There is some evidence that between 45% and 76% of these incidents could be prevented.3

Despite the increasing awareness of the magnitude of this problem, the knowledge base about patient safety in the primary care context is still limited and mostly focused on understanding the nature of safety problems and their frequency. Evidence about the factors that can determine their occurrence is, however, very sparse.4 An increased risk of experiencing safety problems and harm may be associated with both patient clinical characteristics (such as the complexity5 of their conditions or of their therapeutic regimens) and practice characteristics (such as staffing or the practice meeting other criteria for quality of care). This information would be crucial for the development of interventions to reduce avoidable harm. Practice-level predictors of patient safety outcomes could help target interventions aimed at improving patient safety in practices, while individual clinical predictors would help us identify patient groups particularly susceptible to benefiting from targeted interventions.

Previous studies have suggested that adverse drug events are more frequent in the elderly6 and in patients following more complex drug regimens.7 Errors in diagnosis and treatment have been associated with patients’ ethnicity, educational attainment and health status.8 Similarly, it has been observed that smaller practices, those with a higher proportion of elderly patients and those located in areas with higher levels of social deprivation are at increased risk of adverse events.9

However, there are limitations with the available evidence. Previous studies have not simultaneously examined patient and practice factors associated with safety events. The extent to which the occurrence of safety problems is due to patient characteristics or to practice characteristics is largely unknown. Surveys of patients about the healthcare received from their practices generate data that have a hierarchical or multilevel structure (patients ‘nested’ within practices). Analysis of this type of data should therefore use multilevel modelling approaches that take appropriate account of the clustered nature of the data and enable exploration of the sources of variation at each level. This is important because different practices are likely to attract patients with particular characteristics. A further limitation of available evidence is that most of the research in the area of primary care patient safety has relied on information supplied by healthcare professionals, and the views of patients themselves have been seldom taken into account.10 Evidence suggests that patients are sensitive to, and able to recognise, a range of problems in healthcare delivery,11–14 some of which are not identified by traditional systems of healthcare monitoring.15 16 Some studies have been based on patient-reported information, but were limited by the lack of valid and reliable patient-centred instruments to capture patients’ experiences and outcomes of patient safety.17 A patient-centred tool to measure patient safety in general practices — the Patient Reported Experiences and Outcomes of Safety in Primary Care (PREOS-PC) — has been recently developed and validated.18 It allows for a comprehensive assessment of patient perceptions, experiences and outcomes of patient safety in primary care.

The aim of this study was to explore the extent to which factors at the level of the practice and the patient are associated with safety problems and harm as reported by patients themselves.

Methods

Study design and participants

We recruited practices in five regions in the North, Centre and South of England, using purposeful sampling to ensure variation in terms of list size and levels of deprivation. Data on patient-reported patient safety measures and on patient characteristics were obtained using a cross-sectional survey.18 The survey was sent in June 2014 to a computer-generated random sample of 150 registered patients (18 years old or older) at each participating practice with a covering letter and a prepaid return envelope. Ethical approval was granted by Nottingham Research Ethics Committee (Reference 13/EM/0258; July 2013). The survey included the PREOS-PC questionnaire,18 the EuroQOL 5D-5L19 and questions about patient sociodemographic and clinical characteristics described in more detail below.

Practice-level characteristics were obtained from the NHS Digital (Health and Social Care Information Centre).20 We also collected information on professional safety climate with the PC SafeQuest21 from 30 practices.

Dependent variables

We built predictive models for three outcomes indicative of different aspects of patient safety: (1) experiences of patient safety problems (errors), (2) harm and (3) overall perceptions of patient safety in the practice. We measured these with relevant scales from the PREOS-PC instrument relating to patient perceptions and experiences of the safety of their primary healthcare over the past 12 months. Details of the three scales are provided in online supplementary appendix 1.18 In short, experiences of safety problems were measured with an 11-item scale (Cronbach’s α=0.75) capturing information about whether or not the patients experienced specific types of safety problems (response categories being ‘No’, ‘Only once’, ‘Yes, more than once’). Harm was measured with a four-item scale (Cronbach’s α=0.89) capturing information about whether or not the patients experienced specific types of harm (response categories being ‘Not at all’, ‘Hardly any’, ‘Yes, somewhat’, ‘Yes, a lot’, ‘Yes, extreme’). Overall perception of patient safety was measured with a single-item visual analogue scale ranging from 0 (completely unsafe) to 10 (completely safe). Scale scores were calculated as the percentage of the maximum score achievable over all items combined, with scores ranging from 0 to 100. For multi-item scales, where responses were missing for more than 50% of the items the whole scale was scored as missing; otherwise a score was derived based on the available items without any imputation.

Supplementary Material

Supplementary data

Practice-level independent variables

Practice-level data obtained from NHS Digital (Health and Social Care Information Centre)20 included data on size (number of registered patients); case-mix (proportion of patients aged above 65); long-term condition caseload (derived by summing the registers for all conditions in the quality and outcomes framework (QOF) and dividing by list size); rurality (based on population density of the practice postcode); social deprivation (based on practice postcode and estimated using the Index of Multiple Deprivation 2010)22; workforce (number of GPs, proportion of male GPs, proportion of GPs aged below 35, proportion of GPs aged above 50); and an independent estimate of the quality of care provided by the practice based on the QOF overall score.23

In addition, a measure of patient safety activation at the practice level was included by calculating practice mean scores on an 11-item PREOS-PC scale measuring the degree to which practices create an adequate environment to ensure the delivery of safe healthcare (see online supplementary appendix 1). Professional safety climate was measured by the PC SafeQuest.21

Patient-level independent variables

Patient characteristics were collected as part of the postal questionnaire. Sociodemographic variables included age, gender, ethnicity, educational attainment, occupational status, being native to the UK and speaking English as a first language. Clinical variables included a measure of self-reported health status (EQ-5D-5L),19 number of long-term conditions and number of medications currently taken. We also included two measures of practice utilisation: times seen a GP during the last 12 months and years registered with the GP surgery. Finally we included a measure of patient activation in respect to safety, a two-item PREOS-PC scale measuring the extent to which patients are proactively involved in trying to prevent medical errors and avoidable harm (see online supplementary appendix 1).

Statistical analyses

Linear multilevel (patients within practices) multivariate regression models, undertaken using the Stata V.13 xtmixed command, were used to investigate relationships between each of the three dependent variables and patient and practice characteristics, using practice as a random effect in all models and accounting for the clustering of patients within practices. For each outcome an initial series of univariate analyses were conducted to examine the relationship with each independent variable in turn. Next, all variables found to have a univariate relationship with the outcome at α=10% (to avoid premature exclusion) were entered together into a multilevel multivariate regression analysis, using α=5% as the threshold for statistical significance in these multivariate models. Variance inflation factors were computed to check for multicollinearity between the included variables. Scores for all continuous variables (dependent and independent) were standardised in order to facilitate interpretation of the regression coefficients as standardised beta coefficients. Patient scores on all three outcomes demonstrated highly skewed distributions (see online supplementary appendix 2). Therefore, to check the robustness of our findings to non-normality and non-constant variance, the significance of the predictor variables in our final multivariate models was validated using non-parametric, bootstrapped, percentile-based p values, based on bootstrap samples of 10 000.24

As a final step we calculated variance partition coefficients for each outcome. Variance partition coefficients represent the proportion of total variance in an outcome that is due to differences occurring at each level. In other words, a high coefficient at practice level indicates that more of the variation in the model is due to differences between practices than between patients. We first calculated variance partition coefficients in a model with random intercepts at the practice level and no explanatory variables (raw coefficients), then after adjustment for patient characteristics statistically significant in our final models (patient-adjusted coefficients), and finally after adjustment for characteristics of the practice and patients that were statistically significant in our final models (fully adjusted coefficients).

We included patients in the multilevel models only if they had complete data on all the explanatory variables. However, the number of patients providing data for each outcome varied, so comparisons between the different outcomes should be made with caution. In a sensitivity analysis we reran the final multilevel models using multiple imputation to impute missing values across the full set of cases. A maximum of 20% of data were missing for any given variable. We used multivariate chained equations in Stata V.13 to generate 10 imputed data sets and then ran Stata’s xtmixed command on each imputed data set in the same manner as we had before (see above) and pooled the results across the 10 analyses.

Results

Sample characteristics

A total of 45 practices were recruited. Information about their characteristics is reported in table 1. In comparison with the overall characteristics of all English practices, the participating practices were on average larger (mean list size 8744 vs 7041) and had a slightly higher proportion of non-white patients (18.8% vs 15.9%), but were very similar with respect to gender balance (female participants 50.6% vs 49.1%), proportion of older patients (patients aged above 65 16.5% vs 15.3%) and deprivation (Index of Multiple Deprivation22 score 25.5 vs 24.0).25

Table 1

Characteristics of participating practices and patients

A total of 1244 patients completed and returned the PREOS-PC survey (response rate=18.4%).18 Fifty-four patients were excluded where the respondent had not visited their GP practice in the last 12 months. Compared with the overall characteristics of all patients registered in the 45 participating practices, the 1190 included respondents were more likely to be female (59% vs 51%), aged ≥65 (39% vs 15%) and of ‘white’ ethnicity (91% vs 82%) (see table 1).

Average scores for patient safety outcomes (with high scores indicating high levels of safety) were very high for both experiences of safety problems and harm (>90%) and also high for overall perceptions (>85%) (table 2).

Table 2

Outcome measures of patient safety at the patient and practice level

Univariate analyses

In the univariate analyses four patient-level factors (age, health status, speaking English as a second language and patient activation) were consistently associated with all three outcomes, while gender, educational attainment and time registered in the practice were not statistically associated with any outcome (see online supplementary appendix 3). Only one practice-level variable (practice activation) emerged as a consistent predictor of all three outcomes, while practice deprivation was associated with both experiences of patient safety events and overall rating of patient safety (see online supplementary appendix 4). No associations (α=10% or less) were observed for any other practice-level factor.

Multivariate analyses

The results of the linear multilevel multivariate analyses are summarised in table 3 and below. Variance inflation factors for the included variables in all cases were below a conservative threshold of 4.0,26 indicating that multicollinearity was low.

Table 3

Summary of multivariate analyses between explanatory variables and patient-reported experiences of safety problems, harm and overall rating of patient safety

Higher scores on experiences of safety problems (indicating lower frequency of safety problems and therefore safer healthcare) were associated (p<=0.05) with a higher practice activation score (beta coefficient=0.438) and with increasing age (0.150). Higher harm scores (indicating lower frequency and severity of harm and therefore safer healthcare) were associated with better patient health status (0.229) and increasing age (0.139). Higher scores on patients’ overall perception of safety (indicating a perception of safer healthcare) in their practices were associated at the practice level with higher practice activation scores (0.400).

The fully adjusted models explained 4.5% of the variance for experiences of safety problems, 8.5% for harm and 4.4% for overall perception of patient safety (table 4). Practice characteristics were the only contributor to the explained variance for overall perception of patient safety, whereas patient characteristics were the only contributor to the explained variance for harm. Practice and patient characteristics had a similar level of contribution to the explained variance for experiences of safety problems.

Table 4

Variance explained by patient and practice characteristics

In our sensitivity analyses, results from the multiple imputation using data from all 1190 respondents yielded similar results to the main analysis (see online supplementary appendix 5). The only differences were that with multiple imputation a lower level of patient activation was significantly associated with healthcare being rated as safer on all three outcome measures, whereas seeing a GP more than five times was associated with lower scores on experiences of safety problems but not associated with harm scores.

Post-hoc examination of practice activation

To potentially inform the development of interventions to improve patient-reported safety outcomes, we conducted a number of postprotocol analyses focused on practice activation (which had emerged as a particularly relevant predictor of patient-reported safety outcomes in our main analyses). This included (1) examination of the association between practice activation score and other practice characteristics (see online supplementary appendix 6), and (2) examination of the association between practice activation individual items and our three safety outcome measures (see online supplementary appendix 7).

Lower practice activation scores were significantly associated with higher practice deprivation levels (beta coefficient=−0.116), whereas no statistically significant associations were observed with the rest of practice characteristics examined. The individual items of the practice activation scale most strongly associated with our three safety outcomes included (1) communication about tests and treatments (beta coefficients=0.89 for ‘Experiences of safety problems’ and 0.95 for ‘Overall perception of patient safety’); (2) addressing patients’ safety concerns (0.84, 0.65 and 0.93 for ‘Experiences of safety problems’, ‘Harm’ and ‘Overall perception of patient safety’, respectively); and (3) helping to arrange/organise healthcare (0.84, 0.62 and 0.86, respectively).

Discussion

In this study we used multilevel modelling to investigate predictors of patients’ perceptions of patient safety and experiences of safety problems and harm in English general practices. We identified some patient characteristics (age and self-reported health status) associated with patient-reported experiences of safety problems and harm. One practice characteristic (safety activation) was associated with patient-reported experiences and with overall patient perception of patient safety. Patient and practice characteristics had a similar contribution to explained variance for our three outcome measures.

Strengths and limitations

This is the first large-scale study examining predictors of patient safety in general practices as perceived by patients. Patients’ perceptions and experiences were measured using valid and reliable scales. By taking account of the hierarchical nature of the data, our study has provided estimates of the influence of practice and patient-related characteristics on patient-perceived patient safety that are likely to be more realistic than those from earlier single-level analyses.

Our study has some limitations. First, the overall response rate was low (18.4%), with some subgroups being particularly under-represented, most notably younger and male. This may have resulted in non-response bias, particularly if response propensity was associated with the outcomes being collected,27 although other factors, such as under-response by younger males, may have mitigated against this. In this study, patients who experienced a safety event may have been more inclined to respond to the questionnaire, resulting in overestimation of overall rates of safety problems and harms. Given the low response rate, one important consideration is whether the predictive factors we identified could simply be artefacts of non-response. Key predictive variables of higher (ie, better) outcome scores were older age, better health status and higher patient-reported practice activation. For non-response to account for these relationships would require individuals in the converse subgroups (younger, in poorer health and/or with a lower opinion about practice safety activation) to be less likely to respond when they regard outcomes to be good. Our response rate was lower among younger people, and it is conceivable that the younger individuals who responded were more likely to have experienced a safety event, thus lowering the mean outcome scores for this subgroup. To also account for our other results in this way would require the same to be true for individuals in poorer health or with a low opinion of safety activation at their practice, although such relationships are less obvious.

A second limitation was the need to exclude patients with missing data on explanatory variables, such that only around two-thirds of patients were included in the analyses, which may introduce bias if data are not missing completely at random. To some extent, we overcame this limitation by using multiple imputation in our sensitivity analyses. However, the loss of data reduced the statistical power to detect differences, as did the skewed nature of the outcome variables. Third, this study is based on patient-reported information, and no review of medical records was performed, so no comparison against actual adverse events was possible. However some of the outcomes examined in this study are best reported by patients, both for processes (eg, communication with patients and coordination between professionals) and for harm (eg, pain, harm to mental health). Fourth, this is a cross-sectional study, and reverse causality is plausible in some instances. Longitudinal studies are needed to confirm our findings. Finally, our measures included as ‘safety problems’ a broad range of problematic experiences that are not always considered to involve safety as such. Missed appointments or lack of communication or coordination between providers can be seen as contributory factors to safety incidents rather than safety problems per se, but we deliberately treated these as safety problems because evidence from qualitative studies suggests that they represent areas of major concern for patients.28–34

Interpretation of findings and comparison with previous literature

Identifying practice characteristics associated with a higher risk of safety problems and harm is necessary because it allows the development of targeted interventions aimed at improving patient safety in practices. In this study we observed that patients registered in practices where safety activation levels were reported to be higher were less likely to report experiences of safety problems and perceived their practices as safer. This association has strong face validity, and also supports the construct validity of the scales. Increasing the levels of practice activation for patient safety (which seems to be particularly necessary in those practices with higher levels of deprivation, according to our postprotocol analyses) appears as a promising target for developing interventions to reduce medical errors and avoidable harm. In particular, communication of the expectations and results for tests and treatments to patients, helping to arrange/organise healthcare and acknowledging patient concerns appear to be the areas of practice activation most related to safety outcomes. Interventions focused on those areas are particularly promising.

Identifying clinical characteristics that expose patients to a higher risk of safety events is important for identifying groups of patients who can benefit from targeted interventions. In our study, poorer self-reported health status was associated with increased likelihood of reporting harm. Poorer health status may increase the risk of harm because of (1) a higher exposure to healthcare (needed to tackle their increased health problems) and (b) a higher vulnerability to harm (ie, a lower threshold may be needed to trigger harm in patients with poorer health). Also, given the cross-sectional nature of this study, a reversed association is also plausible (ie, patients may have poorer health as a result of the harm suffered). Finally, given that we used a measure of harm based on patient-reported information, it is also possible that patients with poor health status could have perceived iatrogenesis even when it did not exist — that is, when their poor health status simply progressed.

Identifying patient sociodemographic characteristics associated with patient-reported safety outcomes may help to identify differential reporting, which is useful for interpreting results from the administration of self-report instruments. In our study, only age was associated with our patient safety outcome measures. It has been previously reported that elderly patients are more likely to report a favourable perception of care,35–40 and that — independent of the actual care received — older patients are generally more accepting and more reluctant than younger patients to make negative judgements.41–43 Although empirical evidence about the potential reasons for this ‘age effect’ is lacking, a number of suggestions have been made, including that frail elderly patients often see themselves as a burden on their families and society and might feel they are not deserving of attention,39 or that older patients are less inclined to question what they are told than younger patients.40 A failure to adjust practice safety levels for patients’ age could result in systematic misrepresentation of the performance of practices that provide for particular patient groups. This would be a particular concern where comparison across providers using self-reported measures is linked to financial incentives, as practices working in challenging circumstances could be further disadvantaged by loss of investment. However a real difference in safety levels based on patients’ age cannot be ruled out. Some studies have suggested that older patients may be treated in a ‘more thorough and responsive manner’ than younger patients,44–46 which would minimise the risk of safety incidents in this group. If that was the case (ie, lower scores reported by younger patients reflect lower safety levels), then adjusting for patients’ age would mean that inequitable healthcare provision is not detected. Further research is needed to understand whether the observed age difference is due to differential reporting or to actual differences in healthcare provision.

Conclusions

Using a multilevel approach we found that patients in practices where the level of safety activation was perceived to be lower were more likely to report less safe healthcare. Patients with poorer health status were more likely to report harm, and younger patients were more likely to report experiences of safety problems and harm. The development of interventions aimed at improving patient-reported safety outcomes would benefit from focusing on the identified groups of patients and practices.

Supplementary Material

Supplementary data

References

Footnotes

  • Contributors All authors were involved in the conception and design of the study. BB conducted the statistical analyses under the supervision of DR. IRC drafted the manuscript. All authors were involved in the interpretation of the data, contributed towards critical revision of the manuscript and approved the final draft. JMV had full access to all of the data and had final responsibility for the decision to submit for publication. JMV is the guarantor of this work.

  • Funding This research is part-funded by the National Institute for Health Research School for Primary Care Research (NIHR SPCR). The views expressed are those of the authors and not necessarily those of the NIHR, the NHS or the Department of Health.

  • Competing interests All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/coi_disclosure.pdf and declare no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous 3 years, and no other relationships or activities that could appear to have influenced the submitted work.

  • Ethics approval University of Nottingham ethics committee.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement PREOS-PC data are available under request to the study guarantor.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.