Article Text

PDF

Raising up the voices of the closest observers of care
  1. Naomi S. Bardach
  1. Department of Pediatrics and the Philip R. Lee Institute for Health Policy Studies, University of California, San Francisco, CA 94118, USA
  1. Correspondence to Dr Naomi S. Bardach, Department of Pediatrics, University of California, San Francisco, CA 94118, USA; naomi.bardach{at}ucsf.edu

Statistics from Altmetric.com

Patients and family members are the closest observers of care, with their focus on one patient in one hospital bed or clinic room. While patient-centred care is a well-accepted domain of quality, our ability to gather and use the patient and family member perspective to improve care is still relatively limited.

One potential source of data for this perspective is online reviews of care from social media sources such as Yelp, the online review site, and Facebook, the popular social networking site. These online sources of consumer-generated content likely exert greater influence than the scientifically validated measures of quality published on government-sponsored public reporting websites, such as the Consumer Assessment of Healthcare Providers and Systems (CAHPS) and Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) measures. The influence of online sources of patient-generated assessments of care likely reflects a number of factors: Yelp and Facebook receive far greater traffic1 than public reporting websites2; people frequently use social media for healthcare information3 4; the number of online reviews for healthcare providers has increased substantially over time5 6; and because the data are more engaging—Yelp and Facebook reviewers write narrative stories, which consumers find more compelling than aggregated quantitative experience scores.7 8

Given the increasing presence of online consumer-generated content, multiple studies over the past decade have examined whether the consumer-generated data are meaningful and whether they contribute to better informed consumers. Do they steer people to high-performing providers, thereby improving the quality of care for consumers, or to low-quality choices, leading to poor care and potentially worse outcomes?

The paper by Gaudet Hefele et al9 in this issue of BMJ Quality & Safety adds to this literature by looking at nursing homes, a care setting not previously examined and for which consumer choice may have lasting repercussions. The study describes Facebook use for nursing homes in Maryland and Minnesota and assesses relationships between Facebook star ratings and more scientifically validated measures of patient experience (professionally conducted surveys and in-person interviews) and Nursing Home Compare 5-star ratings. Studies to date have generated mixed results on the question of whether better online review scores are associated with better patient experience or higher performance on other process and outcome quality scores.10–14 This study adds to the mix, finding no correlations between the Facebook ratings of nursing homes and the Nursing Home Compare 5-star ratings, and weak or no correlations for individual domains from the other, more validated, patient experience measures.

However, other studies have found meaningful relationships between online consumer-generated ratings and other measures of quality. For instance, one prior study found that higher Yelp star scores were associated with better HCAHPS scores, lower hospital readmissions and lower mortality. Another study found that UK’s National Health Services online patient ratings on cleanliness were associated with lower infection rates.1 10

Given the popularity of such websites and the studies indicating that their data are associated with meaningful outcomes, we should not decide to ignore online reviews or to simply minimise their effect. Instead, innovative analyses of patients’ and families’ online reviews may allow us to leverage these online observations to improve health delivery and outcomes. The following sections describe two critical questions to start this endeavour.

Question 1: how do people use different social media sites?

One potential reason for the mixed findings across studies of social media sites is that people visit various sites for different reasons. For instance, a Yelp reviewer visits the website specifically to post a review. A visitor to a nursing home or hospital Facebook page might be gathering information, ‘Like’-ing something or reading a post. Thus, the mixed evidence regarding associations with online ratings and outcomes could result in part from the different uses of the websites involved. For instance, associations between online review scores and patient experience and outcome measures have more consistently emerged from analyses of online review sites, such as Yelp or the National Health Services patient review site, for which the only reasons for visiting the sites are to post or find reviews. It may be that people visiting websites for other, more positive, reasons, who happen to then leave a review, are more likely to leave positive reviews. This may lead to a more limited association with outcomes, due to an overall more positive set of reviews. The findings of Gaudet Hefele and colleagues support this hypothesis, as the Facebook scores were an average of 4.5, whereas in prior work on Yelp reviews of hospitals the average star score was 3.3.1

In another recent study in BMJ Quality & Safety, Griffiths and Leaver14 found that aggregated scores across multiple different consumer-generated sources (NHS Choices, Patient Opinion, Facebook and Twitter) predicted in-person Care Quality Commission inspection outcomes for hospitals with exceptionally high accuracy.14 This study demonstrates that aggregating patient experience ratings offers a useful approach to help guide patients in choosing providers and even potentially regulators and payors. However, as with any composite approach, we need to understand the contribution of the individual components. Better data on website users’ intentions will allow more nuanced interpretations of ratings from each website and may help us understand which sites contain more valid data on healthcare quality.

Question 2: what are the strengths and weaknesses of different types of reviewers?

Another important area for exploration is the distinct perspectives that different reviewer types may bring. For instance, some evidence suggests that friends or family members more often comment about patient safety than patients.15 This finding, also presented in greater narrative detail in Atul Gawande’s book ‘Being Mortal’,18 makes intuitive sense since family members are often more awake and alert during a hospitalisation and so potentially more likely to be detail-oriented about safety issues. They are also the ones who leave the bedside to go home and worry about loved ones remaining in the hospital. Online review websites could consider gathering data on reviewers’ relationships to the patient, in order to do exploratory analyses regarding the associations with outcomes for different reviewer types.

Public reporting websites in the USA such as Hospital Compare and Physician Compare could also consider incorporating these findings into their approach to measuring patient experience. Their current approach primarily seeks patient feedback only. Although the nursing home data presented by Gaudet Hefele et al9 come from caregivers, neither the domains for nursing home patient experience surveys nor for HCAHPS and CAHPS include safety. Given the recent findings that patients and family members report more medical errors, verified by physician review, compared with healthcare provider-initiated reports (5% of the verified patient-reported errors),16 this represents a missed opportunity to better understand and address the persistent problem of patient safety.

It is also likely that patients with more experience (eg, a patient with congestive heart failure and multiple hospital admissions) have different types of observations than patients with less experience17 (eg, a previously well patient admitted for acute appendicitis or first-time heart attack). The differences in observations may manifest in two ways. Experienced patients and their caregivers, with a larger ‘dataset’ of observations, as well as intimate knowledge of their own disease and situation, may more easily distinguish events that deviate from a pattern. This may make them more likely, for instance, to identify a medication error as well as potentially more likely to note it in a review since they are confidently familiar with their standard medication dosing. In addition, experienced patients and their caregivers, having seen multiple aspects of the healthcare delivery system and often having sought care at more than one system, may more easily identify systems issues. For instance, in a local family advisory council meeting, a parent compared two hospitals where her child had been admitted, remarking that one had better safety equipment to decrease the risk of falls for her child, and that it would make her feel safer if both hospitals had the better equipment.

The implications of potential differences in online reviewers by level of experience are two-fold. Online narratives from more experienced patients may provide system-level observations and savvy advice for otherwise potentially less experienced online consumers. In addition, this heterogeneity of perspectives may partly explain the mixed results regarding the association of online reviews and provider quality. If the mix of experience varies across websites (eg, Facebook has fewer experienced patients leaving reviews compared with Yelp), the websites with more experienced reviewers may have an increased number of more savvy observations that inform and drive the relationship between outcomes and review scores. Whether such a difference in observations between experienced and inexperienced patients exists represents a testable hypothesis for future research. If confirmed, then websites may want to consider amplifying the voices of experienced patients, or more proactively eliciting reviews from them.

In summary, despite mixed evidence as to the validity of online reviews, their influence remains and will likely continue to grow. Thus, we need to better understand how websites with online reviews differ from one another and the strengths and weaknesses of different types of reviewers. Ideally such research would occur in partnership with online review website sponsors. Currently, the vast majority of providers probably ignore online reviews. This option will probably become less tenable in the coming years. Even if it were, though, we will better serve patients if we learn to harness the potential power of these increasingly available sources of patient experience data to improve healthcare delivery systems.

Acknowledgments

The author thanks Astrid Guttmann, Sunitha Kaiser and Eugene Bardach for their insightful comments on earlier drafts of this editorial. They did not receive compensation for their contributions.

References

View Abstract

Footnotes

  • Handling editor Kaveh G Shojania

  • Competing interests None declared.

  • Provenance and peer review Commissioned; internally peer reviewed.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles