Article Text

Download PDFPDF

Harnessing the cloud of patient experience: using social media to detect poor quality healthcare
  1. Felix Greaves1,2,
  2. Daniel Ramirez-Cano2,
  3. Christopher Millett1,
  4. Ara Darzi2,
  5. Liam Donaldson2
  1. 1Department of Primary Care and Public Health, Imperial College London, London, UK
  2. 2Centre for Health Policy, Imperial College London, London, UK
  1. Correspondence to Dr Felix Greaves, Department of Primary Care and Public Health, Imperial College London, Reynolds Building, Charing Cross Campus, Imperial College London, London W6 8RP, UK;felix.greaves08{at}imperial.ac.uk

Abstract

Recent years have seen increasing interest in patient-centred care and calls to focus on improving the patient experience. At the same time, a growing number of patients are using the internet to describe their experiences of healthcare. We believe the increasing availability of patients’ accounts of their care on blogs, social networks, Twitter and hospital review sites presents an intriguing opportunity to advance the patient-centred care agenda and provide novel quality of care data. We describe this concept as a ‘cloud of patient experience’. In this commentary, we outline the ways in which the collection and aggregation of patients’ descriptions of their experiences on the internet could be used to detect poor clinical care. Over time, such an approach could also identify excellence and allow it to be built on. We suggest using the techniques of natural language processing and sentiment analysis to transform unstructured descriptions of patient experience on the internet into usable measures of healthcare performance. We consider the various sources of information that could be used, the limitations of the approach and discuss whether these new techniques could detect poor performance before conventional measures of healthcare quality.

  • Patient satisfaction
  • Patient-centred care
  • Quality measurement
  • Information technology
View Full Text

Statistics from Altmetric.com

Social media, healthcare quality and learning from other industries

Scandals such as the serious failures in standards of care that came to light at Stafford Hospital in the English National Health Service1 show that a new approach is needed to identify services that are delivering unacceptably poor service. The subsequent enquiry into that hospital found that the problems were mostly discovered through post hoc analysis of mortality, process and patient experience data. The advent of social media and new technology potentially opens a door to insights into care (both positive and negative) unfiltered by traditional methods of healthcare data capture and analysis. For the first time, the voice of the patient may be heard with clarity and immediacy.

Patients already post information about the quality of healthcare on the internet, whether on Twitter, blogs, rating websites or other social media. We describe this growing body of information on the internet as a ‘cloud of patient experience’. In a world where sharing experience of life and life-events has brought a new dimension to communication and networking and an inherent intolerance of paternalism and secrecy in governments and institutions that serve the public, we argue that capturing patients’ opinions of their healthcare on the internet in real time could act as an early warning of poor clinical care and over time transform the relationship between care provider and recipients of their services. Traditionally, patients’ experience of hospitals has been measured by annual, often paper based, surveys. Data are available infrequently and costly to collect. Capture, automated analysis and aggregation of social media content could happen on a daily basis, at low cost and could provide a tool for continuous service monitoring.

Outside the health sphere, information from social media is being used for a wide variety of analytical and signal detection tasks, sometimes characterised as part of a ‘Big Data’ revolution.2 Recent examples include attempts at predicting election results,3 the next Hollywood blockbuster4 or the closing price of the stock market.5 In healthcare, social media data are already used but, to date, have been largely limited to infectious disease surveillance. The use of Google search activity to detect spikes of flu activity6 is one example and, more recently, Twitter analysis has helped to monitor disease frequency in cholera and other disease outbreaks.7 ,8 This use of digital information to answer health problems has been called ‘infodemiology’ and ‘infoveillance’.9 There has been little application to quality and patient safety so far but the potential is much greater given the huge volume of healthcare experiences in settings that remain stable for often decades at a time.

More and more people are using the internet as a platform to describe their care in both the USA and UK.10 ,11 In the USA, the public is increasingly willing to engage with their healthcare online. Of the 85% of adults using the internet,12 48% looked at social networking sites daily,12 34% had read about someone else's health experience on a website and 15% had consulted online reviews of medical facilities.13 Previous studies have found significant associations between ratings left on websites and clinical outcomes, with better rated hospitals having lower mortality and healthcare associated infection rates.14 ,15 Early research has demonstrated that sentiment analysis of content in patients’ comments about their care on the internet is feasible.16 ,17 Sentiment analysis involves taking unstructured, often free-text information, and using software to judge whether the information is broadly positive or negative. Alemi and colleagues have called for real time patient satisfaction surveys using these analyses.16 Cambria and others believe that individual experiences could be ‘crowd validated’ by aggregating various sources of unstructured information from patients online.18

The use of social media data to detect the spike of a definite event such as a disease epidemic has been demonstrated across different contexts and shown to be possible. The use of this source of data to track and interpret the subjective nature of patient experience is a more complex task, and has not yet been explored in detail. However, the growing cloud of patient experience, combined with the promise of sentiment analysis techniques developed outside healthcare, presents an exciting opportunity to understand better the quality of health organisations and systems. The most immediate and feasible application of this approach would be in flagging poor performance. There is a parallel with the use of surveillance systems for infectious diseases to detect an ‘outbreak’ or ‘epidemic’ threshold. The consistent appearance of concerns about a hospital captured through social media, when compared with similar hospitals, could indicate problems that official statistics were not yet picking up or indeed were not capable of even recognising. Of course, as with any quality signal, or trigger tool, there would have to be rigorous protocols for further investigation and action that could distinguish the real from the spurious.

How to capture and use the ‘cloud of patient experience’

To build up this social, or soft, intelligence, two technically different and complex tasks are necessary: harvesting data and then processing it into useful information. The harvesting part involves collecting the free text from as many open sources of comment about healthcare providers as possible: social networks, Twitter, discussion fora and rating websites. Anywhere where people talk about their experience of care online is a potential seam of information to mine. The process would involve identification of appropriate websites, and then pulling relevant information off them on a regular, automated basis using specialised software: an information extraction process known in computer science as ‘scraping’.19

In the data processing part, the patients’ free text descriptions of their care would be converted into a social intelligence dataset by analysing the collected statements for sentiment and reliability, transforming them into an aggregated, quantitative measure of experience for each provider. This applies algorithmic processes (natural language processing) to extract useful information from the data retrieved. Natural language processing has been used in other patient safety contexts, for example, in the automated detection of postoperative complications in an electronic medical record.20 Key themes searched for might be cleanliness or emotions such as anger, joy or sadness. The resulting data could then be used to rate specific aspects of care at each hospital, such as the hospital environment and the quality of their interactions with staff. This information, taken together with traditional patient surveys, could then be used by health system regulators to identify poor performance, when a certain warning threshold is crossed, and by managers and clinicians to address areas for improvement.

Limitations

There are substantial technical and logistic problems in any sensing system. There is potential for selection bias as patients who choose to talk about their care online may not be representative of those attending healthcare facilities. Use of rating websites and Twitter tends to be associated with higher socio-economic status and younger age groups.21 ,22 Hospitals serving certain populations may receive less attention on social media than others. Some sources of data lend themselves better to automatic processing than others. The comments left on ratings websites, and those posted on patient discussion fora, are often rich in material directly relevant to their healthcare, but tend to appear infrequently. By contrast, Tweets are necessarily brief, containing less contextual information about healthcare and have their own grammar and syntax. These characteristics present real challenges for processing but the growing popularity of Twitter means that it should be seen as an opportunity for healthcare, not a threat. New methods will need to be developed to capture information on healthcare from this source. The autocatalytic quality of some social media, where ideas are repeated on the basis of their popularity, reinforcing views, rather than providing new information, is another trap for the unwary in interpreting findings. We have listed potential data sources and their relative merits and drawbacks in table 1.

Table 1

Potential sources of information for the ‘cloud of patient experience’

The analytic component is also a technical challenge: machines struggle to read and understand comments accurately; software finds comments preceded by negatives difficult to interpret. Sarcasm and irony, a feature of the British and US cultures, are almost impossible to process. Even if the sentiment within messages could be collected and interpreted perfectly, it is inevitable that there will be some gaming of the system, involving either inflating a provider's own service or denigrating a rival's. There is also a risk that any system like this will throw up false positives. Precautions to prevent wrongful reputational damage will be important. The threshold for an alert may need to be set high initially, or hospitals with sporadic ‘chatter’ about them online may come under the spotlight inappropriately. Experience with methodologies should allow the approach to such matters to be clarified later. Ultimately the challenge will be to pick out signals from the noise.

Testing the hypothesis

Will it be possible to detect institutional poor performance via social media in a valid and consistent way? A simple approach to test the initial validity of the data would be to compare the online cloud of comments, after collection and processing, with conventional measures of patient experience such as routine paper based surveys. Comparison between structured patient rating data online and patient surveys has already shown significant associations at the hospital level.15 ,23 A more sophisticated method to examine the potential as an early warning system would be to identify, through traditional means, future cases of poorly performing hospitals, and collect corresponding data on how patients described these institutions on the internet. The capacity of these modern sources of data as predictors of poor quality care could then be calculated, including assessing the sensitivity and specificity of the alert process. In addition, it might be possible to compare the timings at which the various signals of poor quality start to emerge to see if this approach might give a head start against conventional measures such as death rates. Retrospective analysis of previous care quality failures is not feasible, as use of social media is so new. However, an analysis of the next set of health organisations to be hit by scandal might prove a natural experiment.

Conclusions

We doubt that aggregated real time patient feedback would be a perfect test of clinical performance for an organisation. However, considering the lack of engagement with, and the potential inaccuracy of, the current metrics such as standardised mortality ratios, we believe there is a need to explore alternatives. The soft intelligence provided by this proposed approach—capturing and processing the cloud of patient experience—offers another way to look at health quality; and not just clinical quality but areas such as dignity and respect, cleanliness of the care environment, timeliness and efficiency of care, as well as ideas for improvement. At a time where regulatory organisations are stretched and struggling to complete their basic workload,24 the ability to channel the wisdom of patients in real time to create unique insights into the quality and safety of care, without expensive new infrastructure, is appealing. Furthermore, taken in conjunction with measurement of other metrics of patient outcome, it is possible to imagine a national early warning system that would highlight poorly performing hospitals faster than is done at present. Ultimately, this early warning system could become part of a national, real time monitoring tool for hospital performance. Over time, such a system could also pick up positive messages about excellence that could be built on.

Social media analytics and Big Data are ideas that have been growing in commercial management in recent years. The quality improvement movement in healthcare has often been at its innovative best when adopting ideas from other industries—be they the safety processes of aviation or the rigour and consistency of manufacturing production lines. Perhaps this field presents another opportunity to borrow and adapt the best ideas from other areas and potentially develop a system to make the current wave of quality and regulatory failures less likely in the future.

References

View Abstract

Footnotes

  • Contributors All authors contributed to generating the ideas in this article. FG wrote the draft and all authors edited several iterations including the final version. DR provided information and advice on sentiment analysis and natural language processing.

  • Funding Imperial College London is grateful for support from the National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care scheme, the National Institute for Health Research Biomedical Research Centre Funding scheme, and the Imperial Centre for Patient Safety and Service Quality.

  • Competing interests AD was Parliamentary Under-Secretary of State (Lords) in the UK Department of Health from 2007 to 2009. LD was Chief Medical Officer, England from 1997 to 2010. FG and LD are Twitter users (@felixgreaves, @donaldsonliam). The other authors declare no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous 3 years; and no other relationships or activities that could appear to have influenced the submitted work.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement No data available.

  • Ethical approval Not needed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles