Social media, healthcare quality and learning from other industries

Scandals such as the serious failures in standards of care that came to light at Stafford Hospital in the English National Health Service1 show that a new approach is needed to identify services that are delivering unacceptably poor service. The subsequent enquiry into that hospital found that the problems were mostly discovered through post hoc analysis of mortality, process and patient experience data. The advent of social media and new technology potentially opens a door to insights into care (both positive and negative) unfiltered by traditional methods of healthcare data capture and analysis. For the first time, the voice of the patient may be heard with clarity and immediacy.

Patients already post information about the quality of healthcare on the internet, whether on Twitter, blogs, rating websites or other social media. We describe this growing body of information on the internet as a ‘cloud of patient experience’. In a world where sharing experience of life and life-events has brought a new dimension to communication and networking and an inherent intolerance of paternalism and secrecy in governments and institutions that serve the public, we argue that capturing patients’ opinions of their healthcare on the internet in real time could act as an early warning of poor clinical care and over time transform the relationship between care provider and recipients of their services. Traditionally, patients’ experience of hospitals has been measured by annual, often paper based, surveys. Data are available infrequently and costly to collect. Capture, automated analysis and aggregation of social media content could happen on a daily basis, at low cost and could provide a tool for continuous service monitoring.

Outside the health sphere, information from social media is being used for a wide variety of analytical and signal detection tasks, sometimes characterised as part of a ‘Big Data’ revolution.2 Recent examples include attempts at predicting election results,3 the next Hollywood blockbuster4 or the closing price of the stock market.5 In healthcare, social media data are already used but, to date, have been largely limited to infectious disease surveillance. The use of Google search activity to detect spikes of flu activity6 is one example and, more recently, Twitter analysis has helped to monitor disease frequency in cholera and other disease outbreaks.7 ,8 This use of digital information to answer health problems has been called ‘infodemiology’ and ‘infoveillance’.9 There has been little application to quality and patient safety so far but the potential is much greater given the huge volume of healthcare experiences in settings that remain stable for often decades at a time.

More and more people are using the internet as a platform to describe their care in both the USA and UK.10 ,11 In the USA, the public is increasingly willing to engage with their healthcare online. Of the 85% of adults using the internet,12 48% looked at social networking sites daily,12 34% had read about someone else's health experience on a website and 15% had consulted online reviews of medical facilities.13 Previous studies have found significant associations between ratings left on websites and clinical outcomes, with better rated hospitals having lower mortality and healthcare associated infection rates.14 ,15 Early research has demonstrated that sentiment analysis of content in patients’ comments about their care on the internet is feasible.16 ,17 Sentiment analysis involves taking unstructured, often free-text information, and using software to judge whether the information is broadly positive or negative. Alemi and colleagues have called for real time patient satisfaction surveys using these analyses.16 Cambria and others believe that individual experiences could be ‘crowd validated’ by aggregating various sources of unstructured information from patients online.18

The use of social media data to detect the spike of a definite event such as a disease epidemic has been demonstrated across different contexts and shown to be possible. The use of this source of data to track and interpret the subjective nature of patient experience is a more complex task, and has not yet been explored in detail. However, the growing cloud of patient experience, combined with the promise of sentiment analysis techniques developed outside healthcare, presents an exciting opportunity to understand better the quality of health organisations and systems. The most immediate and feasible application of this approach would be in flagging poor performance. There is a parallel with the use of surveillance systems for infectious diseases to detect an ‘outbreak’ or ‘epidemic’ threshold. The consistent appearance of concerns about a hospital captured through social media, when compared with similar hospitals, could indicate problems that official statistics were not yet picking up or indeed were not capable of even recognising. Of course, as with any quality signal, or trigger tool, there would have to be rigorous protocols for further investigation and action that could distinguish the real from the spurious.

How to capture and use the ‘cloud of patient experience’

To build up this social, or soft, intelligence, two technically different and complex tasks are necessary: harvesting data and then processing it into useful information. The harvesting part involves collecting the free text from as many open sources of comment about healthcare providers as possible: social networks, Twitter, discussion fora and rating websites. Anywhere where people talk about their experience of care online is a potential seam of information to mine. The process would involve identification of appropriate websites, and then pulling relevant information off them on a regular, automated basis using specialised software: an information extraction process known in computer science as ‘scraping’.19

In the data processing part, the patients’ free text descriptions of their care would be converted into a social intelligence dataset by analysing the collected statements for sentiment and reliability, transforming them into an aggregated, quantitative measure of experience for each provider. This applies algorithmic processes (natural language processing) to extract useful information from the data retrieved. Natural language processing has been used in other patient safety contexts, for example, in the automated detection of postoperative complications in an electronic medical record.20 Key themes searched for might be cleanliness or emotions such as anger, joy or sadness. The resulting data could then be used to rate specific aspects of care at each hospital, such as the hospital environment and the quality of their interactions with staff. This information, taken together with traditional patient surveys, could then be used by health system regulators to identify poor performance, when a certain warning threshold is crossed, and by managers and clinicians to address areas for improvement.

Limitations

There are substantial technical and logistic problems in any sensing system. There is potential for selection bias as patients who choose to talk about their care online may not be representative of those attending healthcare facilities. Use of rating websites and Twitter tends to be associated with higher socio-economic status and younger age groups.21 ,22 Hospitals serving certain populations may receive less attention on social media than others. Some sources of data lend themselves better to automatic processing than others. The comments left on ratings websites, and those posted on patient discussion fora, are often rich in material directly relevant to their healthcare, but tend to appear infrequently. By contrast, Tweets are necessarily brief, containing less contextual information about healthcare and have their own grammar and syntax. These characteristics present real challenges for processing but the growing popularity of Twitter means that it should be seen as an opportunity for healthcare, not a threat. New methods will need to be developed to capture information on healthcare from this source. The autocatalytic quality of some social media, where ideas are repeated on the basis of their popularity, reinforcing views, rather than providing new information, is another trap for the unwary in interpreting findings. We have listed potential data sources and their relative merits and drawbacks in table 1.

The analytic component is also a technical challenge: machines struggle to read and understand comments accurately; software finds comments preceded by negatives difficult to interpret. Sarcasm and irony, a feature of the British and US cultures, are almost impossible to process. Even if the sentiment within messages could be collected and interpreted perfectly, it is inevitable that there will be some gaming of the system, involving either inflating a provider's own service or denigrating a rival's. There is also a risk that any system like this will throw up false positives. Precautions to prevent wrongful reputational damage will be important. The threshold for an alert may need to be set high initially, or hospitals with sporadic ‘chatter’ about them online may come under the spotlight inappropriately. Experience with methodologies should allow the approach to such matters to be clarified later. Ultimately the challenge will be to pick out signals from the noise.

Testing the hypothesis

Will it be possible to detect institutional poor performance via social media in a valid and consistent way? A simple approach to test the initial validity of the data would be to compare the online cloud of comments, after collection and processing, with conventional measures of patient experience such as routine paper based surveys. Comparison between structured patient rating data online and patient surveys has already shown significant associations at the hospital level.15 ,23 A more sophisticated method to examine the potential as an early warning system would be to identify, through traditional means, future cases of poorly performing hospitals, and collect corresponding data on how patients described these institutions on the internet. The capacity of these modern sources of data as predictors of poor quality care could then be calculated, including assessing the sensitivity and specificity of the alert process. In addition, it might be possible to compare the timings at which the various signals of poor quality start to emerge to see if this approach might give a head start against conventional measures such as death rates. Retrospective analysis of previous care quality failures is not feasible, as use of social media is so new. However, an analysis of the next set of health organisations to be hit by scandal might prove a natural experiment.