Article Text
PDF
Abstract
Introduction Emergent evidence suggests that patients can identify and report safety issues while in hospital. However, little is known about the best method for collecting information from patients about safety concerns. This study presents an exploratory pilot of three mechanisms for collecting data on safety concerns from patients during their hospital stay.
Method Three mechanisms for capturing safety concerns were coproduced with healthcare professionals and patients, before being tested in an exploratory trial using cluster randomisation at the ward level. Nine wards participated, with each mechanism being tested over a 3-month study period. Patients were asked to feed back safety concerns via the mechanism on their ward (interviewing at their bedside, paper-based form or patient safety ‘hotline’). Safety concerns were subjected to a two-stage review process to identify those that would meet the definition of a patient safety incident. Differences between mechanisms on a range of outcomes were analysed using inferential statistics. Safety concerns were thematically analysed to develop reporting categories.
Results 178 patients were recruited. Patients in the face-to-face interviewing condition provided significantly more safety concerns per patient (1.91) compared with the paper-based form (0.92) and the patient safety hotline (0.43). They were also significantly more likely to report one or more concerns, with 64% reporting via the face-to-face mechanism, compared with 41% via the paper-based form and 19% via the patient safety hotline. No mechanism differed significantly in the number of classified patient safety incidents or physician-rated preventability and severity.
Discussion Interviewing at the patient's bedside is likely to be the most effective means of gathering safety concerns from inpatients, potentially providing an opportunity for health services to gather patient feedback about safety from their perspective.
- Incident reporting
- Patient-centred care
- Patient safety
Statistics from Altmetric.com
Introduction
Patient involvement in patient safety has become increasingly important to policy makers, health professionals and researchers alike, with an emerging consensus that patients can be meaningfully involved in their safety in various ways, and across a range of healthcare settings.1 ,2 Furthermore, a series of high profile reports3–5 have served to emphasise the need to elicit and respond to patient feedback about safety. Such reports are focused primarily on the acute care setting where patients do seem to be in a position to report patient safety incidents (PSIs) during periods of acute illness, with various studies collecting information about safety experiences from patients receiving care in hospital.1 ,6
To date, most published work has captured patient safety reports retrospectively, using survey methods and usually after discharge.1 Some authors have commented on the potential of such retrospective reporting to introduce ‘recall bias’ in the reporting process,7 ,8 although the exact nature of the biases affecting patients in their reporting of safety-related information either close to an event or after a period of time remains unclear. However, it is clear that, to date, ‘hot reporting’, that is, capturing patient-reported safety concerns as close as possible to the event, has not been explored.
A further issue in previous studies of patient reporting has been the structured approach to collecting patient data, usually based on a checklist approach of known categories of patient safety events.1 While this is clearly a useful way of collecting safety information, it limits patient responses to the range provided by the content of the checklist. However, given the patient's key position as observer of their entire care trajectory,9 it is intuitive to think that they might be able to report on a variety of safety issues that might not otherwise be obvious or available to ward staff, or fit within the normal ‘clinical risk paradigm’.2 Furthermore, authors have already commented on the broader definition that patients have of ‘safety events’ or ‘patient safety incidents’ in comparison with many clinical staff.7 Therefore, it would seem plausible that patients may already be able to provide healthcare organisations with information that does not fit within the existing definitions of ‘medical error’ or ‘adverse events’, but may prove useful for organisational learning. On this basis, it is perhaps important that tools for capturing patient-reported safety concerns are not limited to issues viewed through a strictly clinical patient safety lens.
Another feature of the extant literature on patient involvement in safety reporting is the lack of patient involvement in the development of research materials or data capture tools. However, it has been noted that to be successful there should be a ‘collective responsibility’ for the development of patient safety systems, which position patients as central to the process alongside clinicians.10
Recent work has considered patients’ willingness to report safety experiences using different methods of data capture, concluding that patients report a greater willingness to speak to researchers (compared with reporting to a local or national reporting system).11 However, this study was limited to patients’ intentions, rather than documenting their actual behaviour using different methods of capturing patient reports. Given the known low correlations between intentions and behaviour,12 it is important for researchers to move beyond simply asking patients what they may do, and observe what they actually do under real-life conditions. On this basis, perhaps studies should not ask ‘can patients report?’, but ‘can patients report in a system designed to collect this information routinely in a clinical setting?’
This paper presents a novel study designed to meet some of these identified gaps in knowledge, by examining the efficacy of three different (patient-derived) mechanisms in capturing safety concerns from patients within an acute hospital setting, across a variety of specialities. Specifically, we set out to investigate the following research questions:
Can patients relay their safety concerns using a variety of different mechanisms for capturing such data?
Which mechanism might be the most successful in collecting patient-reported safety concerns?
Are there differences in the types of patient-reported safety concerns captured by the different mechanisms, in terms of physician-rated classification as a PSI, or the associated degree of preventability or severity?
Method
Study design
The study presented here was an exploratory pilot using the principles of cluster randomisation.
Setting
This study was undertaken in a large acute teaching hospital based in the north of England, UK, between 1 March and 31 May 2011.
Sample
Patients were recruited from the participating hospital trust. Nine wards from across the hospital agreed to participate. Wards were purposively sampled to achieve a balance across the following specialties: surgery, medicine and cancer services (to represent day-case patients). Recruited wards were: (i) vascular surgery; (ii) urological surgery; (iii) general surgery; (iv) renal, gastroenterology and rheumatology; (v) respiratory medicine, diabetes and endocrinology; (vi) cardiology; (vii) haematology (outpatients only); and (viii) and (ix) two chemotherapy units (outpatients only).
Development of mechanisms for capturing patient reports, terminology used and questions to elicit patient-reported safety concerns
The approach taken for developing ideas for the three mechanisms to capture patient reports, as well as the terminology used to describe patient-reported safety events and the questions to elicit this information from patients, was embedded in the principles of ‘co-design’,13 where contributions from patients, healthcare professionals and researchers were of equal value. The methods for this development (which was part of a wider programme of research) have been outlined previously,14 but are summarised in figure 1.
Process of developing terminology, questions and mechanisms for eliciting and capturing patient-reported safety concerns.
Development proceeded with a series of focus groups with patient representatives and healthcare professionals, within which the following questions were considered and consensus achieved: (i) the terminology for capturing information from patients about specific safety incidents; (ii) the questions required to elicit information from patients about specific safety events; and (iii) three potential mechanisms for capturing this information from patients in a hospital setting. Following the focus groups, a smaller working group was established (composed of two patient representatives, two researchers (JKOH, GA) and two senior healthcare professionals) with a remit to ratify the suggested questions for eliciting information from patients, and undertake further development of the mechanisms into workable systems. The working group met three times between September and December 2010.
Terminology used
In response to the acknowledged criticism of previous work in this area (that terminology for patient safety-related events or incidents is not standardised),1 ,6 it was agreed that one aspect of the working group's remit would be to agree on the terminology for describing what we wanted participants to report. There was consensus from the healthcare professionals and patient representatives that PSIs would not be understood by patients, and that it may also lead to only harm events being reported, at the expense of events that had the potential for harm, that is, ‘near misses’ or ‘close calls’. It was agreed therefore that the term used to describe patient-reported safety events to patients would be ‘safety concerns and experiences’, as it was felt by the group that this was inclusive of the potential for harm, and specifically focused on safety.
Questions to elicit information from patients
It was agreed that open rather than closed questions would be used, due to the potential for closed questions to limit the range of patient responses, a previously identified criticism of this body of work.1 The design of the questions was also informed by a review of previous studies where patients were asked to report safety events or concerns.1 ,6 Questions were designed to elicit information about a participant's safety concern or experience in terms of (i) where it happened; (ii) who was involved; and (iii) what happened. These questions are provided in detail in the online supplementary appendix 1.
Mechanisms for capturing patient-reported safety concerns
The three mechanisms that were put forward by the focus groups were: (i) face-to-face interviewing by researchers, (ii) a paper-based form and (iii) a ‘patient safety hotline’—a centralised phone line which could be called by patients from their ward or on their mobile telephone. The working group was asked to take these ideas and produce three workable ‘tools’ that could be tested within a hospital setting, but that crucially would be a fair representation of the ideas that emerged from the focus groups. Final versions of the three tools were ratified by members of the working group before the study commenced, with all accompanying patient information reviewed by the patient representatives for accuracy and comprehension. Figure 1 provides detailed information on the three developed mechanisms.
Procedure
Randomisation of wards
After securing agreement from the nine participating wards within the hospital, wards were assigned to a condition (mechanism) using a random number generator.15 Randomisation at the level of the individual participant (rather than cluster randomisation) was felt to be inappropriate due to (i) the prohibitive pragmatic considerations of installing the infrastructure required for the mechanisms (phone lines and drop-boxes) within each ward, (ii) the possibility of contamination across participating patients and (iii) the potential confusion arising from multiple mechanisms available on each ward. Randomisation was stratified to ensure that each of these specialties was represented within each of the three conditions (mechanisms).
Recruitment of participants
Recruitment followed the same process across all nine wards. All wards were visited by one or more researcher or research nurses at least three times each week across the 3-month study period. At each visit, the researcher or research nurse would first speak to the ward clerk, or staff at the central nurses’ station on the ward, to ascertain which patients were not approachable on the basis of illness acuity or infection control. Patients were also not routinely approached if they were disorientated or confused, asleep or had visitors at their bedside. When patients were approached, they were first given a brief description of the study and then left with a concise, patient-friendly leaflet summarising the study and what participation involved. After a ‘cooling off’ period of at least 24 h, patients who had not been discharged were followed up to ascertain if they wished to participate and given the opportunity to ask questions. Outpatients were given the same opportunity for a ‘cooling off’ period as inpatients. Following the initial approach to patients on outpatient units, if interest in participating was indicated, a further meeting (to take consent) was planned based on when they were next attending the unit. In some cases the time between their outpatient visits was considerable, and this meant that some of the interested patients were then unable to participate within the study period. All patients wishing to participate were provided with a participant pack providing information about the mechanism that they would be using to report. They were also asked to sign a consent form indicating their agreement to participate, and were asked some basic demographic information relating to them and their current hospital stay (age, gender, ethnicity, first language, date of admission). In addition, patients were asked to indicate whether they had been an inpatient in the hospital over the previous 5 years, and whether they were receiving on-going hospital treatment. From these two questions a ‘combined prior experience’ score was created, with the response range of 0–2, where higher scores indicated a greater amount of prior hospital experience. Participant information packs were also made available in Urdu, and non-English-speaking patients were approached by researchers who could speak Mir-Puri and Urdu.
Analysis
Categorising the content of patient-reported safety concerns
The first step was to explore the nature of patient-reported safety concerns and organise these data thematically. To do this, categories were developed using an inductive and iterative process of thematic analysis.16 This process involved two research nurses and one researcher (CC, LT, JKOH) first individually identifying the emergent themes from the patient reports, before meeting to achieve consensus. In order to understand the spread of the patient-reported safety concerns across the categories generated, each report was then ‘back-coded’ into one of the categories, first individually by three members of the research team, with consensus then reached on discussion as to the final code for each report.
Classification of patient reports
All patient-reported safety concerns were reviewed using a two-stage process. First, reports were reviewed by two research nurses independently, to ascertain how many met the definition of a PSI. The definition used was that adopted by National Health Service (NHS) England: ‘any unintended or unexpected incident which could have or did lead to harm for one or more patients receiving NHS care’.17 All patient reports that were judged to meet this definition were then reviewed by two physicians using a categorisation process adapted from previous studies.18 ,19 First, the reviewing physicians independently assessed each report as to whether it met the definition of a PSI. Those reports categorised as a PSI were then further rated against the standard risk indices of (i) preventability (using a 4-point scale: 1=‘definitely not preventable’ through to 4=‘definitely preventable’;19 and (ii) severity (using a 5-point scale: 1=‘negligible’ through to 5=‘catastrophic’). Severity was rated as the actual rather than the potential severity of the PSI, and preventability was concerned with the event rather than associated harm. Physician review was undertaken across two rounds. The first was an independent review, with a second review undertaken collectively to discuss whenever there were disagreements in scores. Following the second collective review, the inter-rater agreement was 85% for categorisation of a PSI. Of those categorised as a PSI, the inter-rater agreement for the other ratings undertaken was (i) 77% for the likelihood of preventability and (ii) 75% for the degree of severity. Only those reports that were jointly agreed on as meeting the definition of a PSI as part of the collective second round physician review were taken forward into the analyses.
Statistical analysis
Quantitative data were analysed using a number of different methods. The difference between conditions (mechanisms) in the number of reports per patient was explored using an analysis of covariance (ANCOVA) controlling for age, gender, duration of hospital stay and the combined degree of prior experience score. Significant between-condition effects were explored post-hoc using the Bonferroni Test. Two χ2 analyses were conducted to explore differences between conditions in the two categorical dependent variables, namely: (i) likelihood of reporting (ie, did the patient report or not?) and (ii) number of patient reports classified as a PSI (yes or no). Two Kruskal–Wallis analyses were conducted to explore differences between conditions in the two ordinal dependent variables, namely: (i) the degree of preventability in reports classified as a PSI (four response options); and (ii) the degree of severity in reports classified as a PSI (five response options).
This study was approved by an NHS Research Ethics Committee.
Results
Sample
A total of 432 patients were approached and provided with an information leaflet about the study. Of these, 178 patients agreed to participate. The overall response rate was 41%, although rates varied across mechanisms (face-to-face interviewing, 44%; paper-based form, 41%; patient safety hotline, 37%). The difference in response rates between mechanisms was non-significant in a chi-square test (χ2 (2) = 1.170, p=0.56). The sample demographics are presented in table 1.
Sample demographics
Number of reports per patient
There was a significant effect of condition (mechanism) on number of reports per patient, after controlling for the effects of the covariates F(2, 154)=8.292, p<0.01. Post-hoc tests showed that patients in the face-to-face interviewing condition provided significantly higher numbers of reports compared with those in the paper-based form condition (t(154)=−3.064, p<0.01) and those in the patient safety hotline condition (t(154)=−3.781, p<0.01). The mean number of reports per patient across the three mechanisms are displayed in table 2.
Summary scores for number of reported safety concerns per patient, likelihood of reporting, number of classified PSIs, and preventability and severity ratings
Likelihood of reporting
There was a significant association between the condition (type of reporting mechanism) and whether a patient-reported one or more safety concerns: χ² (2)=19.626, p<0.01. Of the patients in the face-to-face interviewing condition, 64% reported one or more safety concerns, compared with 41% in the paper-based form condition and 19% in the patient safety hotline condition. Overall, nearly half (49%) of all patients reporting a safety concern were in the face-to-face condition.
Number of patient reports classified as a PSI
No association was found between condition and the number of reports classified as a PSI. Table 2 displays the number of PSIs by type of reporting mechanism.
Preventability and severity of patient reports classified as a PSI
A summary of the preventability and severity ratings by condition (and as an associated percentage of the total number of reported safety concerns) are detailed within table 2. Due to the low numbers of classified PSIs in the patient safety hotline condition, it was decided to drop this from the analyses, and compare the face-to-face interviewing and paper-based form conditions using only two Mann–Whitney U analyses for preventability and severity ratings. No significant differences were found between the conditions for either the preventability or severity ratings of the classified PSIs.
Nature of patient-reported safety concerns
The categorisation of the content of the patient-reported safety concerns is summarised in table 3. To further illustrate these, table 4 provides typical examples of the patient reports from each of the five overarching categories.
Categorisation of patient-reported safety concerns by number of reports and classified PSIs
Example patient safety reported concerns
Discussion
This study set out to test three mechanisms for capturing patient-reported safety concerns. It is clear that patients can report safety concerns across a variety of mechanisms, but that the most successful means of collecting patient-reported safety concerns is through facilitated discussion at the patient's bedside. However, it would appear that no mechanism differs significantly in terms of either the number of classified PSIs collected or their assessments of preventability. Our findings also suggest that across all mechanisms, classified PSIs were judged to be ‘probably’ or ‘definitely’ preventable, and of low severity.
This study is the first of its kind to address a fundamental question for many patient engagement activities: how should we collect information pertaining to quality and safety from patients? Therefore, our findings advance the literature by suggesting that embedding questions within a conversation with patients—held by someone external to the ward and during their stay in hospital—is likely to provide health services with the most information on patients’ safety concerns. Earlier work had begun to explore the likelihood of patients using various mechanisms for eliciting safety information,11 but this is the first known study to ask patients to actively use different mechanisms within a hospital ward. In an international context of prioritising patient engagement, it will be increasingly useful for organisations to consider our findings when making decisions about how to seek direct feedback from patients about their safety concerns and experiences.
The findings also raise a number of interesting issues. First, patients seem to be able to report on a wide range of patient safety issues. While this result is in keeping with previous work in this area,6 we believe this is notable given that we provided little structure for patients about what safety events they might report, which was a departure from the approach taken in the previous literature.1 It is also of note given the importance placed on patient engagement as part of managing quality and safety, within recent UK policy documents.3–5 It has been argued that patients might usefully be regarded as ‘smoke alarms’ for healthcare organisations,20 with ‘their reports of safety issues…the best early warning system we have for detecting the point at which poor care deteriorates into care that is clearly dangerous’ (p.16).21 With this in mind, it will be important for those seeking feedback to resist the temptation to restrict patients to a defined set of response formats deemed important from a clinical perspective, and to take the rather more patient-centred approach of letting them report concerns about safety from their unique perspective. An unintended consequence of such patient-centredness is, however, that a healthcare provider could receive a wide range of concerns from patients that cross traditional boundaries between comfort, hospitality, quality of care and patient safety, the latter being perceived by some patients as incorporating their personal safety too. An aim of achieving consensus between patients and clinical staff regarding what constitutes a patient safety concern (or indeed a PSI) is perhaps unrealistic. Rather, the emphasis might be placed on the recognition that patients’ safety concerns can reveal the antecedents to potential or actual harm events: for example, a shower door failing to close will likely mean a wet floor outside the shower which could lead to a PSI such as a fall. Similarly, an elderly patient who is unsettled by other patients’ antisocial behaviour and cannot sleep as a result might have an increased chance of a number of adverse clinical outcomes (eg, falls, increased length of stay) owing to resultant fatigue.22 ,23
Related to this, the second point is that there was no significant difference in the number of classified PSIs across the three mechanisms. However, given the lack of structure in the reporting format, it is perhaps unsurprising that we find a range of harm and non-harm events across all mechanisms. In essence, across all mechanisms, patients were central to deciding what to report on the basis of what was important to them. It might be argued that the conversations at the patients’ bedside were simply better at drawing out greater volumes of safety concerns.
A third issue raised by the findings concerns the timing of seeking feedback from patients. Much of the literature to date has collected information from patients at or after discharge, with little attention paid to the possibility of bias in the recollection of events that such a delay introduces.1 In testing mechanisms for asking patients for feedback about care during their hospital stay, we were in effect testing the possibility that patients can ‘hot report’, that is, spontaneously report as and when they experience safety concerns. Therefore, one interpretation of our findings might be that the prospect of true ‘hot reporting’ is perhaps less realistic for health services. The reality of the face-to-face discussions was that most of them happened at the point of consent, with few participants asking for a further interview. This means that while this study provides a potential tool for systematically collecting patient-reported safety concerns, and provides the first known empirical evidence of data collected only within an acute care setting, it still falls short of true ‘hot reporting’ where patients provide information as close to experiencing a safety concern as possible. The reality is that little is known empirically about the ‘optimum’ point at which patients should be asked for their feedback about quality and safety, and the likely biases that may distort this information. It is now clear that further research is needed to answer these important questions, which will inform a wide variety of patient engagement activity across health services globally.
There are a number of strengths to this work. First, the study design meant that we tested each mechanism across different hospital settings and patient groups, thus allowing us to more confidently generalise our findings outside of the study population. Second, our approach to developing the ideas and structures for the mechanisms as a collaboration between patient representatives, healthcare professionals and researchers meant that the final versions were embedded within, and reflective of, the needs and constraints of each of these stakeholders. Third, the mechanisms were actively implemented within each of the participating wards for the study period, and as such our outcomes are actual behaviours, as opposed to the likelihood or willingness to engage with a potential mechanism. This is an important distinction given the known low correlations between intentions to undertake a behaviour and actually undertaking the behaviour.12
Our work has a number of limitations. This study was carried out in one acute teaching hospital in England. Nevertheless, our sample of wards included a range of specialties covering medicine and surgery, as well as outpatient facilities. Second, patients were excluded from participating in the study if the ward team representative believed factors such as their level of illness and level of mental capacity militated against their participation. This may have resulted in patients who are especially vulnerable to PSIs due to the complexity of their care,24 and their relative inability to communicate,25 not being given the opportunity to comment on their care. However, we argue that approaching patients who were judged to be especially ill or lacking the capacity to feedback their patient safety concerns may have placed an inappropriate physical or indeed mental burden on them; this position was also taken by the research ethics committee that approved this study. Our subsequent, commissioned work in this area now includes the option for patients’ families or carers to speak on their behalf if the patient is incapacitated in any way. Third, patient concerns excluded by nurses as incidents at stage 1 may have been included by medical staff, given the opportunity. Although we aligned our classification of PSIs with the commonly applied method of a two-stage classification process reported elsewhere,26 and used previously in the patient reporting literature,27 the reported process was based on incidents identified through case note review and not patient reports. In the future, it may be valuable to further explore nurses’ and doctors’ decisions to exclude or include patient-reported concerns as incidents and any interprofessional differences. Moreover, the categorisation of patient-reported safety concerns has not generated mutually exclusive categories and although valuable in revealing the patient perspective, it should be judged as formative. If patient reporting schemes become more commonplace and the volume of data increases, it is likely that the ease of classification and categorisation will also improve. Finally, as this was an exploratory pilot, we did not collect data on the resources required for each method. Future studies should attempt to ascertain the costs associated with collecting safety concerns from patients to undertake a costs–consequences analysis. Finally, the study hospital has one of the highest percentages of hospital admissions from a non-white, non-British population, reflecting the local demography. We only recruited 11% of this population and therefore the findings may not be representative of all sectors of the local patient population. Additional work in this area should consider the option of using a broad range of specially trained multilingual researchers to collect such data or alternatively creating patient-facing information technologies which facilitate the recruitment of such patients.
Conclusion
Previously published work in this area has been based on patients’ retrospective reports of safety-related information; the information in these reports having typically been generated by structured surveys. Our work represents a departure from this approach. Mindful of the tendency to locate PSI definitions in the clinical risk paradigm, we gave patients licence to define safety on their terms. We now know that hospitalised patients can report their concerns using a variety of mechanisms, but facilitated discussion at the bedside appears to be the preferred and most fruitful option. This approach potentially provides health services with a systematic method through which patients can meaningfully contribute to the range of safety intelligence gathered, reflecting the rapidly emerging policy focus within the UK and beyond.
Acknowledgments
We would like to thank Dr Vicky Brown and Dr Michael McCooe for their help in reviewing and classifying the patient reported safety concerns. We also acknowledge the support of the Yorkshire Quality and Safety Research Group and the unique contribution of the patient panel at both the Bradford Institute for Health Research and Newcastle University.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
- Data supplement 1 - Online supplement
Footnotes
Twitter Follow Jane O'Hara at @janekohara
Contributors JOH drafted the manuscript. JOH and GA were responsible for the design and conduct of the study. CR managed the process of review for patient-reported safety concerns and the dataset as a whole. JW and IW developed the protocol and had an overall view of the project. CR, CC, LT and ID were involved in the conduct of the study and recruited patients into the study. All authors reviewed and edited the manuscript and agreed on the final draft.
Funding This paper presents independent research funded by the National Institute for Health Research (NIHR) under its Programme Grants for Applied Research Programme (RP-PG-0108-10049). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.
Competing interests None declared.
Ethics approval NHS REC.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement We are happy to share anonymised data.