You are here

  • Home
  • Archive
  • Volume 24, Issue 1
  • Patients as teachers: a randomised controlled trial on the use of personal stories of harm to raise awareness of patient safety for doctors in training

Article Text

Article menu
Original research
Patients as teachers: a randomised controlled trial on the use of personal stories of harm to raise awareness of patient safety for doctors in training
Free
  1. Vikram Jha1,
  2. Hannah Buckley2,
  3. Rhian Gabe2,
  4. Mona Kanaan2,
  5. Rebecca Lawton3,
  6. Colin Melville4,
  7. Naomi Quinton3,
  8. Jools Symons5,
  9. Zoe Thompson6,
  10. Ian Watt2,
  11. John Wright3
  1. 1Medical Education and Undergraduate School of Medicine, University of Liverpool, Liverpool, UK
  2. 2Department of Health Sciences, University of York, York, UK
  3. 3Bradford Institute for Health Research, Bradford Teaching Hospitals NHS Foundation Trust, Bradford, UK
  4. 4Warwick Medical School, University of Warwick, Warwick, UK
  5. 5Leeds Institute of Medical Education, University of Leeds, Leeds, UK
  6. 6Leeds Metropolitan University, Leeds, UK
  1. Correspondence to Professor Vikram Jha, Medical Education and Undergraduate School of Medicine, Cedar House, Ashton Street, University of Liverpool, Liverpool L69 3GE, UK; medvj{at}liv.ac.uk

Abstract

Background Patient safety training often provides learners with a health professional's perspective rather than the patient's. Personal narratives of health-related harm allow patients to share their stories with health professionals to influence clinical behaviour by rousing emotions and improving attitudes to safety.

Aim This study measured the impact of patient narratives used to train junior doctors in patient safety.

Methods An open, multi-centre, two-arm, parallel design randomised controlled trial was conducted in the North Yorkshire East Coast Foundation School (NYECFS). The intervention consisted of 1-h-long patient narratives followed by discussion. The control arm received conventional faculty-delivered teaching. The Attitude to Patient Safety Questionnaire (APSQ) and the Positive and Negative Affect Schedule (PANAS) were used to measure the impact of the intervention.

Results 142 trainees received the intervention; 141 the control teaching. There was no evidence of a difference in post-intervention APSQ scores between the groups. There was a statistically significant difference in the underlying distribution of both post PA (positive affect) and post NA (negative affect) scores between the groups on the PANAS (p<0.001) with indications of both higher PA and NA scores in the intervention group.

Conclusions Involving patients with experiences of safety incidents in training has an ideological appeal and seems an obvious choice in designing safety interventions. On the basis of our primary outcome measure, we were unable to demonstrate effectiveness of the intervention in changing general attitudes to safety compared to control. While the intervention may impact on emotional engagement and learning about communication, we remain uncertain whether this will translate into improved behaviours in the clinical context or indeed if there are any negative effects.

Trial registration number Grant reference no. RP-PG-0108-10049.

  • Medical education
  • Patient safety
  • Health professions education

https://doi.org/10.1136/bmjqs-2014-002987

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    There is increasing evidence that training in patient safety improves knowledge and processes of healthcare.1 The recently published Francis and Berwick reports2 ,3 that followed investigations into poor standards of care in a UK hospital emphasised the need to develop patient safety interventions as part of mandatory training for healthcare professionals. A number of these interventions are still new4 ,5 and need further evaluation to inform educators on how to engage students and recent graduates in learning about safety. There is also a tendency to focus on issues such as causes of safety lapses, root-cause analysis of incidents, and the need to promote an organisational patient safety culture.5 This provides learners with a view of patient safety through a health professional rather than a patient ‘lens’ with limited emphasis on the impact of safety lapses on the patient and their families, and little or no involvement of patients in the design or delivery of training. There is a drive to involve patients more explicitly in medical education.6 A systematic review of patient involvement in teaching showed effectiveness in terms of increased learner satisfaction and improved communication skills among healthcare professionals.7 A possible role for such involvement is in patient safety training as part of educational interventions.3–5 ,8 Patient narratives are now widely employed in health professionals’ training, allowing patients to share their own health-related stories with professionals to facilitate the development of clinical knowledge and skills.9 ,10 Patient safety is a particularly appropriate area for narrative-based teaching as it provides an opportunity for patients to share their own real lived experiences of lapses in safety resulting in harm to themselves and/or their families. These real stories, brought to the classroom, allow the exploration of factors causing the error, have the potential to increase awareness of the personal impact of such errors on the patient, and facilitate a wider discussion of safety issues. There is only limited research exploring the use of patient narratives in a safety context. However, preliminary research suggests that this may be a feasible and acceptable method for raising awareness of patient safety concerns among healthcare professionals.8 ,10

    The association between improvements in safety culture (mainly organisational) and climate (mainly staff attitudes and behaviours) and their impact on patient outcomes and staff behaviour is unclear due to the complex nature of this subject and limited empirical research in this area.11 However, attitudes do influence behaviour12 and are increasingly being used as outcome measures for patient safety interventions.13 ,14 In addition, emotions influence behaviour either directly through affect (short-term likes/dislikes) or indirectly through cognitive changes in behaviour.15

    Real stories about patient safety tend to trigger strong emotional responses in the listener16 which, in turn, facilitate greater engagement with17 and better retention of the learning messages.18 An important measure of such an emotional response is the impact narratives have on a humanistic approach to patients including empathy19–21 and how this leads to greater patient-centredness.

    A conceptual framework by Kumagai20 used theories of empathy and moral development to study the use of patient illness narratives in medical education. This work, based on that of Mezirow,22 ,23 utilises the concept of transformative learning where there is a ‘process of effecting change in a frame of reference’.23 These frames of reference include previous experiences, values and feelings, which structure how we make meaning of new learning. Transformative learning encourages ‘critical reflection of assumptions, validating contested beliefs through discourse and taking action on one's reflective insight and critically assessing it’.23 Kumagai's framework is based on the assumption that doctors develop an understanding of the ‘meaning of medicine’ based on the nature of their training and therefore medical education should focus on influencing the way people learn the meaning of medicine. Stories are used to communicate the meaning of individual experiences between people and can therefore be used to shape training of junior doctors. According to Kumagai, patient stories may facilitate the development of empathy in trainee doctors in three ways:

    • Narratives allow doctors to appreciate what it is like to experience an illness and, particularly when delivered face-to–face, may help to develop an ‘interpersonal link’ in affective, cognitive and experiential domains. This helps develop a perspective towards patient experiences which would be difficult in paper-based scenarios where there is no patient interaction.

    • Narratives help communicate meaning by triggering fundamental emotional responses such as loss, anger, jealousy, guilt and sadness. This may arouse a sense of urgency in the learners to explore the causes of the patient’s suffering and highlight moral dilemmas in terms of inequality and social justice.

    • Narratives allow learners to identify better with the patient. If the ideas and beliefs expressed by the patient are not congruent with their own, there may be a dissonance in terms of emotions or cognition. This dissonance stimulates reflection on one’s values and attitudes and stimulates discussions on humanistic and ethical practice and patient care.

    In this study, we adopted the framework suggested by Kumagai to deliberately use emotional stories from patients to enhance the learning experience of trainees24 ,25 and to provide the learners with a greater understanding of safety from the patient's perspective.26 We wanted to explore if facilitating trainees to reflect on the patient stories and on their own experiences could influence their own beliefs, attitudes and intention of future behaviour.

    Aim and objectives

    This National Institute for Health Research (NIHR)-funded study aimed to measure the impact of patient narratives used to train junior doctors in patient safety.

    The primary objective was to measure attitudes towards patient safety using the Attitudes to Patient Safety Questionnaire (APSQ).12 A secondary objective was to measure the short-term emotional response to the patient stories using the PANAS (Positive And Negative Affect Schedule).27

    Methods

    Trial design

    An open, multi-centre, two-arm, parallel design randomised controlled trial (RCT) was conducted in North Yorkshire East Coast Foundation School (NYECFS) using five centres: SBH (Scarborough), HH (Hull), YH (York), GH (Grimsby) and SCH (Scunthorpe). There were 20 days of teaching, organised on 10 days during two consecutive academic years—2011 and 2012 (four at HH, three at YH and one at each of the other sites per year); HH and YH employ a higher number of Foundation Year 1 (FY1) trainees compared to the other sites and therefore require more training days. Within each centre, FY1 trainees in their first year following graduation were individually randomised on a 1:1 basis to the control or intervention arm. In the control arm, participants received standard faculty-delivered teaching on patient safety; in the intervention arm participants received teaching facilitated by patients. The background information and full details of the intervention can be found in the trial protocol28 and are also described briefly below.

    Participants

    All 155 FY1 trainees in 2011 and 158 FY1 trainees in 2012 from NYECFS were eligible to participate. In NYECFS, trainees are allocated to mandatory training days (run between January and March each year) at one of the five centres based on their employing hospitals. The intervention was delivered by the ‘patient group’ consisting of six patients and five carers who had experienced harm during healthcare due to lapses in communication or teamwork, diagnostic errors or poor decision-making. The outcomes of the incidents included psychological or emotional stress and sometimes permanent harm and even deaths.

    Intervention

    The full details of the intervention were published previously.28 ,29 Patient and carers, recruited from the National Patient Safety Agency, Action for Victims of Medical Accidents and advertisements in the local press, were included if they had experience of suffering harm or error to themselves or their families during healthcare. Four preparatory Patient Learning Journey workshops prepared the patient group for the teaching programme.29 The workshops created a confidential supportive environment where the group could comfortably share their experiences, bond with each other, identify key aspects to include within the narrative, and adopt a learner-centred approach.22

    The intervention consisted of two sessions of 1 h each, developed collaboratively with the patient group and delivered in small groups of between seven and 10 trainees. Each session was for 1 h, this included one patient narrative that lasted for approximately 15–18 min followed by facilitated discussion between the patients and the FY1 doctors. The narratives were used to focus on specific issues surrounding the individual patient story as well as more generic issues of safety (see box 1 as example). Emphasis was given to analysis of inadequate care and its causes. Each narrative included a factual description of what happened and reflections about the experience of medical error or inadequate care: what went wrong and why, the impact of the error, and what could be done better. During the discussion (co-facilitated by VJ, JS and the patients), trainees reflected on the narrative, identified emergent patient safety themes from the stories, and explored their own attitudes and beliefs about patient safety. They also shared their own experiences of safety-related incidents as professionals and as patients or carers. The learning objectives common to both the control and intervention groups and derived from the UK Foundation Programme Curriculum were adhered to throughout the sessions and issues related to the objectives were discussed even if they did not naturally arise during the discussion. The trainees were encouraged to discuss any issues related to distress or upset caused by the narratives with the researchers. The patients were provided with emotional support by JS, who debriefed them after each session.

    Box 1

    Narrative excerpt

    “When the same senior medics that saw her previously with that comforting and attentive attitude, walked into her room the following day, I felt an immense sense of relief. Because for the first time in 11 years, of all we had been through together, I felt completely out of my depth. However, this time their concerned attitude appeared to me to be replaced with annoyance. Annoyance that this girl, that should simply be riding the storm, should be back in hospital, and even put on antibiotics. In a dismissive tone, the medic ordered that the antibiotics should be stopped and instructed both her and I that she simply had to be patient. I was by now, scared for L, and I was scared that despite the medic's comprehensive knowledge and expertise, they did not have the knowledge of L, and did not seem to grasp her state of health. They of course did not know her normal reactions, her strength of character. They were not privy to the smallest of symptoms or responses that occurred when they were not present, but I was. So I tried to share all this with them to try and convey that she was not merely a little unwell, but very poorly. I knew of course, logically that glandular fever wasn't treatable but I was getting concerned that something else would be missed that was treatable. However on trying to inform them of my fears, I felt instantly dismissed, and that I was nothing more than an overprotective mother. And L, who simply could not understand that she just had to be patient and must learn to cope. Over the next week, L was in so much discomfort, she slept for maximum of 12 h. Twelve hours out of 168. That's 156 h of constant pain, but this was of no concern. Her face was so swollen that her eyes were the size of golf balls which rendered her blinded for 3 days. This was of no concern. Her face so cracked and scaled, the cream I tried to rub in sat in the crevices created. No one could tell me why her skin was like this. To me, it was like she was being poisoned from the inside/out. But it was of no concern. The bruised marks continued to spread, it was of no concern. Her throat was so sore, that two analgesia tablets broken in half would take me 15 min to administer.

    Her pain so severe that one of her hands would twist my arm around in pain as she clawed at the sheet with the other hand. Analgesia via her IV drip was refused as something we don't do on the paediatric ward.

    From the seventh day of her second admission, L had been vomiting and feeling very ill. And although her spleen and liver had reduced in size to the touch, at 3 pm, she started humming to herself. When I asked her what she was doing, she whispered, ‘It comforts me.’ She then started talking nonsense, ‘Animals need one energy level, I need another.’ I felt as if she was going in and out of consciousness and I was losing her. Trying hard not to scare her, I told her to stay with me and I called for the medical team. When they arrived over 2 h later, and after three requests, they felt this confusion was because she was tired. She was asked two questions to assess her orientation to place, which she answered correctly. But these were done with her back toward them as she faced the wall of her room. They never once saw her face. They said they would place her on 4 h in neurological obs, but they were not concerned. The first set of observations was carried out by a student nurse. Only 2 h later, L suffered a fit, a cardiac arrest, multi-organ failure, and DIC (disseminated intravascular coagulopathy). At this point in time, L was only on paracetamol and ibuprofen administered via me. Her IV drip had been discontinued 24 h previously because of her swelling. Within hours of being admitted to intensive care as an emergency, L was diagnosed with bacterial toxic shock syndrome, this having been the cause of her complete collapse. And on admission to intensive care where she immediately received every treatment imaginable, the ICU doctor said to me, ‘She is a very ill girl.’ I could hear myself saying, ‘I know that. I had been telling them for a week, but they wouldn't listen.’ But for as much as my soul was screaming, my head was saying, ‘What is the point in telling the intensive care? It is too late.’ Later that night, another doctor asked me in a confused manner, with accusation in his tone, ‘Why did you leave it for so long before bringing her to hospital?’ On telling him I hadn't, he went deathly quiet….

    The control group received a clinician-led teaching session using PowerPoint presentations and small group work; the group size was similar in the intervention group. Researchers (ZT, JS) observed the control group sessions to compare content with the intervention session. In one session, typical of the training, trainees were shown a PowerPoint presentation on the General Medical Council (GMC) regulations and their guidance on the duties of a doctor. A number of safety scenarios developed by the trainers were then presented and discussed in small groups. The scenarios comprised ethical/legal dilemmas, self-awareness of limitations, and how to prevent errors. Further discussions took place around how to stay up to date with developments in the profession and the importance of accurate record keeping and communicating with patients. The order of the session might have implied a hierarchy of importance in the topics covered: regulatory and procedural material came first, followed by ethical/legal issues, with communication with patients and record keeping/handovers in the latter part of the session.

    Pilot study

    A pilot study29 tested the feasibility of developing and implementing the patient-led intervention, its acceptability among patients and trainee doctors, and the practicality of delivering the intervention within an established training programme. The suitability of the outcome measures was also assessed leading to confidence in the sensitivity of the APSQ to detect changes in attitudes and knowledge in both groups. Following the pilot, the intervention was refined to create smaller group sizes for participants and a clearer safety focus for each story. In addition to the APSQ, the PANAS was included to capture the emotional response that the patient narratives evoked among the learners. Finally, trainees suggested key learning points on safety at the end of the session that they would take away from their teaching, a further measure of the acceptability of the training among trainees.

    Outcomes

    Primary outcome

    The primary outcome was attitude to patient safety measured by the APSQ (see online supplementary appendix 1),13 a 26-item questionnaire addressing patient safety attitudes. An essential component of safety culture in an organisation is the attitudes of practitioners to safety including responsibility for reporting errors and appreciation of causes of errors, areas that are included in the APSQ. The APSQ was originally designed with final year medical students, which is likely to make it applicable to FY1 trainees at the start of their clinical practice. The psychometric properties of the APSQ have been studied using exploratory factor analysis. The primary endpoint was the overall score on the APSQ completed immediately following the training.

    All participants completed a hard copy of the APSQ on three occasions: before the training (baseline), immediately after training before the two groups had a chance to meet and discuss their sessions (post), and between 3 and 6 weeks after training (follow-up). Each participant was given a unique ID number that could be used to link the questionnaire data over the three time points. The follow-up period varied as it relied on the availability and convenience of the participants. The pilot study had highlighted problems with relying on trainees to complete follow-up questionnaires online. For this study, researchers made group appointments with trainees and travelled to individual hospitals to allow trainees to complete the questionnaires. Participants gave each item on the APSQ a score between 1 (strongly disagree) and 7 (strongly agree). For most items a high score indicated a positive attitude to patient safety. For seven items (questions 11, 13, 14, 15, 16, 17 and 25) a high score indicated a negative attitude and hence scoring was reversed during analysis.13 An overall APSQ score may range between 26 and 182, with high scores indicating a positive attitude to patient safety.

    Secondary outcome(s)

    The APSQ was also scored using subscales suggested by the analyses conducted by Carruthers et al.13 These subscales are formed through grouping the 26 items by topic into nine groups and summing the score for each group. The topic groupings and corresponding question numbers are shown in table 1. The groupings were: patient safety training received (clinical training helping understanding of causes of medical error), confidence in reporting errors (openness/no-blame culture in the workplace), working hours as a cause of errors (shorter working hours, shifts), error inevitability (all doctors make errors), professional incompetence as cause of error (carelessness), disclosure responsibility (reporting error), team functioning (multidisciplinary working), patient responsibility in reducing errors (greater involvement) and importance of training (teaching on safety). Self-reported knowledge about patient safety was an additional subgroup added for the purpose of this study to capture how knowledgeable participants felt in relation to seven aspects of patient safety.

    View this table:
    Table 1

    APSQ (Attitude to Patient Safety Questionnaire) question groupings

    A further secondary outcome of this study was the effect on participants’ mood as assessed by the PANAS,27 a reliable and validated30 20-item questionnaire (see online supplementary appendix 2) that assesses mood and emotional engagement in terms of emotional state. It consists of 10 items on the Positive Affect (PA) scale (attentive, interested, alert, excited, enthusiastic, inspired, proud, determined, strong and active) and 10 on the Negative Affect (NA) scale (distressed, upset, hostile, irritable, scared, afraid, ashamed, guilty, nervous and jittery). The PANAS was independently completed by all participants on two occasions, once before the training session (baseline) and once immediately after it (post).

    Scores for PA and NA were derived separately for both baseline and post session results. Participants gave each of the 20 items on the PANAS a score between 1 (‘Very slightly or not at all’) and 5 (‘Extremely’). An overall PA score may range between 10 and 50, with a high score indicating a more positive mood in the participant. An overall NA score may range between 10 and 50, with a high score indicating a more negative mood.

    The trainees from both the intervention and control groups were also asked to suggest three learning points that they would take away from the session that they would try and implement into their practice. The lists were analysed and compared to measure suggested differences in learning outputs between the groups.

    Sample size

    Initially, the RCT was due to run over one academic year involving approximately 150 participants (15 participants on each of the 10 training days). The 15 participants would be individually randomised into either the control or intervention group that is, seven to eight participants in each arm for each training day. Despite this being an individually randomised trial, it was considered prudent to allow for a clustering effect within each centre using an intra-cluster correlation co-efficient of 0.05. Assuming seven individuals per group and combining this with the intra-cluster correlation resulted in a design effect of 1.3. Dividing the number of participants in the trial by this design effect gave an effective sample size of approximately 115. Little to no attrition was expected as the study formed part of compulsory training for participants and randomisation occurred directly before teaching. No increment for attrition was therefore made. The effective sample size of 115 gave 80% power to detect an effect size of approximately 0.53 should one exist.

    Randomisation and blinding

    Simple randomisation was carried out at an individual level at each centre on a 1:1 basis once consent was obtained. This was done on-site on the training day by an independent administrator using a randomisation sequence generated using randomly ordered envelopes containing allocations. The assignment in the first envelope was given to the first individual as defined by registration and so on. Trainees who did not consent to the study received the same teaching as the control group and did not complete any questionnaires. Participants completed the baseline APSQ before randomisation and the PANAS baseline questionnaire immediately before the training session started. As the sessions had different formats, participants were aware of their assignment.

    Analysis

    Statistical methods

    Analysis was conducted in STATA (V.13) following the principles of intention to treat.

    Statistical testing was conducted at the 5% significance level using two-sided tests and consisted of t tests where normality could be assumed and Wilcoxon rank sum tests where it could not.

    For the primary analysis, a regression-based approach compared differences in overall post APSQ score between the two randomised groups after adjustment for baseline APSQ score and training centre. As a sensitivity analysis, the primary analysis was repeated twice: once without adjusting for training centre and once without adjusting for baseline APSQ score. A t test was used to compare post APSQ scores in individuals with and without baseline data to assess whether data were missing completely at random.

    Secondary analyses for the APSQ compared differences in each post APSQ subscale mean score between the two randomised groups using Wilcoxon rank sum tests where normality could not be assumed and t tests where normality could be assumed. Appropriate testing was also used to compare difference in relation to the additional knowledge subscale between the two groups. All analyses were repeated to compare differences in follow-up APSQ scores between the two groups.

    Analysis on the PANAS involved using Wilcoxon rank sum tests to compare post PA and post NA scores between the intervention and control groups.

    Qualitative methods

    Qualitative analysis31 of the lessons learned was conducted initially on the data set from 2011. Two researchers (NQ, ZT) independently coded the points made by the trainees; these could not be blinded as trainees were easily identifiable by their group. Any discrepancies were resolved by discussion. Two of the patients also analysed the data to interpret the lessons learned in light of the coding frame provided. This provided a consensus on the reliability of the data set. The 2012 data were analysed using the same coding frame; no further subthemes were elicited at this stage. Participants listed a variable number of ranked responses: the majority listed three lessons learned but there were one, two and four responses as well as some individuals who did not complete this element within the data set. Where the lesson learned could be interpreted in a number of different ways, this was resolved after discussion within the research team (NQ, ZT, JS, VJ).

    Results

    Baseline data

    A total of 313 individuals were eligible to participate (figure 1); of these five declined (1.6%) and 25 (8.0%) did not attend. Of the 283 participants who attended and consented to participate, 141 were randomised to the control and 142 to the intervention arm. There was a large amount of missing or incomplete baseline information for the APSQ (49.8%); completion was better in 2011 (18.0% of 150 were missing data) than in 2012 (85.7% of 133 were missing data). In those who completed the APSQ at baseline, the mean baseline scores were similar between allocated groups (see online supplementary appendix 3) and between centres (see online supplementary appendix 4).

    Figure 1
    Figure 1

    Participant flow diagram. APSQ, Attitude to Patient Safety Questionnaire; FU, follow-up; PANAS, Positive and Negative Affect Schedule.

    The PANAS had a much smaller proportion of completely missing baseline data, varying among the centres from 0.0% to 22.9% (see online supplementary appendix 4). Baseline scores were similar by allocated group for both PA and NA (see online supplementary appendix 3). Baseline total PA score was similar among the centres with the exception of GH, which had a lower total mean of 18.88 (SD 6.08). Baseline NA score was also similar across centres.

    APSQ results

    Results from analyses conducted on the overall APSQ scores are shown in table 2. The primary analysis was conducted on 122 individuals—56 from the intervention group and 66 from the control group. There was no evidence of a difference in post-training APSQ scores between the allocated groups, with a non-significant increase of 0.17 in overall post score (95% CI −3.96 to 4.29, p=0.94) for participants in the intervention group compared with those in the control group. This relates to a Cohen's d effect size of 0.01 (95% CI −0.35 to 0.37). Analysis to assess for difference in follow-up APSQ scores was conducted on 90 individuals (37 intervention, 53 control). There was no evidence of a difference between the allocated groups, with a non-significant decrease of 1.04 in overall follow-up score (95% CI −5.16 to 3.07, p=0.62) for participants in the intervention group compared with the control group. This relates to a Cohen's d effect size of −0.09 (95% CI −0.50 to 0.33).

    View this table:
    Table 2

    APSQ (Attitude to Patient Safety Questionnaire) results

    Statistical testing on all post APSQ subscale means was conducted on 236 individuals (126 intervention, 110 control). Only ‘patient involvement’ showed a statistically significant difference between the two groups (p<0.01), implying that the intervention may have changed attitudes about the importance of the role of patients in patient safety more than the standard teaching. No difference between the randomised groups was found in relation to the additional knowledge subscale.

    Statistical testing on the follow-up (3–6 weeks after training) APSQ subscale means was conducted on 184 individuals (88 intervention, 96 control). There was no evidence of a statistically significant difference between the two randomised groups on any of the subscale scores. One likely explanation is that there was communication between the two groups after the intervention.

    When the primary analysis was repeated without adjustment for centre as a sensitivity analysis, results were consistent (p=0.99). Given the amount of missing baseline data, a post hoc analysis repeated the primary analysis without adjustment for baseline APSQ score. Analysis was conducted on 236 individuals; a non-significant increase of 1.60 (95% CI −2.19 to 5.39, p=0.41) for those in the intervention group compared with those in the control group was found.

    A t test used to compare differences in post APSQ score between those with complete baseline data and those without provided no evidence that data were not missing completely at random (p=0.92). This remained the case when each randomisation group was considered separately.

    PANAS results

    Wilcoxon rank sum tests were used to investigate differences by allocated group for both the PA and NA scores and were conducted on 263 individuals (138 intervention, 125 control). There was evidence of a statistically significant difference in the underlying distribution of both post PA and post NA scores between the two randomised groups (p<0.001) with indications of both higher PA and NA scores in the intervention group. The control group showed an average decrease of 0.13 (SD 6.24) in overall PA score from baseline, while there was an average increase of 3.70 (SD 6.75) in the intervention group; this difference between the randomised groups of 3.83 points relates to an effect size of 0.59 (95% CI 0.32 to 0.85). In terms of overall NA score, there was an average increase of 0.21 (SD 3.47) from baseline for individuals in the control group, while for those in the intervention group there was an average increase of 3.11 (SD 5.96); this difference between the randomised groups of 2.9 relates to a Cohen's d effect size of 0.58 (95% CI 0.31 to 0.84).

    Learning points

    The codes derived from the analysis were first grouped into 26 categories. These were then grouped into five overarching themes:

    • Risk management and governance (report more errors/near misses, working safely, clinical governance, risk assessment/management).

    • Learning about error (decision making, attention to detail, minimising error).

    • Communication (speaking up about errors, communication in teams, involve patients in discussions, asking for help, no room for arrogance as a doctor).

    • Processes related to patient safety (documentation in notes, relying on gut feeling, attention to detail, follow-up on clinical investigations) and

    • Role of education (increasing knowledge, safe prescribing, ethical practice).

    The ranked frequencies of each theme, when the two randomised groups were compared, demonstrated a difference in the focus of central learning issues. Participants in the control arm ranked items in risk management and governance (pertaining to elements of reporting errors and near misses) and process (document accurate notes, attention to detail, follow-up results) most highly, whereas those in the intervention group ranked items pertaining to communication (with both patients and colleagues) most highly. Highlighted within this was the lack of a perception of emphasis on communication by trainees in the control arm. Allied with this theme was the appreciation of the necessity of challenging senior colleagues to prevent errors and of speaking up and honesty after an error has occurred.

    Discussion

    There are few rigorous evaluations of different methods of delivering training in patient safety. In this study we compared two forms of training, a standard classroom method and a patient narrative approach, using an RCT design. In education research, the impact of training is often difficult to measure reliably due to the multitude of variables that need to be considered and a lack of clarity on outcome measures. In evaluating patient safety interventions, there are a few validated measures of safety climate,32 ,33 but these tend to measure organisational culture rather than individual attitudes. Subsequent to the start of this study, new measures to test individual junior doctors’ knowledge of and attitudes towards patient safety are being reported.14 These may need to be adopted as outcome measures in future educational interventions on patient safety.

    The APSQ and PANAS baseline scores were similar in the two groups. The mean attitude scores were strongly positive towards patient safety, and in keeping with similar cohorts reported in the literature.34 For the APSQ subscales, as anticipated, there was a significantly increased score for the intervention group in the area of patient involvement in reducing error. While the intervention did not appear to impact on general patient safety attitudes, it did lead to significantly higher scores for the intervention group in the area of patient involvement in reducing error. It is possible that the lack of identification of a difference in general safety attitudes was due to the study being in effect underpowered as, despite the sample size having been doubled, the amount of missing baseline APSQ data affected the numbers analysed. Results were, however, consistent when the primary analysis was repeated without adjustment for baseline APSQ score.

    In keeping with Kumagai's framework, the patient narratives helped ‘communicate meaning’ by evoking an emotional response among the participants. What was interesting was that the narratives seemed to trigger both PAs and NAs, indicating emotional and/or cognitive concordance as well as dissonance among the trainees. In general, doctors learn to distance themselves from their own and their patients’ emotions.35 However, medical educationalists are now reminding us that medicine ultimately involves interaction with real people, with real emotions36 and awareness of emotions, and how to deal with them should be as much part of medical education as the development of clinical skills.

    The emphasis on health organisational and policy focus on the technical aspects of risk management by participants in the control group contrasted with the focus on communication and the patient in the intervention group. This highlights the need for safety training programmes to review their objectives in order to ensure that both aspects of patient safety are addressed. Future programmes may need to combine both the professional and patient input to maximise their impact on safety awareness.

    The trial had a number of strengths. It demonstrated the feasibility of implementing an RCT, which is unusual in studies on patient involvement in education. It also brought the focus back on the patient as the centre of patient safety interventions, allowed collaborative working between patients and researchers, and emphasised to junior doctors the terrible impact of safety incidents on patients and their families.

    There are difficulties with the design and implementation of randomised trials in education, resulting in a limited amount of literature reporting such studies.37 Randomised trials in education may suffer from performance bias (ie, the teacher's approach may be different due to the novelty of the intervention) as well as a placebo effect due to learners perceiving the novel method of teaching as better.38 However, randomising students to different interventions allows educationalists to examine differences due to intervention alone rather than other variables. In this trial, attempts were made to standardise the intervention across the 10 sites. This was done by trying to adhere to the broad learning outcomes for the sessions, using the same team of researchers to facilitate all the sessions, and asking the patients to maintain consistency in their narratives. However, the nature of the intervention meant that there were variations in the key safety issues emerging from the patient stories, with subsequent variations in discussions. There were also expected variations in the examples of safety incidents that the trainees brought up for discussion. The irrelevance of post-randomisation blinding in educational trials has been discussed previously.

    For the control arm, while the teaching materials were identical for each control group session, the clinician tutor varied between teaching sites. This may have resulted in a variation in emphasis within each session, influenced by the beliefs, values and personal experiences of the clinician.

    This trial was also limited by the amount of missing or incomplete baseline APSQ data leading to a reduction in power. We wished to measure the attitude of trainees before randomisation and preferably a few weeks before the teaching session. As we had no control over this, a number of trainees turned up to the session without having completed the APSQ. On the other hand, there were very few missing data for baseline PANAS probably because the trainees completed this just before the teaching session.

    This trial was conducted in one Foundation School and a very high proportion of attendees consented across the five centres. While there is no reason to believe that FY1 trainees in NYECFS are different from elsewhere in the UK, generalisability of the results cannot be confirmed due to a lack of demographic information. The low number of trainees who declined to participate in the trial provided some evidence of the acceptability of the intervention.

    Conclusions

    This study demonstrates a successful implementation of an RCT in medical education. Involving patients with experiences of safety incidents in patient safety training has an ideological appeal and seems to be an obvious choice in designing safety interventions. However, on the basis of our primary outcome measure we were unable to demonstrate that the intervention was any more effective than standard teaching in changing general attitudes to patient safety. While the intervention may impact on emotional engagement and learning about communication, we remain uncertain whether emotional engagement will translate into improved behaviours in the clinical context or indeed if there are any negative effects.

    Acknowledgments

    We would like to thank all the patients, carers and FY1 trainees who participated in this trial. We would also like to thank all the administrative staff of NYECFS for their help.

    References

    1. ↵
      1. Wong BM,
      2. Etchells EE,
      3. Kuper A,
      4. et al
      . Teaching quality improvement and patient safety to trainees: a systematic review. Acad Med 2010;85:1425–39.
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      Department of Health. Report of the Mid Staffordshire NHS Foundation Trust Public Inquiry. UK: Department of Health, 2013.
    3. ↵
      Department of Health. A promise to learn—a commitment to act. Improving the Safety of Patients in England. National Advisory Group on the Safety of Patients in England. UK: Department of Health, 2013.
    4. ↵
      1. Sandars J,
      2. Bax N,
      3. Mayer D,
      4. et al
      . Educating undergraduate medical students about patient safety: priority areas for curriculum development. Med Teach 2007;29:60–1.
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      1. Ahmed M,
      2. Arora S,
      3. Tiew S
      . Building a safer foundation: the Lessons Learnt patient safety training programme. BMJ Qual Saf 2014;23:78–86.
      OpenUrlAbstract/FREE Full Text
    6. ↵
      General Medical Council. Patient and public involvement in medical education. Advice supplementary to tomorrow's doctors. UK: GMC, 2013.
    7. ↵
      1. Jha V,
      2. Quinton ND,
      3. Bekker HL,
      4. et al
      . Strategies and interventions for involvement of real patients in medical education: a systematic review. Med Educ 2009;43:10–20.
      OpenUrlCrossRefPubMed
    8. ↵
      1. Conway J
      . Getting boards on board: engaging governing boards in quality and safety. Jt Comm J Qual Patient Saf 2008;34:214–20.
      OpenUrlPubMed
    9. ↵
      1. Repper J,
      2. Breeze J
      . A review of the literature on user and carer involvement in the training and education of health professionals. Int J Nurs Stud 2007;44:511–19.
      OpenUrlCrossRefPubMedWeb of Science
    10. ↵
      1. Peat M,
      2. Entwistle VA,
      3. Hall J,
      4. et al
      . A scoping review and approach to appraisal of interventions intended to involve patients in patient safety. J Health Serv Res Pol 2009; 15(Suppl 1):17–25.
      OpenUrl
    11. ↵
      The Health Foundation. Research Scan: Does improving safety culture affect patient outcomes? UK: The Health Foundation, 2011.
    12. ↵
      1. Ajzen I
      . From intentions to action: a theory of planned behaviour. In: Kuhl J, Beckmann J, eds. Action-control: from cognition to behaviour. Heidelberg: Springer, 1985:11–39.
    13. ↵
      1. Carruthers S,
      2. Lawton R,
      3. Sandars J,
      4. et al
      . Attitudes to patient safety amongst medical students and tutors: developing a reliable and valid measure. Med Teach 2009;31:e370–6.
      OpenUrlCrossRefPubMedWeb of Science
    14. ↵
      1. Durani P,
      2. Dias J,
      3. Singh HP,
      4. et al
      . Junior doctors and patient safety: evaluating knowledge, attitudes and perception of safety climate. BMJ Qual Safe 2013;22:65–71.
      OpenUrlAbstract/FREE Full Text
    15. ↵
      1. Baumeister RF,
      2. Vohs KD,
      3. DeWall CN,
      4. et al
      . How emotion shapes behavior: feedback, anticipation, and reflection, rather than direct causation. Pers Soc Psychol Rev 2007;11:167–203.
      OpenUrlAbstract/FREE Full Text
    16. ↵
      Patient Safety First. Leadership for safety ‘How to’ guide supplement: Using patient stories with Boards. http://www.patientsafetyfirst.nhs.uk (accessed 10 Oct 2013).
    17. ↵
      1. Charon R
      . Narrative medicine: honoring the stories of illness. New York, NY: Oxford University Press, 2010.
    18. ↵
      1. Schwabe L,
      2. Wolf OT
      . Stress prompts habit behavior in humans. J Neurosci 2009;29:7191–8.
      OpenUrlAbstract/FREE Full Text
    19. ↵
      1. Charon R
      . The patient-physician relationship. Narrative medicine: a model for empathy, reflection, profession, and trust. JAMA 2001;286:1897–902.
      OpenUrlCrossRefPubMedWeb of Science
    20. ↵
      1. Kumagai AK
      . A conceptual framework for the use of illness narratives in medical education. Acad Med 2008;83:653–8.
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      1. Finset A
      . Conceptual explorations on person-centered medicine 2010. Emotions, narratives and empathy in clinical communication. Int J Integr Care 2010;10(Suppl):e020.
      OpenUrlPubMed
    22. ↵
      1. Mezirow J
      . A critical theory of adult learning and education. Adult Educ 1981;32:3–24.
      OpenUrlCrossRefWeb of Science
    23. ↵
      1. Mezirow J
      . Transformative learning: theory to practice. New Dir Adult Contin Educ 1997;74:5–12.
      OpenUrl
    24. ↵
      1. Kasman DL,
      2. Fryer-Edwards K,
      3. Braddock CH III.
      . Educating for professionalism: trainees’ emotional experiences on IM and pediatrics inpatient wards. Acad Med 2003;78:730–41.
      OpenUrlCrossRefPubMedWeb of Science
    25. ↵
      1. Pitkala KH,
      2. Mantyranta T
      . Feelings related to first patient experiences in medical school. A qualitative study on students’ personal portfolios. Patient Educ Couns 2004;54:171–7.
      OpenUrlCrossRefPubMedWeb of Science
    26. ↵
      1. Taylor JS
      . Learning with emotion: a powerful and effective pedagogical technique. Acad Med 2010;85:1110.
      OpenUrlCrossRefPubMedWeb of Science
    27. ↵
      1. Watson D,
      2. Clark LA,
      3. Tellegen A
      . Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol 1988;54:1063–70.
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      1. Winterbottom AE,
      2. Jha V,
      3. Melville C,
      4. et al
      . A randomised controlled trial of patient led training in medical education: protocol. BMC Med Educ 2010;10:90.
      OpenUrlCrossRefPubMed
    29. ↵
      1. Jha V,
      2. Winterbottom A,
      3. Symons J,
      4. et al
      . Patient-led training on patient safety: a pilot study to test the feasibility and acceptability of an educational intervention. Med Teach 2013;35:e1464–71.
      OpenUrlCrossRefPubMed
    30. ↵
      1. Crawford JR,
      2. Henry JD
      . The Positive and Negative Affect Schedule (PANAS): construct validity, measurement properties and normative data in a large non-clinical sample. B J Clin Psychol 2004;43:245–65.
      OpenUrlCrossRefPubMedWeb of Science
    31. ↵
      1. Ritchie J,
      2. Spencer L
      . Qualitative data analysis for applied policy research. In: Bryman A, Burgess RG, eds. Analysing qualitative data. London: Routledge, 1994:172–94.
    32. ↵
      1. Sexton JB,
      2. Helmreich RL,
      3. Neilands TB,
      4. et al
      . The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res 2006;6:44.
      OpenUrlCrossRefPubMed
    33. ↵
      1. Colla JB,
      2. Bracken AC,
      3. Kinney LM,
      4. et al
      . Measuring patient safety climate: a review of surveys. Qual Safe Health Care 2005;14:364–6.
      OpenUrlAbstract/FREE Full Text
    34. ↵
      1. Wetzel AP,
      2. Dow AW,
      3. Mazmanian PE
      . Patient safety attitudes and behaviors of graduating medical students. Eval Health Prof 2012;35:221–38.
      OpenUrlAbstract/FREE Full Text
    35. ↵
      1. Ahluwalia S,
      2. Murray E,
      3. Stevenson F,
      4. et al
      . ‘A heartbeat moment’: qualitative study of GP views of patients bringing health information from the internet to a consultation. Br J Gen Pract 2010;60:88–94.
      OpenUrlAbstract/FREE Full Text
    36. ↵
      1. Shapiro J
      . Does medical education promote professional alexithymia? A call for attending to the emotions of patients and self in medical training. Acad Med 2011;86:326–32.
      OpenUrlCrossRefPubMedWeb of Science
    37. ↵
      1. Baernstein A,
      2. Liss HK,
      3. Carney PA,
      4. et al
      . Trends in study methods used in undergraduate medical education research, 1969–2007. JAMA 2007;298:1038–45.
      OpenUrlCrossRefPubMedWeb of Science
    38. ↵
      1. Parks T
      . Randomized controlled trials in medical education. J R Soc Med 2009;102:214.
      OpenUrlFREE Full Text

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    Footnotes

    • Contributors VJ made a substantial contribution to the conception and design of the research and to the acquisition, analysis and interpretation of data for the work, and wrote the paper. HB, RG, MK and RL made substantial contributions to the analysis and interpretation of data for the work. CM, NQ, JS, ZT, IW and JW made substantial contributions to the design of the research and to the acquisition, analysis, and interpretation of data for the work. All authors revised the paper critically for important intellectual content and gave final approval of the version to be published. They agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

    • Funding This work was supported by the National Institute for Health Research (NIHR) under the Programme Grants for Applied Research Programme (Grant Reference Number RP-PG-0108-10049). The views expressed in this paper are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.

    • Competing interests None.

    • Ethics approval Ethics approval was granted by the National Research Ethics Committee in February 2010.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement Qualitative data from the study are available at the Bradford Institute for Health Services Research.

    Linked Articles