Article Text
PDF
Abstract
Purpose To develop a patient safety curriculum and evaluate its impact on medical students' safety knowledge, self-efficacy and system thinking.
Methods This study reports on curriculum development and evaluation of a 3-day, clinically oriented patient safety intersession that was implemented at the Johns Hopkins School of Medicine in January 2011. Using simulation, skills demonstrations, small group exercises and case studies, this intersession focuses on improving students' teamwork and communication skills and system-based thinking while teaching on the causes of preventable harm and evidence-based strategies for harm prevention. One hundred and twenty students participated in this intersession as part of their required second year curriculum. A pre–post assessment of students' safety knowledge, self-efficacy in safety skills and system-based thinking was conducted. Student satisfaction data were also collected.
Results Students' safety knowledge scores significantly improved (mean +19% points; 95% CI 17.0 to 21.6; p<0.01). Composite system thinking scores increased from a mean pre-intersession score of 60.1 to a post-intersession score of 67.6 (p<0.01). Students had statistically significant increases in self-efficacy for all taught communication and safety skills. Participant satisfaction with the intersession was high.
Conclusions The patient safety intersession resulted in increased knowledge, system-based thinking, and self-efficacy scores among students. Similar intersessions can be implemented at medical, nursing, pharmacy and other allied health schools separately or jointly as part of required school curricula. Further study of the long-term impact of such education on knowledge, skills, attitudes and behaviours of students is warranted.
- Curriculum
- students
- health occupations
- medical errors
- communication
- safety
- communication
- health professions education
- medical education
- patient safety
- teamwork
- patient outcomes
- medication error
- health care quality
- adverse events, epidemiology and detection
- anaesthesia
- checklists
- critical care
- evidence-based medicine
Statistics from Altmetric.com
- Curriculum
- students
- health occupations
- medical errors
- communication
- safety
- communication
- health professions education
- medical education
- patient safety
- teamwork
- patient outcomes
- medication error
- health care quality
- adverse events, epidemiology and detection
- anaesthesia
- checklists
- critical care
- evidence-based medicine
Far too many patients suffer preventable harm from medical errors, adding needless patient suffering and healthcare costs.1 An estimated 2.9–16.6% of hospitalised patients are subject to at least one adverse event and half of those are judged to be preventable.2–5 Though the number of preventable deaths is uncertain, approximately 100 00 people die each year in the USA from healthcare-associated infection,6 another 100 000 from venous thromboembolism,7 8 and between 40 000 and 80 000 from diagnostic errors,9 making medical error one of the leading causes of death. Despite efforts to remedy this problem over the last decade, there has been limited progress in reducing preventable harm.10
A major contributor to preventable harm is the lack of training for clinicians in the science of patient safety. The science of safety includes identifying and mitigating hazards, improving teamwork and communication (T&C), and ensuring patients receive recommended practices. The science of safety is based on human factors and systems engineering, sociology, psychology and health services research, disciplines often absent from the faculties of American medical colleges. The Institute of Medicine has called for education reform of health professions to advance healthcare safety and quality.11 The Liaison Committee on Medical Education has required medical schools to provide ‘specific instruction’ in communication skills including ‘communication with patients, families, colleagues, and other health professionals’. The Association of American Medical Colleges has incorporated objectives around quality of care in its Medical School Objectives Project.12
Generally, training in patient safety is underdeveloped. Knowledge on safety is low among medical students and trainees,13 and reluctance to disclose or report errors persist.14 Physicians underestimate the health system's contribution to medical errors.15 Few medical schools include patient safety in their required curricula.16 17 Training in T&C is underdeveloped and faculty development in this area is deficient.18 Consensus about the goals and curricular content areas for patient safety training is still largely lacking. Published content has included medical errors, incident reporting, risk management and system causes of patient harm.19–21 Recent papers have highlighted the need to introduce training on situational awareness and effective T&C skills.22–24
We have previously described a 6 h patient safety curriculum that we have taught to first-year medical students.25 In this article, we report on the development, implementation and evaluation of a new clinically oriented, 3-day patient safety curriculum. The curriculum aims to enable students to practice safely and effectively engage in improving system safety later in their professional training. Central foci of the curriculum are T&C strategies to safeguard patients against harm and analytical frameworks and tools to help students advance their system-based thinking. The hypothesis was that in a pre–post assessment, medical students who receive this curriculum would show an increase in their safety knowledge, self-efficacy and system thinking.
Methods
Study design
The overall design of this study was curriculum development, implementation and pre–post assessment of a cohort of 120 second year medical students at the Johns Hopkins School of Medicine (JHSOM). Students were expected to participate as part of their required curriculum.
Beginning in 2004, we developed and taught a 6 h safety curriculum over a 3-week period.25 In 2009, the JHSOM started implementation of a new curriculum.26 The new curricular design incorporated horizontal strands and intersession time periods where focused learning can take place over 3–4 days about specific topics, which is then further buttressed by additional elements throughout medical school training. Under that design, a horizontal Safety and Quality strand was introduced which allows for integrating learning about safety and quality in a developmentally appropriate manner. A Patient Safety Intersession that is scheduled during the second year serves as an anchor ‘course’ for the Safety and Quality strand. Given the new curricular design, and the interim advances in patient safety, we elected to formally review our prior safety curriculum and develop a new one. In this paper, we describe the new curriculum that we developed and implemented for the Patient Safety Intersession. The new curriculum was developed using a published six-step curriculum development model,27 as detailed below.
Problem identification and general needs assessment
We conducted a review of the medical literature on patient safety, which demonstrated the magnitude of the patient safety problem, common causes of patient harm, and evidence-based prevention strategies.
Needs assessment of targeted learners
Previously, we conducted a survey of third-year medical students, residents and nurses, asking what medical students should know or know how to do prior to starting their clinical clerkships in order to practice safely.28 The survey revealed that students need orientation to safety and infection control issues, medical records and clinical information systems. They also need to communicate well with their multidisciplinary team, offer task assistance and deliver focused briefings on patient status. Based on the literature review and targeted assessment, we developed a conceptual model for the intersession (see figure 1). The model depicts the main mechanisms of patient harm29–39 (eg, medication errors, pulmonary embolism, falls etc) and places T&C issues at the centre to signify its contribution to the majority of patient harm events.40–44
Patient safety intersession conceptual model.
Goal and specific measurable objectives
With a focus on T&C and system thinking, the curriculum has three overarching goals:
To describe how medical errors may occur, how we can learn from them and how we can prevent their recurrence at the healthcare provider level, team level and system level.
To provide the necessary knowledge and skills to practice safely as individual providers and within the healthcare team.
To advance system-based thinking as a means for improving patient safety and quality of care. This includes helping the learners develop ‘lenses’ to see systems and to understand basic principles of designing safety systems.
The curriculum objectives, educational methods and content are shown in online appendix 1.
Educational strategies
Educational methods varied based on the session objectives and were designed to be consistent with the adult learning theory focusing on learner engagement and content relevance.45 Methods included case-based learning, small group activities, role play exercises and simulations.
Implementation
Intersession faculty were selected from multiple disciplines including generalist and specialist physicians, health services researchers, nurses, healthcare safety experts, infection control practitioners and other allied healthcare professionals.
On each of its three half days, the intersession programme started with a full class session followed by two 90 min breakout sessions. The full class sessions covered key topics in safety science. Guided by the curriculum conceptual model (figure 1), the breakout sessions were designed to introduce students to main harm mechanisms and provide them with the necessary knowledge and skills to practice safely as individual providers and within the healthcare team (online appendix 1).
Evaluation and feedback
We used several methods to evaluate this curriculum, using both internal and external evaluators.
The primary outcome variables were system thinking, self-efficacy and safety knowledge. Secondary outcome variables included student intentions to apply safety practices and satisfaction with the intersession.
To measure system thinking we used the system thinking scale (STS). This is a validated scale composed of 20 items on a 0–4 Likert-type scale.46 The STS composite score ranges from 0 to 80.
Self-efficacy was measured using nine ‘I know how to’ statements that students scored on a 1–5 Likert-type scale (see table 2). Safety knowledge was measured using a 19-item test. Most of the knowledge test and self-efficacy items have been pilot tested in earlier safety and ward preparation courses.13 25 28 Intentions to apply safety practices were measured via agreement ratings to the following two statements: “I will speak up about any safety concerns I have about my patients” and “I plan to use the ‘Teach Back’ method to ensure that my patients understood my instructions”. Satisfaction was measured using a standard JHSOM post course. The survey includes general satisfaction questions, students' usefulness ratings and general comments.
Evaluation procedure
Three evaluations were conducted: a pre–post intersession evaluation of student knowledge, awareness of safety problems, self-efficacy and systems thinking; a post-intersession assessment of student intentions to apply safety practices and satisfaction; a review by the JHSOM's Student Assessment and Program Evaluation (SAPE) committee 1 month after the intersession as part of the regular evaluation process for JHSOM courses.
All student tests and surveys were administered online. The satisfaction survey was administered separately in an anonymous manner. The Johns Hopkins Institutional Review Board approved this study.
Data analysis
We conducted a descriptive analysis using means, medians and 95% CIs.
A total knowledge and STS composite score were calculated for each student pre and post intersession. Scores were only compared for students with both pre data and post data (n=105). Differences in knowledge scores were compared using a paired t test and those for STS scores with Wilcoxon signed ranks test. Each of the student self-efficacy items were compared statistically using the Wilcoxon signed ranks test (table 2).
Student ratings of their intention to apply safety practices were dichotomised into an ‘agree’ or ‘strongly agree’ category versus all other responses. Students' ratings of usefulness of intersessions were also dichotomised and a percentage of ‘useful’ or very ‘useful’ ratings were calculated for each session. Data were analysed using Microsoft Office Excel and Minitab 15 Statistical Software (Minitab Inc., State College, PA).
Results
A total of 119 students participated in the intersession held on 3–5 January 2011. The mean age of students was 24.40 years (SD=2.25) and 53% were men and 47% were women. Students' prior patient care experience included participation in a 10-month longitudinal clerkship during which they worked with a primary care provider for one half day per week. In addition, eight students reported that they had received training in patient safety or quality improvement, seven students reported having participated in improvement projects and four students reported both.
Survey response rates were 94% for the pre-knowledge and post-knowledge tests, 88% for the pre-self-efficacy and post-self-efficacy and system thinking survey and 32% for the satisfaction survey.
Safety knowledge and system thinking
Table 1 depicts mean pre-intersession and post-intersession safety knowledge and composite STS scores with 95% CIs on the difference.
Safety knowledge and system thinking scores pre and post the patient safety intersession. Johns Hopkins School of Medicine, January 2011
Self-efficacy ratings pre and post the patient safety intersession (n=105). Johns Hopkins School of Medicine, January 2011
The 19% increase in mean knowledge scores from pre to post intersession was significant (t=16.25, p≤0.001). Additionally, the 7.4 increase in mean composite STS scores was also significant (t=6.55, p≤0.001).
Self-efficacy and intentions to apply taught practices
Table 2 depicts students' pre-intersession and post-intersession mean self-efficacy ratings in applying the various skills taught in the intersession.
Students had statistically significant increases in self-efficacy ratings for all nine assessed skills. Ninety-five per cent of students reported that they plan to use the ‘teach back’ technique to ensure patient understanding and 85% reported that they will speak up about any safety concerns on their patients.
Student satisfaction and committee review
Students who responded to the anonymous satisfaction survey were highly satisfied with the intersession. Intersession quality was rated as ‘excellent’ or ‘very good’ by 92% of responding students. Students perceived intersession activities to be highly useful. The percentage of students rating sessions as ‘useful’ or ‘very useful’ ranged from 77% to 95% for the didactic sessions, from 74% to 100% for small group activities and from 79% to 97% for simulation activities. Box 1 lists select quotes from students' feedback.
Select quotes from students' qualitative feedback. Johns Hopkins School of Medicine, January 2011
“I thought the intersession did a good job of having us participate in a variety of interesting activities. The structure of brief lecture and then cycling through various activities, was effective for me.”
“Overall, the intersession was excellently constructed and highly relevant to medical care. I greatly enjoyed the emphasis on small group learning activities as opposed to lectures as well as the effort to provide all of the information in clinical context.”
“The elderly simulation was very well done, and I liked that it was entirely interactive.”
“I would consider this the best intersession yet. The activities were very well coordinated and chosen as well as very engaging. I felt illuminated to the systems nature of patient safety, more aware of the mistakes waiting to happen around as well as empowered to be a better advocate for my patients. I am quite impressed that such a wealth of practical knowledge as well as widening of horizons happened in a mere three half-days. I really liked the 'systems approach' ideas and the hands-on activities.”
“I liked the case discussions and some of the simulations. What I liked least was that there were mandatory activities at 8:00 AM.”
“My favourite parts of the intersession were the error reporting and communication lab small groups. They were excellently run, and I came out of them feeling like I had gained concrete skills.”
“The course was useful AND it was fun. To date, there has not been a better balance between lecture/small group activities/simulation centre etc… ”
“This was an incredibly valuable and interesting intersession. I greatly appreciated having the opportunity to learn critical information that will enable me to communicate better with patients and healthcare colleagues as well as safety precautions to prevent infection spread and prevent falls. All of the information in this course was highly relevant to helping us transition to the wards, and truly makes me feel more ready to handle problems and challenges that I will encounter when I start my rotations.”
“As patient safety is the most important priority in our future practice of medicine, I'm happy that we had a good exposure to some of the key concepts on this topic. All of the small group activities were really helpful in understanding and experiencing some of the practical issues surrounding patient safety in healthcare practice.”
The SAPE committee review was highly complementary, describing the patient safety intersession as a ‘terrific success’ and providing an ‘exceeds expectation’ rating on intersession objectives, innovation and scholarship, and educational methods.
Discussion
In this study of an innovative patient safety curriculum, we found a significant pre–post increase in safety knowledge, self-efficacy for a variety of safety skills and ‘system thinking’ among medical students. The study results are encouraging, suggesting the ability to impact safety knowledge, skills and system thinking during a brief intersession.
Although the Institute of Medicine report To Err is Human had attributed the largest majority of medical errors to ‘system’ defects, medical providers still largely think of such errors as problems of ‘individual’ providers.15 47 Gaining a ‘system’ perspective is crucial to being able to analyse medical errors and create system safeguards. Prior literature has pointed to the need to help providers acquire ‘system lenses’ to ‘see hazards’ and design ‘system’ solutions.48 In this intersession, we taught medical students early in their professional training and aimed to empower them with analytic tools to advance their system thinking. This offers them the opportunity to ‘practice’ and ‘master’ this way of thinking during their clinical clerkships. To that end, we have coordinated with the clinical clerkship directors, and embedded clerkship exercises for students to help reinforce system thinking. For example, in their pediatrics clerkship, students are asked to identify a medication error, explore its root causes and propose solutions to prevent its recurrence. Other activities that we have pilot-tested include observation of operating room briefings and debriefings in surgery, review of diagnostic errors in neurology, and identification of barriers to the provision of evidence-based therapies in medicine.
Teaching about safety and T&C improvement, like other behavioural and social content, has been challenged with being perceived by the medical students as a ‘soft’ science; that is, removed from the ‘hard core’ biomedical topics they learn in their pre-clinical and clinical years.21 49 This study demonstrated high student satisfaction. We attribute that to three key factors: the topic of safety was approached as a ‘science’ with a defined set of principles and theories, and supported with published literature, yet made clinically relevant via ‘story telling’ of real patient harm events; all safety principles were discussed in an interactive and clinical context thus helping students understand their relevance to clinical care; students were empowered through learning a specific set of skills to personally use to maximise the safety of their own patients. The latter point is particularly important as it allows students to engage in self-improvement efforts and become part of the solution regardless of their ability to participate in system-level safety interventions. Recent literature has pointed to the ‘pendulum going too far’ regarding safety being viewed as largely a system problem with minimal role for individual accountability.50 51
Prior studies have demonstrated increases in knowledge levels and self-efficacy after participating in short courses on safety.19 20 25 52 53 Curricular content have included the epidemiology of medical errors, risk management, medical incident analysis, safety culture, systems theory, human factors engineering and error disclosure.21 In this study, we demonstrated that a broader curriculum that included, in addition to aforementioned content, teaching on harm prevention strategies with a focus on skill acquisition via role play and simulation, was effective and highly valued by students. The curriculum conceptual model that draws on the various mechanisms by which patient harm may occur allows for framing the teaching of safety principles in a clinically relevant context, thus reducing the challenge medical schools have typically encountered in teaching socio-behavioural content.49
This study had some limitations. One limitation is the absence of a formal competency assessment for taught skills. This was due to lack of time and resources for conducting an observed standardised clinical exam. Another limitation is that student ratings are subject to social desirability bias. To minimise this effect, we communicated to students prior to the start of intersession that the surveys have no right and wrong answers and are being administered only for programme evaluation purposes. Long-term assessment of knowledge retention and implementation of taught safety skills would be needed to assess the potential impact of this course on students' future clinical practice and patient safety. Towards that end, we are conducting a 1-year follow-up assessment of student knowledge retention, their implementation of taught skills and strategies, and their experiences while doing so.
Given its brief duration, this intersession could be easily integrated into the curricula of existing medical schools and its content is also relevant to other healthcare professions, thus allowing it to be adapted for teaching as an interprofessional learning experience. In fact, three nursing students attended this intersession as an independent study course and responded favourably to its content. Modified versions of this intersession could also be taught for medical professionals at various levels (eg, for medical students, residents and as continuing medical education for attending physicians). This type of multilevel implementation of safety education could enhance curriculum effectiveness via increased role modelling of safe practices by senior medical professionals and subsequent encouragement of more junior professionals (eg, medical students and incoming interns) to also do so.
Successful implementation of this curriculum, however, requires attention to faculty development. The small group sessions require faculty that are comfortable teaching the ‘science’ of patient safety. It took several years at our institution to achieve this and some schools may not have similar resources. Motivated safety advocates and quality improvement specialists can present a valuable resource to draw upon and include in faculty development efforts.
Conclusions
We designed, implemented and evaluated a 3-day patient safety intersession as part of the required second-year medical school curriculum. Participants increased their knowledge, safety skills self-efficacy and system thinking scores. Students particularly enjoyed the simulation and skill-based activities taught by a multidisciplinary group of instructors. Similar intersessions can be implemented at medical and other healthcare professions schools. Further study of the long-term impact of similar courses on knowledge, skills, attitudes and behaviours of students is warranted.
Acknowledgments
The authors wish to thank all Johns Hopkins University and Hospital faculty, staff and physicians in training who have helped us in development and/or teaching of this course. In particular, we thank Sean Berenholtz, Brad Winters, Simon Mathews, Kathy Wagner-Kosmakos, HeeWon Lee, Maria Cvach, Quiera Tidwell, Eileen Kasda, Polly Ristaino, Donna Fellerman, Elaine Nowakowski and Carol Fleishman. We would also like to thank the Office of Curriculum, Hospital Epidemiology and Infection control team, the Simulation Lab team, and the standardised patients who helped us during the course sessions and Drs Pam Lipsett, Pat Thomas and members of the SAPE committee for their review of this course.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Download Supplementary Data (PDF) - Manuscript file of format pdf
Footnotes
Other disclosures: The lead author of this paper ‘had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis’.
This manuscript is not under consideration at any other journal. All those who qualify to be authors of this work are listed above and meet criteria for authorship. The authors take responsibility for and agree with the data presented and report no conflict of interest. No similar manuscript based on this study has been submitted or published.
Funding This intersession was supported by funds from the Johns Hopkins School of Medicine.
Competing interests None.
Ethics approval Johns Hopkins Institutional Review Board.
Provenance and peer review Not commissioned; externally peer reviewed.