Article Text
PDF
Abstract
Background The purpose of this study was to evaluate the impact of a patient-safety curriculum administered during a paediatric clerkship on medical students' attitudes towards patient safety.
Methods Medical students viewed an online video introducing them to systems-based analyses of medical errors. Faculty presented an example of a medication administration error and demonstrated use of the Learning From Defects tool to investigate the defect. Student groups identified and then analysed medication errors during their clinical rotation using the Learning From Defects framework to organise and present their findings. Outcomes included patient safety attitudinal changes, as measured by questions derived from the Safety Attitudes Questionnaire.
Results 108 students completed the curriculum between July 2008 and July 2009. All student groups (25 total) identified, analysed and presented patient safety concerns. Curriculum effectiveness was demonstrated by significant changes on questionnaire items related to patient safety attitudes. The majority of students felt that the curriculum was relevant to their clinical rotation and should remain part of the clerkship.
Conclusions An active learning curriculum integrated into a clinical clerkship can change learners' attitudes towards patient safety. Students found the curriculum relevant and recommended its continuation.
- Medical education
- medication error
- safety culture
- medication safety
Statistics from Altmetric.com
The Association of American Medical Colleges (AAMC) Medical School Objectives Project report recommends that medical schools deliver patient safety education to undergraduates.1 Additionally, the Liaison Committee on Medical Education (LCME) surveys patient safety content in all accredited medical school curricula as part of the standards for the educational programme of medical schools.2 Much of the published literature about patient safety education is directed towards graduate medical education and practising physicians.3–5 Despite increasing calls to educate medical students about these issues, few medical schools have developed and evaluated patient safety curricula.6–11 A recent national survey of US and Canadian Medical Schools suggests that very few schools have explicit patient safety curricula, and most existing curricula are based in lectures or small group discussions.12
Medication errors in hospitals have been reported to occur with alarming frequency with the great majority involving junior doctors recently graduating from medical school.13 We asked our students to seek out such errors during their clinical experiences and taught them a systems-based approach to analyse the error. We wanted to offer students an experiential opportunity where they could apply recently learnt information during their daily activities to establish the connections between information and application. Prior work has suggested that interactive modalities are preferred to more passive methods such as lectures.10 14 In this paper, we describe the development and implementation of a novel curriculum focussing on medication errors within a paediatric clerkship at a large academic medical centre. We hypothesised that this exercise would impact medical student attitudes towards patient safety. Safety attitudes contribute to the safety climate of a healthcare institution and are believed to lead to improvements in overall patient safety.15 16
Methods
Learner population
In the 2008–2009 academic year, 108 medical students were asked to participate in and evaluate the curriculum as part of their 9-week paediatric clerkship. Students included those in their second, third and fourth years. This study was based at Johns Hopkins Children's Center, which is an urban 174-bed academic tertiary care paediatric hospital in Baltimore, Maryland. Students spend 4½ weeks in an inpatient setting and 4½ weeks in an outpatient setting. The Johns Hopkins Institutional Review Board deemed this curriculum improvement study exempt.
Curriculum development
The Johns Hopkins University School of Medicine introduced patient safety teaching into the preclinical curriculum in 2004.17 Our current project focused on extending this education into the clinical years by involving all the core clerkships, each focussing on a different aspect of patient safety. Based upon input from content experts (MRM, DGB) and education experts (RD, MB), we developed a curriculum to teach the systems-based process of identifying and remediating potential medication errors during the paediatric clerkship. We focused on medication errors owing to the complex nature of paediatric dosing and the relatively high rate of medication errors compared with other types of errors in paediatrics, as well as the potential for greater harm in this population.18 19 We developed our curriculum following the six steps of curriculum development as described by Kern et al.20 These steps included: problem identification, needs assessment, development of goals and objectives, choosing educational strategies, implementation, and evaluation and feedback. We sought an experiential learning environment to ground our curriculum in adult learning principles.21 Students worked in small groups to identify and analyse a medication error or near-miss using a tool designed for this purpose. In this way, students would be actively involved in the educational process, see the relevance of this topic to their care of patients, feel empowered to discuss errors, and have the opportunity to direct their learning and reflect on their findings.
Curricular content
Our patient safety curriculum consisted of a 25 min online video introduction to patient safety, two 60 min facilitated ‘large-group’ sessions with 23–25 students, and a self-directed ‘small-group’ exercise with four or five students per group (table 1). The first large group session was delivered during a regularly scheduled weekly didactic session, while the second large group session was separately scheduled approximately 1 week before the end of the clerkship.
Investigation of a medical error curriculum description, Johns Hopkins University School of Medicine, 2008–2009
Students were asked to view an online video introduction to systems-based analysis delivered by one of our faculty (MRM). Subsequently, in the first large-group session, we presented an example of a medication error and demonstrated the use of the Learning From Defects (LFD) tool to investigate the defect. We posted our completed LFD tool online for students to see as an example. The LFD tool is similar to a root cause analysis, and a tutorial on the use of this tool has been previously published.22 The LFD tool requires users to focus on positive factors that help to prevent harm as well as factors that contribute to failure.23 The tool has three sections: an explanation of ‘what happened’; a section on ‘why did it happen’ prompting users to consider eight potential contributing/mitigating factors (patient factors, care giver factors, team factors, training factors, information technology factors, local environment and institutional environment); and a section to propose specific actions to reduce the likelihood of recurrence. Students were assigned to teams of five with each group instructed to identify an actual medication error, or near-miss, during the remainder of their clinical rotation. During the period of error identification, half the students are rotating in an ambulatory setting and half in an inpatient setting. Using the LFD framework, student groups delivered 10 min presentations of their findings in the second large-group session at the end of the clerkship and submitted a completed LFD tool to the faculty for review. Students were awarded 5% points to their final grade in the clerkship based solely on participation (completion of LFD tool and participation in the presentation).
Outcome measures
To assess the impact of our curriculum, we generated a brief survey instrument using a subset of items from the Safety Attitudes Questionnaire (SAQ) as a starting-point. The SAQ is a widely used survey measuring six attitudinal domains (team climate, safety climate, job satisfaction, stress recognition, perceptions of management and working conditions) which utilises a standard five-point Likert scale, ranging from disagree strongly to agree strongly.16 Although SAQ items are designed to elicit a consensus assessment from a particular unit or work group, our students were dispersed across several clinical units and groups, and we thus modified questions to focus on the individual attitude of the respondent. A round-table discussion by investigators generated the 10 SAQ-related items selected from the teamwork climate and safety climate domains of the SAQ. The questionnaire was distributed after the small group presentations near the end of the clerkship and required 10 min to complete. We did not collect any identifying information, and students were free to return an incomplete form. We analysed students' perceived attitudinal changes with a retrospective pre–post design used in other published educational interventions.24 25 The theory supporting this type of design is that when participants are asked to respond to a question about how much they know about a particular subject after they have some basic knowledge of the subject itself, they are better able to accurately reflect on the degree of change in knowledge or attitude, and these responses are less subject to response shift bias which threatens the validity of traditional pre then post designs.26 27
Student satisfaction with the curriculum was assessed with additional questions developed by the investigator team using the same five-point scale. Students rated their attitudes towards the online content, the relevance of the curriculum to paediatrics and whether this curriculum should continue in the clerkship.
Data analysis
Data analysis was performed using Stata, version 9.2 (StataCorp LP). We evaluated differences in responses to the retrospective pre and post groups. We chose to analyse responses to the SAQ by collapsing scores of 4 (agree) and 5 (strongly agree), and comparing these with any other response (1–3) or collapsing scores of 1 (strongly disagree) and 2 (disagree) and comparing these with any other response (3–5) depending on the direction of change to allow for a more uniform presentation of data for reverse-worded questions. This is presented as ORs with 95% CIs. Frequencies and simple means with standard deviations, where appropriate, were calculated.
Results
Medication errors identified
One-hundred and eight students completed this required curriculum, and all students provided responses to the survey. All student groups (25 total) identified and presented a patient safety concern with an analysis of the causative factors and suggestions for mitigating errors (examples presented in table 2).
Case examples from small group exercises
Student attitudes towards patient safety
Student groups reported increased appreciation about the impact of teamwork and continuity of care on medical errors with several of the groups identifying communication during transitions of care as a major contributing factor. Reported changes in student knowledge and attitudes about safety were significant for most questions derived from the SAQ. Specifically, medical students reported an increased awareness that they, as well as others, will make errors that have the potential to harm patients in the future and that disruptions in continuity of care can be detrimental to patient safety. Students reported that this exercise made them feel more comfortable reporting issues to the appropriate authorities and that their suggestions on safety would be acted upon (table 3).
Medical student responses to questionnaire items
Student satisfaction
Students were largely satisfied with the curriculum, with 76% recommending that the session continue. Two-thirds were satisfied with the delivery of content through an online video, but only 53% reported viewing it. Most (79%) report that this curriculum is relevant to paediatrics (table 3).
Discussion
It is feasible to deliver an experiential patient safety curriculum focused on medication errors as part of a core clinical clerkship. There are sufficient numbers of medication errors observed by students during a clinical rotation to provide opportunities to place these errors into their proper, systems-based context. The curriculum is valued by students and brings about short-term changes in their attitudes related to patient safety which may potentially lead to overall improvements in patient safety. Medical students begin to build their attitudes about patient safety during their training, and their clinical clerkship experiences may present the ideal ‘teachable moments.’
While completing this exercise, all medical students enrolled in the paediatrics clerkship become aware of a medical error or near-miss in the process of taking care of their assigned patients. As a result, students had the opportunity to report to our hospital's electronic adverse event and near-miss reporting system. Many students reported being unaware of such error-monitoring systems and felt empowered by the process of entering a report. Our results demonstrate a 3.1-fold increase in the odds of students agreeing with the statement, ‘I feel comfortable reporting issues to appropriate authorities.’ Previous studies have suggested that medical students are frequently exposed to medical errors, yet only half reported the error to a resident or attending physician, and only 7% reported using an electronic error-reporting system.10 An important barrier to disclosure may be the medical hierarchy which can promote student intimidation and discourage error reporting. This may be a particularly important component of the safety climate at teaching institutions. Making this a mandatory exercise can help overcome student reluctance to openly discuss medical errors and may help improve the safety culture of involved institutions. Ideally, students should feel comfortable bringing concerns to attention, and the student voice should serve, if nothing else, as a mechanism of redundancy in identifying errors or near-miss errors. There were occasions in which the exercise described in this manuscript led to the reporting of a patient safety concern that would not otherwise have been uncovered. In one instance during the process of reviewing a patient's medications, a student discovered that an incorrect weight had been entered into the electronic record, and thus weight-based medication calculations were incorrect. This student suggested that this curricular exercise led to increased scrutiny of the process of medication delivery resulting in the discovery. Medical students may serve as an important untapped resource in identifying and preventing patient harm.
It is noteworthy that students feel that medical errors are not adequately addressed by existing systems and are not handled as well as they would have previously thought. Their response to the statement, ‘Medical errors are handled appropriately in hospitals’ was lower after this exercise. This should not be a surprising finding, as the process of evaluating an error and proposing a solution may help students realise how complex the interacting systems are. This finding is reinforced by their response to, ‘I expect there to be adequate mechanisms in place to insure safe care’ which also was associated with lower agreement after the exercise. In addition, students may not perceive the degree to which different healthcare systems (eg, an institution and a community-based private pharmacy) are able to partner effectively to reduce errors.
Our study has limitations. We chose to employ a retrospective pre–post design which can be subject to recall bias as well as effort justification bias wherein participants report improvement to justify the effort they put into the programme, though such a design is generally believed to be less susceptible to response shift bias. Also, we did not collect any information about students' prior clinical experiences or years of training, and our results may be susceptible to maturation bias. However, all students had the same preclinical education in patient safety, and this curriculum was the only additional content on medication administration errors. Another limitation includes our decision to alter a subset of questions from the original SAQ; as such, the published psychometrics of the SAQ are not directly transferable to the survey we developed. Also, we surveyed students immediately after completing the curriculum, and we are thus unable to comment on the durability of the attitudinal changes. Additionally, our assessment of student satisfaction and safety attitudes is arguably narrow in focus. However, we attempted to present some evidence of skills assessment by way of presenting case examples in table 2, as these demonstrate students' abilities to identify the ways in which various factors contributed to their identified error or near-miss as well as student's abilities to suggest mechanisms to decrease the likelihood of recurrence. In this way, we hope to offer a first step towards teaching and assessing some of the harder to reach educational competencies such as systems-based practice.28 While we did not formally assess for potential learner harm or second victim effect as a direct result of these exercises, we provided a safe peer-review forum for students to present their findings. Our impression was that the students were more attentive to the complexities of the interacting systems that allow errors to occur which helped to minimise blame at the individual provider level. Lastly, we recognise that this is the experience of a single clerkship at a single institution, and institutional culture around this topic is likely to affect the feasibility of implementing this curriculum more widely. Despite these limitations, our findings may be generalisable to other medical curriculum programmes. Our student body is made up of diverse students from across the nation, and their clinical encounters took place in multiple teaching locations, including inpatient and outpatient settings in academic and community-based venues.
We continue to present this curriculum in our paediatric clerkship, but we no longer ask the students to watch the online video introducing them to patient safety concepts prior to the first large group session, as only half of the students elected to view it. This may be due to the perceived redundancy of the information presented, as students are exposed to these concepts in the preclinical years.
Conclusion
We developed an innovative and experiential curriculum that can be successfully delivered to medical students during a clinical clerkship and results in short-term changes in student attitudes towards patient safety. There were a sufficient number of medication administration errors to allow all student groups to identify and analyse an error or near-miss during their clinical experiences. The Learning From Defects tool appears to be well suited to providing a framework for students to learn how to identify and analyse medical errors. This work adds to the current body of medical literature about patient safety curriculums, and further investigation is warranted to assess the durability and generalisability of our findings.
Acknowledgments
The authors gratefully acknowledge PJ Pronovost and H Aboumatar, for their participation in the development of this curriculum, and B Sexton, for his assistance with the manuscript. We also are indebted to G Dover, for his leadership in engendering an institutional culture, which encourages discussion of errors at all levels of medical training and healthcare delivery.
References
Footnotes
Competing interests None.
Ethics approval Ethics approval was provided by the Johns Hopkins University Institutional Review Board.
Provenance and peer review Not commissioned; externally peer reviewed.