Article Text

Download PDFPDF

Integrating patient safety into health professionals’ curricula: a qualitative study of medical, nursing and pharmacy faculty perspectives
  1. Deborah Tregunno1,
  2. Liane Ginsburg2,
  3. Beth Clarke3,
  4. Peter Norton4
  1. 1School of Nursing, Queen's University, Kingston, Ontario, Canada
  2. 2School of Health Policy and Management, York University, Toronto, Ontario, Canada
  3. 3Bridgepoint Health, Toronto, Ontario, Canada
  4. 4Department of Family Medicine (Emeritus), University of Calgary, Calgary, Canada
  1. Correspondence to Dr Deborah Tregunno, School of Nursing, Queen's University, 92 Barrie Street, Kingston ON, Canada K7L 3N6; tregunno{at}queensu.ca

Abstract

Background As efforts to integrate patient safety into health professional curricula increase, there is growing recognition that the rate of curricular change is very slow, and there is a shortage of research that addresses critical perspectives of faculty who are on the ‘front-lines’ of curricular innovation. This study reports on medical, nursing and pharmacy teaching faculty perspectives about factors that influence curricular integration and the preparation of safe practitioners.

Methods Qualitative methods were used to collect data from 20 faculty members (n=6 medical from three universities; n=6 pharmacy from two universities; n=8 nursing from four universities) engaged in medical, nursing and pharmacy education. Thematic analysis generated a comprehensive account of faculty perspectives.

Results Faculty perspectives on key challenges to safe practice vary across the three disciplines, and these different perspectives lead to different priorities for curricular innovation. Additionally, accreditation and regulatory requirements are driving curricular change in medicine and pharmacy. Key challenges exist for health professional students in clinical teaching environments where the culture of patient safety may thwart the preparation of safe practitioners.

Conclusions Patient safety curricular innovation depends on the interests of individual faculty members and the leveraging of accreditation and regulatory requirements. Building on existing curricular frameworks, opportunities now need to be created for faculty members to act as champions of curricular change, and patient safety educational opportunities need to be harmonises across all health professional training programmes. Faculty champions and practice setting leaders can collaborate to improve the culture of patient safety in clinical teaching and learning settings.

  • Patient safety
  • Nurses
  • Medical education
  • Continuing education, continuing professional development
  • Pharmacists

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

Amidst increased interest in patient safety curricular innovation, there is a shortage of research that critically examines the perspectives of faculty who are on the ‘front lines’ of curricular change. While there has been a growth in patient safety curricular resources, much of the early work is limited to theoretical frameworks1 and curriculum guides,2 including WHO's Patient Safety Curriculum Guide for Medical Schools3 and WHO's Multiprofessional Patient Safety Curriculum Guide.4 More recently, empirical work has examined the impact of specific curricular interventions, such as incident reporting and error disclosure, within single institutions and/or a single professional group.5–7 While there is limited research that addresses critical perspectives of faculty who are on the ‘front-lines’ of curricular innovation, patient safety and quality improvement training literature continues to recognise the critical role faculty play in moving the patient safety agenda forward and in the development of safe practitioners.5

Given increased demands for patient safety competency among health professionals (HP) at entry to practice, we recently conducted a large, cross-sectional survey of new graduates in medicine, nursing and pharmacy in the Canadian province of Ontario to gain insight on the extent of patient safety competency among new HPs. Our findings suggested that the culture of the clinical training environments in which we educate and acculturate new HPs seems to hamper patient safety learning.8 ,9 In the current study, we report on medical, nursing and pharmacy teaching faculty perspectives regarding the factors that influence curricular integration and the preparation of safe practitioners.

Methods

Sampling and data collection

At the outset of the project, our goal was to gain insight into the integration of patient safety content into undergraduate curriculum from regular faculty members (ie, faculty members who do not have a specific role as patient safety champions in their respective programmes). To identify a sample of nursing and pharmacy faculty to participate in the study, a list of full-time faculty members was compiled from the web sites of the two pharmacy and six university nursing programmes located in three geographic areas in southern Ontario (Southwestern Ontario, Greater Toronto Area, Eastern Ontario). A request to participate in the study, which outlined the purpose of the study and eligibility criteria (ie, faculty members with full-time academic appointment who spend at least 40% of their time teaching, preferably clinical courses), was sent by email to all faculty on the compiled list.

The following approach was taken to recruit medical faculty. First, given the breadth of specialties and the fact that most medical faculty are situated in care settings, we initially focused on academic departments of general internal medicine in the same three geographic areas listed above. We compiled a list of faculty members from the departmental web sites and sent two separate emails to their university email addresses requesting participation in the study. Given the lack of response to this approach, we requested participation from the undergraduate and graduate programme directors, and clerkship directors in the three geographic areas in southern Ontario.

The lead author responded to the email responses from all potential participants to determine eligibility and to schedule interviews. In total, 28 people responded to our request for participation (n=6 medicine; n=10 pharmacy; n=12 nursing), and interviews were scheduled with the first eligible participants from each discipline until we completed 20 interviews divided relatively evenly across the three disciplines. Twenty interviews are typically sufficient to achieve saturation10 ,11 which is the point in data collection when no new or relevant information emerges with respect to the question of interest.

Between May 2010 and February 2011 semistructured interviews were conducted with 20 faculty members engaged in HP education (n=6 medicine from three universities; n=6 pharmacy from two universities; n=8 nursing from four universities). Participants had over 15 years of practice experience in their respective professions. With the exception of the two part-time pharmacy faculty who were the only participants responsible for the integration of patient safety within the curriculum, all participants were engaged in undergraduate and postgraduate education. Consistent with the goal of our larger study, we were most interested in gaining perspective about education regarding the sociocultural aspects of patient safety central to a number of key patient safety competency frameworks12 (ie, working in teams, recognising and responding to adverse events, culture of safety). The questions used to guide the interviews focused on participants’ experience with teaching patient safety (see box 1). The semi-structured interviews were conducted in person by the lead author (DT), lasted approximately an hour, and were audio recorded and transcribed verbatim. Written consent was obtained from each participant. The study received approval from the Human Participants Review Committee in the Office of Research Ethics at York University in Toronto and at Queen's University (Health Sciences Research Ethics Board).

Box 1

Interview guide

  • As you consider the preparation of safe practitioners in your discipline (ie, physician, nurses, pharmacists), what are the key challenges for safe (medical, nursing, pharmacy) practice?

  • How do you support student learning about the delivery of safe care (policies and standard practices) and sociopolitical aspects of patient safety (communicating effectively, managing safety risk, understanding human and environmental factors, recognising and responding to adverse events and culture of safety)?

  • How is patient safety integrated into your curriculum?

  • What factors contribute to the integration of patient safety, especially the sociopolitical aspects (listed above) into preregistration curricula?

Analysis

We initially read through the interview transcripts to comprehend its essential features. Preliminary analysis involved open coding to generate a range of key themes that emerged from the data. The initial codes were then organised into provisional categories to build a coding framework divided into major themes and subthemes.13 Two authors conducted all aspects of the coding (DT and BC), first independently and then categories were compared and discussed until consensus was reached. Thematic categories were all induced from the data; however, discussion with other research team members during data analysis ensured data were triangulated with other quantitative study data12 and helped to generate a comprehensive and holistic account of the complexities associated with integrating patient safety into undergraduate health professions curricula.

Findings

Emergent themes

Seven themes emerged in three areas: (1) Challenges to Safe Practice: (i) clinical safety areas; (ii) priority setting (see table 1); (2) Challenges in Preparing Safe Practitioners: (iii) culture of the practice setting, (iv) formal versus informal teaching, (v) faculty preparation, (vi) authenticity (see table 2); (3) Faculty Concerns: (vii) academic-practice gap (see table 3).

Table 1

Challenges to safe practice themes and narrative data

Table 2

Challenges in preparing safe practitioners themes and narrative data

Table 3

Faculty concerns categories and narrative data

Challenges to Safe Practice

Clinical safety areas. Faculty identified a variety of disciplinary focused challenges to safe practice. Pharmacists and physicians identified a narrow band of key patient safety issues, with pharmacists unanimously focusing on safe medication practices, and physicians focusing on communication between medical staff and communication with other members of the multidisciplinary team. By contrast, nursing faculty identified a broader set of threats to patient safety, from the fast pace of care, including rapid changes in patient condition and doctors’ orders, to intraprofessional and interdisciplinary communication and conflict. Pharmacy faculty described the dilemma associated with equipping their students to transition into practices, described as ‘working in the trenches’, and of the increased accountability to prepare future pharmacists to be agents of change. This was seen as especially important for students who eventually practice in proprietary pharmacies, which were described as lagging in patient safety infrastructure and improvement. Pharmacists also spoke of the rapid change in medical practice and the challenges faculty and students face maintaining knowledge of technical and therapeutic advances.

Priority setting. Study participants also discussed competing demands and priority setting in undergraduate curricula that are already overcrowded. In terms of limitations of current curricula, pharmacists emphasised limited exposure of medical students to pharmacology and toxicology content, while nursing faculty tended to talk about the need for expanding nursing curriculum in their personal areas of clinical expertise (eg, gerontology, mental health). Both pharmacy and medical faculty indicated that external accrediting bodies are increasingly requiring the inclusion of patient safety content in undergraduate curricula as well as in postgraduate training programmes in medicine. Specifically, physicians spoke about the requirement outlined by The Royal College of Physicians and Surgeons of Canada in the CanMEDs framework (http://www.royalcollege.ca/portal/page/portal/rc/canmeds/framework), while pharmacists spoke about the need to integrate patient safety into new curriculum to achieve the educational outcomes defined by the Association of Faculties of Pharmacy of Canada. Additionally, medical faculty expressed concerns about the pedagogical approach to medical education and the importance of helping students develop critical thinking and information management skills. As stated by one physician, ‘we have to stop shovelling information and help them to learn how to detect, investigate and solve a problem’.

Challenges in preparing safe practitioner 1

Culture of the practice setting. Participants identified a number of environmental and situation-based realities that complicate the preparation of safe practitioners. Study participants spoke about how ‘real learning’, or the integration of theory into practice required for safe, competent practice, takes place in clinical practice settings, and they affirmed the importance of a strong patient safety culture in the teaching and learning environment. Nursing faculty are fully aware of the differences that exist in safety culture in the various sites that are used by their programmes for clinical education; as described by one faculty member, some sites have a culture of ‘hide and don't reveal’ where staff ‘bury and don't talk about critical issues’. In other settings, students may experience hyper-vigilant staff, where ‘everything is picked up on and staff can be quite brutal to students’. Nursing faculty noted that students witness unsafe practices and unsafe care in some clinical settings that differ from what they are taught in the classroom setting. Nursing faculty also noted that some safety practices, such as medication error reporting, vary from site to site and that it's difficult for students and clinical instructors to know the expectations in different sites.

Noting the difference between the culture of the classroom and the practice setting, pharmacy faculty suggested that students have a higher level of comfort in the classroom, where they can engage in dialogue rather than in some clinical settings where they report being intimidated by medicine. The notion of intimidation by medical staff was also raised by medical participants who suggested that ‘hierarchical difference in medical knowledge’ might contribute to other HPs’ reluctance to question medical decisions. It may also boil down to what one physician stated: ‘some people are just more approachable than others’. Physician respondents suggested that it is easier to address technical issues of patient safety (eg, infection control, hand hygiene) than more highly charged sociopolitical aspects of patient safety, such as communication, power and conflict.

Formal versus informal teaching. Participants described informal and formal approaches to patient safety teaching. First, medical and nursing faculty overwhelmingly portrayed patient safety as being integral to ‘all aspects’ of (informal) clinical education and practice. Nursing faculty described informal curriculum in the clinical practice settings as focused on technical elements of competent practice that underpin nursing care, such as nursing assessments, care planning, infection control, pressure ulcers and so on.

Medical faculty also focused on technical aspects of patient safety, citing issues, such as sterile barrier precautions, hand washing, prevention of venous thromboembolism, pressure ulcers and blood stream infections. Physician participants explained that much of the patient safety education they are involved with takes place informally during daily rounds, grand rounds, mortality and morbidity review, and interaction with students and residents on an individual as-needed base. They also spoke about the development of daylong workshops designed to address specific CanMEDS roles, such as health advocate and communicator.

A smaller number of participants spoke about the lack of awareness by colleagues about the systems perspective of patient safety, and of the challenges associated with changing formal curriculum, specifically the challenge of negotiating with ‘self interested’ colleagues who express limited interest and time to develop expertise outside their specialty focus. One pharmacy faculty member expressed that not all faculty need to have expertise in the science of safety, stating, ‘I would love to have a patient safety expert come into a clinical classroom course … who is an expert in … culture of safety, and this person could really talk to the students about the issues.’ By contrast with medical and nursing study participants, pharmacy participants focused on the way in which patient safety has been integrated into formal curricula. In both pharmacy programmes represented in this study, part-time faculty members who are experts in patient safety were specifically hired to integrate patient safety curriculum across the programme of study. A spiralling curricular approach has been used to integrate comprehensive content focused on the science of patient safety (ie, based on the Canadian Patient Safety Institute patient safety Competencies Framework), in which constructs are introduced in the first year and re-examined each subsequent year in increasing depth and complexity. Formal curricular content is delivered by safety content experts.

Faculty preparation. Recognising that the quality of the student experience is highly dependent on the pedagogical approach and faculty knowledge of patient safety, study participants raised concerns about the extent to which current faculty members and clinical preceptors are adequately prepared to teach and mentor in the area of patient safety. In the same way that there is variation in safety culture in clinical practice settings, faculty suggest a high degree of variation in clinical preceptors, noting that some are well prepared and are committed to student learning, while others are not fully engaged with students and do not participate in in-services and workshops to enhance their competencies in either content or pedagogy. It is important to note that, only when prompted did medical and nursing faculty we interviewed speak of the need for more formalised teaching about the science of patient safety, teamwork and conflict (and then they spoke of the need for faculty development to address these issues). Neither nursing nor medical faculty knew of specific patient safety courses, either required or elective, within their respective curriculum and faculties. One pharmacy faculty member spoke about how the interprofessional education department at their university was developing an optional patient safety elective.

Authenticity. Study participants suggested that the classroom fails to provide an authentic learning experience for HP students. Recognising the challenges associated with developing competencies related to interdisciplinary communication, conflict, power and teamwork, most of the respondents suggested that these issues are best addressed in the clinical practice setting where concrete examples can be used. And while noting that simulation is increasingly used to augment clinical education for nurses, nursing faculty spoke of challenges creating simulations that accurately reflect the complexity of the practice environment, and noted that students are more concerned about their performance in the clinical setting than in the simulated environment, because if they don't do something right in clinical practice, they will ‘get kicked out’, whereas, if they don't do something in the classroom, ‘what is going to happen?’

Faculty concerns

Academic-practice gap. Only the nursing participants spoke of the gap between the classroom and clinical practice settings. In particular, they spoke of challenges maintaining their own clinical competence in the context of rapidly changing practice requirements. They also spoke of pressures, such as time constraints, for teaching in an already overcrowded curriculum, increased enrolments, growing competition for clinical placements, and lack of control and a sense of vulnerability around securing appropriate placement opportunities for their students. As one faculty member stated, ‘we can't expect too much from them or do anything that will make them mad at us, or we will run the risk of losing the placement.’

Discussion

This study begins to explore medical, nursing and pharmacy faculty perspectives on the integration of patient safety into HP curricula. Here we discuss three key findings. First, while we did not set out to highlight divergent disciplinary perspectives about patient safety education, our results suggest differences in perspectives on key challenges to safe practice. Simply stated, physicians emphasised personal responsibility for communication, pharmacists focused on the complexity of drugs, and nurses on the environment of care. Nursing and medical faculty spoke about teaching elements of competent practice, where pharmacy faculty focused on the integration of a much broader set of ‘safety science’ concepts (ie, sociopolitical) in formalised classroom teaching. Additionally, pharmacy's use of ‘safety experts’ recognises that their own faculty members may not be the only, nor necessarily the best, people to teach patient safety.6 Moving forwards, a learning culture focused on what the students ‘need to learn’ rather than depending on research faculty who teach what they ‘know and love’14 is likely warranted. This finding highlights that each discipline functions within highly specialised roles with discrete professional competencies and, as such, there are relatively few faculty with the knowledge and skills required to teach the breadth of patient safety-related content.15

Second, study findings demonstrate the way in which external regulatory requirements such as those set out in the Royal College Physicians and Surgeon's CanMEDs framework and the Association of Faculties of Pharmacy of Canada’ educational outcomes can be effective levers for priority setting and curricular change.16 The recent call by the Royal College of Physicians and Surgeons of Canada to embed patient safety into the accreditation standards for postgraduate medical education programmes provides one example. In the context of curricular change, in addition to the ‘forcing’ function of regulation, regulatory changes may also be necessary to ‘permit’ certain behaviours by innovative practitioners and decision makers who strive to make changes in settings where dominant cultures are more resistant to change.17

Third, our results show that the culture of patient safety in clinical teaching environments may thwart the preparation of safe practitioners. At the outset of this project, we were focused on gaining insight into the stated and formally endorsed curriculum, and while participants provided some information about formal teaching, the dominant discourse focused on informal influences in clinical practice settings. Participants’ descriptions of student experiences in clinical practice settings are consistent with work showing the clinical settings in which we train HPs are characterised by harmful power imbalances, disrespectful treatment, and faculty-learner incivility.18 This alerts us to the role of hidden curriculum, which refers to the processes and pressures that fall outside of the formal curriculum, and which are influenced by the predominant culture19 consists of shared tacit assumptions and reflects ‘the way things are done around here’.20 In the case of patient safety, the predominant cultures that influence teaching and learning are the professional cultures and the culture of the organisations within which clinical teaching and learning take place. Our findings suggest that the hidden curriculum is a thriving part of the HP’s teaching and learning environment. While faculty members need to examine the cultural assumptions and socialisation processes that impact patient safety teaching and learning, this analysis is neither easy nor emotionally neutral. Patient safety curriculum could, however, benefit from mechanisms that challenge professionals’ underlying values, assumptions and mental models. We propose interprofessional training opportunities for faculty and trainees in order to develop a common curriculum that will address professional culture differences, deliver safety education in a more ‘authentic’ environment and, hopefully, counteract harmful effects of hierarchical power dynamics that exist between different HP groups21 and that hamper learning and competence.

Practice implications

The absence of significant patient safety content in HP curricula is fairly clear in the literature. In the 10-year period since release of the 1999 Institute of Medicine report, curricula that aimed to teach residents or medical students about quality improvement or patient safety concepts identified in a systematic review22 were primarily from a single medical school or residency programme and had fewer than 10 contact hours. The recent Leape Foundation report suggests that substantive attention to safety and improvement science and the kind of sociocultural aspects of patient safety we inquired about in this study are ‘missing in part, or totally, from the typical medical school curriculum’.23 While the literature offers a number of patient safety curriculum frameworks and examples of curricular approaches, there is little evidence that education of other HP groups is any further ahead. Our data suggest some of the factors that are hampering patient safety in HP education.

In addition to strategies that address structural curricular issues (ie, greater number of faculty with expertise in patient safety24 and closer links between academic staff and managers in healthcare settings,25) strategies to address problematic aspects of informal and hidden curriculum will be essential for overcoming challenges we identified for preparing safe practitioners. Interprofessional faculty development, strengthened academic-service partnerships, and the use of dedicated clinical teaching units, all with an emphasis on promoting a strong and positive patient safety culture, may be useful for responding to negative aspects of hidden curriculum and may provide the basis for an innovative interdisciplinary clinical education model for enhancing patient safety competency development. This approach may help address the contextual influence of training in the clinical setting that weakens nurses’ confidence in key sociocultural aspects of patient safety.26

Finally, attention to broader healthcare contextual influences agreed to be critical for making healthcare safer will be important for improving the culture of the settings in which trainees gain their practical experience. Attention to strong patient safety focused leadership at all levels, including board,27 ,28 executive29 and departmental,30 continues to be among the most promising levers for creating a strong culture of safety. More practical recommendations and next steps required to facilitate curricular innovation are summarised in box 2. These focus on the policy and faculty levels. We suggest, for instance, establishing entry to practice patient safety competencies and incorporating assessment of those competencies into licensure exams, as this will compel HP education programmes to more fully address safety. On the faculty level, education around patient safety is what is needed most.31 ,32

Box 2

Next Steps for Patient Safety Curricular Innovation

  • Policy implications

  • Establish entry to practice patient safety competencies, and accreditation standards, for each health professional discipline.

  • Incorporate assessment of patient safety competency into licensure exams.

  • Examine current courses to determine where it is best to integrate patient safety content

  • Use existing patient safety curricular resources (eg, WHO's Multi-professional Patient Safety Curriculum Guide9).

  • Faculty implications

  • Identify faculty champions who will be responsible for curricular integration of patient safety

  • Conduct interprofessional faculty development workshops for both classroom and clinical practice setting educators

  • Create opportunities for faculty members and practice setting leaders to work together to enhance patient safety education in clinical settings

Limitations

The present study was limited to one Canadian jurisdiction and the results may, to some extent, reflect contextual factors that are not shared by other international jurisdictions. Moreover, variation in the structure of each discipline's education no doubt contributes to different perspectives identified in this study. While HP programmes at Canadian universities undergo accreditation and are accountable to prepare students to meet externally established criteria for entry to practice, they also experience significant independence with respect to curricular innovation in their approach to teaching and learning and the Canadian HP education and regulatory contexts are not dissimilar from most other western countries.

A second potential limitation is that our sample of interview participants represents a small convenience sample of those faculty members responsible for undergraduate education or those with a particular interest in safety. Their perspectives are, at the very least, likely to be more ‘safety informed’ than those of the typical HP faculty preceptor. It is unlikely that a more representative sample of HPs faculty would be as knowledgeable about, or aware of, issues pertaining to patient safety in HP education. Indeed, the further away we move from practitioners with knowledge of safety science, the more the conversation tends to focus on technical safety, such as safe medication practice, hand hygiene and other infection control practices. Further examination is needed to determine why we had such a difficult time recruiting medical faculty to participate in a study of patient safety in HP education. Nonetheless, to our knowledge, the present study is unique in its exploration of the faculty perspective, and helps inform our understanding of formal and informal factors that influence the preparation of safe practitioners. Future research is required to better understand how faculty can address these factors, in both the classroom and clinical settings, in order to enhance their students’ patient safety competence and, ultimately, improve patient safety outcomes.

Conclusions

Continued integration of patient safety into HP curricula is critical for the preparation of safe HPs, yet the extent to which patient safety competencies will become embedded into HPs’ education depends on whether we build capacity among faculty who are engaged in both formal and informal teaching. It will also depend on recognition that curricular change is very slow, and that widespread change may only happen through leveraging accreditation and regulatory requirements. We propose a major effort to engage interprofessional faculty at all levels and in all clinical teaching settings to harmonise patient safety curricular priorities and educational opportunities across HP training programmes. Additionally, curricular innovation needs to engage healthcare leaders at all levels and practices settings, who recognise the potentially damaging effects of the hidden curriculum and who are committed to improving the culture of patient safety in the teaching and learning environments.

Acknowledgments

The authors also wish to thank Dr Shenda Tanchak, Dr Dan Faulkner, Heather Campbell, Anne Resnick and Della Croteau from the Colleges of Physicians and Surgeons of Ontario, the College of Nurses of Ontario and the Ontario College of Pharmacists who acted as decision makers for the grant. We also wish to thank Dr Jennifer Medves, Associate Dean Faculty of Health Sciences and Director of the School of Nursing, Queen's University for her support for the grant. We thank the editors and reviewers for providing feedback that improved the manuscript.

References

Footnotes

  • Contributors DT designed the study, collected and analysed the data, drafted and revised the paper. She is the guarantor. LG, PN and contributed to the study design, reviewed the data analysis, approved the first manuscript, assisted with revisions. BC managed data transcription and correction, assisted with data analysis, contributed to manuscript writing and revision.

  • Funding This study was funded by a research grant from the Canadian Patient Safety Institute, RFA09-1181-ON. DT was supported with an Ontario Ministry of Health and Long Term Care Senior Nurse Research Award (2009–2012).

  • Competing interests None.

  • Ethics approval The study received approval from the Human Participants Review Committee in the Office of Research Ethics at York University in Toronto and at Queen's University (Health Sciences Research Ethics Board).

  • Provenance and peer review Not commissioned; externally peer reviewed.