Article Text

The ins and outs of change of shift handoffs between nurses: a communication challenge
  1. John S Carroll1,
  2. Michele Williams2,
  3. Theresa M Gallivan3
  1. 1MIT, Sloan School of Management, Cambridge, Massachusetts, USA
  2. 2Industrial and Labor Relations School, Cornell University, Ithaca, New York, USA
  3. 3Patient Care Services, Massachusetts General Hospital, Boston, Massachusetts, USA
  1. Correspondence to Professor John S Carroll, MIT, Sloan School of Management, 100 Main St, E62-318, Cambridge 02142, MA, USA; jcarroll{at}mit.edu

Abstract

Background Communication breakdowns have been identified as a source of problems in complex work settings such as hospital-based healthcare.

Methods The authors conducted a multi-method study of change of shift handoffs between nurses, including interviews, survey, audio taping and direct observation of handoffs, posthandoff questionnaires, and archival coding of clinical records.

Results The authors found considerable variability across units, nurses and, surprisingly, roles. Incoming and outgoing nurses had different expectations for a good handoff: incoming nurses wanted a conversation with questions and eye contact, whereas outgoing nurses wanted to tell their story without interruptions. More experienced nurses abbreviated their reports when incoming nurses knew the patient, but the incoming nurses responded with a large number of questions, creating a contest for control. Nurses' ratings did not correspond to expert ratings of information adequacy, suggesting that nurses consider other functions of handoffs beyond information processing, such as social interaction and learning.

Discussion These results suggest that variability across roles as information provider versus receiver and experience level (as well as across individual and organisational contexts) are reasons why improvement efforts directed at standardising and improving handoffs have been challenging in nursing and in other healthcare professions as well.

  • Safety culture
  • decision making
  • transitions in care
  • patient safety
  • significant event analysis
  • critical incident review

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

Communication quality is a key requirement of effective interdependent work processes in complex work settings such as hospital-based healthcare.1–3 Communication breakdowns were implicated as root causes in over 80% of the sentinel events voluntarily reported by hospitals in 2010.4 Bates and Gawande1 note, ‘failures of communication, particularly those that result from inadequate ‘handoffs' between clinicians, remain among the most common factors contributing to the occurrence of adverse events' (p. 2527). Handoffs often receive among the worst ratings on safety climate.5 At our research site, a large urban teaching hospital, communication breakdowns were identified as a contributing factor in 31% of asserted malpractice claims (Hanscom R, personal communication, 2004). In this paper, we report results of a multi-method study of one communication practice: change of shift handoffs between nurses.

Shift report handoffs require technical communication, that is, the transmission of information about a patient relevant to their condition and care during the next shift. This represents the typical understanding of the primary function of handoffs as information processing.6 Although it seems obvious that a ‘good handoff’ should cover active medical issues, as well as the personal and familial issues relevant to the plan of care, research literature offers no uniform or standardised way of giving a report.6–9 There is some necessary variability due to the content and complexity of the handoff, the professional knowledge and norms of the parties involved, the physical setting and available resources, and additional goals such as learning within the handoff. The Joint Commission10 is advocating handoff standards, as exemplified by templates such as Situation-Background-Assessment-Recommendation,11 but tailored to the specific needs of the unit or work group.12

The quality of communication is not simply equivalent to transmission of technical facts, but also includes all interpersonal behaviours that help create an effective conversation and productive relationships among coworkers. We adopt an approach to handoffs that considers functions in addition to information processing, such as cross-checking assumptions, enacting social interaction and support, transmitting shared norms, and sharing learning.6 ,13

Behaviours that have been labelled relational communication14 contribute to good technical communication by creating perceptions of psychological safety, trust and respect that encourage sharing and learning,15 and positive energy that combats burnout.16 Aspects of relational communication include verbal statements (eg, asking for questions), vocal characteristics (eg, voice tone, turn taking) and non-verbal behaviours (eg, smiles, head nods). For example, both the Joint Commission Handbook17 and the collaborative WHO-JCAHO brochure18 advocate that handoffs include the opportunity for questioning. However, when relational communications are poor, a ritual request for ‘any questions?’ is not likely to improve handoffs.19 Too many questions, irrelevant questions or mistimed questions can be annoying.20 Yet, efforts to standardise technical communication could have unintended consequences for both technical and relational communication, for example, substituting audiotapes or electronic medical records for in-person interaction could minimise attention to unusual information, discourage questions, and reduce opportunities for perspective-taking, trust-building and learning.6

Method

Setting and overview

We conducted a multi-method study on two general medical/surgical units of a large, urban teaching hospital. Each unit had approximately 25 beds and 6–9 nurses per shift. Patients needed high levels of nursing care: 90% or more of the patients whose handoffs we studied required medication management, fluid management, pulmonary management, cardiac and neurologic management, educational intervention, and/or assistance with activities of daily living. Human subjects approval for the study was granted by the hospital Institutional Review Board.

To provide background information about shift report practices, we first conducted individual interviews with nurses on one of these units, and then used the other unit to collect data from: (a) a survey questionnaire to nurses, (b) audio taping of handoffs, (c) direct observation of these handoffs, (d) posthandoff questionnaires to nurses and (e) coding of clinical problems from the nursing records associated with these patient handoffs. This latter unit had a large nurses' lounge in which the nurses congregated to give report at 07:00, 15:00, 19:00 and 23:00 (most nurses had 12-h shifts from 7:00 to 19:00 or 19:00 to 7:00 but some had 8-h shifts). Each outgoing nurse handed off 3–5 patients, generally 1–2 to each of multiple incoming nurses.

Interviews

On the first unit, we conducted 30-min interviews with 12 nurses, seven of whom we classified as more experienced (6 or more years as a Registered Nurse (RN)) and the other five were less experienced, using categories from Benner.21 The interviews used a critical incident technique22 to enquire about a recent handoff that ‘had gone well’ and then a recent handoff that ‘had not gone as well.’ Following responses and probes, each nurse was asked to generalise about what makes for a good handoff and to give any other comments.

Survey questionnaire

Several weeks before starting direct observation of change of shift handoffs in the second unit, a paper-based questionnaire was given to the 28 nurses in the unit (out of 34) who had given consent to participate. The questionnaire took approximately 30 min to complete, and included basic demographic information (eg, years of experience as an RN) as well as other items that are not reported in this paper.

Audio taping and direct observation

We directly observed and audio taped 77 handoffs (ie, 77 patient transitions-in-care) during 40 shift changes. For each shift change, one outgoing nurse who had consented to participate was observed giving handoffs to one or more incoming nurses who had also consented to participate. Although 28 nurses consented to be in the study, only 23 were observed during shift report: 21 were observed as incoming nurses, 15 as outgoing nurses and 13 as both.

For each handoff, both incoming and outgoing nurses wore a special tape recorder with microphones hung from one ear. The tapes were transcribed for content coding of medical issues discussed and adequacy of that discussion, as well as questions asked by each nurse. The tape recorder was a computer prepared by the MIT Media Laboratory in order to be able to code paraverbal information (voice tone, turn taking, etc), separate analyses of which are reported in Waber et al.14 Additionally, one of the first two coauthors was in the room to code non-verbal behaviour (gaze direction, eye contact, joint object focus, smiling, frowning, head nods and shakes, and hand gestures) on a coding form marked off in minutes, which allowed us to assess respectful body language and signs of engagement in the conversation.6

Posthandoff questionnaire

At the end of a handoff or at the end of the shift report, whichever was convenient for the nurse, each observed nurse answered a one-page questionnaire about each handoff she or he had participated in. The questionnaires included ratings of handoff effectiveness and items modified from measures of Psychological Safety15 and Burnout.23 The incoming nurse was also asked if she or he had prior knowledge of the patient (the incoming nurse questionnaire is shown in online appendix A). The questionnaire usually took about 1 min per handoff to complete (eg, if an outgoing nurse had been observed for two handoffs, she or he would complete two questionnaires in 2 min).

Coding of nursing records and transcribed handoffs

An advanced practice RN examined the nursing records for each patient handoff from 48 h prior to the handoff up to the time of the handoff. The records were coded for patient acuity, the hospital's estimate of the nursing resources needed to care for the patient and active medical issues. The 48 h time period was judged sufficient to identify active issues that should have been discussed during the handoff, without going back so far in time that issues would no longer be relevant or there would be unnecessary effort for the coders. This RN also coded the transcribed audiotapes of the handoffs for adequacy of discussion of these medical issues on a 3-point scale (not mentioned, discussed insufficiently, discussed adequately). A second advanced practice RN then reviewed the transcript and analysis and either agreed or disagreed with the proposed rating. Transcripts were also coded by a research assistant for the number of questions asked by the outgoing and incoming nurses.

Results

For narrative clarity, and because many of the important results examine variables across the multiple methods, we first present an overview of the handoffs and then work backwards from effectiveness measures to their antecedents. In particular, the interviews appear in several places to support results from other methods.

Handoff overview

Interviews on the first unit revealed the variability of handoffs across units, nurses and prior experience with the patient. Nurses told us that medical units differed on the availability and size of rooms in which to give report, and that nurses in another medical unit gave a report via tape recorder rather than face-to-face. Individual nurses also have their own way of giving report, often from patterns learnt in nursing school (eg, some go head-to-toe).

On the unit we observed, handoffs averaged 5.4 min per patient, with a range from 2 to 13 min. Typically, the incoming nurse sat at a table reading from the clinical record in a loose-leaf binder and taking personal notes on a single sheet of paper that was later folded and carried during the shift. The outgoing nurse sat or stood next to the incoming nurse and spoke from memory and/or from her or his own personal notes about the patient. Outgoing nurses usually were made aware at the start if the incoming nurse knew the patient by either asking or when incoming nurses volunteered that information. The outgoing nurse did most of the talking, looking towards the incoming nurse, who was writing notes and scanning the clinical record. From time to time, there would be a notable bit of information that would cause the incoming nurse to look up and make eye contact, or the outgoing nurse would signal with a louder voice tone or by touching the incoming nurse that the incoming nurse should attend to this information. Over a third of outgoing nurses asked at the end if there were any questions. Incoming nurses asked 80% of all questions.

Handoff effectiveness measures

We used two measures of handoff effectiveness, self-report and expert-coded. Each nurse answered the posthandoff questionnaire that included agreement or disagreement with the statement, ‘This handoff was effective.’ Note that these ratings capture the nurses' own framing of the functions of the handoff. As shown in table 1, self-reported ratings of effectiveness were quite high (means of 6.72 and 6.49 on 7 point scales where 7=strongly agree). However, ratings by incoming and outgoing nurses did not correlate (r=−0.07). Note that since the unit of measurement is the handoff but nurses appear multiple times in the data, we have chosen not to report statistical significance.

Table 1

Means, SDs and correlations of handoff variables

For the expert coding of handoff effectiveness, we compared active medical issues identified in the nursing records up to 48 h prior to each handoff with discussion of these medical issues in the handoff. Note that this measure of effectiveness is based solely on the information processing function of handoffs. The two advanced practice RNs who coded the handoff transcript for adequacy of discussion of these medical issues agreed on the proposed rating for 250 of 263 issues (95%); 10 of 13 disagreements were resolved by discussion between the two RNs and the remainder were resolved by the clinical research team.

Of the 263 active medical issues identified by our coders from the clinical records, a third were not mentioned in the handoffs, and only 26% of the active medical issues were presented adequately. We calculated handoff effectiveness by averaging across the active issues (each issue was scored 1 if not mentioned, 2 if mentioned but insufficiently discussed and 3 if adequately discussed). This effectiveness score did not correlate with ratings of effectiveness by incoming or outgoing nurses (r=−0.07 and −0.09, see table 1), suggesting that perceived effectiveness was based on factors other than technical information content, as we explore in the next section.

Attributes associated with self-rated handoff effectiveness

In order to understand the differences in ratings of effectiveness by incoming and outgoing nurses, we examined the correlation of those ratings with other ratings on the posthandoff questionnaire, measures of non-verbal behaviour, handoff duration and coding of questions asked during the handoff. Considering the posthandoff questionnaire responses, for both incoming and outgoing nurses, effectiveness correlated strongly with the same three questions (see online appendix A): ‘I felt positive about this handoff’ (r=0.73 and 0.66), ‘I felt comfortable enough to speak up if I perceived a problem during this handoff’ (r=0.70 and 0.62) and ‘I felt a positive connection with the other nurse during this handoff’ (r=0.66 and 0.59). Note that these individual questions are not included in table 1, but the three questions form a reliable scale of ‘Positive Relationship’ (α=0.84 and 0.74 for incoming and outgoing nurses) that, naturally, has a strong correlation with effectiveness ratings (r=0.79 and 0.76, see table 1). The next strongest question was ‘I had all the information I needed’ (r=0.52 and 0.49), indicating that technical communication was important but not as important as the overall sense of the relationship during the handoff.

Non-verbal behaviours were coded per minute; handoffs ranged from 2 to 12 min. We examined averages across the entire handoff and also across only the first 3 min of each handoff (except for two handoffs that were averaged across 2 min). Since these averages were highly correlated and the results changed very little with either average, we report results from only the first 3 min of coding because research has shown that ‘thin slices’ or short observations of behaviour can often be more accurate than longer time periods24 and we thought this would focus the analysis on more comparable aspects of non-verbal interaction. For incoming nurses, higher effectiveness ratings were most strongly associated with more eye contact (r=0.36). For outgoing nurses, higher effectiveness ratings were associated with the incoming nurse gazing less at the outgoing nurse (r=−0.43), fewer questions asked by the outgoing nurse (r=−0.39), less eye contact (r=−0.39), less joint object focus (typically, looking together at the nursing record, r=−0.36), shorter handoff duration (r=−0.28) and fewer questions asked by the incoming nurse (r=−0.23).

Some of the above relationships are supported by data from the interviews on the other unit. Four of the 12 interviewees asserted that a good report was associated with fewer questions being asked, for example, ‘you know you have given a good report when the nurse doesn’t have to ask many questions'; three of these four were more experienced nurses, whereas less experienced nurses seemed to be more concerned with distractions, for example, ‘the worst part is when people are in a rush to go home and things are left out.’

Attributes associated with expert-rated handoff effectiveness

Ratings of effectiveness from comparing transcripts and medical records showed few relationships with questionnaire data, non-verbal data or other indicators. There was a significant association with outgoing nurse identity (F (14,46)=1.92, p<0.05), which may reflect different styles of giving handoff including narrative features, relational behaviours and general likeability. Scores were slightly higher when incoming nurses asked fewer questions (r=−0.20).

Incoming nurse knowledge of the patient

In our interviews on the first unit, nine of the 12 nurses mentioned that reports are shorter if the incoming nurse already knows the patient (Cohen and Hilligoss label this ‘continuing’ vs ‘new’ patient transfer7). For example, one nurse stated, ‘if she knows the patient you don’t go to every single detail, you just give an update.' Consistent with the interviews, our direct observations showed that outgoing nurses shortened their handoff presentations when the incoming nurses knew the patient. In particular, when we split incoming nurses by experience level as RNs (5 years or less vs 6 years or more), we found that only the more experienced nurses were making these adjustments. As shown in table 2, less experienced outgoing nurses did not vary in handoff adequacy or questions received. Experienced outgoing nurses gave less adequate handoffs and received five times as many questions when the incoming nurse knew the patient. Indeed, the ranges of questions asked were non-overlapping: between 0 and 5 when the incoming nurse did not know the patient, and between 8 and 19 when the incoming nurse knew the patient. The pattern of results is similar if we restrict the data to 53 handoffs, eliminating all but the first handoff from the same pair of nurses during the same change of shift.

Table 2

Expert-coded adequacy of handoff, number of questions asked per minute by incoming nurse, and length of handoff as a function of outgoing nurse experience and incoming nurse knowledge of the patient

Box 1 gives a sample set of questions from an incoming nurse who knew the patient to an experienced outgoing nurse. The questions seem to focus on the details of executing the plan of care (‘Did he still have the patch on?’) and preparing to deal with other caregivers (‘Is she a new Intern?’), rather than on critical medical conditions.

Box 1

Questions asked by an incoming nurse (Handoff #17)

  1. What time? I'm not drawing it again.

  2. Who is it? Is she nice?

  3. Did you get a [inaudible] draw?

  4. Oh, he has no access whatsoever?

  5. Normal saline at 150?

  6. Okay, they're piggy backed in?

  7. They're compatible?

  8. He needs another line on top of that?

  9. Is she a new Intern?

  10. Did he still have the patch on?

  11. Huh?

  12. Did he still have the patch on when he left?

  13. Did you give him Tylenol?

Discussion

The effectiveness puzzle

The concept of ‘an effective handoff’ is surprisingly elusive. Cohen and Hilligoss7 state that the ‘many research reports that have accumulated do not converge on any simple characterization of a good handoff’ (p. 37) and Riesenberg et al8 agree ‘there’s little empirical evidence delineating what constitutes best handoff practices' (p. 30). Although we may expect different hospital units and different professions to structure handoffs differently, in this hospital incoming and outgoing nurses seemed to want different things from handoffs, even though 12 h later they reverse roles! Further, neither of their ratings correlates with our expert-coded measure of handoff adequacy. Perhaps this is less surprising when we consider that the experts were coding only technical communication, that is, the factual clinical content of the handoff. The nurses' ratings of handoff effectiveness were much more strongly associated with the experience of a positive relationship during the handoff.

A communication challenge

Our results show that incoming and outgoing nurses experience the handoffs very differently. Incoming nurses prefer handoffs that conform to most theories of effective communication. They appreciated more eye contact and nodded more when they felt the handoff was more effective. For example, a less experienced nurse in our preliminary interviews said that handoffs were ‘good when they are good communicators, someone who looks you in the eye and don't [sic] get distracted.’

However, outgoing nurses appear to want to give their story in their way and go home. They rated handoffs as less effective if there was more eye contact, more gaze directed towards them by the incoming nurse, a longer handoff and more questions asked. The overall impression is that when the incoming nurse was more active in the handoff, it disrupted the flow preferred by the outgoing nurse, lengthened the handoff and generated more questions. Consistent with some of our preliminary interviews, when the incoming nurse is asking more questions, the outgoing nurse feels that something is going wrong with the handoff. As one experienced outgoing nurse said, ‘you are tired… made me feel like why is she asking this stuff, is she trying to trip me up or is she really interested?’

Experienced outgoing nurses in particular seemed to abbreviate the handoff when the incoming nurse knew the patient. But the result was an explosion of questions from the incoming nurse, which suggests that the experienced outgoing nurses were overcompensating for the assumed knowledge of the incoming nurse. As a result, these handoffs became contests for control: the outgoing nurse tried to present a brief report but this frustrated the incoming nurse whose knowledge of the patient enabled even more question asking, which in turn frustrated the outgoing nurse who was trying to tell a succinct story but was repeatedly interrupted.

Implications for practice

Although the research literature suggests that it will be easier and better when nurses give report to others who already know the patient, due to shared mental models,7 we find that these are exactly the situations that may create conflict. Incoming and outgoing nurses rate the reports highly, but they do not agree on which handoffs are more effective, and coding of clinical records suggests that a lot is being left out. Outgoing nurses are placed in an awkward situation of staying late to give report after an 8- or 12-h (or longer) shift. Experienced outgoing nurses shorten their report to nurses who know the patient, but they seem to overshoot by making it too short, which prompts the incoming nurse to ask questions to fill out their understanding, while the outgoing nurse struggles to finish. This seems related to a more general phenomenon that speakers systematically overestimate what listeners understand.25 ,26

Standardisation of some sort is likely to help, especially around the different expectations of incoming and outgoing nurses. Recognition of the problem, and ways to alleviate the demands on the outgoing nurses, could help. Given that nurses are not typically aware of these different expectations, more discussion is needed among nurses and managers about where and how to standardise and where to allow or support variation (such as with more complex patients, less experienced nurses).

However, standardisation can create additional problems. For example, subsequent to our research, this hospital changed the nursing shift report process to take advantage of electronic nursing records. Outgoing nurses now enter patient information into the computer with a standard data format during their shift (and not necessarily at the very end of the shift when they are under time pressure to leave). When incoming nurses arrive, they go to the computer terminals and read about their assigned patients. Before outgoing nurses can leave the building, they are required to ask the incoming nurses taking their patients if they have any questions. Although the new process provides a clear structure with more documentation and reduces the time that overlapping shifts are away from the patients, a pro forma request for questions may not produce effective verbal communication.27 The benefits and challenges of this new handoff process have yet to be formally evaluated.

Limitations

This is an exploratory study of two medical units of one hospital, with its particular norms and patient population. We studied a modest number of nurses and a modest number of patient handoffs. Calculation of statistical significance is complicated given that the same nurses were studied in repeated handoffs, sometimes as incoming nurses, sometimes as outgoing. We present the pattern of data in an exploratory way rather than as a statistical test of hypotheses.

Our survey measures and observations of handoffs were intrusions into the work pattern of the nurses. Approximately a quarter of the nurses declined to participate. Our sense is that our presence did not change the actual shift report work, but even a couple of minutes of extra time spent fiddling with tape recorders and answering posthandoff questions were competing for precious work time.

Our self-report and expert coded measures of handoff effectiveness were both limited in effectiveness. Given that self-report was measured immediately after the handoff, it could not reflect how the nurses would have evaluated the handoffs at the end of shift after discovering what necessary information they did and did not receive. From an information processing viewpoint, a postshift rating might correspond better to the expert coding of handoff effectiveness.

That said, the handoff process is of growing interest precisely because it is so widely utilised throughout healthcare, including physicians handing off across shifts and across departments and inter-profession handoffs from inpatient care to outpatient care, physicians to nurses or physical therapists, and so forth. We believe our results transfer to different contexts because they reflect the basic behavioural processes underlying workplace communication, with a particular emphasis on relational communication.

Conclusion

Shift report handoffs are not like typical conversations, which have a symmetry that our concepts of good communication anticipate. There is tremendous asymmetry between the roles of giving and receiving a report. The outgoing nurse has the information to transmit, and the incoming nurse is taking over responsibility for care of the patient. The incoming nurse is multi-tasking to read the written documentation and also hear the outgoing nurse's report. Each nurse is also distracted, the incoming nurse by the need to get to the patients' bedside and the outgoing nurse by the need to get home.

What incoming nurses value in a handoff conforms to our expectations for good communication, specifically, eye contact and opportunity for questions, but these same features are experienced as interruptions and problems for the outgoing nurse who wants to transfer care and get home. We find a conflict emerging when an experienced nurse gives a short report to an incoming nurse who knows the patient from a prior shift, assuming that the incoming nurse already knows many of the details. Yet, the incoming nurse reacts by asking a large number of questions, and the interaction becomes a contest for control.

Our work reinforces the conclusions of Cohen and Hilligoss,7 Patterson and Wears,6 and Riesenberg et al8 that there is little current agreement on what constitutes a good handoff or on how to standardise handoffs to increase effectiveness. More needs to be done to improve handoffs and to learn from the many innovations now being tried. Our research suggests that efforts to standardise handoffs also should focus beyond the technical information content. Neither incoming nor outgoing nurses' ratings of effectiveness corresponded to expert ratings of technical adequacy of the handoffs. We must be aware of relational communication practices that facilitate transfer of clinical information, development of productive working relationships and creation of a culture that supports effective learning.

Key messages

  • There is considerable variability in nursing change of shift handoffs.

  • Incoming and outgoing nurses see the value of handoffs differently, even though each play both roles repeatedly, and neither nurses' perceptions correspond to expert ratings of information adequacy.

  • More experienced outgoing nurses abbreviate handoffs when incoming nurses know the patient, while incoming nurses who know the patient ask more questions, creating a contest for control.

  • Efforts to standardise handoffs or to pair experienced and less experienced nurses for learning need to recognise the conflicting role demands of handoffs.

  • Relational communication competencies support effective exchange of information and productive working relationships.

Acknowledgments

The authors thank Carol Camooso, Scott Ciesielski, Katie Farraher, Joan Fitzmaurice and Susan Morash for their assistance in collecting the data, and Kate Barba, Vincent Chen and Matias Raby for assistance in coding data. We also thank Adrian Bangerter, Paul Barach, Marlys Christianson, Julie Johnson, Emily Patterson, Sara Singer and Robert Wears for comments on earlier drafts.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Files in this Data Supplement:

Footnotes

  • Funding The work was supported by a seed grant for exploratory research from CRICO/RMF (part of Partners hospitals), but they had no role in the design and conduct of the studies.

  • Competing interests None.

  • Ethics approval Ethics approval was approved by Massachusetts General Hospital

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement There are some additional data from the preobservation questionnaire that is intended to be in another paper, so it is not available at present. The only other data available from this paper are either highly redundant or not very useful. Contact John S Carroll for any requests.

Linked Articles

  • Editorial
    Robert L Wears