Article Text
PDF
Abstract
Background The ability of medical teams to develop and maintain team situation awareness (team SA) is crucial for patient safety. Limited research has investigated team SA within clinical environments. This study reports the development of a method for investigating team SA during the intensive care unit (ICU) round and describes the results.
Methods In one ICU, a sample of doctors and nurses (n=44, who combined to form 37 different teams) were observed during 34 morning ward rounds. Following the clinical review of each patient (n=105), team members individually recorded their anticipations for expected patient developments over 48 h. Patient-outcome data were collected to determine the accuracy of anticipations. Anticipations were compared among ICU team members, and the degree of consensus was used as a proxy measure of team SA. Self-report and observational data measured team-member involvement and communication during patient reviews.
Results For over half of 105 patients, ICU team members formed conflicting anticipations as to whether patients would deteriorate within 48 h. Senior doctors were most accurate in their predictions. Exploratory analysis found that team processes did not predict team SA. However, the involvement of junior and senior trainee doctors in the patient decision-making process predicted the extent to which those team members formed team SA with senior doctors.
Conclusions A new method for measuring team SA during the ICU round was successfully employed. A number of areas for future research were identified, including refinement of the situation awareness and teamwork measures.
- ICU
- teamwork
- team situation awareness
- decision-making
- communication
- culture
- human factors
- shared decision-making
- teams
- teamwork
Statistics from Altmetric.com
- ICU
- teamwork
- team situation awareness
- decision-making
- communication
- culture
- human factors
- shared decision-making
- teams
- teamwork
‘Situation awareness’ (SA) refers to an individual's perception of the information within a task environment, comprehension of its meaning and anticipation of potential future states.1 When medical and nursing staff perform clinical work together, the development of shared and accurate SA between team members (termed ‘team situation awareness’) is important for patient safety.2 3 Team SA has been identified as especially important for the ICU owing to its reliance on multidisciplinary teamwork and complex patient populations.3–5 In particular, team SA is important during daily rounds, a task where ICU teams collaborate to review patients and share information pertinent to specific roles and care tasks (eg, daily goals).6 7 The SA developed during daily rounds will likely influence how team members monitor the patient, prioritise tasks and anticipate urgent events. Teams with mismatching SA for a patient's condition or expected development are susceptible to enacting uncoordinated and erroneous activities.
Team processes related to team members sharing information and perspectives underpin the development of team SA.8–13 Such processes are central to daily rounds; senior doctors lead decision-making by communicating and performing ‘sensemaking’ activities to collect information, diagnose illnesses and understand potential developments.14–16 Open communication is important for developing a shared understanding of patient care plans, and senior doctors influence trainee and nursing staff behaviour during the round.7 17–19 This resonates with aviation research showing teams with suboptimal SA to be characterised by poor information sharing, and the rejection of junior crew member contributions.9 10 20 Daily rounds may support the development of team SA by (1) team members perceiving and sharing patient information (eg, diagnosis, physiological data, team member opinions) and (2) team members interpreting information together to understand patient conditions (influenced by team, experience/expertise, prior knowledge of patient) and anticipate progression.
This article describes the development and trial of a measure of team SA for the ICU round. Although team SA appears important for patient safety,2 3 21 real-life healthcare research on this topic is minimal, with team SA being assessed through team observations.22 23 Although such methods have acknowledged limitations,24 developing non-observational measures of team SA for live environments is highly complex.25 Measurement tools should capture SA as teams perform taskwork (without disrupting performance), and then compare SA between team members. Ideally, SA should be measured against objective situational measures.26 We report on the results of a feasibility study to test the validity and potential utility of an ICU round-specific team SA measure. Furthermore, we explore team SA theory by investigating the relationship between team processes and team SA during patient reviews.
Method
Setting and participants
The study was based in a 16-bed ICU in a UK teaching hospital treating 800 patients per year (mean stay 4.8 days) and with a 21% ICU mortality. A convenience sample of 44 ICU care givers volunteered to participate, consisting of seven senior doctors (consultants), six senior trainee doctors (specialist registrars), 23 junior trainee doctors (basic specialty trainees) and eight senior nurses. No demographic information was collected. Data were collected over 3 months. Owing to shift patterns, the 44 participants combined to form 37 different teams (with overlapping personnel). Ethical approval was given by an NHS research ethics committee.
Procedure
Developing a measure of team SA
The team SA measure was developed with ICU senior physicians and senior nurses, and was intended to be used immediately after the discussion of a patient during the round. The measure assesses team member anticipations for the following four events on a likelihood scale of 0–10027: (1) the patient being discharged from the ICU during the next 48 h (discharge likelihood); (2) the patient deteriorating during the next 48 h (deterioration likelihood); (3) the patient remaining on (or requiring) ventilator support during the next 48 h (ventilation likelihood); and (4) the patient surviving (survival likelihood). Each anticipated event was designed to be comparable with objective patient outcomes, in order that the ability of team members to anticipate future events could be assessed against objective data. For example, discharge, ventilation and survival likelihood were retrospectively verifiable from ICU clinical audit systems. Deterioration likelihood was determined through the sequential organ failure assessment calculation, which is used to track illness severity trajectory.28–31 Compared with much of the SA literature, the focus upon anticipations was unusual, but not unique.21 32–35 Anticipations were investigated in order to (1) develop a standardised SA scale (irrespective of specific pathologies) that allows accuracy to be easily assessed (ie, by evidence of change), (2) make standardised comparisons between team members' SA and (3) avoid priming participant attention/SA by asking them to focus on specific data/illnesses. In addition, anticipating patient outcomes is considered important for ICU decision-making, with an established literature measuring clinical and nursing predictive accuracy.27 36–39
SA data collection
During the daily round, team member SA for each patient was measured immediately at the end of the discussion for that patient. Participants recorded their anticipations (using a PDA device issued to each team member) for discharge likelihood, deterioration likelihood, ventilation likelihood and survival likelihood, alongside their perceived involvement in the patient decision-making process. Structured observations also noted communication events during patient reviews, and patient outcome data were collected. Figure 1 outlines the full study procedure, and the section below describes the team observational measures and involvement question item in greater detail.
Study procedure for measuring team situation awareness and team processes during the intensive care unit (ICU) round.
Measuring team process
An observational protocol was designed to measure team communication events during each patient review (see table 1). First, the number of verbal communications by senior trainee doctors, junior trainee doctors and senior nurses was noted. Trainee and nursing communications during ICU rounds often focus on information provision and sharing,18 40 41 and higher numbers of verbal communications were expected to indicate greater information sharing between team members (potentially enhancing team SA). Second, the number of prompts by senior doctors for contributions from trainees and nursing staff was noted. These were expected to be significant for developing team SA (eg, promoting knowledge sharing, highlighting knowledge gaps) and are considered important for developing patient treatment plans.42 Communication frequencies (and not content) were noted in order to limit the workload associated with managing both the SA and team data-collection process.
Observational categories of data collected during patient reviews
In addition to the team observations, participants reported their perceived involvement (on a 0–100 scale) in the patient decision-making process (involvement). Participants answered the question using the PDA device immediately after recording their anticipations of patient development. During the round, senior doctors are usually the main decision-makers, and they involve team members in the decision-making process in order to gather information on patients (eg, from trainees monitoring patients), to share information (eg, goals) and to educate trainees.7 42 43 Therefore, trainee and nursing involvement during patient reviews may facilitate team SA by (1) increasing levels of perspective and information sharing by trainees and nursing staff; (2) helping trainees and nursing staff to understand the senior doctor's goals and interpretation of information; and (3) allowing senior doctors to identify and resolve gaps in the knowledge/understanding of team members.
Lastly, observations also noted a number of control variables that might be found to influence team processes or team SA (table 1). These included: (1) the length of patient review (shorter reviews may indicate lower levels of patient complexity, or provide less opportunity for information sharing); (2) the number of team members attending each patient review (larger teams may reduce opportunities for interactions between team members); (3) the senior doctor leading the patient review (senior doctors may have different preferences for involving trainees and nurses);44 and (4) the number of times patient reviews were interrupted (potentially disrupting the flow of conversations and understanding being developed by teams).45
Analysis
Team SA
The validity and sensitivity of the individual SA measures were assessed in order to consider their potential for further analysis and development. First, it was expected that the anticipatory measures would regularly detect instances of shared and divergent team SA (and therefore demonstrate variance). Second, it was expected that the measures would be sensitive to expertise, with senior doctors being most accurate in anticipating patient outcomes. Lastly, it was expected that where SA diverged, the anticipations of the majority would be more accurate than the minority. To assess the formation of shared and accurate SA during patient reviews, directions of team member anticipations were compared.46 Where respondents indicated an anticipation to be more than 55% likely, the response was classified as ‘likely.’ Items rated as less than 45% likely were classified as ‘unlikely’ (This margin was introduced after poststudy debriefings found a small number of participants to use the 45–55% range as proxy for ‘uncertain either way’). A team was assessed to have developed shared SA for a patient if all team members' anticipations for the SA measure were the same (ie, all ‘likely’ or ‘unlikely’). To test SA accuracy, the anticipations of teams and individuals were compared with actual patient events and progress.
Team processes
The analysis assessed whether the team process data reflected findings from the ICU team literature showing team member roles and group hierarchies to influence teamwork.17 19 44 47 First, it was expected that trainees and nursing staff would report lower levels of involvement in patient decision-making than senior doctors. Second, it was expected that when team members were more involved in patient reviews, this would be associated with increased communication events. Finally, it was expected that higher numbers of senior doctor prompts for information would be associated with increased team-member communications.
Team SA and team processes
Finally, for the appropriate measures, exploratory analyses investigate whether team processes measured during the patient review predict the strength of team SA convergence. To measure convergence, we emulated previous team cognition research48 49 and calculated the average squared Euclidian space between team-member anticipations of likely patient progression (on a 0–100 scale). Based on the distances between team member anticipations, a team SA index was calculated. To explore team SA theory on the relationship between team processes and team SA,2 regression analyses investigate the relationship between team processes and team SA within the ICU team. It is hypothesised that higher-level involvement, communication and prompts for information will predict the strength of team SA convergence. Furthermore, considering the structure of ICU teams (with senior doctors as decision-makers who share information and delegate tasks/goals to specific team members42), a dyadic analysis using regression investigates whether the involvement and communication behaviours of trainees and senior nurses predict the degree to which they share SA with the senior doctor (termed subteam SA).
Results
A total of 37 ICU teams provided data on 105 patients. Of the patients, 70% survived, 53% were not on ventilation after 48 h, 36% were discharged after 48 h, and 47% showed a deterioration in condition within 48 h.
Team SA
For each SA item, table 2 shows the percentage of patients for which team members formed shared anticipations. Binomial tests found teams to form shared SA more often than chance (all p<0.05) for all individual items except deterioration likelihood, which showed a greater variance. In addition, Pearson χ2 tests found senior doctors to be more accurate in anticipating deterioration likelihood than junior trainees (p<0.05, with a moderately strong Cramer V effect size of 0.26) and senior nurses (p<0.05, with a moderate Cramer V effect size of 0.23). For items where all team members formed shared SA (n=252), anticipations were correct for 71% of items. For items where team members did not form shared SA, 77% had a single team member diverging from the group (senior nurses 31%, senior doctors 27%, junior trainees 25% and senior trainees 17%). In these cases, the majority grouping was more likely to be correct in their predictions (p<0.01). When in a minority, senior doctor anticipations were incorrect for 58% of items.
Proportion of patients for which the intensive care unit team formed shared anticipations of patient progression, and the accuracy of anticipations for predicting patient outcomes
Team process data
A one-way ANOVA found trainees and nursing staff to report lower levels of involvement than senior doctors (p<0.001). In addition, self-ratings of involvement during patient reviews were found to correlate with the verbal communications made by nursing staff (p<0.05), senior trainees (p<0.01) and junior trainees (p<0.01). Finally, senior doctor prompts for information correlated with verbal contributions from nursing staff (p<0.05), senior trainees (p<0.01) and junior trainees (p<0.01). See table 3 for correlations and descriptive statistics.
Pearson correlations and descriptive statistics for measures of team member involvement, communication behaviours (per minute) and senior doctor prompts for input (per minute) during patient reviews
Teamwork processes and team SA
The analysis between team processes and team SA convergence focused on deterioration likelihood. This item showed considerable variance in the extent to which teams formed shared SA, and appeared more sensitive to expertise for predicting outcomes. A hierarchical regression was conducted, with (1) reported involvement of team members during patient decision-making being regressed onto team SA for deterioration likelihood, (2) team member communication behaviours being introduced to the regression and (3) the inclusion of control variables (see table 1). The regression was non-significant.
Further analysis investigated whether the involvement and communication behaviours of trainees and senior nurses predicted the degree to which they formed convergent subteam SA (table 4). A series of hierarchical regressions was performed for each team member/senior doctor dyad, with (1) reported involvement of team members during patient decision-making being regressed onto subteam SA for deterioration likelihood, (2) team member communication behaviours being introduced to the regression and (3) inclusion of control variables. Senior trainee involvement during patient decision-making predicted senior doctor/senior trainee subteam SA for deterioration likelihood (R2=0.07, p<0.05), but this was non-significant when including communication behaviours and control variables. Junior trainee involvement during patient decision-making predicted senior doctor/junior trainee subteam SA for deterioration likelihood (R2=0.18, p<0.001). Communication behaviours and control variables explained no further variance. Analysis of the senior nurse/senior doctor dyad yielded a non-significant regression.
Hierarchical regression of senior trainee/senior doctor and junior trainee/senior doctor subteam situation awareness for deterioration likelihood on measures of team process during the intensive care unit round
Discussion
This study tested a method for measuring team SA during the ICU round, and a number of comments can be made on the results.
Measurement of team SA
The study measured team SA by focusing upon shared anticipations for aspects of patient progression. More often than chance, ICU teams were found to develop shared anticipations of discharge, ventilation and survival likelihood. When teams did not form shared SA, a majority view was most accurate for predicting patient outcomes. Teams formed shared anticipations of deterioration likelihood for less than half of patients, with senior doctors' predictions being the most accurate. On reflection, the measures of discharge, ventilation and survival likelihood had clear physical/visible outcomes or were dependent upon system factors (eg, discharge), and were therefore reasonably predictable (eg, most patients survive ICU). Patient deteriorations may have required greater expertise to predict (and therefore be a more effective measure of SA), as they are subtle (eg, not physically obvious), measured by a variety of clinical data, can occur early during a patient stay and are caused by a variety and combination of illness and treatment factors.28–30
The handheld computers were efficient for measuring team SA during the round, and could be adapted to handover processes or preoperative checks, or focus on anticipations of specific patient developments (eg, sepsis). Notably, despite team members being present during patient discussions, teams frequently formed different anticipations for patient progression. We cannot establish whether this was due to differences in the information perceived (or comprehended) during patient reviews, but it may indicate anticipations to be an alternative diagnostic of SA for teams where members have differing roles, knowledge and expertise. Furthermore, it is not possible to demonstrate that team members understood and answered the SA questions consistently. For example, the implications of team members being uncertain or having anticipations that agree, but vary in strength, are unclear. This reflects debate on the meaningfulness of comparing team member cognitions.50 The study also found examples of teams forming shared and incorrect anticipations of patient progression. This could arise from a dominant senior doctor leading opinion. It was notable that senior doctors' anticipations were incorrect for approximately half of items where they were found to hold divergent anticipations to the team. This may highlight the importance of multidisciplinary collaboration for forming accurate shared SA during patient reviews, while also pointing towards the limitations of using anticipations of highly complex and uncertain patients to measure team SA.
Team processes and team SA
Team processes were only found to predict subteam SA at the dyadic level (between senior doctors and trainees) for deterioration likelihood. Trainee involvement during patient decision-making processes may help develop subteam SA by (1) promoting information sharing by trainees; (2) helping trainees to understand the senior doctor's goals and to interpret information using the senior doctors expertise; and (3) allowing senior doctors to identify and remedy gaps in trainees' knowledge and understanding. This is consistent with team SA theory in healthcare,2 with strong team SA emerging when team members share information pertinent to roles and tasks. Senior doctors report the engagement of trainees as particularly important when distributing tasks during the round.42 However, interpretation of the results is restricted owing to limitations in the data analysis. The analysis aggregated perceptions of SA to the group level, with each patient treated as an individual data point. Data were collected from 37 unique teams with overlapping team members (eg, senior doctors and junior trainees), and the non-uniformity of data excluded hierarchical linear modelling. Thus, the regression analysis does not test whether certain combinations of team members are particularly proficient at forming shared SA. In addition, extraneous variables may have influenced the study, including audibility of discussions and underlying patient complexity. These limitations reflect complexities in the collection of team SA data within live healthcare environments. Even within the (relatively) controlled environment of an ICU ward round, the structure and nature of changing ICU teams, complex patient profiles and environmental constraints influence how team SA data can be collected and analysed.
The team process data reflected findings from the ICU team performance literature,16 with team member involvement during decision-making being influenced by role. Informal poststudy discussions found nurses to report a lack of opportunity/need to contribute to decision-making, and difficulties in finding physical space to be involved. Junior trainees reported feeling involved when presenting patients, contributing opinions/information to the decision-making processes or participating in the diagnosis of new patients. In addition, junior trainees and nursing staff reported ‘confirming’ patient review outcomes with senior trainees after the round if they were unclear on an aspect of patient management. To encourage involvement, senior doctors reported using eye contact, requests for information/contributions and temporary delegation of decision-making to senior trainees (who generally reported high levels of involvement). While team members agreed their involvement in patient decision-making was important for building shared SA, they acknowledged that this was not always possible (eg, during very high workloads).
Future research will use structured qualitative observations to measure team processes, with a focus on specific processes identified as important for team performance. The current study found trainee involvement to contribute to the development of subteam SA. This was not the case for senior nurses, who reported being the least involved in rounds, and were observed to make limited verbal contributions. Yet, monitoring and anticipatory skills are important for nursing practice in both surgical and intensive care teams,21 51 52 and future research should investigate why team members do not become involved in patient decision-making, and identify the leadership behaviours and team skills that might increase participation. Future SA research may also wish to further identify the shared information requirements of teams, and dyads of team members, for different tasks/scenarios (ie, to understand optimal team SA). It may also investigate why teams develop shared but inaccurate SA during the round, alongside the implications for performance, and methods for re-establishing and maintaining accurate SA. This will allow future studies to explore whether there is a predictive relationship between team SA and patient outcomes, and to use team SA as a measure of team performance during the round.
Conclusions
This study successfully piloted an experimental method for measuring team SA during the ICU round. Further development is required to refine the SA measures, with potential applications to a number of domains. Furthermore, the research may indicate a need to develop concepts of team SA within healthcare, and to link them with patient outcomes. Research needs to reflect the nature of developing shared SA in hierarchical multidisciplinary teams where training is ongoing, and team members have different responsibilities, knowledge and experience.
Acknowledgments
The authors would like to thank the ICU staff who supported and participated in the study. We would also like to thank the reviewers for their advice in redrafting the article.
References
Footnotes
Funding This work was supported by the College of Life Sciences and Medicine, University of Aberdeen.
Competing interests None.
Ethics approval Ethics approval was provided by the Medical Research Ethics Council.
Provenance and peer review Not commissioned; externally peer reviewed.