The performance of teams in complex systems is fundamental to safe and efficient work in high risk environments.^1,2 Surprisingly, research and development of interdisciplinary teamwork in surgery is comparatively scarce, despite the need to improve the quality and safety of surgery.^3–7 The theory and methods of team assessment developed elsewhere provide a valuable foundation for research on the surgical teams.^8–13 However, teamwork assessments developed in one domain may not be immediately transferable to another.¹⁴ Surgical teams operate in a unique and complex environment, and the assessment of surgical teams must take account of that setting.

Healthcare teams have been examined from a number of different perspectives. Team assessment has, for instance, addressed the functional roles of doctors¹⁵ and nurses,¹⁶ the effects of leadership style in surgical teams,¹⁷ and the specific impact of anaesthetists on team performance.^18,19 Communication has been assessed in UK National Health Service nursing teams,²⁰ medical students,²¹ operating room staff,^22–24 and intensive care unit staff.²⁵ Other studies have surveyed attitudes of medical staff to error and stress,²⁶ and identified discrepant attitudes toward safety between nurses and physicians in intensive care units.²⁷ Research has also focused on the team process, examining, for example, procedural deviations and errors in neonatal cardiac surgery.²⁸ These studies have generally confined their investigation to particular aspects of team performance, suggesting that a full understanding of healthcare teams will require investigation from several different perspectives.

There is a need for in depth analysis of all aspects of the performance of surgical teams, but particularly for observational studies. Ideally, an observational assessment of team performance in surgery would be based on a theoretical model, or at least on a precise specification of tasks. Unfortunately, there is no clear specification of the precise team behaviours required for even the most basic intra- or inter-disciplinary teamwork, although it is clear that the process has many routine elements. The observation and measurement of routine process is a useful foundation for studying interdisciplinary teamwork, and provides a basis for assessing more subtle behaviours and development of a formal validated assessment method.

The aim of this paper is to provide a description, explanation, and discussion of the theoretical and practical issues of developing observational measures of team performance in surgery. We begin by proposing a model of team performance as a prelude to discussing measurement issues.

A MODEL OF SURGICAL TEAM PERFORMANCE

The input−process–output model of team performance (fig 1),²⁹ established in aviation and the UK health service,³⁰ and prominent in team theory literature^8,9,31 provides a useful framework for studying surgical teams. The model permits predictions and hypotheses about the relationships between team factors and their effect on team performance. A surgical team has specific tasks and objectives, composition, structure, conditions of work, human and technological resources and competencies, collectively termed as input factors. The surgical team process involves a set of complex interactions amongst team members, their technology, and their patients. Those interactions involve members carrying out planned tasks and responding to unexpected clinical events. Various social, cultural, political, and ergonomic factors also affect team performance.^32–35 These factors operate at the psychological level in terms of the knowledge, skills, beliefs, expectations, understanding, and attitudes of individuals, which in turn affects their behaviour, co-operation, and communication.^11,36–38 In short, a hierarchy of interrelated factors affects team performance, some intrinsic and some extrinsic to the team.

METHODS OF MEASURING TEAM PERFORMANCE

There is no firm consensus on how to measure teamwork, partly owing to the lack of empirical data to validate measures³⁹ and because different teams require different types of assessment. Tesluk et al⁴⁰ and Annett¹⁰ provide useful tabulations of the main methods of assessing teams, which are questionnaires/surveys, observations, interviews, and archival data. Observational measures of tasks and behaviours are a common form of team assessment, although they cannot reveal all relevant facets of team performance. Cognitive processes may be inferred from questionnaires or observation; constructs include team orientation,⁴¹ team cohesion,⁸ team affective tone,⁴² team climate,⁴³ team role selfperception⁴⁴ and team mental models.^36–38 A thorough assessment of team performance in the surgical process would ideally comprise a combination of data from multiple sources, addressing all disciplines and their inter- and intra-relations,⁴⁵ supported by both quantitative and qualitative data.^46,47 The need for objective observational measures is paramount, however, particularly in surgery, where a focus on process by observation may provide a basis for interdisciplinary team assessment and training.

DEVELOPING OBSERVATIONAL MEASURES OF TEAM PERFORMANCE IN SURGERY

Specifying tasks as a basis for assessing team performance

Tasks and their management form the basis of many observational assessments (for example, the TARGETS methodology⁴⁸), as the behaviours required of a team depend upon them. Tasks need to be considered in context and to be organised with respect to that context, not least for practicality of data collection; the first step is to specify the beginning and end points of the process.

Given that surgical teams operate within a larger organisational system, the definition of team process is somewhat arbitrary. However, it is common to refer to three phases of surgical process: preoperative, intra-operative, and postoperative. The preoperative phase accounts for preparation and planning and the process that leads to preparation for the operation. The intra-operative phase starts from the point of incision to the point of closure, and the postoperative phase starts from the point of closure to the recovery transfer. Each phase may be further divided by stages, defined in table 1.

The next step in developing task measures is to define the objectives of each stage; categorising tasks as to whether they concern categories of patient, environment, equipment, or information transfer (table 2 shows possible categories to assess). A list of subsidiary goals and tasks within each category then follows (fig 2 shows an example of tasks to be checked on spreadsheet format). A successful transition from one stage to another will depend on the completion of tasks in preceding stages.

The basic tasks carried out by surgical teams vary between operations and are seldom clearly defined and specified. A starting point is to refer to guidelines and best practices outlined by each disciplinary group. There are a number of sources of guidance and information on operating room procedures when drawing up the task list for observational measures. The UK Department of Health Modernisation Agency’s Step Guide to Improving Operating Theatre Performance provides a valuable template (see Appendix). There is also a wealth of independent research for obtaining guidelines, a sample of which is cited in the appendix.

An observational teamwork assessment could focus only on tasks involving the whole team, as opposed to those carried out by individual team members; however, all tasks are ultimately the responsibility of the team, and we have therefore included all clinically relevant tasks. Consistent with that (systems) view is the suggestion that when a team fails to detect or correct an individual’s error it becomes a team error.^2,49 An observational team performance assessment should focus on tasks that are essential to team process, regardless of whether those tasks can be completed by one individual or by more than one.

Specifying teamwork behaviours as independent measures of team performance

The completion of tasks, while necessary for effective team performance in surgery, does not completely capture the quality and efficacy of the team’s work. Team measures in other areas have also sought to capture behaviours such as communication, co-operation, leadership, and other aspects characteristic of highly performing teams.

There are numerous models of teamwork, which vary in the behaviours defined, depending on the type of work, goals, tasks, and functions of the teams.⁵⁰ Dickinson and McIntyre⁴¹ proposed a generalised model of teamwork derived from a review of the team literature.⁵¹ Their model is consistent with an input−process–output system of team performance and therefore particularly suitable for application to surgery. Their model has the following dimensions:

• Team orientation accounts for the attitudes and cohesion of the team.

• Team leadership refers to provision of direction, assertiveness, and support among team members.

• Communication refers to the quantity and quality of information exchanged among members of the team.

• Team monitoring refers team observation and awareness of process.

• Team feedback refers to the quality of information provided in response to communications and performance of others.

• Backup behaviour involves assistance provided to among team members, supporting others and correcting errors.

• Co-ordination refers to team’s performance enhancement of function through managing and timing activities and tasks.

Team orientation is difficult to observe, but is closely related to co-operation. Backup behaviour we have regarded as an aspect of co-operation. Similarly, team feedback may be viewed as a component of communication. We therefore began with five areas of observable behaviour of a team adapted from Dickinson and McIntyre’s model: co-operation, co-ordination, leadership, monitoring, and communication.

There are various methods of constructing behavioural rating scales,⁴¹ behavioural markers,¹⁴ and event based measures,¹¹ which vary in purpose and measurement specificity. A starting point for behavioural assessment is to use behavioural summary scales: broad summary statements of behaviour to rate performance. The summary scales are ordinal: each scale point relating to a certain level of quality of a given teamwork component, determined by various definitions (the Shorter Oxford Dictionary⁵²) of a component (table 3 shows an example of a leadership scale). Team assessment can also be directed at specific individuals, as in the use of behavioural markers.¹⁴

A crucial practical point is that the more specific the behaviours rated the larger the assessment becomes. There is a limit to what can be observed in the time available. Moreover, focusing on fixed behaviours may occur at the expense of observing and recording unexpected behaviours, in response to crises or other unexpected events. Inevitably, there is a trade off between detail and scope when assessing behaviours in real time by observation. One alternative is to assess behaviour with structured rating scales and semistructured forms comprising a behavioural event record, where the observer notes effective and ineffective exemplar behaviours, relating to each behavioural dimension.⁴¹ The final decision on the design of an assessment will always in the end be dependent on its purpose.

OTAS IN PRACTICE: AN OVERVIEW

The practical aspects of collecting data on team performance are addressed in this section, with a brief report on an initial study aimed at testing the feasibility of recording tasks and rating behaviours of surgical teams concurrently. We constructed a generic assessment applicable to different operations focused on patient safety related tasks and team behaviours. The assessment was constructed in a hierarchical manner, moving from phases, stages, and task categories to individual tasks. Behavioural rating scales were constructed based on those extrapolated from Dickinson and McIntyre’s model: communication, co-operation, co-ordination, leadership, and monitoring (fig 2). Behavioural event forms, for noting effective and ineffective behaviours or events as they occurred, were also included to provide support for global behavioural ratings.

Two observers, a psychologist and surgeon, collected data using a combination of handheld computer with MS Excel spreadsheet and pen and paper to record observations of events and behaviours. The following describes the task checks and behaviour recording in more detail. Table 1 shows the categories of data acquisition for the two observers, and fig 2 shows an example of the spreadsheet task checklist.

Pre-operative assessment

In practice, the observers (1 and 2) entered the operating room before the patient arrived. Observer 1 checked pre-operative planning tasks, such as whether patient notes had been prepared. Observer 1 also checked whether appropriate equipment and instruments were available, whether the anaesthetic equipment had been checked and if the anaesthetic logbook was up to date. Communications among staff concerning patient consent, co-morbidity, and special requirements, such as allergy to latex, were also checked. In Pre3, Observer 1 checked for specific tasks that are carried out during patient preparation ready for incision (Op1), such as the use of Ted stockings, armboards, warming blanket, and pressure point protectors. Equipment readiness, such as the diathermy and suction apparatus, was also checked.

Observer 2 began observing teamwork behaviours and noting as they occurred using a behavioural event form with headings for each behavioural dimension, rows for participants role identity, and an abbreviation key. In the final preoperative phase, Observer 2 used the behaviour summary scales (see table 3 for leadership scale) to provide ratings for the overall impression of team behaviour, supported by the semistructured record of observed behaviours.

Intra-operative assessment

Observer 1 and Observer 2 continued with checks and behavioural event recording during the intra-operative phase. Intra-op1 is a crucial stage, when the whole team must be fully prepared for incision. Observer 1 checked that the patient was draped and painted with Betadine, and whether equipment settings and placement were appropriate. Observer 1 also checked that the surgeon and anaesthetist confirmed verbally that the incision could be made. Meanwhile, Observer 2 continued with recording task related behaviours. During Intra-op2, the operation proper, Observer 1 checked whether correct instruments were used, whether there were sufficient swabs and sharps, and whether patient condition was monitored. Observer 1 also checked for essential communication tasks between surgeon and anaesthetist. During the Intra-op stage, Observer 2 continued recording behaviours, which were usually stable as the surgeon operated with assistance from the scrub nurse, supported by surgical assistants and circulating nurses. During Intra-op3, Observer 1 checked for blood loss analysis, swab and instrument checks, correct suturing material, and essential communications between surgeon and anaesthetist. Observer 2 continued as before, and towards closure rated team behaviours with the behavioural summary scale.

Post-operative assessment

From closure and anaesthetic reversal, Observer 1 recorded patient ‘set down’, specifically whether the patient’s airway was maintained, pressure and diathermy areas checked and oxygen mask fitted, and patient cleaned. Safe transfer to trolley was also recorded by Observer 1. Observer 2 continued with behaviour recording, noting particularly team members’ assistance in safe patient transfer to trolley. Post 2 is the final stage for observation, where the patient is transferred from the operating theatre to recovery. Observer 1 and Observer 2 followed the anaesthetist and nurse to the recovery room, where transfer was observed. Observer 2 entered the recovery room before the patient to observe the action and communication provided by the recovery team upon patient and entry of theatre staff. Observer 1 checked that the patient notes and x rays accompanied the patient and that adequate fluids and analgesia were administered to the patient. Observer 1 also checked that the patient was made comfortable and that information regarding the operation was transferred from theatre staff to recovery staff. Observer 2 recorded the observed behaviour among the relevant team members and rated team behaviours for the postoperative phase accordingly.

ILLUSTRATIVE DATA

Our initial assessment of 50 general operations provided a wealth of basic data, which will be reported separately. In this paper, we report sample data only, to illustrate the potential richness of the assessment and the findings it produces. Fig 3 illustrates average operative stage duration and variation in 50 cases. Fig 4 shows how measures of behaviour might vary depending on the team assessed and the phase of assessment.

DISCUSSION

Developing a valid observational assessment of surgical team performance is a substantial undertaking. Team performance in routine objectives and tasks depends on team behaviours that currently lack specification. Researchers and practitioners need observational data to develop models of performance, and to assess teamwork behaviour. Concurrent assessment of specific tasks and behaviours covers the principal aspects of surgical team performance although considerable further work is needed to define and validate the measures described here.⁵⁷ We believe however, that the detailed objective account of task completion is well balanced by the more subjective, interpersonal behaviour ratings. In the meantime, assessment of tasks and global level behaviour in interdisciplinary surgical teams may, to a limited degree, be useful for discriminating performance of surgical teams.

We acknowledge that the choice of tasks was necessarily selective. It could be argued that task oriented measures using guidelines are simplistic and do not account for the complexity of team performance.^53,54 However, from a patient safety perspective, there must be a minimal set of tasks that assessors can specify, such as confirmation of surgical site and laterality. We found that with an organised dataset and structured assessment, many tasks could be observed and checked. Points in team process, defined by stage transition, served as useful prompts to move from one stage to another. Thus, a staged checklist is a valid objective measurement of team performance in surgical process, where movement from one stage to another is conditional upon certain basic practices. We therefore see potential value in using a staged task checklist for research assessment and for training and practice, and perhaps also in the form of a manageable staff checklist for use during operations.

Rating team behaviour in surgical process was feasible and an important complement to simply recording task completions. We found, however, that in rating behaviour across different stages within a phase, we were losing potentially valuable variation in performance measurement within and between disciplinary teams. Thus, a further improvement would be to differentiate intra-team and inter-team performance on different teamwork behaviours. Fig 5 shows the team interactions that might be assessed. A thorough assessment of team performance may need to account for differences among individuals too. If summing individual performance is invalid, then researchers must develop sophisticated models that account for interactive variance within-team (Tesluk et al⁴¹ and Dickinson and McIntyre⁴¹ provide useful review and discussion on aggregating individual scores for team assessment).

Research through team training and training through team research

The introduction to this paper remarked upon the importance of understanding the surgical team process more fully in order to provide explicit models of team process, against which teams can be compared,⁵⁵ especially given the introduction of new technology and procedure in surgery. Valid measures of teamwork behaviour could be obtained from an interactive team training and research paradigm. Staff might specify task dependent behaviours themselves in simulation. This would entail staff supplying statements midsimulation about what they expect or need from other members during particular tasks, in respect of teamwork components. For instance, the surgeon might state that they expect the nurse to be aware of and anticipate the requirement for particular instruments. Likewise, the nurse may expect the surgeon to communicate unanticipated instruments or provisions before they are needed during the operation, to enhance co-ordination. A full understanding of team performance in multidisciplinary healthcare will only be gained if we engage the knowledge of all disciplines.^45,55 This process of behavioural assessment could form part of a training and research exercise at the local level and contribute to construction of models of team performance globally, through natural selection.

Cautionary notes

In the case of observational assessment of surgical teams, there are clear benefits to collaborative research between research specialists and surgical domain experts, in ‘being present’ in the operating theatre and in training scenarios. However, in such collaboration, researchers may become participants and staff may become researchers, potentially confounding the data recorded.

Researchers and practitioners can of course both provide impartial and objective assistance in improving safety and quality of theatre processes. However, external advisors bring their own values, agendas, perspectives, and language to the surgical setting which should be considered. For instance, there is a real possibility of practitioners departing from academic discipline to the degree that their work lacks a firm theoretical basis. At the other extreme, academics may at a distance fail to account for experiences of participants in the so-called ‘real world’.⁵⁶

There are also costs and benefits to the relationships among researchers and those they work with in organisations; they should be realised and carefully managed. Indeed, research into, and improvement of, hospital teamwork will depend on the co-operation among associations, government agencies, researchers and staff, where the principles of teamwork should also apply.⁵⁷ An action research paradigm, where researchers attempt to solve work problems in collaboration with staff and practitioners with a firm theoretical basis, is a discipline in need of development.⁵⁶

CONCLUSIONS

In summary, research into teamwork in surgery is at a very early stage, and research is needed from a number of different perspectives, with different levels of analysis. Interdisciplinary teamwork and team training is needed to integrate tasks and behaviours of the three main disciplinary groups. At the local level, team research through team training and simulation may contribute to evolving models of surgical team performance. OTAS may serve as a useful framework for those purposes and as a measure of evaluating and validating methods of team training both in simulation and in the operating theatre.

APPENDIX 1: GUIDANCE ON RECOMMENDED THEATRE PRACTICE AND TEAM PERFORMANCE

National Confidential Enquiry into Patient Outcome and Death. Functioning as a team. The 2002 report of the national confidential enquiry into peri-operative deaths. Available at: http://www.ncepod.org.uk

Association of Anaesthetists of GB & Ireland. Guidelines. Available at: http://www.aagbi.org/guidelines.html

The American Society of Anaesthesiologists. Policy statements on practice parameters, standards and rules for anaesthesiologists. Available at: http://www.medana.unibas.ch/eng/educ/standard.htm

British Orthopaedic Association. Recommendations on sterile procedures in operating theatres. Available at: http://www.boa.ac.uk/BOAsterilerec.htm

The Royal College of Surgeons of England. Good surgical practice. 2002. Available at: http://www.rcseng.ac.uk/services/publications/publications/pdf/gsp2002.pdf

National Electronic Library for Health. Guidelines finder. Available at: http://www.nelh.nhs.uk

NHS Modernisation Agency. Stepguide to improving operating theatre performance. June 2002. Available at: http://www.modern.nhs.uk/scripts/default.asp?site_id=28&id=6706