Article Text
PDF
Abstract
Background Interruptions and multitasking are implicated as a major cause of clinical inefficiency and error.
Objective The aim was to measure the association between emergency doctors' rates of interruption and task completion times and rates.
Methods The authors conducted a prospective observational time and motion study in the emergency department of a 400-bed teaching hospital. Forty doctors (91% of medical staff) were observed for 210.45 h on weekdays. The authors calculated the time on task (TOT); the relationship between TOT and interruptions; and the proportion of time in work task categories. Length-biased sampling was controlled for.
Results Doctors were interrupted 6.6 times/h. 11% of all tasks were interrupted, 3.3% more than once. Doctors multitasked for 12.8% of time. The mean TOT was 1:26 min. Interruptions were associated with a significant increase in TOT. However, when length-biased sampling was accounted for, interrupted tasks were unexpectedly completed in a shorter time than uninterrupted tasks. Doctors failed to return to 18.5% (95% CI 15.9% to 21.1%) of interrupted tasks.
Conclusions It appears that in busy interrupt-driven clinical environments, clinicians reduce the time they spend on clinical tasks if they experience interruptions, and may delay or fail to return to a significant portion of interrupted tasks. Task shortening may occur because interrupted tasks are truncated to ‘catch up’ for lost time, which may have significant implications for patient safety.
- Communication
- emergency department
- quality of care
Statistics from Altmetric.com
Hospital environments have been characterised as interrupt-driven with high multitasking loads.1 The combination of multitasking (carrying out multiple tasks simultaneously) and interruptions is a potent latent source of clinical error.2–4 Yet we have a very limited understanding of their impact in real-world settings.5 6 Simulations and analyses of major negative events in the aviation5 and nuclear power7 industries have identified interruptions as a significant hazard for task errors.
Experimental studies show that interruptions can trigger cognitive failures, including lapses in attention, memory or perception.8–11 Further, interruptions add significantly to cognitive load, increase stress and anxiety, inhibit decision-making performance and increase task errors.11 12 Interruptions can cause an individual to forget items in working memory, leading to errors such as failing to complete or initiate tasks, or to repeat tasks.8 11 Switching between tasks also imposes cognitive penalties, such as resumption lag, the time penalty associated with task-switching. Experimentally, resumption lag can double when a task switch is externally forced via an interruption.10
Emergency physicians working in acute, pressurised settings have been shown to experience high rates of interruption and multitasking.13–16 Their environment is dynamic, characterised by resource and time constraints, and has been identified to be at greater risk of errors than many other settings.17 The aim of this investigation was to measure the association between emergency physicians' rates of interruption and task completion times and rates.
Methods
We conducted an observational time and motion study in the emergency department (ED) of a 400-bed metropolitan teaching hospital. Forty of the 44 doctors (91%) working on weekdays between 08:00 and 18:00 during July 2006 to January 2007 agreed to participate and provided written consent. The sample comprised five emergency physicians, seven medical registrars (one declined), 21 resident medical officers (one declined) and seven interns (two declined).
A trained researcher observed individual clinicians as they undertook their normal work activities and recorded tasks using a PDA (Personal Digital Assistant). In total, 131 observation sessions lasting 1.6 h on average, and 210.45 h in total, were completed. Inter-rater reliability tests were performed between the study observer and a test observer, achieving an 88% or higher agreement on data items. The study was approved by the hospital and the University of New South Wales Human Research Ethics committees.
We applied an observational method and multidimensional classification of clinical work encoded on a PDA based on previous work.18–21 Table 1 shows the 10 task categories. Some had sub-categories for greater detail (table 1). Tasks are automatically time-stamped when selected and the observer records whom the subject is with and information tools used. Interruptions and multitasking are recorded using buttons on the PDA. Interruptions were defined as situations where a doctor ceased a current task in order to attend to an external stimulus—for example, a doctor stops writing a prescription to answer a colleague's question. If the doctor continued to write the order while answering her colleague, the activity would be recorded as multitasking.
Work tasks and definitions
Tasks suspended following an interruption are visible as tabs on the PDA to allow recording if these tasks are resumed. This permits measurement of interrupted tasks recommenced and calculation of the total ‘time on task.’ Time on Task (TOT) is defined as the amount of time to complete a task and is obtained by adding the time spent on all fragments of a task if it was interrupted. Tasks completed by the end of the observation session are referred to as ‘completed tasks.’
Statistical analysis
Descriptive statistics were calculated to identify proportions of time on specific tasks, and multitasking and interruption rates. Comparisons between task types or clinician role were made using analysis of variance and χ2 analysis. Times are presented as proportions of minutes (eg, 2.06 min) or as minutes and seconds (eg, 2:06). The 95% CIs for the average TOT in each work category and by doctor seniority were calculated using the large sample approximation of mean ±1.96 times SE.
In studying the impact of interruptions on the total time required to complete a task, it is important to account for length-biased sampling, that is the longer it takes to complete a task, the greater the chance of that task being interrupted. This assumes that interrupting tasks arrive at random, according to some stochastic process independent of the task being interrupted. Thus, when we examine the hypothesis that interruptions increase TOT, the outcome may be confounded by length-biased sampling. In other words, an association between interruption and longer tasks may arise because of length-biased sampling. We developed a method (see detailed technical note), for assessing the impact of length-biased sampling on TOT, based on a simplifying assumption that interruptions arrive according to a Poisson process, with a constant rate of interruptions aggregated for the analysis presented here.
We also examined factors which may be associated with the frequency of interruptions and assessed the rate of interruptions across time during the study. Indicators of patient load on the ED included numbers of patients admitted on the day of the observation session, presentations to the ED 1 h prior and during the observation session, average age of patients in the hour immediately prior to the observation session, numbers of admissions from the ED to the hospital (a proxy for the severity of patients), numbers of medical staff rostered to the ward in the hour prior to and during the observation session, and the combination of registrars and junior medical officers working at the time of the session.
Results
Interruptions, multitasking and task time distributions
Over the 210.45 h of observation, 9588 individual task actions were recorded. The total TOT was 237.9 h, indicating that on average physicians spent 12.8% of their time multitasking. There was no significant difference in multitasking by seniority (emergency physicians 14.4%, registrars 13.2%, residents 12.3% and interns 9.4%; p=0.24).
Four work categories (direct care, indirect care, professional communication and documentation) accounted for over 80% of total observed time and were the most frequent tasks (table 2). The average task completion time for these task categories decreased with increasing physician seniority for direct care (p<0.0001), indirect care (p<0.0001) and documentation (p=0.0002), while for professional communication, there was no difference by seniority (p=0.9826) (table 3).
Proportion of time and average time per work task category
Comparison of average task completion times by task type category
Overall, clinicians were interrupted 6.6 times per hour. Interruptions occurred most frequently during documentation, with 47% of discharge summary documentation tasks and 40% of other documentation tasks interrupted. Direct and indirect care tasks were interrupted at similar rates (17% and 19% respectively). Physicians were least likely to be interrupted while engaged in professional communication (5%) or social activities (2%). Emergency physicians and interns had the highest interruption rate (7.1 interruptions per hour). Registrars were interrupted 6.6 times per hour and residents 6.1 times.
Associations between interruptions and time on task (TOT)
Of the 9588 task actions, 8369 (87.3%) marked the beginning of a new task, and the remaining 1219 (12.7%) were fragments undertaken to continue an earlier interrupted task. Of these 8369 new tasks, 7488 (89.5%) were not interrupted, 7.3% were interrupted once, 1.9% were interrupted twice and 1.4% were interrupted three or more times. In an extreme case, one task consisted of 14 fragments (due to 13 interruptions).
Without adjusting for length-biased sampling, we found that TOT was significantly positively associated with number of interruptions. The mean TOT for tasks without interruption was 1:26 (95% CI 1:23 to 1:29). The mean TOT doubled with one interruption and increased by 493% to 7:04 with three or more interruptions (table 4).
Time on task (min:s) by interruption frequency
These results were consistent across each of the task type categories, namely interrupted tasks were associated with longer TOT in all cases (table 5).
Task time (min:s) by task category and interruption status with 95% CIs
We also examined ED patient load factors, which may be associated with the frequency of interruptions, and assessed the rate of interruptions across time during the study. We found that none of these patient load factors was significant in explaining the observed fluctuations in interruption rates across sessions when incorporated into a generalised linear model for counts, using a Poisson response distribution or a negative binomial response distribution, or using a quasi-likelihood assuming overdispersion in the variance relative to the mean for Poisson responses.
Characteristics of the doctors participating in the study were recorded, including their clinical experience, training and position. Of these, doctors' experience and seniority were statistically significant when a negative binomial response distribution was assumed for the counts of interruptions in each session. Details of these analyses are available in the online technical note. Additionally, we investigated if morning or afternoon, or day of the week, impacted upon interruption rates in sessions and found these did not. Thus, interruption rates cannot be reliably predicted by the factors measured. Other factors are required to capture the complex manner in which interruptions arise in the emergency department setting. We identified a temporal relationship in relation to the interruption rate, which was also not explained by these factors. Further discussion of this issue is provided in the technical note
Effect of interruptions on time on task adjusting for length-biased sampling
We adjusted for potential length-biased sampling. Table 6 compares the observed average TOT with that predicted under length-biased sampling with no additional penalty due to interruptions. As we found that the frequency of interruptions varied by observation sessions, with a substantial decrease in rates over time, we selected three groups (sessions 1 to 15, 16 to 66 and 67 to 131) as the basis for combining session data in order to obtain a reasonable amount of data for calculating the cumulative distributions and the mean comparisons for table 6. While the interruption rate is not strictly homogeneous within these three groups, it is far more homogeneous than across the whole 131 sessions combined. (Further discussion of this issue may be found in the online technical note.)
Comparison of observed average total time on task (TOT, min) with that predicted under length biased sampling and no additional penalty due to interruptions
The results in table 6 show that, regardless of the hourly interruption rate (from low 3.3/h to high 15.4/h), the effect of the number of interruptions (1, 2 or 3+) on observed TOT was the same, namely TOT increased as interruptions increased. However, accounting for length-biased sampling, the average TOT for each category of interruption observed was lower than expected due to length-biased sampling. Thus, controlling for length-biased sampling, we found clinicians in fact spent a shorter amount of time on interrupted tasks than on tasks which were not interrupted.
Figure 1 shows the comparison of cumulative distribution functions of total TOT predicted under length-biased sampling with that observed aggregated over all 131 observation sessions. The null hypothesis being tested was that the observed TOT for interrupted tasks was not significantly different from that expected as a result of length-biased sampling. The vertical lines mark the mean values for observed actual TOT (solid line) and predicted TOT (dotted line) listed in table 6. Table 6 and figure 1 clearly demonstrate that the null hypothesis was rejected. The average TOT for interrupted tasks was significantly shorter than expected when length-biased sampling is accounted for. Thus, when interrupted, clinicians completed tasks in a shorter time than they did for tasks not interrupted.
Comparison of cumulative distribution functions of total time on task predicted under length-biased sampling with that observed aggregated over all 131 observation sessions. Vertical lines mark the mean values listed in table 6. H0, null hypothesis.
Association between interruptions and task completion
Of the 881 new interrupted tasks, 166 (18.8%) were not completed by the end of the observation session, compared with 113 (1.5%) of uninterrupted tasks not completed by session end (χ2=734.88, df=1, p<0.0001). Of the 166 interrupted tasks not completed, 163 (98.2%) involved physicians not returning to complete the task. The remaining three interrupted tasks involved a physician returning to a task but not completing this task by the end of the session.
We also investigated if inclusion of tasks that were interrupted and not resumed, or were terminated at the end of session, would impact our conclusions concerning length-biased sampling. Details are available in the accompanying technical note online and show that our results are not impacted in any substantial way by inclusion of non-resumed tasks (which are likely to have a shorter total time on task).
Discussion
Our results show that task completion times are shorter for interrupted tasks than for tasks with no interruptions when controlled for length-biased sampling. For tasks experiencing one interruption, completion time was almost half that of uninterrupted tasks. Within a clinical setting, there are very few studies of the relationship between interruptions and task completion times, and none have dealt with the problem of length-biased sampling.
Our study is unable to provide an explanation for these results, though several explanations are possible. Most likely is that clinicians, once interrupted, return to the original task but compensate for the time ‘lost’ in interruption by hastening task completion. This could be done by a faster work rate, reducing the effort spent on task elements, or even dropping task elements. Other possible explanations include that emergency clinicians are refusing interruptions for long or complex tasks, or it may be that shorter tasks are more likely to attract interruptions relative to longer tasks, for example because of the settings in which they occur, or the perceived availability of the clinician by others.
Experimental studies provide some insights, but these reveal conflicting ideas about the effects of interruptions on performance.22 However, there is experimental evidence which is consistent with our findings. Speier et al23 interrupted undergraduate students in a laboratory setting, attempting simple and complex computer tasks. They found that both simple and complex tasks, when interrupted, were completed more quickly but with similar accuracy to uninterrupted tasks. However, those completing complex tasks with a high frequency of interruptions had a significantly lower accuracy and significantly lower task completion times than those completing complex tasks with a low interruption frequency. They concluded that these results suggested that interruptions of high frequency are processed very differently from those at a low frequency.23
In a more realistic experiment, Mark et al22 simulated an office environment, using 48 university students, to test the effect of interruptions on the completion of email tasks. They found that interrupted subjects completed tasks more quickly than those not interrupted. Those who were interrupted also wrote much shorter email responses. Subjects' ratings of their stress, frustration and feelings of being time-pressured were all significantly higher among interrupted subjects. As a result, Mark et al22 proposed a hypothesis similar to that which we suspect may be operating in the ED, that as individuals are interrupted, they compensate by working faster and cutting corners. In our study, the only outcome indicator available was the proportion of completed tasks, which we found was lower for tasks that were interrupted. Importantly, Mark et al's results provide an indication of the negative effects of interruptions on subjects.
As is regularly noted in experimental studies, the controlled laboratory design is both a great strength and weakness. Its strength is the ability to control the nature and frequency of tasks and interruptions. A weakness is its inability to represent the full complexity of real-world work environments. For example, in experiments, interruptions are generally devoid of any social cues, such as the status of the person doing the interruption.23 Further, there is usually no time pressure on subjects compared with real-world clinical settings, which may have a significant effect. For example, Cellier et al24 attempted to introduce time pressures in an experimental situation of the effects of interruptions on computer tasks. They found that under low time constraints, subjects aimed for accuracy, while under high time constraints they aimed for speed.24 Thus, there appears to be some convergence between experimental results and our findings in the clinical setting.
Clearly, interruptions and multitasking are unavoidable in busy clinical environments, and patient care may be compromised if clinicians were unable to use these tactics in their work.
While it was not possible to measure the outcomes of the interruptions observed, the experimental findings reported here demonstrate some of the negative effects of interruptions on task performance. Our own research has demonstrated that interruptions to hospital nurses significantly increase the rate and severity of medication administration errors.4 Thus, the uncontrolled and untrained use of interruptions in clinical practice is an expensive and dangerous strategy, and the need to develop clinical processes that minimise unnecessary interruption and multitasking is strong.2
Our study has also raised some valuable analysis considerations when examining data of this type, clearly demonstrating how incorrect conclusions could be drawn by failing to control for length-biased sampling.
Limitations
The results relate to one organisation potentially limiting generalisability. Where possible, we compared our results with other studies to assess the consistency of the general pattern of work in our study emergency department. Overall, we found that emergency clinicians spent the greatest proportion of their time (29%) on direct patient care, which is consistent with studies of emergency physicians in the UK (22–30%)25 and Canada (28%).26 Our interruption rate was similar to the rate experienced in other countries where rates of 6.9/h,13 6.2/h,27 6.7/h 28 and 6.9/h16 have been reported. This suggests similarities in the nature of clinical emergency communication work patterns. Differences between senior and junior emergency physicians are consistent with Brennan et al,29 who found that work productivity for first-, second- and third-year US emergency medicine residents increased significantly as they moved through their training.
Our study examined weekday work. The results may not be representative of evenings or weekends. Physicians observed may have changed their behaviour as a result of being observed. If so, the activities likely to be subject to bias are tasks such as social activities which could be under-represented.
Future research
Our results support the hypothesis that the highly interruptive nature of busy clinical environments may have a negative impact on patient safety. If subsequent research confirms these data, attention should be devoted to substantial redesign of clinical work processes to minimise these expensive task-management behaviours. There are several areas of further analyses which could be pursued to gain an even greater understanding of this issue. This could include analyses to understand the contribution of censored tasks due to observational sessions ending and the impact on TOT calculations and potentially clustering effects by physician.
Acknowledgments
We thank the doctors who participated in the study.
References
Supplementary materials
Web Only Data qshc.2009.039255
Files in this Data Supplement:
Footnotes
This work was completed while NK was a Research Fellow at the Health Informatics Research and Evaluation Unit, Faculty of Health Sciences, University of Sydney, Australia.
Funding The study was funded by grants from the Hospital Contribution Fund (HCF) Health and Medical Research Foundation and the National Health and Medical Research Council (NHMRC) Program grant 568612. The funding sources had no role in the design, conduct, or reporting of this study.
Competing interests None.
Ethics approval Ethics approval was provided by the the hospital and the University of New South Wales Human Research Ethics committees.
Provenance and peer review Not commissioned; externally peer reviewed.