Design, application and testing of the Work Observation Method by Activity Timing (WOMBAT) to measure clinicians’ patterns of work and communication☆
Introduction
Evidence regarding how health information technologies influence patterns of clinical work and support efficient work practices is limited. A systematic review [1] published in 2005 uncovered 23 studies since 1984 which examined the impact of system use on clinicians’ (doctors’ and nurses’) time. These studies in general adopted either work sampling or time and motion methods. Only six (26%) examined work on general wards, all in US hospitals, while the remainder focused on specialized settings (e.g. ICU and general practice). Overall, studies which compared electronic with paper systems and calculated task time per patient or consultation, reported that computer use increased time required to complete tasks. Studies which examined task time across multiple patients or working shifts found computer use was more time efficient than paper-based systems [1]. Lo et al. [2] more recently observed specialists using either paper or computer systems within outpatient clinics and reported no significant difference in time spent per patient visit. A similar but smaller study [3] measuring time taken for hand-written and computer prescriptions in an ambulatory setting in the US also found no significant difference in average time per task for physicians.
Studies of changes in work distribution and communication patterns following system use are less prevalent, but do include evidence of changes. For example in the US, where nurses usually transcribe hand-written medication orders, CPOE eliminates this task [4], [5]. Following the introduction of a CPOE system clinicians may sequence work differently. As Callen [6] found, clinicians reported “if you are waiting for something on the computer you go and do something else”. This may result in, for example, batching the ordering of patients’ tests to one time of the day. Shu et al. [7] found interns in a US hospital spent more time alone and less time with other doctors after system introduction. A French hospital study [8] of doctor–nurse communication around medications showed that a CPOE system, in comparison with paper-based medication records, resulted in a move from synchronous communication to asynchronous. This introduced opportunities for misunderstandings and increased the extent to which nurses had to make assumptions about orders. Carpenter and Gorman [9] also reported a tendency for doctors in a US hospital to talk with nurses less about medication orders following system implementation.
Understanding these shifts in patterns of communication between clinicians are important as poor communication wastes time, threatens patient care and may be one of the major causes of preventable adverse events in clinical practice [10]. Any potential negative consequences of changes in communication patterns may be more than offset by the improvements in information exchange provided by having legible, easily accessible information which computerized systems afford clinicians. However until we have better quality data about how systems enhance or disrupt existing patterns of clinical work and communication we cannot move to re-design work practices or systems in ways which avoid any possible negative outcomes.
This research agenda needs to continue to progress beyond answering the question, does use of a computer save a clinician time, to questions about how patterns of work are re-arranged in response to the introduction of health technologies. Where time is released, or additional time consumed, how do clinicians re-distribute their time among work tasks? What amount of variation exists among different clinical sub-groups and do work tasks get re-distributed across groups? For example, if senior clinicians are found to spend less time in patient ordering following computerization is this because the system is efficient or because they have re-allocated this task, either explicitly or implicitly, to their junior colleagues? We need to examine how system use interferes with communication processes and as Gorman et al. [11] suggest, ensure that such systems “… facilitate care without interfering with or eliminating aspects of the process that are essential to high reliability performance in the face of urgency, uncertainty and interruptions” (p. 383).
We require studies which investigate whether changes in patterns of clinical work result in improved care delivery, patient outcomes and the work experiences of health professionals. These questions require a multi-method approach [12], and work measurement studies form an important component of such investigation. Researchers should be able to build upon previous work undertaken in different settings and countries. This requires standardization of measurement approaches and the adoption of valid and reliable measurement tools. Major factors identified for the paucity of evidence in this area are the limitations and varieties of methods used [1], [13], [14], the difficulties of capturing the non-linear and interruptive nature of clinical work [15], [16], and the lack of consistency in the application of rigorous research methods.
Our objective was to design, apply and test an observational method for capturing clinician work and communication patterns which incorporated an electronic data collection tool. The purpose of the tool was to allow efficient, accurate and reliable data collection, while also capturing a greater level of work complexity than previous paper-based methods have allowed. Results from a small number of previous studies [2], [17], [18], [19] suggest that the use of handheld computers (including personal digital assistants—PDAs) may be useful for this task, but researchers have presented minimal information about application and reliability issues relating to these tools. Building upon our previous work designing a paper-based, multi-dimensional work classification tool for nurses [14], we sought to investigate how much additional detail and task complexity data we could add using a PDA without reducing data accuracy or reliability. In this paper we detail: the development of these methods and tool; the application of the method in a study of 52 nurses in an academic hospital and a summary of key findings; and tests of the reliability and validity of data produced. This research comprised the first stage in a study to measure the impact of a commercial electronic medication management system (e-MMS) on doctors’ and nurses’ work and communication patterns.
Section snippets
Design of a multi-dimensional work task classification
Our first objective was to develop a work task classification system which would be incorporated in the PDA data collection tool. As a basis for this we used a multi-dimensional work measurement classification which we had applied in a paper-based work-sampling study of nurses that showed high levels of inter-rater reliability and face validity [14]. We extended the classification to contain greater detail about work tasks which previous literature indicated may be most susceptible to change
Reliability and validity of data
Inter-rated reliability testing showed all data collectors maintained 85% or higher (range 85–98%) in the field. The work task classification allowed 99% of all tasks to be classified with only 1% of tasks allocated to the ‘other’ category. The patterns of work task time distribution for enrolled nurses and registered nurses were compared to determine whether expected differences were reflected in the observational data. As Fig. 3 shows enrolled nurses spent a greater proportion of their time
Discussion
Our aim was to design research methods and a tool to allow efficient, reliable and accurate data collection about clinicians’ patterns of work and communication, which could potentially be used across specialties, professional groups, settings and countries. The Work Observational Method by Activity Timing (WOMBAT) proved to be a reliable means for collecting valid, complex, multi-dimensional data about nurses’ work and communication patterns, based on the indicators we measured. High levels of
Conclusions
Capture and analysis of clinicians’ work and communication patterns are vital if we are to further understand the full and far-reaching effects of electronic systems. The use of the WOMBAT approach allows us to go further than gathering clinicians’ perceptions of their time in various tasks, providing valid and reliable measurements of task time distribution and with whom and how tasks are undertaken. Further it enables us to distinguish between multi-tasking and interruptions and the
Acknowledgments
This research was funded by an ARC linkage grant in partnership with NSW Health. JW is supported by an NHMRC Fellowship. We thank M. Williamson, L. Kearney, K. Nguyen, and J. Pich for their involvement in various stages of the research.
References (21)
The impact of electronic health records on time efficiency of physicians and nurses: a systematic review
J. Am. Med. Inform. Assoc.
(2005)Electronic health records in specialty care: a time-motion study
J. Am. Med. Inform. Assoc.
(2007)The impact of e-prescribing on prescriber and staff time in ambulatory care clinics: a time-motion study
J. Am. Med. Inform. Assoc.
(2007)- et al.
Enabling technologies promise to revitalize the role of nursing in an era of patient safety
Int. J. Med. Inform.
(2003) Return on investment for a computerized physician order entry system
J. Am. Med. Inform. Assoc.
(2006)The people and organisational issues which impact on the diffusion of IT systems in healthcare facilities
School of Public Health and Community Medicine
(2006)Comparison of time spent writing orders on paper with computerized physician order entry
Impact of CPOE on doctor–nurse cooperation for the medication ordering and administration process
Int. J. Med. Inform.
(2005)- et al.
What’s so special about medications: a pharmacist’s observations from the POE study
Proc. Am. Med. Inform. Assoc. Symp.
(2001) Communication among health professionals
BMJ
(1998)
Cited by (112)
A longitudinal time and motion study quantifying how implementation of an electronic medical record influences hospital nurses’ care delivery
2021, International Journal of Medical InformaticsA Time and Motion Analysis of Nursing Workload and Electronic Health Record Use in the Emergency Department
2021, Journal of Emergency NursingImpact of an electronic health record on task time distribution in a neonatal intensive care unit
2021, International Journal of Medical InformaticsCitation Excerpt :Participants were provided with an information leaflet and allowed to study it before being asked to sign a consent form, after which they were assigned a unique study identification number. Microsoft Excel was used to design a local version of a pre-validated workflow tool [21,22]. Tasks were grouped into broad categories of direct care, professional communication, reviewing of patient charts, documentation, and medication-related tasks, as these tasks have been used in previous work sampling studies of HCPs [20,21].
A pre–post interventional study to reduce time spent on clinical documentation by nurses and midwives
2024, Journal of Advanced NursingTime-motion observations to characterize the developmental environment in a paediatric post-acute care hospital
2024, Child: Care, Health and DevelopmentUnderstanding the workflow of nurses in acute and subacute medical wards: A time and motion study
2023, Journal of Clinical Nursing
- ☆
This paper is based upon a presentation and paper from 12th World Congress on Medical Informatics. Editors: Kuhn KA, Warren JR, Leong T. Amsterdam: IOS Press, 1083–1087; Westbrook JI, Ampt A, Williamson M, Nguyen K, Kearney L. (2007). Methods for measuring the impact of health information technologies on clinicians’ patterns of work and communication.
- 1
This research was undertaken while A. Ampt was a researcher at the Health Informatics Research & Evaluation Unit, Faculty of Health Sciences, University of Sydney, Australia.