Article Text
PDF
Abstract
Background Conceptual models have been developed to address challenges inherent in studying health information technology (HIT).
Method This manuscript introduces an eight-dimensional model specifically designed to address the sociotechnical challenges involved in design, development, implementation, use and evaluation of HIT within complex adaptive healthcare systems.
Discussion The eight dimensions are not independent, sequential or hierarchical, but rather are interdependent and inter-related concepts similar to compositions of other complex adaptive systems. Hardware and software computing infrastructure refers to equipment and software used to power, support and operate clinical applications and devices. Clinical content refers to textual or numeric data and images that constitute the ‘language’ of clinical applications. The human–computer interface includes all aspects of the computer that users can see, touch or hear as they interact with it. People refers to everyone who interacts in some way with the system, from developer to end user, including potential patient-users. Workflow and communication are the processes or steps involved in ensuring that patient care tasks are carried out effectively. Two additional dimensions of the model are internal organisational features (eg, policies, procedures and culture) and external rules and regulations, both of which may facilitate or constrain many aspects of the preceding dimensions. The final dimension is measurement and monitoring, which refers to the process of measuring and evaluating both intended and unintended consequences of HIT implementation and use. We illustrate how our model has been successfully applied in real-world complex adaptive settings to understand and improve HIT applications at various stages of development and implementation.
- Human factors
- information technology
Statistics from Altmetric.com
Introduction
An ongoing challenge to the design, development, implementation and evaluation of health information technology (HIT) interventions is to operationalise their use within the complex adaptive healthcare system that consists of high-pressured, fast-paced and distributed settings of care delivery. Many conceptual models of user interaction, acceptance and evaluation exist,1 2 but most are relatively limited in scope. Given the dearth of models that are specifically designed to address safe and effective HIT development and use, we have developed a comprehensive, sociotechnical model that provides a multidimensional framework within which any HIT innovation, intervention, application or device implemented within a complex adaptive healthcare system can be studied. This model builds upon and bridges previous frameworks, and is further informed by our own work to study the safe and effective implementation and use of HIT interventions. In this paper, we describe the conceptual foundations of our model and provide several examples of its utility for studying HIT interventions within real-world clinical contexts.
Background
Previous analyses of HIT interventions have been limited by a lack of conceptual models that have been specifically developed for this purpose. Examples of models previously applied by HIT investigators include Rogers' diffusion of innovations theory,3–5 Venkatesh's unified theory of acceptance and use of technology,6–9 Hutchins' theory of distributed cognition,10–14 Reason's Swiss Cheese Model15–17 and Norman's seven-step human–computer interaction (HCI) model.18–20 Although all of these models account for one or more important facets of technology implementation, we believe that the scope of each model limits its utility to address the full range of factors that should be considered in the design, development, implementation, use and evaluation of HIT interventions. For example, these models were not specifically designed to address the complex relationships between the HIT hardware, software, information content and HCI. Furthermore, while most of these models provide general guidance to study the high-level aspects of HIT implementation within a given clinical environment, none of them includes a measurement and monitoring infrastructure (eg, methods to routinely collect data, create or review reports or conduct surveillance of outcomes). Based on these limitations, our aim was to develop a more comprehensive model to integrate specific technological and measurement dimensions of HIT with other sociotechnical dimensions (eg, people, workflow, communication, organisational policies, external rules and regulations).
Previous health information technology-focused sociotechnical systems models
Four related sociotechnical models have been particularly influential in providing the foundation of our proposed model. First, Henriksen's model addresses (1) individual provider characteristics; (2) the nature or complexity of the work or task performed; (3) the physical environment where care takes place; (4) the human–system interfaces involved; and (5) various characteristics of the organisation (social, environment and management).21 Second, Vincent's framework for analysing risk and safety proposes a hierarchy of factors that can potentially influence clinical practice.22 Third, Carayon's Systems Engineering Initiative for Patient Safety model23 identifies three domains: (1) characteristics of providers, their tools and resources, and the physical/organisational setting; (2) interpersonal and technical aspects of healthcare activities; and (3) change in the patient's health status or behaviour. Finally, Harrison et al's Interactive Sociotechnical Analysis framework provides an excellent broad overview of the complex, emergent, inter-relationships between the HIT, clinicians and workflows within any healthcare system.24
While these sociotechnical models include a ‘technology’ component, none break down the ‘technology’ into its individual components to enable researchers to dissect out the causes of particular HIT implementation or use problems, or to help identify specific solutions. We have found that many HIT problems we are studying revolve around the interplay of hardware, software, content (eg, clinical data and computer-generated decision support) and user interfaces. Failing to acknowledge these specific technology-specific elements or attempting to treat them separately can hinder the overall understanding of HIT-related challenges. For example, the ‘content’ dimension of our model accounts for much of what informaticians do, that is, studying the intricacies of controlled clinical vocabularies that provide the cognitive interface between the inexact, subjective, highly variable world of biomedicine and the highly structured, tightly controlled, digital world of computers.25 A well-constructed, robust user interface vocabulary can make all the difference in the world to a busy clinician struggling to quickly and accurately enter a complex clinical order for a critically ill patient,26 and it is important to distinguish this aspect of technology from others that may contribute to additional challenges (eg, a user interface ie, difficult to navigate, an order entry application, ie, slow to respond, or computers that are only available at the main nursing station). Failure to do so, for example, leads to general statements such as ‘clinicians struggled with the new technology’ or ‘it takes clinicians longer to complete their tasks using the new technology’ without providing any insight into specific causes of the problems or their solutions. In this example, without a multidimensional understanding of the technological dimensions of the failed IT application, the researcher may conclude incorrectly that the hardware, application software or user was responsible, when in fact a poorly designed or implemented clinical vocabulary might have been the root of the problem.
Finally, the preceding models do not account for the special monitoring processes and governance structures that must be put in place while designing and developing, implementing or using HIT—for example, identifying who will make the decision on what, when and how clinical decision support (CDS) interventions will be added27; developing a process for monitoring the effect of new CDS on the systems' response time28; building tools to track the CDS that is in place29; developing an approach for testing CDS; defining approaches for identifying rules that interact; developing robust processes for collecting feedback from users and communicating new system fixes, features and functions; and building tools for monitoring the CDS system itself.30
Moving towards a new sociotechnical model for HIT
To overcome the limitations of previous models, we propose a new sociotechnical model to study the design, development, use, implementation and evaluation of HIT (figure 1). Our comprehensive eight-dimensional model accounts for key factors that influence the success of HIT interventions. A major assumption of our model is that the eight dimensions cannot be viewed as a series of independent, sequential steps. As with other components of complex adaptive systems, these eight interacting dimensions must be studied in relation to each other. Clearly, several of our model's components are more tightly coupled than others—for example, the hardware, software, content and user interface are all completely dependent on one another. However, all the other social components also exert strong influences on these technical components.
Illustration of the complex inter-relationships between the eight dimensions of the new sociotechnical model.
In our model, one cannot expect to gain an in-depth understanding of the intricacies of complex HIT interventions simply by integrating the results of studies performed within any single dimension of the model.31 Rather, HIT interventions must be understood in the context of their simultaneous effects across multiple dimensions of the model. For instance, a recent evaluation of a national programme to develop and implement centrally stored electronic summaries of patients' medical records in the UK revealed their benefits to be lower than anticipated and cautioned that complex interdependencies between many sociotechnical factors at the clinical encounter-,organisational- and national level are to be expected in such evaluations.32 These study findings are illustrative of how and why our proposed model could be useful.
The eight dimensions are detailed in the following sections.
Hardware and software computing infrastructure
This dimension of the model focuses solely on the hardware and software required to run the applications. The most visible part of this dimension is the computer, including the monitor, printer and other data-display devices along with the keyboard, mouse and other data-entry devices used to access clinical applications and medical or imaging devices. This dimension also includes the centralised (network-attached) data-storage devices and all of the networking equipment required to allow applications or devices to retrieve and store patient data. Also included in this dimension is software at both the operating system and application levels. Finally, this dimension of the model subsumes all the machines, devices and software required to keep the computing infrastructure functioning such as the high-capacity air-conditioning system, the batteries that form the uninterruptable power supply that provides short-term electrical power in the event of an electrical failure and the diesel-powered backup generators that supply power during longer outages.
In short, this dimension is purely technical; it is only composed of the physical devices and the software required keeping these devices running. One of the key aspects of this dimension is that, for the most part, the user is not aware that most of this infrastructure exists until it fails.33 For example, in 2002, the Beth Israel Deaconess Medical Center in Boston experienced a 4-day computer outage due to old, out-of-date computer equipment coupled with an outdated software programme designed to direct traffic on a much less complex network. Furthermore, their network diagnostic tools were ineffective because they could be used only when the network was functioning.34
Clinical content
This dimension includes everything on the data-information-knowledge continuum that is stored in the system (ie, structured and unstructured textual or numeric data and images that are either captured directly from imaging devices or scanned from paper-based sources).35 Clinical content elements can be used to configure certain software requirements. Examples include controlled vocabulary items that are selected from a list while ordering a medication or a diagnostic test, and the logic required to generate an alert for certain types of medication interactions. These elements may also describe certain clinical aspects of the patients' condition (eg, laboratory test results, discharge summaries or radiographic images). Other clinical content, such as demographic data and patient location, can be used to manage administrative aspects of a patient's care. These data can be entered (or created), read, modified or deleted by authorised users and stored either on the local computer or on a network. Certain elements of the clinical content, such as that which informs CDS interventions, must be managed on a regular basis.36
Human–computer interface
An interface enables unrelated entities to interact with the system and includes aspects of the system that users can see, touch or hear. The hardware and software ‘operationalise’ the user interface; provided these are functioning as designed, any problems with using the system are likely due to HCI issues. The HCI is guided by a user interaction model created by the software designer and developer.37 During early pilot testing of the application in the target clinical environment, both the user's workflow and the interface are likely to need revisions. This process of iterative refinement, wherein both the user and user interface may need to change, must culminate in a human–computer interaction model that matches the user's modified clinical workflow. For example, if a clinician wants to change the dose of a medication, the software requires the clinician to discontinue the old order and enter a new one, but the user interface should hide this complexity. This dimension also includes the ergonomic aspects of the interface.38 If users are forced to use a computer mouse while standing, they may have difficulty controlling the pointer on the screen because they are moving the mouse using the large muscles of their shoulder rather than the smaller muscles in the forearm. Finally, the lack of a feature or function within the interface represents a problem with both the interface and the software or hardware that implements the interface.
People
This dimension represents the humans (eg, software developers, system configuration and training personnel, clinicians and patients) involved in all aspects of the design, development, implementation and use of HIT. It also includes the ways that systems help users think and make them feel.39 Although user training is clearly an important component of the user portion of the model, it may not by itself overcome all user-related problems. Many ‘user’ problems actually result from poor system design or errors in system development or configuration. In addition to the users of these systems, this dimension includes the people who design, develop, implement and evaluate these systems. For instance, these people must have the proper knowledge, skills and training required to develop applications that are safe, effective and easy to use. This is the first aspect of the model that is purely on the social end of the sociotechnical spectrum.
In most cases, users will be clinicians or employees of the health system. However, with recent advances in patient-centred care and development of personal health record systems and ‘home monitoring’ devices, patients are increasingly becoming important users of HIT. Patients and/or their care givers may not possess the knowledge or skills to manage new health information technologies, and this is of specific concern as more care shifts to the patient's home.40
Workflow and communication
This is the first portion of the model that acknowledges that people often need to work cohesively with others in the healthcare system to accomplish patient care. This collaboration requires significant two-way communication. The workflow dimension accounts for the steps needed to ensure that each patient receives the care they need at the time they need it. Often, the clinical information system does not initially match the actual ‘clinical’workflow. In this case, either the workflow must be modified to adapt to the HIT, or the HIT system must change to match the various workflows identified.
Internal organisational policies, procedures and culture
The organisation's internal structures, policies and procedures affect every other dimension in our model. For example, the organisation's leadership allocates the capital budgets that enable the purchase of hardware and software, and internal policies influence whether and how offsite data backups are accomplished. The organisational leaders and committees who write and implement IT policies and procedures are responsible for overseeing all aspects of HIT system procurement, implementation, use, monitoring and evaluation. A key aspect of any HIT project is to ensure that the software accurately represents and enforces, if applicable, organisational policies and procedures. Likewise, it is also necessary to ensure that the actual clinical workflow involved with operating these systems is consistent with policies and procedures. Finally, internal rules and regulations are often created in response to the external rules and regulations that form the basis of the next dimension of the model.
External rules, regulations and pressures
This dimension accounts for the external forces that facilitate or place constraints on the design, development, implementation, use and evaluation of HIT in the clinical setting. For example, the recent passage of the American Recovery and Reinvestment Act of 2009, which includes the Health Information Technology for Economic and Clinical Health Act, makes available over $20 billion for healthcare practitioners who become ‘meaningful users’ of health IT. Thus, the American Recovery and Reinvestment Act introduces the single largest financial incentive ever to facilitate electronic health record (EHR) implementation. Meanwhile, a host of federal, state and local regulations regulate the use of HIT. Examples include the 1996 Health Insurance Portability and Accountability Act, recent changes to the Stark Laws and restrictions on secondary use of clinical data. Finally, there are three recent national developments that have the potential to affect the entire healthcare delivery system in the context of HIT. These include: (1) the initiative to develop the data and information exchange capacity to create a national health information network41; (2) the initiative to enable patients to access copies of the clinical data via personal health records42; and (3) clinical and IT workforce shortages.43
System measurement and monitoring
This dimension has largely been unaccounted for in previous models. We posit that the effects of HIT must be measured and monitored on a regular basis. An effective system measurement and monitoring programme must address four key issues related to HIT features and functions.44 First is the issue of availability—the extent to which features and functions are available and ready for use. Measures of system availability include response times and percentage uptime of the system. A second measurement objective is to determine how the various features and functions are being used by clinicians. For instance, one such measure is the rate at which clinicians over-ride CDS warnings and alerts. Third, the effectiveness of the system on healthcare delivery and patient health should be monitored to ensure that anticipated outcomes are achieved. For example, the mean HbA1c value for all patients with diabetes in a practice may be measured before and after implementation of a system with advanced CDS features. Finally, in addition to measuring the expected outcomes of HIT implementation, it is also vital to identify and document unintended consequences that manifest themselves following use of these systems.45 For instance, it may be worth while to track practitioner efficiency before and after implementation of a new clinical charting application.46 In addition to measuring the use and effectiveness of HIT at the local level, we must develop the methods to measure and monitor these systems and assess the quality of care resulting from their use on a state, regional or even national level.47 48
Relationships and interactions between our model's components
Our research and experience have led us, and others, to conclude that HIT-enabled healthcare systems are best treated as complex adaptive systems.49 The most important result of this conclusion is that hierarchical decomposition (ie, breaking a complex system, process or device down into its components, studying them and then integrating the results in an attempt to understand how the complete system functions) cannot be used to study HIT.50 As illustrated by the evaluation of centrally stored electronic summaries in the UK, complex interdependencies between various sociotechnical dimensions are to be expected, and our HIT model (had it existed at the time) might have potentially predicted some of them and allowed them to address them prior to go-live rather than in the evaluation stages of the project. Therefore, one should not view or use our model as a set of independent components which can be studied in isolation and then synthesised to develop a realistic picture of how HIT is used within the complex adaptive healthcare system. Rather, the key to our model is how the eight dimensions interact and depend on one another. They must be studied as multiple, interacting components with non-linear, emergent, dynamic behaviour (ie, small changes in one aspect of the system lead to small changes in other parts of the system under some conditions, but large changes at other times) that often appears random or chaotic. This is typical of complex adaptive systems, and our model reflects these interactions.
For example, a computer-based provider order entry (CPOE) system that works successfully on an adult, surgical nursing unit within a hospital may not work at all in the nearby paediatric unit for any number of potential reasons, including: (1) hardware/software (eg, fewer computers, older computers, poor wireless reception, poor placement); (2) content (eg, no weight- or age-based dosing, no customised order sets or documentation templates); (3) user interface (eg, older workforce that has trouble seeing the small font on the screen); or (4) personnel (eg, no clinical champion within the medical staff). However, each of these dimensions has a potential relationship with one or more of the other dimensions. For instance, computers may have been few or old because of some organisational limitations, there may be no customised order sets because clinician-users did not agree on how best to do it, and there was no clinical champion because the organisation did not provide any incentive for the additional time this role would entail. Other reasons could include problems with the user interface, and the communication and workflow related to how nurses process new medication orders using the EHR and record administration of medications. These issues, in turn, may have been due to organisational policies and procedures. For example, the unit governance committee may have decided not to approve a request for mobile computers, with the result that nurses spent more time away from patients and therefore had a slower workflow related to processing new orders. The preceding example illustrates the interaction of six dimensions of our model: hardware/software, clinical content, user interface, people, workflow and organisational policies. Additionally, some form of monitoring could have detected these issues. In summary, our model provides HIT researchers with several new avenues of thinking about key technology components and how these dimensions can be accounted for in future research.
New HIT model in action in real-world settings
The following sections illustrate how we have used the sociotechnical model of safe and effective HIT use within our research. In an attempt to describe how the model can be applied across the breadth of HIT research and development, and to provide examples of different systems and interventions that can be analysed within this new paradigm, we highlight key elements of our model in the context of several recent projects.
HIT design and development
The design and development of CDS interventions within clinicians' workflow present several challenges. We conducted several qualitative studies to gain insight into the eight dimensions of our model during the development of a CDS tool within a CPOE application. This CDS intervention was designed to alert clinicians whenever they attempted to order a medication that was contraindicated in elderly patients or one that had known serious interactions with warfarin. We used several methods, including focus groups, usability testing and educational sessions with clinician users,51 to identify issues related to hardware/software, content, interface, people, measurement, workflow/communication, and internal policies and procedures. These efforts helped us, for example, to understand the need to meet with the organisation's Pharmacy and Therapeutics (P & T) committee (ie, internal policy) to convince them to modify the medication formulary as well as the information technology professional (ie, people) who was responsible for maintaining the textual content of the alerts (ie, font size, contents and order of the messages) to fit within the constraints of the alert notification window (ie, user interface) which eliminated the need to train clinicians to use the horizontal scrolling capability. This is just one simple example of how use of the eight-dimensional model paid huge dividends during the development and implementation stages of this highly successful project.52 53
HIT implementation
In a recent article, we described lessons that could be learnt from CPOE implementation at another site.54 One of the most important conclusions from this implementation was that problems could, and often do, occur in all eight dimensions of the model (see table 1).56
Illustration of how the eight dimensions of our sociotechnical model have been used to analyse different health information technology-related interventions and how other dimensions might need to be addressed for every dimension
HIT use
Safe and effective use of an EHR-based notification system involves many factors that are addressed by almost all dimensions of our model.57 58 This CDS system generates automated asynchronous ‘alerts’ to notify clinicians of important clinical findings. We examined communication outcomes of over 2500 such alerts that were specifically related to abnormal test results. We found that 18.1% of abnormal lab alerts and 10.2% of abnormal imaging alerts were never acknowledged (ie, were unread by the receiving provider). Additionally, 7–8% of these alerts lacked timely follow-up, which was unrelated to acknowledgement of the alert.
Despite a notification system that ensured transmission of results, it was concerning that abnormal test results did not always receive timely follow-up, even when acknowledged. This study revealed complex interactions between users, the user interface, software, content, workflow/communication and organisational policies related to who was responsible for abnormal test follow-up. Our findings thus highlighted the multiple dimensions of our model that need to be addressed to improve the safety of EHR-based notification systems and perhaps other forms of CDS (see table 1).59–62 We are now applying the sociotechnical model to study barriers, facilitators and interventions for safe and effective test result notification through EHRs.
HIT evaluation
Our model recently provided us with guidance in HIT evaluation, reminding us that however technologically savvy we make our patient care processes, we must also monitor their impact, effectiveness and unintended consequences carefully. We recently evaluated why, despite implementation of an automated notification system to enhance communication of fecal occult blood test results, providers did not take follow-up actions in almost 40% of cases.63 Again, our findings highlighted multiple dimensions corresponding to our sociotechnical model. For instance, we found that clinician non-response to automated notifications was related to a software configuration error that prevented transmission of a subset of test results but we also found that if the institution was using certain types of workflows related to test performance and that if organisational procedures for computerised order-entry of fecal occult blood tests were different, the problem may not have occurred. Thus, we found our multidimensional approach, which accounted for interactions, to be useful for comprehensive evaluation of HIT after implementation.
Conclusions
The eight dimensions of the safe and effective HIT use model introduced in this manuscript establish a new paradigm for the study of HIT. We have successfully applied this model to study several HIT interventions at different levels of design, development, implementation, use and evaluation. We anticipate that additional study of the eight dimensions and their complex interactions will yield further refinements to this model and, ultimately, improvements in the quality and safety of the HIT applications that translate to better health and welfare for our patients.
Acknowledgments
We thank D Espadas and A Esquivel, for their help in creating the graphical depiction of the model, the reviewers of this paper, for their constructive criticism, and A Bradford, for editorial assistance.
References
Footnotes
Funding This research was supported in part by the National Library of Medicine R01-LM006942, NIH K23 career development award (K23CA125585), the VA National Center of Patient Safety, Agency for Health Care Research and Quality and in part by the Houston VA HSR&D Center of Excellence (HFP90-020).
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.
Linked Articles
- Introduction