Anatomy of a failure: A sociotechnical evaluation of a laboratory physician order entry system implementation

Abstract

Objective

To investigate the human, social and organizational issues surrounding a Computerized Physician Order Entry system for Laboratory ordering (CPOE-L) implementation process and to analyze their interrelated effects on the system implementation failure in an academic medical setting. Second, to provide lessons learned and recommendations on to how to manage challenges of human, social and organizational nature surrounding CPOE-L implementations.

Methods

The themes surrounding CPOE introduction were identified by a heuristic analysis of literature on CPOE implementations. The resulting set of themes was applied as a reference model for 20 semi-structured interviews conducted during the CPOE-L implementation process with 11 persons involved in the CPOE-L project and in reviewing all CPOE-L related project documentation. Data was additionally gathered by user questionnaires, by user discussion rounds and through an ethnographical study performed at the involved clinical and laboratory departments. In analyzing the interview transcripts, project documentation and data from user questionnaires and discussion rounds a grounded theory approach was applied by the evaluation team to identify problem areas or issues deserving further analysis.

Results

Outlined central problem areas concerning the CPOE-L implementation and their mutual relations were depicted in a conceptual interpretative model. Understanding of clinical workflow was identified as a key theme pressured by organizational, human and social issues ultimately influencing the entire implementation process in a negative way. Vast delays in CPOE introduction, system immaturity and under-functionality could all be directly attributed to a superficial understanding of workflow. Consequently, final CPOE integration into clinical and laboratory workflows was inhibited by both end-users as well as department managers and withdrawal of the CPOE-L system became inevitable.

Conclusion

This case study demonstrates which human, social and organizational issues relevant to CPOE implementation cumulatively led to a failure outcome of the CPOE-L pilot introduction. The experiences and considerations described in this paper show important issues for CPOE systems to be successfully introduced and to be taken into account in future CPOE implementations. Understanding and consideration of (clinical) workflow aspects by project managers and the involved clinical organization is of extreme importance from the very start of a CPOE implementation process.

Introduction

Though potential benefits abound [1], [2], [3], successful implementation of Computerized Physician Order Entry (CPOE) systems for electronic entering and retrieving of medical orders is known to be difficult and expensive [4], [5]. Failure of CPOE introduction seems to be tied to the current lack in understanding of current clinical practices surrounding order creation and of how to integrate CPOE into the apparent complexity of order workflow [6]. Sociotechnical approaches that focus on this interrelation of organizational environment and technology are considered valuable in enhancing the understanding of the CPOE implementation process [7]. This paper describes the evaluation of a failed implementation of a CPOE system for ordering laboratory tests (CPOE-L) at a large university hospital, the Academic Medical Center (AMC) in Amsterdam. For a good understanding of the issues influencing the CPOE-L project's failure, the aim of this study is to characterize the nature of involved key issues, explore their interrelations, and use these insights to support new implementation efforts in our institution and possible other institutions. Frameworks or models that focus on analyzing factors that influence the adoption of Information and Communication Technology (ICT) have been applied in research for decades. Models such as those of Delone and McLean (IS Success model [8]), Seddon (variance model of Information System success [9]), Mirani and Lederer (framework to measure benefits derived from IS projects [10]), Grover (framework for measuring IS effectiveness [11]) and Smithson and Hirscheim (conceptual framework for IS evaluation [12]) analyze the impact, success and effectiveness of a system based on different classes of measures such as, among others, system quality, information quality, service quality, organizational impact, productivity and system usage. However, these models of ICT success cannot be applied in analyzing the variables and interpret their mutual interrelations influencing a complete system implementation process, from feasibility study to a first pilot phase. Delone and McLean themselves for example argue that ‘top management support’ and ‘user involvement’ may cause success rather than being a part of success and that their model does not take these variables into account. Other issues that might be relevant to an implementation study, such as the culture and organizational characteristics are also not included in these IT success models. Sociotechnical based frameworks could yet offer more insight into the sociotechnical transfusion of Information Technology (IT) in healthcare and its adoption by its intended users. However, existing frameworks such as the IT Adoption Model (ITAM) focuses on the individual users’ perspective and attributes of users’ system adoption but fails to operationalize the organizational, human and social aspects surrounding a system development and (first) implementation effort [13]. The review by Van Der Meijden et al. [14] was likewise aimed at identifying attributes to assess the success of electronic health care systems after implementation, not at revealing variables and their mutual interdependencies that might have influenced the software development life cycle and in the end may explain the nature of the cause of systems’ successes or failures.

We were specifically interested in analyzing and interpreting the variables and problem areas that may cause CPOE success or failure in order to explain what went wrong during the CPOE-L implementation process. Our research focus is on interpreting human, social and organizational problems encountered in the entire CPOE-L implementation process, from feasibility to pilot implementation. We interpret our results by developing a three layer conceptual model based on the analysis of factors influencing CPOE implementation described by Ash et al., Kuperman, Sittig and Massaro [4], [15], [16], [17], [18], [19], [20]. Research of these factors dates back to 1970 and so the knowledge base of CPOE implementations has been built on over more than 25 years of experience [21], [22].

Recent studies on CPOE implementation underscored the importance of understanding clinical workflow as a key issue for effective CPOE adoption in clinical practice [6]. But although success and failure factors for CPOE implementation have been thoroughly explored in studies on CPOE adoption, the influence of the human, social and organizational issues surrounding an implementation process has only limited been examined in the literature [23]. The main objective of this study is to investigate the impact and interrelation of experienced implementation problems on the whole CPOE-L implementation process from feasibility study to pilot introduction in the Academic Medical Center of Amsterdam. In doing so, we provide lessons learned and recommendations for acting upon challenges concerning human, social and organizational issues during a complete CPOE implementation process.