You are here

Article Text

Article menu

Download PDFPDF

Original research
The impact of a tele-ICU on provider attitudes about teamwork and safety climate
Free
  1. M Y L Chu-Weininger1,
  2. L Wueste2,
  3. J F Lucke2,
  4. L Weavind3,
  5. J Mazabob3,
  6. E J Thomas2
  1. 1School of Health Information Sciences, University of Texas Health Science Center at Houston, Houston, Texas, USA
  2. 2Medical School, University of Texas Health Science Center at Houston, Houston, Texas, USA
  3. 3The Memorial Hermann Healthcare System, Houston, Texas, USA
  1. Correspondence to Eric J Thomas, University of Texas Houston Medical School, 6431 Fannin, MSB 1.122, Houston, TX 77030, USA; eric.thomas{at}uth.tmc.edu

Abstract

Objective To measure teamwork and safety climate in three intensive care units (ICUs) before and after remote monitoring by intensivists using telemedicine technology (tele-ICU).

Design Controlled pre tele-ICU and post tele-ICU cross-sectional survey.

Setting ICUs in two non-teaching community hospitals and one tertiary care teaching hospital.

Subjects ICU physicians and nurses.

Interventions Remote monitoring of ICU patients by intensivists.

Outcome measurements Teamwork Climate Scale (TWS), a Safety Climate Score (SCS) and survey items related to tele-ICU.

Main results The mean (SD) teamwork climate score was 69.7 (25.3) and 78.8 (17.2), pre and post tele-ICU (p=0.009). The mean SCS score was 66.4 (24.6) and 73.4 (18.5), pre and post tele-ICU (p=0.045). While SCS scores within the ICUs improved, the overall SCS scores for these hospitals decreased from 69.0 to 65.4. Three of the non-scaled items were significantly different pre and post tele-ICU at p<0.001. The item means (SD) pre and post tele-ICU were: “others interrupt my work to tell me something about my patient that I already know” 2.5 (1.2) and 1.6 (1.3); “I am confident that my patients are adequately covered when I am off the unit” 3.2 (1.3) and 4.2 (1.1); and “I can reach a physician in an urgent situation in a timely manner” 3.8 (1.2) and 4.6 (0.6).

Conclusions Implementation of a tele-ICU was associated with improved teamwork climate and safety climate in some ICUs, especially among nurses. Providers were also more confident about patient coverage and physician accessibility, and did not report unnecessary interruptions.

  • Teamwork
  • safety culture
  • critical care
  • intensive care
  • tele-ICU
  • telemedicine
  • quality of care

https://doi.org/10.1136/qshc.2007.024992

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Intensive care unit (ICU) patients cared for by physicians trained in critical care medicine (intensivists) have lower mortality rates than ICU patients cared for by other types of physicians.1 However, there is a shortage of intensivists and a growing number of ICU patients.2 Telemedicine technology (tele-ICU) is being used to alleviate this problem by connecting remotely located critical care physicians and nurses to multiple ICUs that lack intensivists.3 These tele-ICUs allow the remote care team access to real-time vital signs, laboratory data, on-site care giver notes, computerised decision support, and audio and video of patients' rooms. This enables them to participate in many aspects of patient care, ranging from rapid assessment and treatment of unstable patients, to implementation of routine evidence-based guidelines.

    Initial research has shown a favourable, albeit inconclusive, impact on patient outcomes.3–6 Regarding care giver perceptions, a study of a virtual critical care intervention that provided hospital emergency department staff with timely access to specialists and intensivists found that the on-site (emergency department) providers perceived positive teamwork relationship with the remote consults.7 However, little is known about how a tele-ICU may affect ICU physicians' and nurses' attitudes and perceptions in the outlying units about teamwork climate and safety climate. The impact of the tele-ICU is likely to be substantial: the remotely located tele-ICU physicians and nurses become new team members, ones who communicate and intervene with the on-site team through a new and complicated technology. Such technology and clinician deployment can fundamentally change communication and teamwork for better, or for worse.8 In addition, the tele-ICU should result in other changes that improve the quality and safety of care.9 ,10 Nevertheless, new technologies also often introduce unexpected errors in care processes.11

    We believed that implementation of the tele-ICU technology could substantially affect the teamwork climate and safety climate of the units. These measures are part of a unit's safety culture, which has been defined as “the product of individual and group values, attitudes, perceptions, competencies and patterns of behaviour that determine the commitment to, and the style and proficiency of, an organisation's health and safety management”.12 Teamwork climate and safety climate, the primary outcome measures in this study, are comprised of the provider attitudes and perceptions relevant to safety and teamwork. These attitudes are one important part of the broader safety culture of a clinical area. We measured them using the teamwork climate and safety climate scales from an extensively used and validated survey, the Safety Attitudes Questionnaire.13–17

    We hypothesised a priori that implementation of the tele-ICU would be associated with an improvement in safety climate because the frontline providers would have immediate access to intensivists, learning would improve, and overall safety and quality of care would improve. For example, if a physician or nurse in an outlying unit can easily seek input from a remote intensivist, and when they know that their patients are being monitored and cared for by remote nurses and intensivists, their attitudes about safety in the unit should improve. We also hypothesised that teamwork climate would worsen as the frontline providers struggled to incorporate the remote providers into their daily communication and work patterns. The tele-ICU was not just a passive monitoring device. The remote physicians and nurses frequently contacted the bedside care givers to ask questions, offer advice and initiate treatments—all without the benefits of face-to-face communication. Therefore, we suspected that there would be difficulties incorporating this new team member.

    Materials and methods

    Setting

    We conducted the study in a large healthcare system located in the Gulf Coast region of the United States. The system used tele-ICU that is the proprietary eICU technology developed by VISICU (Phillips Holdings USA Inc, Andova, Massachusetts, USA). The study began after the system had implemented the eICU in some hospitals, and the study ended before full implementation throughout the system. Therefore, not all ICUs in the system are included in the study. When the study began, the system planned to remotely monitor 99 ICU beds in eight ICUs of five of the system's 10 hospitals. The remote monitoring facility was staffed by two intensivists from noon to 7:00 Monday through Friday and 24 h a day on Saturday and Sunday, and four registered nurses and two administrative technicians 24 h a day, 7 days a week. Each intensivist collaborated with two nurses and one technician to monitor half of the ICU beds. Physicians and nurses each used a computer workstation with multiple LCD monitors to manage patient care. Among many functions, the workstation displayed early warning signals on abnormality in a patient's status (Smart Alerts®) and allowed the clinician to see live video of patients, monitor real-time vitals and manage clinical information. This study was approved by the institutional review board of the University of Texas Health Science Center at Houston.

    Survey description and background

    The survey instrument consisted of a six-item Teamwork Climate Scale (TWS), a seven-item Safety Climate Scale (SCS) (Appendix A), and an additional eight teamwork items and six safety climate items that were not part of the scales but have been retained because of the unique information they elicit. These two scales and items are from the psychometrically validated Safety Attitudes Questionnaire,13 which was refined from the Intensive Care Unit Management Attitudes Questionnaire.18 These healthcare surveys were adapted from surveys originally developed for commercial aviation.19 ,20 The teamwork and safety climate survey existed as a stand-alone survey separate from the full Safety Attitudes Questionnaire before the planning of this study. This survey was created so that a shorter survey would be available and because the teamwork climate and safety climate constructs are the ones most often correlated with patient outcomes (see the Discussion section).

    In addition, we added 12 new items that addressed workflow and quality-of-care issues (WQC scale) that could be directly related to the tele-ICU. The response option to all items was a 5-point Likert scale (“disagree strongly”–“agree strongly”) with an option for “not applicable”. We also asked respondents to indicate their position, years of experience in the organisation, gender and ethnic group.

    Survey administration

    We distributed surveys to physicians and nurses who worked in three critical care units of three hospitals. Other ICUs in the system were not surveyed because they implemented the tele-ICU before this study began, or were not going to implement it in the near future. The study ICUs and hospitals included a six-bed medical ICU in an 84-bed non-teaching community hospital (unit A), a 14-bed surgical ICU in a 520-bed non-teaching community hospital (unit B) and a 20-bed shock trauma ICU in a 657-bed tertiary care teaching hospital (unit C). The trauma unit was staffed only by intensivists; the other units were “open”. We surveyed all nurses who worked in these ICUs and the physicians who admitted at least one patient per week to the ICU.

    We administered pre tele-ICU surveys during the month prior to implementation of the tele-ICU (June 2005 for two ICUs and July 2005 for one ICU), and post tele-ICU surveys during the fourth month of tele-ICU implementation. A research nurse (LW) distributed surveys to nurses during staff meetings and during breaks in regular work hours. Physicians received three mailings, each 7–10 days apart. A US$5.00 gift certificate was included in the first mailing.

    This healthcare system had been annually administering the SCS as a stand-alone survey to all hospitals since 2003. We used these results from the study hospitals to control for secular trends in SCS scores. The system was not administering the TWS before our study.

    Data analysis

    We analysed the data using the statistical programming environment R, V.2.5.0.21 We scored each scale by first converting the five-point Likert scale to a 100-point scale as follows: 1=0, 2=25, 3=50, 4=75 and 5=100. Negatively worded items were reverse scored so that the higher scores reflected a more positive response. Responses to each item in a scale were summed, then divided by the number of items in that scale to create a scale score that ranged from 0 to 100. Extensive exploratory and confirmatory factor analyses had already been performed on these scales13; thus, it was not repeated. Internal consistency was measured using Cronbach α.

    We calculated the means and standard deviations of scale scores for all providers in the units for the pre and post tele-ICU surveys. Differences between the means of the preintervention and postintervention groups were tested using Welch's (unequal variances) two-sample t test. We also calculated means and SD of the 5-point Likert scale for the 26 non-scaled items. To assign an acceptable type I error rate and to control for family-wise error, we used the Bonferroni procedure and divided the α of 0.05 by the 26 comparisons to yield an α of 0.002. “Not applicable” responses and missing values were treated as missing data.

    Results

    We distributed 118 surveys pre tele-ICU and 118 post tele-ICU. The response rates were 71% and 60%, yielding sample sizes of 84 and 71, respectively. Most respondents were white (42% and 49% pre and post), women (51% and 58%), registered nurses (70% and 69%) and with 3–12 years of critical care experience (52% and 52%) (table 1). Missing values, including “not applicable” responses, per item (if any) ranged from 1.2% to 7.1% pre tele-ICU and 1.4% to 4.3% post tele-ICU, except the variable gender (20% and 14.3%, respectively) (table 1).

    View this table:
    Table 1

    Demographic characteristics of survey participants pre and post tele-ICU

    The Cronbach α's of TWS and SCS were 0.9 pre tele-ICU and 0.8 post tele-ICU. The 12 workflow and quality-of-care items created for this study had α=0.82, and exploratory factor analysis found that none of these items loaded onto the TWS or SCS (thus indicating the measurement of a unique construct). The mean (SD) TWS score was 69.7 (25.3) pre tele-ICU and 78.8 (17.2) post tele-ICU (p=0.009). The mean (SD) SCS score was 66.4 (24.6) pre tele-ICU and 73.4 (18.5) post tele-ICU (p=0.045) (table 2). While the SCS scores within the studied ICUs improved, the overall SCS scores for these hospitals, which served as control for secular trends, decreased from 69.0 pre tele-ICU to 65.4 post tele-ICU.

    View this table:
    Table 2

    Teamwork and safety climate scores compared among units and between nurses and physicians

    TWS scores for each unit pre and post tele-ICU were: unit A 71.2 and 86.7 (p=0.052), unit B 81.0 and 74.7 (p=0.171) and unit C 62.2 and 79.1 (p=0.003). SCS scores for each unit pre and post tele-ICU were: unit A 75.3 and 75.0 (p=0.968), unit B 73.6 and 68.6 (p=0.368) and unit C 60.3 and 76.1 (p<0.005). TWS scores for providers, pre and post tele-ICU were: nurses 67.9 and 79.3 (p=0.006) and physicians 74.9 and 77.1 (p=0.739). SCS scores for providers, pre and post tele-ICU were: nurses 66.5 and 75.4 (p=0.032) and physicians 66.0 and 68.7 (p=0.679) (table 2).

    Three of the items that were not part of the teamwork and safety climate scales were significantly different pre and post tele-ICU at p<0.001 (table 3). The item means (SD) pre and post tele-ICU on the 5-point Likert response scale were: “others interrupt my work to tell me something about my patient that I already know” 2.5 (1.2) and 1.6 (1.3); “I am confident that my patients are adequately covered when I am off the unit” 3.2 (1.3) and 4.2 (1.1); and “I can reach a physician in an urgent situation in a timely manner” 3.8 (1.2) and 4.6 (0.6).

    View this table:
    Table 3

    Non-scaled survey items and pre and post tele-ICU

    Discussion

    Implementation of a tele-ICU was associated with improved teamwork and safety climate among nurses in some of the monitored units. These improvements occurred in the context of no improvement in overall hospital-level safety climate scores, thus increasing the likelihood that the ICU improvements were related to the tele-ICU instead of a secular trend towards improved scores. We also found that the providers in the monitored units were more confident that their patients were adequately covered when they were off the unit, that they felt it was easier to contact physicians for urgent issues and that they were not being interrupted unnecessarily. This is the first assessment of how a tele-ICU may affect the teamwork climate and safety climate of monitored units. Given that technological interventions of this magnitude often cause unintended negative consequences,11 ,22–24 our results should be reassuring to others who are interested in this technology.

    There are several plausible explanations for why the tele-ICU improved teamwork and safety climate. The safety climate scale measures providers' comfort with reporting safety concerns, learning from errors and obtaining feedback on performance. Anecdotally, the tele-ICU facilitates learning and feedback because the remotely located ICU providers often give feedback about care to the ICU physicians and nurses. The much greater improvement in TWS and SCS scores among nurses compared to physicians post tele-ICU was likely due to the fact nurses are the frontline providers who received interruptions and interacted most with the remote team members, where learning occurred. Alternatively, our sample of physicians may have been too small to detect a difference. It is unclear why one unit improved more than others, but this provides further data to support the notion that unit-level factors often determine provider responses to quality improvement efforts and new technologies. Additional research is needed to elucidate these factors.

    The tele-ICU is also used to improve compliance with evidence-based guidelines. Greater collaboration between remote intensivists and on-site ICU providers may enhance compliance with best practices and quality indicators in the ICU.25 ,26 These activities could also lead to more positive responses to the safety climate items such as, “I would feel safe being treated here as a patient”.

    Regarding teamwork climate, we were surprised to see an improvement so soon after implementation. We expected that it would take several months for the new team member (the tele-ICU) to be integrated. However, the TWS is comprised of items that ask about speaking up, asking questions and getting help. The tele-ICU can facilitate all of these important communication activities because it was available to frontline providers 24 h a day, 7 days a week.

    Teamwork climate and safety climate have become important measures of safety and quality in hospitals. In fact, the Joint Commission and the National Quality Forum both recommend that hospitals annually measure their safety climate using a survey like the one used here. Higher safety climate scores are correlated with fewer catheter-related bloodstream infections.13 Higher teamwork climate scores are associated with better patient outcomes and lower nurse turnover rates.27 ,28 Thus, our findings also lend some indirect support to an earlier study that found reduced ICU and hospital mortality after implementation of a tele-ICU.3

    The strengths of this study include the use of a validated survey, inclusion of three ICUs that varied in size and type of patients, and a study design that allowed us to lessen the likelihood that secular trends caused the improvement in safety climate, although results did not necessarily rule out potential temporal confounders (if any) on teamwork climate. Our results may not be generalisable to other types of ICUs or to hospitals and healthcare systems that implement the technology in a significantly different manner. The generalisability issue is even hinted at in our own data—statistical power limits any firm conclusions; however, it appeared that the overall improvement in teamwork score was driven by a large improvement in one of the three ICUs. Generalisability is also limited, as discussed, because the acceptance and impact of new technology is mediated by the local managerial and leadership methods used to implement the technology.29–31 For example, implementation of a remote ICU in Pennsylvania may have strategically enhanced remote on-site team collaboration because the nurses rotated relationships prior to tele-ICU and on-site care.5 Other strategies such as building team relationships prior to tele-ICU implementation might also induce differences in teamwork outcomes.8 We used a survey that included 14 items that were not statistically part of the Safety Climate or Teamwork Climate Scale. They had been retained because during original development of the survey they were believed to be conceptually related to safety climate and teamwork, and prior users of the survey found them informative. Additional analyses revealed that seven of these items were not correlated with the scales and may provide additional information. Thus, the survey may contain seven uninformative items.

    Conclusions

    Implementation of a tele-ICU was associated with improved teamwork and safety climate in some of the monitored units. These findings provide some support to previous studies that documented positive effects of tele-ICU technology. However, tele-ICU technology is expensive, difficult to implement and has limited evidence for its ability to improve quality of care. More research is needed to fully understand the impact of tele-ICUs on patient care.

    Acknowledgments

    The authors thank Craig Johnson, PhD, for assistance provided for this article.

    Appendix A Teamwork climate and safety climate scale items

    Teamwork climate

    1. Nurse input is well received in this clinical area.

    2. In this clinical area, it is difficult to speak up if I perceive a problem with patient care.

    3. The physicians and nurses here work together as a well-coordinated team.

    4. Disagreements in this clinical area are resolved appropriately.

    5. It is easy for personnel here to ask questions when there is something they do not understand.

    6. I have the support I need from other personnel to care for patients.

    Safety climate

    1. I would feel safe being treated here as a patient.

    2. I am encouraged by my colleagues to report any patient safety concerns I may have.

    3. The culture in this clinical area makes it easy to learn from the errors of others.

    4. I receive appropriate feedback about my performance.

    5. Medical errors are handled appropriately here.

    6. I know the proper channels to direct questions regarding patient safety in this clinical area.

    7. In this clinical area, it is difficult to discuss errors.

    References

    1. ↵
      1. Pronovost PJ,
      2. Angus DC,
      3. Dorman T,
      4. et al
      . Physician staffing patterns and clinical outcomes in critically ill patients: a systematic review. JAMA 2002;288:2151–62.
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. Angus DC,
      2. Kelley MA,
      3. Schmitz RJ,
      4. et al
      . Committee on Manpower for Pulmonary and Critical Care Societies (COMPACCS): caring for the critically ill patient. Current and projected workforce requirements for care of the critically ill and patients with pulmonary disease: can we meet the requirements of an aging population? JAMA 2000;284:2762–70.
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Breslow MJ,
      2. Rosenfeld BA,
      3. Doerfler M,
      4. et al
      . Effect of a multiple-site intensive care unit telemedicine program on clinical and economic outcomes: an alternative paradigm for intensivist staffing. Crit Care Med 2004;32:31–8.
      OpenUrlCrossRefPubMedWeb of Science
    4. ↵
      1. Zawada ET,
      2. Kapaska D,
      3. Herr P,
      4. et al
      . Prognostic outcomes after the initiation of an electronic telemedicine intensive care unit (eICU) in a rural health system. S D Med 2006;59:391–3.
      OpenUrlPubMed
    5. ↵
      1. Rabert AS,
      2. Sebastian MC
      . The future is now: implementation of a tele-intensivist program. JONA 2006;36:49–54.
      OpenUrlCrossRef
    6. ↵
      1. Breslow MJ
      . Remote ICU care programs: current status. J Crit Care 2007;22:66–76.
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      1. Brear M,
      2. Westbrook J,
      3. Coiera E
      . Virtual Critical Care Unit (ViCCU) clinical and organizational evaluation final report . Sydney, NSW, Australia: University of New South Wales, Center for Health Informatics, 2006. http://www.chi.unsw.edu.au/CHIweb.nsf/page/ViCCU (accessed May 2007).
    8. ↵
      1. Rufo RJ
      . Virtual ICUs: foundations for healthier environments. Nurs Manag 2007;38:32–9.
      OpenUrl
    9. ↵
      1. Aderholt B
      . From a distance: saving lives through remote care. Health Manag Technol 2007;28:26–9.
      OpenUrlPubMed
    10. ↵
      1. Witzke AK
      . The eICU program transforms the ICU by enabling an intensivist-directed model of care. Nurs Crit Care 2006;1:51–60.
      OpenUrl
    11. ↵
      1. Koppel R,
      2. Metlay JP,
      3. Cohen A,
      4. et al
      . Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005;293:1197–203.
      OpenUrlCrossRefPubMedWeb of Science
    12. ↵
      1. Nieva VF,
      2. Sorra J
      . Safety culture assessment: a tool for improving patient safety in healthcare organizations. Qual Saf Health Care 2003;12(Suppl 2):ii17–23.
      OpenUrlAbstract/FREE Full Text
    13. ↵
      1. Sexton JB,
      2. Helmreich RL,
      3. Neilands TB,
      4. et al
      . The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res 2006;6:44. http://www.biomedcentral.com/1472-6963/6/44/abstract (accessed 2 Mar 2007).
      OpenUrlCrossRefPubMed
    14. ↵
      1. Huang DT,
      2. Clemeont G,
      3. Sexton JB
      . Perceptions of safety culture vary across the intensive care units of a single institution. Crit Care Med 2007;35:165–76.
      OpenUrlCrossRefPubMedWeb of Science
    15. ↵
      1. Sexton JB,
      2. Makary MA,
      3. Tersigni AR,
      4. et al
      . Teamwork in the operating room: frontline perspectives among hospitals and operating room personnel. Anesthesiology 2006;105:877–84.
      OpenUrlCrossRefPubMedWeb of Science
    16. ↵
      1. Colla JB,
      2. Bracken AC,
      3. Kinney LM,
      4. et al
      . Measuring patient safety climate: a review of surveys. Qual Saf Health Care 2005;14:364–6.
      OpenUrlAbstract/FREE Full Text
    17. ↵
      1. Thomas E,
      2. Sexton J,
      3. Neilands T,
      4. et al
      . The effect of executive walk rounds on nurse safety climate attitudes: a randomized trial of clinical units. BMC Health Serv Res 2005;5:28.
      OpenUrlCrossRefPubMed
    18. ↵
      1. Thomas E,
      2. Sexton J,
      3. Helmreich R
      . Discrepant attitudes about teamwork among critical care nurses and physicians. Crit Care Med 2003;31:956–9.
      OpenUrlCrossRefPubMedWeb of Science
    19. ↵
      1. Helmreich R,
      2. Merrrit A,
      3. Sherman P,
      4. et al
      . The flight management attitudes questionnaire (FMAQ). NASA/UT/FAA Report 93-94. Austin, Texas, USA: NASA/University of Texas, 1993.
    20. ↵
      1. Helmreich R
      . Cockpit management attitudes. Hum Factors 1984;26:583–9.
      OpenUrlAbstract/FREE Full Text
    21. ↵
      R Development Core Team. R: a language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, 2007. ISBN 3-900051-07-0. http://www.R-project.org (accessed 23 May 2007).
    22. ↵
      1. de Mol B
      . Designing medical technology. Saf Sci 2007;45:283–91.
      OpenUrlCrossRefWeb of Science
    23. ↵
      1. Balka E,
      2. Doyle-Waters M,
      3. Lecznarowicz D,
      4. et al
      . Technology, governance and patient safety: systems issues in technology and patient safety. Int J Med Inform 2007;76(Suppl 1):35–47.
      OpenUrlCrossRef
    24. ↵
      1. Sitting DF,
      2. Ash JF,
      3. Zhang J,
      4. et al
      . Lessons from “unexpected increased mortality after implementation of a commercially sold computerized physician order entry system”. Pediatr 2006;118:797–801.
      OpenUrlFREE Full Text
    25. ↵
      1. Badawi O
      . Greater collaboration between remote intensivists and on-site clinicians improves best practice compliance. Crit Care Med 2006;34:A20.
    26. ↵
      1. Youn BA
      . ICU process improvement: using telemedicine to enhance compliance and documentation for the ventilator bundle. Chest 2006;130:226S.
      OpenUrl
    27. ↵
      1. Baggs J,
      2. Schmitt M,
      3. Mushlin A,
      4. et al
      . Association between nurse–physician collaboration and patient outcomes in three intensive care units. Crit Care Med 1999;27:1991–8.
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      1. DeFontes J,
      2. Surbida S
      . Preoperative safety briefing project. Perm J 2004;8:21–7.
      OpenUrl
    29. ↵
      1. Lee S,
      2. Kim KJ
      . Factors affecting the implementation success of internet-based information systems. Comput Hum Behav 2007;23:1853–80.
      OpenUrlCrossRefWeb of Science
    30. ↵
      1. Roggow DJ,
      2. Tracy MF
      . Clinical nurse specialist leadership in computerized provider order entry design. Clin Nurse Spec 2005;19:209–14.
      OpenUrlPubMedWeb of Science
    31. ↵
      1. Goldstein MK,
      2. Coleman RW,
      3. Tu SW,
      4. et al
      . Translating research into practice: organizational issues in implementing automated decision support for hypertension in three medical centers. JAMIA 2004;11:368–76.
      OpenUrlAbstract/FREE Full Text

    Footnotes

    • Funding Sponsored by the Agency for Healthcare Research and Quality grant number R01 HS15234 and NIH Grant UL1 RR024148 (CTSA). Other funders: NIH and AHRQ.

    • Competing interests None.

    • Ethics approval This study was conducted with the approval of the University of Texas Health Science Center at Houston.

    • Provenance and peer review Not commissioned; externally peer reviewed.