Skip to main content

Advertisement

SpringerLink
Log in

An Evaluation of the Veterans Health Administration’s Clinical Reminders System: A National Survey of Generalists

  • Original Article
  • Published:
Journal of General Internal Medicine Aims and scope Submit manuscript

Abstract

BACKGROUND

The Veterans Health Administration (VHA) is a leader in developing computerized clinical reminders (CCRs). Primary care physicians’ (PCPs) evaluation of VHA CCRs could influence their future development and use within and outside the VHA.

OBJECTIVE

Survey PCPs about usefulness and usability of VHA CCRs.

DESIGN AND PARTICIPANTS

In a national survey, VHA PCPs rated on a 7-point scale usefulness and usability of VHA CCRs, and standardized scales (0–100) were constructed. A hierarchical linear mixed (HLM) model predicted physician- and facility-level variables associated with more positive global assessment of CCRs.

RESULTS

Four hundred sixty-one PCPs participated (response rate, 69%). Scale Cronbach’s alpha ranged from 0.62 to 0.82. Perceptions of VHA CCRs were primarily in the midrange, where higher ratings indicate more favorable attitudes (weighted standardized median, IQR): global assessment (50, 28–61), clinical/situational specificity (29, 17–42), integration with workflow/workload (39, 17–50), training (50, 33–67), VHA’s management of CCR use (67, 50–83), design/interface (53, 40–67), perceived role in CCR use (67, 50–83), and self-efficacy (67, 57–78). In a HLM model, design/interface (p < .001), self-efficacy (p < .001), integration with workflow/workload (p < .001), and training (p < .001) were associated with more favorable global assessments of CCRs. Facilities in the west as compared to the south (p = .033), and physicians with academic affiliation (p = .045) had less favorable global assessment of CCRs.

CONCLUSIONS

Our systematic assessment of end-users’ perceptions of VHA CCRs suggests that CCRs need to be developed and implemented with a continual focus on improvement based on end-user feedback. Potential target areas include better integration into the primary care clinic workflow/workload.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1

Similar content being viewed by others

References

  1. Shea S, DuMouchel W, Bahamonde L. A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting. J Am Med Inform Assoc. 1996;3(6):399–409.

    PubMed  CAS  Google Scholar 

  2. Balas EA, Weingarten S, Garb CT, Blumenthal D, Boren SA, Brown GD. Improving preventive care by prompting physicians. Arch Intern Med. 2000;160(3):301–8.

    Article  PubMed  CAS  Google Scholar 

  3. Cannon DS, Allen SN. A comparison of the effects of computer and manual reminders on compliance with a mental health clinical practice guideline. J Am Med Inform Assoc. 2000;7(2):196–203.

    PubMed  CAS  Google Scholar 

  4. Hoch I, Heymann AD, Kurman I, Valinsky LJ, Chodick G, Shalev V. Countrywide computer alerts to community physicians improve potassium testing in patients receiving diuretics. J Am Med Inform Assoc. 2003;10(6):541–6.

    Article  PubMed  Google Scholar 

  5. Dexter PR, Wolinsky FD, Gramelspacher GP, et al. Effectiveness of computer-generated reminders for increasing discussions about advance directives and completion of advance directive forms: a randomized, controlled trial. Ann Intern Med. 1998;128(2):102–110.

    PubMed  CAS  Google Scholar 

  6. Dexter PR, Perkins S, Overhage JM, Maharry K, Kohler RB, McDonald CJ. A computerized reminder system to increase the use of preventive care for hospitalized patients. N Engl J Med. 2001;345(13):965–70.

    Article  PubMed  CAS  Google Scholar 

  7. Wagner EH, Austin BT, Von Korff M. Organizing care for patients with chronic illness. Milbank Q. 1996;74(4):511–44.

    Article  PubMed  CAS  Google Scholar 

  8. Sequist TD, Gandhi TK, Karson AS, et al. A randomized trial of electronic clinical reminders to improve quality of care for diabetes and coronary artery disease. J Am Med Inform Assoc. 2005;12(4):431–7.

    Article  PubMed  Google Scholar 

  9. Patterson ES, Nguyen AD, Halloran JP, Asch SM. Human factors barriers to the effective use of ten HIV clinical reminders. J Am Med Inform Assoc. 2004;11(1):50–9.

    Article  PubMed  Google Scholar 

  10. Perlin JB, Kolodner RM, Roswell RH. The Veterans Health Administration: quality, value, accountability, and information as transforming strategies for patient-centered care. Am J Manag Care. 2004;10(11 Pt 2):828–36.

    PubMed  Google Scholar 

  11. Demakis JG, Beauchamp C, Cull WL, et al. Improving residents' compliance with standards of ambulatory care: results from the VA Cooperative Study on Computerized Reminders. JAMA. 2000;284(11):1411–6.

    Article  PubMed  CAS  Google Scholar 

  12. Fung CH, Woods JN, Asch SM, Glassman P, Doebbeling BN. Variation in implementation and use of computerized clinical reminders in an integrated healthcare system. Am J Manag Care. 2004;10(11 Pt 2):878–85.

    PubMed  Google Scholar 

  13. Agrawal A, Mayo-Smith MF. Adherence to computerized clinical reminders in a large healthcare delivery network. Medinfo. 2004;11(Pt 1):111–4.

    Google Scholar 

  14. Doebbeling BN, Vaughn TE, McCoy KD, Glassman P. Informatics Implementation in the Veterans Health Administration (VHA) healthcare system to improve quality of care. AMIA Annu Symp Proc. 2006:204–8.

  15. Weeks WB, Yano EM, Rubenstein LV. Primary care practice management in rural and urban Veterans Health Administration settings. J Rural Health. 2002;18(2):298–303.

    Article  PubMed  Google Scholar 

  16. Saleem JJ, Patterson ES, Militello L, Render ML, Orshansky G, Asch SM. Exploring barriers and facilitators to the use of computerized clinical reminders. J Am Med Inform Assoc. 2005;12(4):438–47.

    Article  PubMed  Google Scholar 

  17. Web-based user interface evaluation with questionnaires. http://www1.acm.org/perlman/question.cgi. Accessed August 23, 2006.

  18. Yano EM, Simon B, Canelo I, Mittman B, Rubenstein LV. 1999 VHA survey of primary care practices. VA HSR&D Center of Excellence for the Study of Healthcare Provider Behavior; 2000. Report No.: 00-MC12.

  19. Yano EM, Asch SM, Phillips B, Anaya H, Bowman C, Chang S, Bozzette S. Organization and management of care for military veterans with HIV/AIDS in Department of Veterans Affairs Medical Centers. Mil Med 2005;170(11):952–9.

    PubMed  Google Scholar 

  20. McClure DL. Veterans Affairs: Subcommittee Questions Concerning the Department’s Information Technology Program. http://www.gao.gov/new.items/d01691r.pdf. Accessed April 10, 2007.

  21. Gaul GM. Revamped Veterans’ Health Care Now a Model. Washington Post 2005;Sect. A01.

  22. Zheng K, Padman R, Johnson MP, Diamond HS. Understanding technology adoption in clinical care: clinician adoption behavior of a point-of-care reminder system. Int J Med Inform. 2005;74(7–8):535–43.

    Article  PubMed  Google Scholar 

  23. Walter LC, Bertenthal D, Lindquist K, Konety BR. PSA screening among elderly men with limited life expectancies. JAMA. 2006;296(19):2336–42.

    Article  PubMed  CAS  Google Scholar 

  24. Nutting PA, Rost K, Smith J, Werner JJ, Elliot C. Competing demands from physical problems. Arch Fam Med. 2000;9(1059):1059–64.

    Article  PubMed  CAS  Google Scholar 

  25. Werner RM, Greenfield S, Fung CH, Turner BJ. Measuring quality of care in complex patients: findings from a conference organized by the Society of General Internal Medicine. http://www.sgim.org/PDF/SGIMReports/MeasuringQualityCareFinalReportOct42006.pdf. Accessed December 4, 2006.

  26. Tai-Seale M, McGuire TG, Zhang W. Time allocation in primary care office visits. Health Serv Res. 2007;42(5):1871–94.

    Article  PubMed  Google Scholar 

  27. Werner RM, Asch DA. Clinical concerns about clinical performance measurement. Ann Fam Med. 2007;5(2):159–63.

    Article  PubMed  Google Scholar 

  28. Mayo-Smith MF, Agrawal A. Factors associated with improved completion of computerized clinical reminders across a large healthcare system. Int J Med Inform. 2007;76(10):710–6.

    Article  PubMed  Google Scholar 

  29. McGlynn EA. Intended and unintended consequences: what should we really worry about? Med Care. 2007;45(1):3–5.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors are grateful to Barbara Simon, MA for her assistance with the survey, Mingming Wang, MPH for her assistance in programming the facility data, and Joya Golden for her administrative support of the project. This research was supported by the Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service (TRX 02-216). A VA HSR&D Advanced Career Development Award supported Dr. Asch, and a VA HSR&D Merit Review Entry Program Award supported Dr. Patterson. The views expressed in this article are those of the authors and do not necessarily represent the view of the Department of Veterans Affairs.

Conflict of Interest Statement

Drs. Glassman, Asch, and Doebbeling are staff physicians at their respective VHA facilities. After completing this project, Dr. Fung became an employee at Zynx Health Incorporated. Dr. Fung was a staff physician at a VHA facility during the study.

Author information

Authors and Affiliations

  1. VA Greater Los Angeles Healthcare System, Los Angeles, USA

    Constance H. Fung MD, MSHS, Jerry S. Tsai MS, Armine Lulejian MPH, Peter Glassman MBBS, MSc & Steven M. Asch MD, MPH

  2. David Geffen School of Medicine at UCLA, Los Angeles, USA

    Constance H. Fung MD, MSHS, Peter Glassman MBBS, MSc & Steven M. Asch MD, MPH

  3. VA Getting at Patient Safety (GAPS) Center, Cincinnati, USA

    Emily Patterson PhD

  4. Ohio State University, Columbus, USA

    Emily Patterson PhD

  5. VA Center of Excellence on Implementing Evidence-based Practices, Richard L. Roudebush VA Medical Center, Indianapolis, USA

    Brad N. Doebbeling MD, MSc

  6. Indiana University Center for Health Services and Outcomes Research, Regenstrief Institute, Indianapolis, USA

    Brad N. Doebbeling MD, MSc

  7. Department of Medicine, Indiana University School of Medicine, Indianapolis, USA

    Brad N. Doebbeling MD, MSc

Authors
  1. Constance H. Fung MD, MSHS
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jerry S. Tsai MS
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Armine Lulejian MPH
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peter Glassman MBBS, MSc
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Emily Patterson PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Brad N. Doebbeling MD, MSc
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Steven M. Asch MD, MPH
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Constance H. Fung MD, MSHS.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fung, C.H., Tsai, J.S., Lulejian, A. et al. An Evaluation of the Veterans Health Administration’s Clinical Reminders System: A National Survey of Generalists. J GEN INTERN MED 23, 392–398 (2008). https://doi.org/10.1007/s11606-007-0417-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11606-007-0417-8

KEY WORDS

Search

Navigation