Article Text

Download PDFPDF
Usability evaluation of order sets in a computerised provider order entry system
  1. Julie Chan1,
  2. Kaveh G Shojania2,
  3. Anthony C Easty1,
  4. Edward E Etchells2,3,4
  1. 1Institute of Biomaterials and Biomedical Engineering, University of Toronto, Toronto, Ontario, Canada
  2. 2Centre for Patient Safety, University of Toronto, Toronto, Ontario, Canada
  3. 3Centre for Health Services Sciences (CHSS), Sunnybrook Research Institute, Toronto, Ontario, Canada
  4. 4Department of Medicine, Information Services, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada
  1. Correspondence to Dr Edward E Etchells, Department of Medicine, Information Services, Sunnybrook Health Sciences Centre, University of Toronto, 2075 Bayview Avenue, Room H469, Toronto, ON, Canada M4N 3M5; edward.etchells{at}


Background Computerised provider order entry (CPOE) is an important patient safety intervention that has encountered significant barriers to implementation. The usability of a CPOE system plays a significant role in its acceptance. The authors conducted a heuristic evaluation of a CPOE order set system to uncover existing usability issues prior to implementation.

Methods A heuristic evaluation methodology was used to evaluate the usability of a CPOE test order set system. There are 10 heuristic principles, such as error prevention, to help users identify and recover from errors. Evaluators included a staff physician with extensive clinical experience, and three engineers with expertise in heuristic evaluation methodology. The results of the heuristic evaluation were used to create a user centred design prototype.

Results 92 unique heuristic violations were found for the CPOE test order set system, including 35 identified by the clinician and at least one engineer, and 57 of the 92 violations (62%) found only by the clinician. All evaluators identified at least one violation of each of the 10 usability heuristics in their analysis of the CPOE system. A user centred design prototype was created to demonstrate changes that could improve usability.

Interpretation The CPOE test order set system had many usability heuristic violations. Many violations were found by a clinician with knowledge of the heuristic evaluation process. Implementation of the CPOE system was deferred and a new user centred design prototype was developed for future study. The authors recommend conducting heuristic evaluations early in the process of designing, selecting and implementing CPOE systems.

  • Computerised provider order entry
  • computerised order sets
  • usability evaluation
  • human factors engineering
  • medical order entry systems
  • human factors
  • information technology

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.


  • Funding This project was supported by a student grant to Ms Chan from Information Services, Sunnybrook Health Sciences Centre.

  • Competing interests Dr Etchells is currently the President of the OACIS User Group (OUG). OUG is a voluntary association of healthcare organisations that use the OACIS system. Dr. Etchells receives no payments or inducements from the owners of OACIS. We provided a preliminary copy of our report to the makers of OACIS.

  • Provenance and peer review Not commissioned; externally peer reviewed.