Effect of predefined order sets and usability problems on efficiency of computerized medication ordering

Objectives: To study the effect of predefined order sets on the efficiency of computerized medication ordering, and to analyze the effect of different types of usability problems on ordering efficiency.

Methods: Crossover study to comparing the efficiency of two methods of ordering (with and without use of predefined order sets) in a laboratory setting using a computerized physician order entry system (CPOE). The excess number of mouse clicks and keystrokes (the difference in number of mouse clicks and keystrokes needed by each physician and the minimally required numbers to accomplish the ordering tasks) for each method was measured and per physician, occurrences of usability problems during the task sessions were recorded. Observed usability problems were categorized using Zhang et al.'s heuristic principles of good user interface design. The effect of different types of usability problems on the excess number of mouse clicks and keystrokes was statistically analyzed.

Results: The median excess number of mouse clicks and keystrokes needed by physicians was 6.2 times lower in the method with predefined order sets (p<0.01). The excess number of mouse clicks and keystrokes was significantly increased by vague and erroneous system messages with a factor of 2.62 (95% CI 2.24-3.07), the use of unfamiliar language and terminology by a factor of 1.28 (95% CI 1.14-1.43), and non-informative system feedback by a factor of 1.15 (95% CI 1.03-1.28), respectively. Other categories of usability problems had little influence on ordering efficiency.

Conclusions: Predefined order sets can improve the efficiency of computerized ordering by reducing the excess number of mouse clicks and keystrokes. However, the efficiency of computerized ordering can be significantly impaired by usability problems due to vague and incorrect system messages, unfamiliar language, and non-informative system feedback.