Table 1

Operational definitions used to classify ease of entry of ‘error scenario’ test orders

How easy was it to place the order?
1	2	3	4	5

Easy	Minor workarounds	Some protections	Difficult	Impossible

1. Easy ▸ End user successfully and quickly entered the erroneous order ▸ No alerts/warnings ▸ No workarounds or additional mouse clicks required ▸ Order ‘sailed through’ (order simply accepted as if was a normal error)
2. Minor workarounds ▸ End user is able to enter the order fairly easily ▸ No alerts/warnings ▸ Requires some kind of additional workarounds (eg, needed to adjust default dosing or enter all or part of the order in free text, or use of comments field to complete order)
3 Some protections ▸ End user is able to enter the order ▸ ‘Passive’ alerts/warnings appear – Warning appears but it can be ignored (no over-ride required) – Warning appears but can over-ride with single mouse-click (this includes selecting a reason for over-ride from pull-down menu) ▸ Typical response from the provider is to say ‘I usually just blow through these [warnings]’ or equivalent.
4 Difficult ▸ End user is able to enter the order, but doing so requires a conscious, concerted effort ▸ ‘Active’ alerts/warnings appear that require additional action from provider (eg, typed reason for over-ride) ▸ Often, typed workarounds and extra mouse clicks are required to over-ride ▸ Order often does not go through on first attempt ▸ Significant time and thought required to enter successfully ▸ Articulated end-user frustration
5 Impossible ▸ Order could not be entered, despite attempted workarounds ▸ No way to enter order in free text comments field ▸ Hard-stop warnings appear or significant changes are required to send to pharmacy (eg, required to d/c order or remove drug/diagnosis) ▸ System is completely ‘bulletproof’, at least in regard to this particular order