Background Concerns with the usability of electronic prescribing (ePrescribing) systems can lead to the development of workarounds by users.
Objectives To investigate the types of workarounds users employed, the underlying reasons offered and implications for care provision and patient safety.
Methods We collected a large qualitative data set, comprising interviews, observations and project documents, as part of an evaluation of ePrescribing systems in five English hospitals, which we conceptualised as case studies. Data were collected at up to three different time points throughout implementation and adoption. Thematic analysis involving deductive and inductive approaches was facilitated by NVivo 10.
Results Our data set consisted of 173 interviews, 24 rounds of observation and 17 documents. Participating hospitals were at various stages of implementing a range of systems with differing functionalities. We identified two types of workarounds: informal and formal. The former were informal practices employed by users not approved by management, which were introduced because of perceived changes to professional roles, issues with system usability and performance and challenges relating to the inaccessibility of hardware. The latter were formalised practices that were promoted by management and occurred when systems posed threats to patient safety and organisational functioning. Both types of workarounds involved using paper and other software systems as intermediaries, which often created new risks relating to a lack of efficient transfer of real-time information between different users.
Conclusions Assessing formal and informal workarounds employed by users should be part of routine organisational implementation strategies of major health information technology initiatives. Workarounds can create new risks and present new opportunities for improvement in system design and integration.
- Qualitative research
- Decision support, computerized
- Health services research
- Implementation science
- Information technology
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
- Qualitative research
- Decision support, computerized
- Health services research
- Implementation science
- Information technology
Hospital electronic prescribing (ePrescribing) systems offer considerable potential to improving the quality, safety and efficiency of care.1 ,2 However, they are not yet part and parcel of everyday healthcare delivery in most UK hospitals.
New practices of delivering care and organising healthcare professional work associated with health information technology (HIT) introduction mean that users need to change existing practices to suit the digital environment.3–5 Workarounds are behavioural/technological strategies that users of technology employ to deal with perceived problems that hinder them in achieving their goal.6–8 They often arise in the context of changed workflows brought about by the new technologies.8 ,9 Workarounds have commonly been viewed as negative in discussions on HIT, as the resulting changes to working practices can pose significant risks to patient safety and organisational functioning.10–12 For example, Koppel and colleagues assessed workarounds in a barcode medication administration system and found that these resulted in a range of adverse consequences including administration of inappropriate medicines, unsuitable doses and incorrect times.12
Some have suggested that workarounds can also wrongly convey that solutions to existing system problems have been found and that managerial interventions are not required.11 However, the prevalence of informal workarounds may reflect weaknesses in organisational learning mechanisms insofar as local problem-solving by individuals and groups is not used as a resource to develop robust well-functioning systems, which is a defining characteristic of high reliability organisations.13–18
Being aware of workarounds and associated possible adverse consequences can help to mitigate risks by effectively redesigning existing workflows and/or technological systems. Conversely, although workarounds are often seen as disruptive and unintended from managerial perspectives as they can reduce the amount of organisational control over practices on the ground,19 they can also be adaptive, particularly when exploiting the innovative capacity of system users and also potentially identifying opportunities to enhance system design and work processes. In this capacity, they can also allow users to mitigate perceived risks to patient safety posed by HITs—for instance, when a system is not well designed and/or suited to the environment in which it is being deployed.20 Workarounds may also be able to help to identify inappropriate policies or business process changes surrounding HIT that may interrupt the delivery of day-to-day care.21 ,22 They can also serve as intentional organisational strategies to help users engage with new systems, for example, by maximising benefits and minimising disruption to existing procedures.23
The lack of existing empirical work and theorising surrounding workarounds in healthcare has repeatedly been emphasised.6 ,24 In particular, there is a need for better conceptual models of the reasons underpinning workarounds.24 To contribute to this evidence base, we examined workarounds as part of a large national evaluation of ePrescribing systems in English hospitals.3 The aims of this work were threefold, namely to investigate: (1) what workarounds were employed by users of ePrescribing systems; (2) why these were undertaken; and (3) how workarounds could create and mitigate risks associated with the implementation and adoption of hospital ePrescribing systems.
This work drew on a large data set we have amassed in recent years undertaking a series of longitudinal, multisite, qualitative case studies of commercially available ePrescribing systems in English hospitals. Previous publications from this data set have related to product diversity and markets,25 ,26 procurement,27–29 integration and interfacing,30 user engagement (Cresswell K, Lee L, Mozaffar H, et al. Sustained user engagement in health information technology. Health Services Research. Invited re-submission), implementation processes3 ,31 and patient and vendor perspectives.32 ,33 The avenues of investigation emerged from the existing literature, initial themes inductively identified from early results across sites, and our own experience from related evaluations of large-scale HIT initiatives.4 Participating hospitals were treated as individual cases.34
Approvals and permissions
We received Institutional Review Board approval from The University of Edinburgh, UK. An application was reviewed by a National Health Service Research Ethics Committee that classed the work as a service evaluation. All necessary regulatory approvals were obtained from case study sites and individuals gave written informed consent to participate. All names and places were anonymised by the authors in order to protect the anonymity.
Our purposeful sampling strategy aimed to identify English hospitals that had implemented, were in the process of implementing, or were planning to implement ePrescribing systems during the conduct of the research (a period of 5 years).35 Prior to implementation of ePrescribing, sites were using paper drug charts that nurses used to administer medication. As there were a limited number of hospitals with ePrescribing systems when our work commenced in 2011, we sampled those hospitals that were planning implementation of systems soonest. Organisations varied in demographic characteristics (urban/rural, teaching/non-teaching), type of system (stand-alone ePrescribing functionality vs as part of an integrated system) and time since implementation began (or was about to begin) (see table 1). Our initial point of contact was the Director of Pharmacy or equivalent.36
Within hospitals, we sampled a range of managers (information technology and clinical implementation team members) who were, or were to be, involved in implementing ePrescribing systems. We then snowball sampled a range of users from these contacts including doctors, nurses, pharmacists, allied health professionals and pharmacy technicians. These varied in levels of seniority and experience with ePrescribing system use.
Methods of data collection included a mixture of interviews, non-participant observations and gathering of documents. This range of data sources permitted us to gain insights into individual expectations and experiences (interviews), changes to work and organisational practices brought about by the implementation (observations) and organisational implementation plans and procedures (documents) (see table 2).37 Interviews were the main source of data and these were conducted by experienced qualitative researchers (KMC, HM, LL) at up to three different time points (see table 2). This longitudinal dimension allowed us to track changes over time and helped to assess temporary and longer-term difficulties encountered as well as strategies employed to address these.38 Interviews were tailored to individual sites, professions and backgrounds, but generally explored topics surrounding experiences and expectations, perceived changes to work practices and organisational functioning, and barriers/facilitators to implementation and adoption.
We also wrote field notes of non-participant observations. These included following individual system users throughout their normal working day and observing how they used the system. Observations also involved attending strategic meetings in order to gain insights into organisational dimensions of implementation strategies.
Organisational documents collected consisted of business cases, implementation plans, work practice maps, risk registers and lessons learnt reports. These provided insight into the practices that were perceived as compromising patient safety.
Interview data were digitally audio-recorded and transcribed verbatim. Data from all sources were uploaded onto NVivo 10.39
We employed a mixture of inductive and deductive analysis. Data were analysed within cases first, followed by cross-case analysis. Initially, the lead researchers (KMC, HM, LL) coded individual case study data, drawing on a system lifecycle model developed as a coding framework (the deductive component).40 This helped to conceptualise the implementation and adoption journey surrounding ePrescribing technology and included temporal stages from initial conception, through implementation, to system optimisation. During this process, we also noted emerging themes that were not captured in our code book, but which occurred frequently across data sources and individual participants (the inductive component).
Data across sites were then examined in designated data analysis meetings, where we drew on the Theory of Workarounds as an analytical lens.6 The Theory of Workarounds is concerned with different types of workarounds, attempting to explain why they occur, and describing their consequences. It suggests that workarounds emerge based on situational characteristics and the goals and needs of different stakeholders. The decision as to whether to employ a workaround is assumed to be based on perceived benefits, risks and costs. The framework therefore offered an appropriate lens to examine emerging issues across sites and contributed to a nuanced analysis of emerging issues from a range of different perspectives.
The credibility of findings was enhanced through triangulating data sources obtained through a range of methods, keeping reflexive field journals to record any potential researcher bias and through active involvement of the extended research team in the analysis. This included examining codes and discussing negative cases.
We conducted 173 interviews, 24 observations and collected 17 documents between December 2011 and March 2015 (see table 2).
We identified different types of workarounds employed by users of the new technology. These often involved using paper and other technologies for intermediary storage of information (see table 3). Workarounds were most commonly associated with perceived limitations in system design, potentially leading to patient safety risks. Our inductive analysis indicated that they could be divided into ad hoc workarounds, which were conceptualised as informal practices employed by users that were not approved by management (including the implementation team and senior clinical managers), and formalised workarounds, which were conceptualised as necessities that were accepted and sometimes actively promoted by management. An overall graphical presentation of our findings is given in figure 1.
Reasons for employing workarounds
Contingent nature of healthcare professional work practices and lack of system flexibility
The main underlying reason for adopting workarounds appeared to be the extraordinary complexity and variability of healthcare workflows, which were not well catered for in existing system designs. As a result, users had to “find shortcuts” (Site D, Pharmacy Technician, T3) or “find ways around” (Site E, Nurse, T2) the way the system defined their workflows. The limited capacity of systems to cater for a diversity of practices (eg, through libraries of pre-programmed options that could be locally configured without rewriting code) was seen as a consequence of these being relatively immature solutions. This was an issue both in relation to stand-alone systems and integrated systems. In addition, information displays often failed to address the exigencies of particular moments of healthcare work, necessitating for example a cumbersome switching between screens.
… at the moment [warfarin and International Normalised Ratio (INR)] run two different screens kind of within the system so [users] need to go and look at what the recent INR's are, go into the drug chart and see what the doses of warfarin have been and go to a different screen to prescribe and that just isn't a tidy way to work. (Site D, Implementation Team, T2)
Compensating for the increase in workloads associated with use of the system
Another underlying reason for employing workarounds was the perceived necessity to compensate for the increased time it took to use the systems. Here we found that workarounds helped users to accommodate the increased time required to perform tasks.
Everything is taking us a lot longer to do…and that's why we're trying to find ways around and just streamlining the process for us so that we can do as much as we can. (Site D, Pharmacy Technician, T2)
In stand-alone systems pharmacists, for instance, would verify orders in batches, as doing each order separately was perceived to take too long, finding a time and perhaps a quiet place (not in the dispensary) where they would not be interrupted.
…we now do like [system] slots where we have to sit and verify the prescription separately in the dispensary because it takes a lot longer to do and needs more concentration than what you would get in the dispensary… by the time you've checked the bloods, checked the other [paper patient notes], converted it, rang the doctor, gone back to it, you need to do that without being constantly interrupted like you would be in the dispensary. (Site D, Pharmacist, T1)
Changes to professional roles and responsibilities resulting from system design
Workarounds were also employed to mitigate perceived changes to professional roles and responsibilities. For instance, consultants in many cases delegated use of the system to their juniors:
I think there are some consultants who still sort of need their inbox to be managed by their secretary … And then when it comes to them doing the ward rounds, they would tell the juniors what to prescribe and what to do so they aren't very hands-on… (Site D, Pharmacist, T1)
Types of workarounds employed
Using other systems as intermediary storage of information
We frequently observed users drawing on other electronic systems as intermediary storage of information. This kind of workarounds was more common with stand-alone systems (ie, those that were not designed to integrate with medical notes), and included for instance using Microsoft Word to store temporarily text that needed to be transferred from the patient administration system.
It's just a [Microsoft] Word document that we cut and paste in, that we save on our desktops… (Site C, Pharmacist, T3)
Emails were also used as intermediaries. For example, a workaround employed by pharmacists related to requesting medication. As each requested medication from a nurse resulted in a printed request form in the dispensary (this process was not fully electronic due to limited functionality being deployed at the time), pharmacists designed a workaround that would allow nurses to request items through informal means (an email) which were then translated by pharmacists into a formal inpatient order on the ePrescribing system. However, this workaround was subsequently abandoned due to potential legal issues and increased workloads.
…the nurses were basically sending us an electronic message to say “please can you supply these four items”… and then someone in pharmacy was translating that onto the inpatient order and printing off one piece of paper requesting those four items. (Site A, Pharmacist, T2)
Similarly, paper was often used as an intermediary means to store information in instances where users did not have easy access to electronic means or were too busy to transfer information electronically. Paper was also used when users wanted to memorise something that needed to be transferred from one screen and to another (store and transmit).
The pharmacist goes back to the computer and puts a note onto a medication on [system]. He then takes a piece of paper and writes on it that he has left a note on [system] and where. He writes “[name of person] please” on it and states that “this is how I tend to communicate with doctors”. This is funny—he is writing a paper note to remind doctors to look at an electronic note! (Site A, Pharmacist, Observation, T1)
Other types of workarounds observed included entering information in batches to save time, and ticking boxes in order to satisfy the demands of the system (see table 3).20 ,41–45
Consequences resulting from workarounds: the introduction of new risks
Workarounds sometimes created new risks, such as delayed access to information or difficulty finding information for other professions. This was the case, for instance, when users did not follow the process dictated by the system in the right order. Here, doctors sometimes prematurely ticked a box in the system and then subsequently changed their mind, which was then not reflected in the system.
…if you do the histories first, it's very quick, if you don't, you've got to put each one in and then it doesn't then come right into the discharge letter so the process is really important but I can't make people do it, the system doesn't make you do it right. (Site D, Consultant, T2)
Similarly, some systems required users to refresh the screen to look at the most recent version of the medication chart. The need to manually refresh the screen meant that the information displayed might not be up to date and this could create the scope for errors.
I'd like to do away with the concept that you have to refresh a drug chart I think that's really, really dangerous because obviously you can train people as much as possible to say refresh it before you do anything but it's still an obvious banana skin lying around just for someone to slip over. (Site D, Junior Doctor, T2)
In some cases, shortcuts resulted in other healthcare professionals not finding the necessary information on the computer system. For example, in one hospital, we learned that nurses would tick a box in the system that stated ‘drug not administered’ (even though the medication was administered but not in line with the timing schedule of the system) as this was seen as the only way to complete a routine on the screen. When prompted for a reason by the system, they would tick another box stating ‘see notes’ to avoid having to complete what they perceived to be cumbersome patient records in the system. No note was however added. This presented issues for the pharmacist reviewing the medications who could then not find out why a medication had not been given.
…what you find is that [nurses] tend to tick any old box that doesn't necessarily relate to why the patient hasn't been given the drug, so the concern is that the common one that they tick is “see notes” and then there's no note generated to say why the drug wasn't given. (Site A, Pharmacist, T1)
Managers stated that appropriate training could to some degree minimise workarounds that were viewed as unnecessary.
I can't impose any workaround I have to work within the parameters of the system and it's about identifying what the issues are and just maybe training people the best way to use the system. (Site C, Pharmacists, T3)
Formalising strategies: management efforts to mitigate risks associated with workarounds
We also identified formalised workarounds that were seen as necessary by the organisation to maintain organisational functioning and patient safety while using an ‘imperfect’ technical system. These might arise because of a lack of integration in stand-alone systems, where limited functionality had been implemented in early integrated system implementations, where there were usability problems, or where adopter organisation practices diverged from those embedded in the software package. They were therefore promoted by management, shared across specialties and expected of users.
We still have frustrations around the basic software itself…but quite often we can develop workarounds… (Site A, Implementation Team, T2)
These formalised workarounds were promoted when persistent problems in technical design threatened patient safety (as opposed to issues associated with increased workloads and changing roles of staff). Some were designed to facilitate more efficient information transfer. For instance, when creating discharge medications on the ward, the system would not automatically alert the pharmacists to initiate the dispensing of discharge medications. Therefore, doctors on the ward and pharmacists agreed that a ‘fake drug’ would be created in the system in order to alert pharmacists that discharge medications were needed.
Now what they [doctors] do is they, on the inpatient order on the screen, they right click “convert to prescription”, and they end it with a discharge medication list that they've created from the inpatient list. Which is obviously just the things they need the patient to go home on, with appropriate course lengths for antibiotics and stuff like that. But there's no way for pharmacy to identify that that's been done. So the way we got around that is we created a fake drug called “TTO [To Take Out] for discharge” so the doctor would complete all the medicines and then prescribe TTO for discharge which appears on the inpatient order. (Site D, Pharmacist, T1)
Similarly, if prescribers wanted to use specific medicines that were not held in the system, then workarounds drawing on paper systems were created to compensate for this.
There are things that we can't do, you know, for example if a prescriber wants to prescribe and it's not on the system then the workaround this is to put in a generic drug and then we write in a note about what the drug is… (Site C, Pharmacist, T3)
In other instances, the system made actions ‘too easy’ for users. This was the case when nurses ordered medication at the click of a button, without checking which medicines had already been ordered as this would involve navigating to another screen. The increased ease of ordering resulted in a greater number of inappropriate and duplicate orders which pharmacists had to verify. As this was viewed as a time-consuming activity in the ePrescribing system, management created a workaround that allowed nurses to order medications in batches by email.
If an order is created using the system then it still needs somebody to go through and cross off the things that aren't needed and we've created a system whereby nurses can identify electronically that they wanted something and send an email saying “we want this, this and this” which obviously we can look at on the electronic system but we just found it loads of duplication, it was too easy to order. (Site C, Pharmacist, T3)
Temporality of workarounds
Improving patient safety was an important motivator for management to help users compensate for known technical limitations by mitigating known risks associated with the system. Many of these were temporary and associated with the gradual adoption and implementation of the system, which necessitated some degree of parallel paper and electronic means being used alongside each other. This was perceived to create risks of error.
A&E [Accident and Emergency] our emergency department still don't use [system]…they still write doses of medications on their casualty sheet, on paper…and we have had incidents where something is prescribed in A&E on paper, the nurses and medics don't realise and they get a second dose within a short time for example. (Site A, Consultant, T1)
In some instances, workarounds were also found to provide a quicker way for organisations to resolve the gap between local working practices and standardised ePrescribing packages than seeking to modify or customise the package.
Yes so it's not how [system supplier] envisage it being used, it's how we are then adapting the system to give it things that we want. And what you'll find is that we'll all do that in a slightly different way…because we've worked a workaround round. (Site C, Pharmacist, T1)
However, finding appropriate workarounds proved difficult or even impossible in some instances where efficient organisational functioning was threatened. In these instances, where workarounds could not be found, hospitals argued for the need to change the system.
… a workaround was brought up which was declined for governance reasons…it actually created a complete new level of work to be done (Site E, Implementation Team, T2)
Workarounds arise from the strategies employed by individuals or groups of users of HIT to mitigate perceived barriers to completing tasks. They can also become incorporated as part of organisational mandates to mitigate risks to patient safety brought about perceived shortcomings in the usability of new systems. We have found that, on occasions, these workarounds created new often unanticipated and unintended risks to patient safety.
The motivations to introduce workarounds were mainly associated with concerns surrounding poor usability, but in line with other work in the area, we also observed other motives for example where personal values were threatened or where systems did not align with professional norms or broader conditions (eg, time pressures).18 ,41 ,42 ,45 Although some types of workarounds observed here are consistent with previous research, particularly in relation to the use of paper and other software systems as intermediaries (table 3),20 ,41–45 and dealing with perceived system limitations, obstacles to work routines, threats to time and pretending compliance;18 ,46–51 we also found some strategies employed by users that involved delegating input of electronic information and compromising the temporal accuracy of electronic systems by deferring information input.
The distinction between informal and formalised workarounds can be particularly helpful in promoting organisational learning, cooperative working and promoting patient safety.13–18 It builds on the notion that workarounds can be used as intentional organisational strategies to help users gradually get used to new systems.23 Problems may arise if these are not addressed in a proactive manner to create more robust and well-considered solutions. If, for instance, consultants keep delegating data input to juniors (something than can be seen as first-order problem-solving), then this staff group may become increasingly overloaded. If management is not aware of this, then appropriate interventions, such as, for example, the employment of data entry clerks, are unlikely to be implemented in time. If, workarounds are formalised then it is easier to track and anticipate emerging systemic issues and embark upon mitigating action to address these. This suggests that workarounds only become a threat to organisational functioning if they are not formalised. The question is when and what to formalise and this could be a fruitful area of future work in this area.
The frequent use of paper and other software systems as intermediaries in both stand-alone and integrated systems may suggest a lack of system integration in the ePrescribing solutions we studied. They may therefore present temporary workarounds that may attenuate as increasing system functionality is implemented and systems are progressively integrated.51
Workarounds can both introduce new risks to safety (most commonly associated with a lack of up to date clinical information) and mitigate risks resulting from poor functionality and/or usability. As they often involve innovative user behaviours, they may therefore also be used as a source of design enhancements.20 However, it is important to keep in mind that workarounds can change over time and may become interdependent. They may also be seen as organisational/user ways to tailor systems to contexts of use and thereby impact on system standardisation, as well as system maintenance and upgrades. As such, they may be conceptualised as part of the tacit skills and cooperative activities through which work is accomplished as an everyday, practical activity.52 ,53 This suggests that rather than trying to demonise these kinds of informal practices, they should be made visible as they may be needed to ‘get a job done’. In doing so, it is important to keep in mind that different viewpoints are at play and views on which action needs to be completed often vary widely across organisational stakeholders.
It is therefore important to track how formal and informal workarounds are operationalised over time. Management tools incorporating such assessments exist and studying their routine use in healthcare settings is an important area for future work. For example, the workaround design system is a tool that can help to anticipate and address undesired workarounds.54 Similarly, the Workaround Process Modeling Notation incorporates workarounds into business process modelling, thereby helping to anticipate how changes to workflows are likely to be dealt with by users.55 However, such tools place the onus of modelling and design on managers, which may lead to inadequate reflection on informal practices. They also rely on the willingness/ability of developers to change systems accordingly, which is not always possible in large-scale commercial systems. In addition, they do not address issues surrounding the deactivation of workarounds once new functionalities are introduced.
We have collected a sizeable longitudinal qualitative data set drawing on a variety of perspectives and data sources. Although this work has provided some important insights into the undertheorised area of workarounds and their consequences in HIT, there are also a number of limitations. First, informal workarounds are hard to research and observe—particularly those that are undesired by management. As a result, we expect that users may not have reported/exhibited many informal workarounds. Second, workarounds can be subjective and hard to define. What constitutes a workaround from a managerial perspective may not necessarily be perceived as such from a user perspective and vice versa. Similarly, healthcare professionals may view workarounds differently. Third, some of the systems implemented in each hospital were originally designed for use in countries other than the UK. This may have resulted in more workarounds used due to international differences in workflows. We also did not gain insights into the nature of workarounds employed with paper systems, so we did not obtain any insights in relation to the nature of workarounds existing with these and how they changed with the introduction of the system.
This work has helped to provide insights into the way users and organisations have coped with the perceived limitations in functionality and usability of existing ePrescribing systems being deployed in English hospitals. Issues with information integration in particular seem to have contributed to a range of workarounds designed to bridge informational gaps. This highlights that rule-based accounts of organisational activities need to move away from viewing workarounds as a risk towards viewing them as a necessity.
The differentiation between informal and formal workarounds now needs to be taken forward in existing management tools designed to anticipate risks and opportunities created through workarounds employed by users. Essential to this will need to be recognition that informal workarounds may require system re-design and/or need to be formalised in order to better anticipate potentially adverse consequences.
We gratefully acknowledge the input from our Independent Programme Steering Committee, which is chaired by Prof Denis Protti and has Prof Sir Munir Pirmohamed, Prof Bryony Dean Franklin, Ms Eva Leach, Ms Rosemary Humphreys, and Ms Ailsa Donnelly as members. We also gratefully acknowledge the input of Rosemary Porteous (RP), who transcribed the discussions. Members of the Programme Team are: Dr Ann Robertson, Prof Jill Schofield, Dr Jamie Coleman, Ms Ann Slee, Prof David Bates, Dr Zoe Morrison, Mr Alan Girling, Mr Antony Chuter, Dr Laurence Blake, Prof Anthony Avery, Prof Richard Lilford, Dr Sarah Slight, Dr Behnaz Schofield, Ms Sonal Shah, Ms Ndeshi Salema, Mr Sam Watson, and Dr Lucy McCloughan.
Contributors AS and RW conceived this work. LL, KMC and HM collected data for this study. KMC led on data analysis and drafting of the manuscript. All authors have commented on various versions of this manuscript and inputted into the analysis.
Funding National Institute for Health Research (RP-PG-1209-10099).
Competing interests None declared.
Ethics approval The work received Institutional Review Board approval from The University of Edinburgh, UK.
Provenance and peer review Not commissioned; externally peer reviewed.