You are here

  • Home
  • Archive
  • Volume 30, Issue 4
  • Work effort, readability and quality of pharmacy transcription of patient directions from electronic prescriptions: a retrospective observational cohort analysis

Article Text

Article menu

Download PDFPDF

Original research
Work effort, readability and quality of pharmacy transcription of patient directions from electronic prescriptions: a retrospective observational cohort analysis
  1. Yifan Zheng1,
  2. Yun Jiang2,
  3. Michael P Dorsch1,
  4. Yuting Ding1,
  5. V G Vinod Vydiswaran3,
  6. Corey A Lester1
  1. 1 Department of Clinical Pharmacy, University of Michigan College of Pharmacy, Ann Arbor, Michigan, USA
  2. 2 Department of Systems, Populations and Leadership, University of Michigan School of Nursing, Ann Arbor, Michigan, USA
  3. 3 Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor, Michigan, USA
  1. Correspondence to Dr Corey A Lester, Clinical Pharmacy, University of Michigan College of Pharmacy, Ann Arbor MI 48109, Michigan, USA; lesterca{at}umich.edu

Abstract

Background Free-text directions generated by prescribers in electronic prescriptions can be difficult for patients to understand due to their variability, complexity and ambiguity. Pharmacy staff are responsible for transcribing these directions so that patients can take their medication as prescribed. However, little is known about the quality of these transcribed directions received by patients.

Methods A retrospective observational analysis of 529 990 e-prescription directions processed at a mail-order pharmacy in the USA. We measured pharmacy staff editing of directions using string edit distance and execution time using the Keystroke-Level Model. Using the New Dale-Chall (NDC) readability formula, we calculated NDC cloze scores of the patient directions before and after transcription. We also evaluated the quality of directions (eg, included a dose, dose unit, frequency of administration) before and after transcription with a random sample of 966 patient directions.

Results Pharmacy staff edited 83.8% of all e-prescription directions received with a median edit distance of 18 per e-prescription. We estimated a median of 6.64 s of transcribing each e-prescription. The median NDC score increased by 68.6% after transcription (26.12 vs 44.03, p<0.001), which indicated a significant readability improvement. In our sample, 51.4% of patient directions on e-prescriptions contained at least one pre-defined direction quality issue. Pharmacy staff corrected 79.5% of the quality issues.

Conclusion Pharmacy staff put significant effort into transcribing e-prescription directions. Manual transcription removed the majority of quality issues; however, pharmacy staff still miss or introduce following their manual transcription processes. The development of tools and techniques such as a comprehensive set of structured direction components or machine learning–based natural language processing techniques may help produce clear directions.

  • pharmacists
  • medication safety
  • human error
  • human factors
  • information technology
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjqs-2019-010405

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    According to the Institute of Medicine report, Preventing Medication Error, more than one-third of 1.5 million reportable adverse drug events happen in outpatient settings every year and they result in healthcare costs of about $1 billion annually in the USA.1 Globally, according to WHO, the cost related to medication errors could be up to around US$42 billion annually.2 Poor patient comprehension of the prescription label is an important root cause of medication administration errors.3 4 Although verbal counselling serves as a useful tool to communicate administration instructions, studies have reported there is not sufficient communication between patients and their prescribers or pharmacists.5–7 Thus, written patient directions on prescription labels remain an important artefact to help patients take medication appropriately.8 Written patient directions on a prescription label may be even more important during the COVID-19 outbreak, as direct contact with patients is discouraged and limited in pharmacies.9

    Patient directions are also commonly referred to as ‘Sigs’ which is the abbreviation for the Latin word ‘Signetur’ meaning ‘let the medication be labeled’. Sigs contain important instructive information on safe medication administration. In electronic prescriptions (e-prescriptions) in the USA, the patient directions are mostly transmitted to pharmacies via a 140-character free-text field in the National Council for Prescription Drug Programs (NCPDP) SCRIPT Standard V.10.6.10 Of note, the NCPDP has recently extended the length of free text from 140 to 1000 characters in SCRIPT Standard V.2012 to accommodate medications with complicated directions.11 The NCPDP Structured and Codified Sig Segment, a technique developed by NCPDP in 2008 to represent Sig components in a standardised format, can also transmit the patient directions between prescribers and pharmacies though it is not intended to replace the free-text patient directions. E-prescription patient directions in the USA are still transmitted via the free-text data field due to potential ambiguities when combining different types of direction segments and an incomplete set of terms available for the codified fields.12 13

    It is convenient and efficient for prescribers to use the free-text field to transmit Sigs with flexibility. However, the free-text directions can also cause quality issues.11 14–16 One of the issues is that the strings used in free text expressing the same concept can vary significantly. In a study assessing the quality and variability of patient directions in e-prescriptions in the outpatient setting, the concept of ‘take 1 tablet by mouth once daily’ had 832 permutations such as ‘take 1 tablet by oral route every day’ and ‘take one (1) tablet by mouth once a day’.13 The variability from the free-text field could consequently lead to interpretation challenges for pharmacy staff that may interrupt the pharmacy workflow, decrease transcription efficiency and negatively affect patient counselling. Moreover, the free-text patient directions from e-prescriptions may lead to quality-related events if they miss important action verbs (eg, take, inhale, inject, etc), dosing, necessary indication information, or contain medical abbreviations, Latin words and medical jargon. It was reported in one study that one in ten free-text directions has quality-related events that could increase medication safety risk and cause workflow interruption.13

    Quality issues are prevalent in patient directions on e-prescriptions despite the implementation of various decision tools for constructing patient directions, including the electronic prescribing system in England and NCPDP Structured and Codified Sig Segment in the USA.12 13 17 For example, one study of 15 English community pharmacies identified that e-prescriptions were associated with higher labelling error rates and failed to decrease direction editing rates and content errors by pharmacy staff compared with other prescription types.17 Without pharmacy staff to edit and double-check prescription directions, patients may not be able to understand and follow the directions on e-prescriptions appropriately. Thus, pharmacy staff play a key role in manually transcribing them into prescription label directions to make directions more understandable for patients. However, this manual transcription process can also cause issues. On one hand, the process can be a burden for pharmacy staff and patients. A study conducted in Finland showed that the process significantly increases pharmacies’ workload and extends patients’ service time.18 On the other hand, it can introduce other quality challenges due to the repetitious nature of the task and the high workload of pharmacy staff. For example, one study in the USA assessed the variability of pharmacy interpretations of physician prescriptions from 85 prescription labels.14 Pharmacy staff omitted the dose frequency on 6% of prescription label directions and included precise administration timing on only 2% of prescription labels.

    Precise transcription is an important part of producing high-quality prescription label directions given the high variability from prescribers and pharmacy staff. Thus, it is critical to better understand the transcription process and evaluate relevant outcomes of transcribing. However, little research has occurred regarding this manual transcription process of e-prescription directions. The objective of this study is to quantify transcription work effort, evaluate the readability level before and after transcription, and assess the quality of directions before and after transcription.

    Methods

    This was a retrospective, observational cohort analysis considering three quality and safety perspectives of the transcription process. First, we describe the effort of pharmacy staff manually transcribing e-prescription directions into prescription label directions with quantitative methods. Second, we assess the readability of patient directions before and after the transcription. Finally, we evaluate the transcription process by assessing patient direction quality in a randomly selected sample. The Institutional Review Board reviewed and approved this analysis.

    Data sources

    Data containing e-prescription and prescription label pairs were obtained from a mail-order pharmacy in the USA. There were 537 710 e-prescription and prescription label pairs from January 2017 through October 2018 in the dataset. There are 19 variables in the original dataset including medication name, national drug code, medication strength, medication directions, medication days’ supply and national provider identifier. Examples of e-prescription directions before and after transcription include

    • ‘1 p.o. q. day’ → ‘take 1 tablet by mouth every day’.

    • ‘take 1 tablet(s) twice a day by oral route’ → ‘take one tablet by mouth twice a day’.

    All data were manipulated using R Studio, V.3.6.1.19 Figure 1 contains a description of the data filtering procedures. The prescriber’s geographical location information was obtained by linking the prescription national provider identifier to the November 2018 the United States National Plan and Provider Enumeration System registry file.20 The medication therapeutic class information was obtained by linking the national drug code of e-prescriptions to the United States National Library of Medicine’s RxClass medication classification system via the RxNorm application program interface.21

    Figure 1
    Figure 1

    Flowchart of data filtering procedures

    Pharmacy staff work effort during the transcription process

    In this study, the edit distance represented the number of keystrokes required to make the patient directions on e-prescriptions equal to the pharmacy prescription labels. It was defined as the minimum number of inserting and deleting operations required to transform one free-text string into another.22 23 The longest common subsequence algorithm was used to calculate the edit distance since it only considers inserting and deleting characters compared with the Levenshtein distance that includes substitution operations. It is a simulative and conservative method to calculate the minimum number of manipulations performed by pharmacy staff to transcribe the e-prescription directions. The Keystroke-Level Model was used to estimate the typing execution time.24 We followed the protocol published in 2001 to calculate the amount of time pharmacy staff may need to transcribe the patient directions from e-prescriptions to prescription labels.25 The research team defined the action sequence using the following steps:

    1. Press ‘Tab’ to get into the editing box: 0.28 s.

    2. If n (n=edit distance) ≥1, initiate the editing (decide to do the task): 1.2 s.

    3. Perform the editing including deleting and inserting: 0.28 s×n.

    4. Press ‘Tab’ to get to the next box: 0.28 s.

    According to the action sequence above, if no edition (n=0), the estimating time for transcription is 0.56 s, which is the time to tab through the editing box. If editing happens, the equation to estimate the time is

    Embedded Image

    Readability assessment

    In this study, the New Dale-Chall (NDC) readability formula was the method of choice.26 We used this metric because it primarily uses a list of ‘easy’ words to determine the percentage of words in a free-text string classified as difficult words (ie, those words not contained in the ‘easy’ NDC word list). Examples of ‘easy’ words include blood, tablet and twice. NDC is based on the two most important factors in the classic readability measurement: semantic (word) difficulty and syntactic (sentence) difficulty. These fit the characteristics of medication patient directions. Other readability metrics require average word and sentence length (Flesch-Kincaid Grade Level), numbers of polysyllabic words and sentences (Simple Measure of Gobbledygook), or the sentence length and the proportion of polysyllabic words (Gunning-Fog Index). These alternative metrics may be inaccurate due to frequent medical abbreviations with fewer syllables and shorter word/sentence length in e-prescription directions.27–29 Consistent with NDC recommendations, percentages of difficult words in the directions instead of exact numbers were used to accommodate samples shorter than 100 words.26 NDC cloze scores for each direction were calculated from these percentages and abstracted to reading levels. NDC cloze scores correlate with reading comprehension where higher NDC cloze scores represent better comprehension. Reading levels were abstracted from NDC cloze scores to different grade levels (eg, 1st grader, college graduate). The lower the reading level, the easier the text is to read and understand. NDC cloze scores were calculated using the formula in the quanteda package in R studio (formula listed in online supplementary appendix) and the reading levels were converted using the table described in online supplementary appendix table S1.19 26 30 An age range was indicated based on the corresponding reading level and statistics from the US National Center for Education Statistics.31 A Wilcoxon test was used the evaluate the effect of transcription on NDC cloze scores.19 32 33

    Supplemental material

    Quality evaluation

    To evaluate the quality of patient directions, a simple random sample of 1000 e-prescriptions was obtained using the sample function in the dplyr package.19 34 E-prescriptions for medical devices (eg, blood glucose testing supplies) were removed due to the lack of suitable evaluation criteria. The quality of patient directions in the random sample was assessed both before (ie, the directions from prescriber e-prescriptions) and after transcription (ie, the directions on prescription labels) using the standards adapted from national E-Prescribing Quality Guidelines.35 The following criteria determined which prescriptions contained quality issues:

    1. Patient directions must include all the required components (ie, action verb, dose, dose unit, route and frequency) unless the patient direction states ‘Take/use as directed/instructed’.

    2. An indication must be included in patient directions if there is ‘PRN/as needed’ stated in the direction.

    3. A qualifier that instructs patients to obtain detailed administration information must be included when the patient direction states ‘Take/use as directed/instructed’. For example, a patient direction that reads ‘use as directed per instructions on package insert’ should be used instead of ‘use as directed’.

    4. The use of all abbreviations (including Latin words), acronyms, symbols and medical jargon must be avoided.

    Two researchers, a Doctor of Pharmacy candidate and a healthcare engineer (YZ and YD), independently annotated the random sample of the e-prescription and prescription-label pairs of directions. The researchers evaluated and labelled the presence or absence of each pre-defined quality criterion. If the same quality issue was present more than one time in a single prescription direction, the issue was counted as one occurrence. Quality issues are reported as a percentage using the number of prescriptions containing that quality issue divided by the total number of patient directions included in the analysis. Inter-rater reliability was calculated using Cohen’s Kappa method for each quality issue category (median: 0.86, 1st–3rd quartile: 0.68–0.97).36 All discrepancies between labelled quality issues were resolved through a consensus-based approach with a third researcher (CAL). The major evaluation variability came from the different interpretations for the definitions of quality issues such as whether ‘at bedtime’ versus ‘every night at bedtime’ contain a frequency of use. McNemar's test was used to test the null hypothesis that the transcription process does not have an effect on the incidence rate for each quality issue category.19 37

    Results

    In total, there were 529 990 e-prescription and prescription-label pairs of 3838 unique medication or medical device products from 65 035 prescribers after the filtering procedures. Table 1 lists the demographic characteristics of the dataset.

    View this table:
    Table 1

    Demographic characteristics of the prescription data

    The edit distance estimated the number of keystrokes for transcribing directions from e-prescriptions to prescription labels by pharmacy staff. Figure 2 describes the distribution of the editing distances. Most e-prescription directions (83.8%) had at least one character edited by pharmacy staff. This analysis found a median (1st–3rd quartile) edit distance of 18 (8–29) per e-prescription during the transcription process. Based on the Keystroke-Level Model, the conservative time estimate that pharmacy staff spent transcribing one e-prescription direction was 6.64 s (minimum: 0.56 s, maximum: 53.28 s, 1st–3rd quartile: 4.00–9.88 s).

    Figure 2
    Figure 2

    Histogram of the edit distance (ie, number of character insertions and deletions during transcription) using longest common subsequence (LCS) algorithm for each electronic prescription

    Before transcription, the median NDC cloze score (1st–3rd quartile) of e-prescription directions was 26.12 (34.79–13.74); equivalent to the average reading level for 11th–12th grade students, who are typically between 16 and 18 years old. After the transcription, the median NDC cloze score increased to 44.03 (46.61–32.03); equivalent to the average reading level for 4th grade students, who are typically between 9 and 10 years old. There was a 68.6% increase in the median cloze score after transcription (median: 26.12 vs 44.03, Wilcoxon signed-rank test, p<0.001). Figure 3 shows the probability densities for NDC cloze scores of prescription directions before and after the transcription.

    Figure 3
    Figure 3

    Probability densities for New Dale-Chall cloze scores of prescription directions before and after the transcription

    Patient directions were evaluated for quality issues and the prevalence was compared before and after transcription. Figure 4 describes and compares the prevalence of each quality issue. Of the 1000 randomly selected e-prescriptions, 966 e-prescriptions were included after removing 34 medical device labels. Before transcription, 497 (51.4%) e-prescriptions contained at least one quality issue, while after transcription, 109 (11.3%) e-prescriptions contained at least one quality issue. This left 395 (79.5%) e-prescriptions with quality issues that were fully resolved by pharmacy staff transcription. However, the transcription process also introduced quality issues. There were 7 (0.7%) patient directions without any quality issues from e-prescriptions that became problematic after manual transcription and, in total, 34 quality issues were introduced by pharmacy staff in the random sample. Pharmacy staff transcription had a significant effect (McNemar’s test, p<0.001) on adding an action verb, dose unit, route and removing abbreviations/Latin words/medical jargon. Online supplementary appendix table S2 reports each quality issue before and after transcription along with example directions from the data.

    Figure 4
    Figure 4

    Incidence rate of quality issues of each category before and after transcription in random 966 prescriptions.

    Discussion

    To the best of our knowledge, this is the first study to characterise the work of pharmacy staff during the e-prescription transcription process in an outpatient pharmacy using the methods of edit distance, keystroke time and NDC readability. Quality issues of the e-prescription directions before and after the transcription were also evaluated and compared in this study.

    Despite the implementation of decision-support systems and various patient direction building tools, many e-prescription directions received by pharmacies have issues that require examination, transcription and double-checking from pharmacy staff around the world. In England, 21.9% of e-prescriptions were edited by community pharmacists to enhance the patient directions, while in Finland, 7.2% e-prescriptions contain errors, ambiguities and other shortcomings.17 18 Most e-prescription directions (83.8%) in our study were edited on review by pharmacy staff. The higher percentage in the USA may result from its decentralised architecture of the e-prescription system, a wide variety of e-prescription order entry system and no national e-prescription database. The e-prescribing systems in England and Finland are more similar, which have a centralised architecture and a national e-prescription database.38

    For each e-prescription, we estimate that pharmacy staff inserted or deleted a median of 18 text characters during the transcription process. This corresponds to 6.64 s spent executing the typing based on the proposed keystroke-level model. Each week, the mail pharmacy involved in this study processed an average of 6023 e-prescriptions. This resulted in around 11.1 hours spent on transcription in a week. To put this in perspective, there were 1.91 billion new e-prescriptions transmitted in the USA during 2018.39 Given that the median pay for one pharmacist is US$60.64/hour and for one pharmacy technician is US$15.72/hour,40 41 manual transcription is a time-consuming and expensive task. A survey of Finnish pharmacists also confirmed that most e-prescription anomalies (errors, ambiguities and other shortcomings) increase pharmacies’ workload and extend patients’ service time.18 The edit distance in our study is a direct but conservative estimation because it measures the minimum number of manipulations. In addition, the keystroke-level model does not include the time pharmacy staff may take for clarifying patient directions. To process e-prescriptions requiring a prescriber consultation, literature review or other interventions, a pharmacist can take an average of 6.07 min to correct an e-prescription with a quality issue.42 The more time pharmacy staff spend transcribing, the less time they may have for other tasks such as product selection verification, evaluating drug–drug interactions, updating allergies or other health information, and providing medication counselling. This can negatively affect medication safety and pharmacy service quality.

    The cloze score of directions from both e-prescriptions and prescription labels was calculated using a traditional readability metric, the New Dale-Chall (NDC) formula.26 43 Based on the NDC readability analysis, e-prescription directions were written at the average reading level 11th–12th grade student who is usually 16–18 years old.31 After transcription by pharmacy staff, the reading level decreases to an average 4th grade student reading level who is usually 9–10 years old.31 This result is important because it verified that the directions on the e-prescriptions from prescribers are difficult for most laypersons and demonstrates the value of the transcription performed by pharmacy staff. Based on estimates from the Institute of Medicine, 90 million adults in the USA have difficulties understanding and acting on health information.44 The directions on prescription labels must be easy to read and understand by patients, so patients can take their medications correctly. Patient understanding of their medication directions on prescription labels is an important part of improving patient medication adherence, improving therapeutic outcomes and decreasing medication safety risks.3 4

    Finally, we evaluated the quality of patient directions with 966 randomly selected e-prescription and prescription-label direction pairs. Our findings showed that 51.4% of e-prescription directions contained pre-defined quality issues and 11.3% of prescription label directions contained at least one quality issue after transcription. Moreover, one patient direction could have multiple quality issues. For example, a patient direction such as ‘Take 1 tab po’ is missing the frequency information and also includes abbreviations like ‘tab’ and ‘po’. When directions on prescription labels are missing essential information, patients may take the medications inappropriately, which may lead to adverse drug events or suboptimal therapeutic outcomes. In addition, it is hard for patients to interpret and follow the direction correctly when the prescription label directions contain abbreviations or medical jargon. Consequently, the trust between pharmacists and patients may be impaired due to the confusion about the content of prescription label directions.45

    The indication is also a key component of the directions for patients and pharmacy staff.46 Patients rely on the indication information to take the medications under the correct condition. Although it is not required, the indication for use is recommended on e-prescription and prescription label directions. In our random sample, only a small number of e-prescriptions (5.0 %) included indication information in patient directions and the finding is consistent with other research in the USA.47 However, national e-prescribing guidelines recommend that indication is a required component for ‘as needed’ or ‘PRN’ directions.35 Our analysis found that 3.5% and 3.6% of patient directions on e-prescriptions and prescription labels, respectively, were missing indications when the prescriptions called for ‘as needed’ directions. Consequently, patients may be at an increased risk of a drug overdose or lack of medication adherence if they do not know the reason to use their ‘as needed’ medication.48 It has been reported that patients are less likely to be adherent to the medication regimen if they believe the medication is not necessary or know little about the medication.49 50 An indication on the prescription label can improve patient knowledge about the timing of medication taking and may lead to more appropriate medication use. Another issue is using ‘Take as directed’ without further referencing the source of the instruction. In the random sample, 4.1% of prescription label directions contained this issue. This issue may confuse patients in terms of where to find instructive information and may result in medication safety risks if patients rely on an incorrect source of instruction.

    During the transcription, pharmacy staff removed most quality issues and improved the readability level. However, the manual transcription process is far from perfect and in some cases introduced certain types of quality issues. In this study, certain quality issues are more likely to be resolved while editing prescription label directions. For example, missing an action verb or route can be resolved without further clarification from the prescriber since pharmacy staff easily infer the relevant information based on their knowledge of the medication (eg, taking by mouth can be inferred if the medication’s dose unit tablet). However, for other quality issues such as no dose, the information may not be inferred, and pharmacy staff may need further clarification from the prescriber. It was reported in one study that more new e-prescriptions necessitate a clarification contact compared with prescriptions in other formats (2.0% vs 1.2%).51 The clarification contacts may result in a delay in patients receiving their medications. Moreover, based on findings in figure 4 and online supplementary appendix, for some advanced quality standards like the required indication with ‘PRN’ or ‘as directed’ with a qualifier, pharmacy staff are less likely to resolve the quality issues and sometimes may introduce the quality issues. Non-compliance with the NCPDP standards or best practice recommended in the e-prescription guidelines by national e-prescribing networks is common. This likely reflects a lack of comprehensive, well-accepted and enforced prescription label standards for directions during the transcription process.

    One potential strategy to improve the efficiency and quality of the transcription process is to standardise the direction components on e-prescriptions by implementing an independent structured and codified data segment in the e-prescribing system. It can decrease the workload of transcription and it may also reduce potential misinterpretations of the prescriber’s intent. However, the structured and codified data is not a required segment in the e-prescription system and the industry has been slow to adopt this functionality. E-prescription patient directions in the USA are still transmitted via the free-text data field.12 13 It was pointed out in one study that the NCPDP Structured and Codified Sig Format should not be immediately used as a federally mandated component of outpatient e-prescription standard due to lack of a complete set of terms available for the codified fields and potential ambiguities.12 In the future, we could use natural language processing and machine learning methods to transcribe the free-text directions from e-prescriptions automatically.

    Of note, this study has important limitations. First, the prescription-label data were from a single mail-order pharmacy and the results may not be generalisable to other pharmacies. However, the e-prescriptions received by the mail-order pharmacy were from across the USA and should represent the wide variety of directions requiring transcription by pharmacy staff. Second, because the edit distance used measures the minimum number of characters inserted or deleted in order to make the e-prescription direction equal to the prescription label direction, it likely underestimates the true transcription effort. For example, the edit distance and time spent on transcription may increase dramatically in practice if pharmacy staff use a more complicated transcription procedure compared with our keystroke-level model such as deleting all characters and re-writing from scratch or making editing typos and corrections. This study did not investigate the relationship between edit distance and medication class. Certain medications or classes may have more complicated regimens that require greater editing of directions (eg, a prednisone taper regimen). In terms of the readability assessment, the validity of the NDC readability formula for short medical text has not been confirmed and it is a surrogate of readability assessment. In the future, patients’ subjective ratings may be used to validate the improvements in readability.

    Conclusion

    Pharmacy staff engage in sustained efforts to transcribe patient directions in order to make them accurate, complete and easier to read. However, achieving this requires significant time editing the e-prescription directions on the part of the pharmacy staff. Besides, despite a large amount of time spent transcribing, more than 1 in 9 directions on prescription labels still had quality issues that may confuse patients when they are deciding how to take their medication. If misinterpreted by patients, these issues may pose a significant safety risk and concern. Increased efforts to develop tools or techniques to help standardise the patient directions such as a comprehensive set of structured direction components or machine learning–based natural language processing techniques could support a more efficient and safe transcription process for pharmacy staff and lead to improved medication use by patients.

    References

    1. ↵
      2. Aspden P ,
      3. Wolcott J ,
      4. Bootman L
      . Institute of Medicine. Preventing medication errors, 2006.
    2. ↵
      2. Aitken M ,
      3. Gorokhovich L
      . Advancing the responsible use of medicines: applying levers for change. SSRN Electron J 2013.
    3. ↵
      2. Shrank W ,
      3. Avorn J ,
      4. Rolon C , et al
      . Effect of content and format of prescription drug labels on readability, understanding, and medication use: a systematic review. Ann Pharmacother 2007;41:783–801.doi:10.1345/aph.1H582 pmid:http://www.ncbi.nlm.nih.gov/pubmed/17426075
      OpenUrlCrossRefPubMed
    4. ↵
      2. Wolf MS ,
      3. Davis TC ,
      4. Shrank W , et al
      . To err is human: patient misinterpretations of prescription drug label instructions. Patient Educ Couns 2007.
    5. ↵
      2. Tarn DM ,
      3. Heritage J ,
      4. Paterniti DA , et al
      . Physician communication when prescribing new medications. Arch Intern Med 2006;166:1855. doi:10.1001/archinte.166.17.1855 pmid:http://www.ncbi.nlm.nih.gov/pubmed/17000942
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      2. Sleath B ,
      3. Roter D ,
      4. Chewning B , et al
      . Asking questions about medication: analysis of physician–patient interactions and physician perceptions. Med Care 1999;37:1169-73. doi:10.1097/00005650-199911000-00009 pmid:10549619
      OpenUrlPubMed
    7. ↵
      2. Stevenson FA ,
      3. Cox K ,
      4. Britten N , et al
      . A systematic review of the research on communication between patients and health care professionals about medicines: the consequences for concordance. Health Expect 2004;7:235–45.doi:10.1111/j.1369-7625.2004.00281.x pmid:http://www.ncbi.nlm.nih.gov/pubmed/15327462
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      2. Hernandez LM
      . Standardizing medication labels: confusing patients less: workshop summary. National Academies Press, 2008.
    9. ↵
      1. Centers for Disease Control and Prevention
      . Considerations for pharmacies during the COVID-19 pandemic. Available: https://www.cdc.gov/coronavirus/2019-ncov/healthcare-resources/pharmacies.html#Strategies [Accessed 17 Apr 2020].
    10. ↵
      1. National Council for Prescription Drug Programs
      . Script implementation recommendations 2019.
    11. ↵
      2. Dhavle AA ,
      3. Rupp MT
      . Towards creating the perfect electronic prescription. J Am Med Inform Assoc 2015;22:e7–12.doi:10.1136/amiajnl-2014-002738 pmid:http://www.ncbi.nlm.nih.gov/pubmed/25038197
      OpenUrlCrossRefPubMed
    12. ↵
      2. Liu H ,
      3. Burkhart Q ,
      4. Bell DS
      . Evaluation of the NCPDP structured and Codified Sig format for e-prescriptions. J Am Med Inform Assoc 2011;18:645–51.doi:10.1136/amiajnl-2010-000034 pmid:http://www.ncbi.nlm.nih.gov/pubmed/21613642
      OpenUrlCrossRefPubMed
    13. ↵
      2. Yang Y ,
      3. Ward-Charlerie S ,
      4. Dhavle AA , et al
      . Quality and variability of patient directions in electronic prescriptions in the ambulatory care setting. J Manag Care Spec Pharm 2018;24:691–9.doi:10.18553/jmcp.2018.17404 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29345553
      OpenUrlPubMed
    14. ↵
      2. Wolf MS ,
      3. Shekelle P ,
      4. Choudhry NK , et al
      . Variability in pharmacy interpretations of physician prescriptions. Med Care 2009;47:370–3.doi:10.1097/MLR.0b013e31818af91a pmid:http://www.ncbi.nlm.nih.gov/pubmed/19194338
      OpenUrlCrossRefPubMedWeb of Science
    15. ↵
      2. Bailey SC ,
      3. Persell SD ,
      4. Jacobson KL , et al
      . Comparison of handwritten and electronically generated prescription drug instructions. Ann Pharmacother 2009;43:151–2.doi:10.1345/aph.1L388 pmid:http://www.ncbi.nlm.nih.gov/pubmed/19126825
      OpenUrlCrossRefPubMed
    16. ↵
      2. Davis TC ,
      3. Wolf MS ,
      4. Bass PF , et al
      . Literacy and misunderstanding prescription drug labels. Ann Intern Med 2006;145:887. doi:10.7326/0003-4819-145-12-200612190-00144 pmid:http://www.ncbi.nlm.nih.gov/pubmed/17135578
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      2. Franklin BD ,
      3. Reynolds M ,
      4. Sadler S , et al
      . The effect of the electronic transmission of prescriptions on dispensing errors and prescription enhancements made in English community pharmacies: a naturalistic stepped wedge study. BMJ Qual Saf 2014;23:629–38.doi:10.1136/bmjqs-2013-002776 pmid:http://www.ncbi.nlm.nih.gov/pubmed/24742778
      OpenUrlAbstract/FREE Full Text
    18. ↵
      2. Timonen J ,
      3. Kangas S ,
      4. Kauppinen H , et al
      . Electronic prescription anomalies: a study of frequencies, clarification and effects in Finnish community pharmacies. J Pharm Health Serv Res 2018;9:183–9.doi:10.1111/jphs.12224
      OpenUrl
    19. ↵
      1. RStudio Team
      . RStudio: integrated development environment for R, 2016. Available: http://www.rstudio.com/
    20. ↵
      NPI files. Available: https://download.cms.gov/nppes/NPI_Files.html [Accessed 15 Nov 2018].
    21. ↵
      1. National Institutes of Health, Services D of H& H. U.S
      . National Library of Medicine RxNav. Available: https://mor.nlm.nih.gov/RxNav/ [Accessed 9 Jul 2019].
    22. ↵
      2. Masek WJ ,
      3. Paterson MS
      . A faster algorithm computing string edit distances. J Comput Syst Sci 1980.
    23. ↵
      2. der Loo MPJ
      . The stringdist package for approximate string matching. 6, 2014: 111–22.
      OpenUrl
    24. ↵
      2. Card SK ,
      3. Moran TP ,
      4. Newell A
      . The keystroke-level model for user performance time with interactive systems. Commun ACM 1980;23:396–410.doi:10.1145/358886.358895
      OpenUrlCrossRef
    25. ↵
      2. Kieras D
      . Using the keystroke-level model to estimate execution times, 2001. Available: http://www-personal.umich.edu/~itm/688/KierasKLMTutorial2001.pdf
    26. ↵
      2. Chall JS
      . Readability revisited: the New Dale-Chall readability formula. Japanese J Educ Media Res 1996.
    27. ↵
      2. Kincaid JP ,
      3. Fishburne Jr RP ,
      4. Rogers RL
      . Derivation of new readability formulas (automated readability index, Fog count and Flesch reading ease formula) for navy enlisted personnel, 1975.
    28. ↵
      2. McLaughlin GH
      . Smog grading: a new readability formula. J Reading 1969;12:639–46.
      OpenUrlWeb of Science
    29. ↵
      2. Gunning R
      . The technique of clear writing. New York: NY McGraw-Hill B Co, 1968.
    30. ↵
      2. Benoit K ,
      3. Watanabe K ,
      4. Wang H , et al
      . quanteda: an R package for the quantitative analysis of textual data. J Open Source Softw 2018;3:774.doi:10.21105/joss.00774
      OpenUrl
    31. ↵
      2. Snyder TD ,
      3. de Brey C ,
      4. Dillow SA
      . Digest of Education Statistics 2017. Natl Cent Educ Stat, 2019.
    32. ↵
      2. Bauer DF
      . Constructing confidence sets using rank statistics. J Am Stat Assoc 1972;67:687–90.doi:10.1080/01621459.1972.10481279
      OpenUrlCrossRefWeb of Science
    33. ↵
      2. Hollander M ,
      3. Wolfe D
      . Nonparametric statistical methods. 2nd Edn, 1999.
    34. ↵
      2. Wickham H ,
      3. Francois R ,
      4. Henry L , et al
      . dplyr: a grammar of data manipulation, 2018. Available: https://cran.r-project.org/package=dplyr
    35. ↵
    36. ↵
      2. McHugh ML
      . Interrater reliability: the kappa statistic. Biochem Med 2012;22:276–82.doi:10.11613/BM.2012.031 pmid:http://www.ncbi.nlm.nih.gov/pubmed/23092060
      OpenUrlPubMed
    37. ↵
      2. Agresti A
      . Categorical data analysis. In: Symmetry model Categ data anal. New York: John Wiley sons, 1990: 350–4.
    38. ↵
      2. Samadbeik M ,
      3. Ahmadi M ,
      4. Sadoughi F , et al
      . A comparative review of electronic prescription systems: lessons learned from developed countries. J Res Pharm Pract 2017;6:3-11.doi:10.4103/2279-042X.200993 pmid:28331859
      OpenUrlPubMed
    39. ↵
      1. SureScripts
      . National progress report. 2018, 2018. Available: https://surescripts.com/news-center/national-progress-report-2018/ [Accessed 12 Feb 2020].
    40. ↵
      1. Bureau of Labor Statistics; U.S. Department of Labor
      . Pharmacists occupational outlook handbook. Available: https://www.bls.gov/ooh/healthcare/pharmacists.htm [Accessed 28 Aug 2019].
    41. ↵
      1. Bureau of Labor Statistics; U.S. Department of Labor
      . Pharmacy technicians occupational outlook handbook. Available: https://www.bls.gov/ooh/healthcare/pharmacy-technicians.htm [Accessed 28 Aug 2019].
    42. ↵
      2. Warholak TL ,
      3. Rupp MT
      . Analysis of community chain pharmacists’ interventions on electronic prescriptions. J Am Pharm Assoc 2009;49:59–64.doi:10.1331/JAPhA.2009.08013
      OpenUrlCrossRef
    43. ↵
      2. Wu DTY ,
      3. Hanauer DA ,
      4. Mei Q , et al
      . Assessing the readability of ClinicalTrials.gov. J Am Med Inform Assoc 2016;23:269–75.doi:10.1093/jamia/ocv062 pmid:http://www.ncbi.nlm.nih.gov/pubmed/26269536
      OpenUrlCrossRefPubMed
    44. ↵
      2. Kindig DA ,
      3. Panzer AM ,
      4. Nielsen-Bohlman L , et al
      . Health literacy: a prescription to end confusion. National Academies Press, 2004.
    45. ↵
      2. Montague E ,
      3. Lee JD
      . Trust in health technologies. Handb Hum factors Ergon Heal care patient Saf 2012;2:281–91.
      OpenUrl
    46. ↵
      2. Fischer MA ,
      3. Vogeli C ,
      4. Stedman M , et al
      . Effect of electronic prescribing with formulary decision support on medication use and cost. Arch Intern Med 2008;168:2433–9.doi:10.1001/archinte.168.22.2433 pmid:http://www.ncbi.nlm.nih.gov/pubmed/19064827
      OpenUrlCrossRefPubMedWeb of Science
    47. ↵
      2. Salazar A ,
      3. Karmiy SJ ,
      4. Forsythe KJ , et al
      . How often do prescribers include indications in drug orders? Analysis of 4 million outpatient prescriptions. Am J Health Syst Pharm 2019;76:970–9.doi:10.1093/ajhp/zxz082 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31361884
      OpenUrlPubMed
    48. ↵
      2. Tarn DM ,
      3. Heritage J ,
      4. Paterniti DA , et al
      . Prescribing new medications: a taxonomy of physician–patient communication. Commun Med 2008;5:195-208.doi:10.1558/cam.v5i2.195 pmid:19644569
      OpenUrlPubMed
    49. ↵
      2. De Young M
      . Research on the effects of pharmacist–patient communication in institutions and ambulatory care sites, 1969–1994. Am J Health Syst Pharm 1996;53:1277–91.doi:10.1093/ajhp/53.11.1277 pmid:8725202
      OpenUrlAbstract
    50. ↵
      2. Horne R ,
      3. Weinman J
      . Patients' beliefs about prescribed medicines and their role in adherence to treatment in chronic physical illness. J Psychosom Res 1999;47:555–67.doi:10.1016/S0022-3999(99)00057-4 pmid:http://www.ncbi.nlm.nih.gov/pubmed/10661603
      OpenUrlCrossRefPubMedWeb of Science
    51. ↵
      2. Åstrand B ,
      3. Montelius E ,
      4. Petersson G , et al
      . Assessment of ePrescription quality: an observational study at three mail-order pharmacies. BMC Med Inform Decis Mak 2009;9:8. doi:10.1186/1472-6947-9-8 pmid:http://www.ncbi.nlm.nih.gov/pubmed/19171038
      OpenUrlCrossRefPubMed

    Footnotes

    • Twitter @dorschmike

    • Contributors CAL and YZ conceptualised the research question, study design and analysis, along with YJ, MPD and VGVV. YZ executed the analysis along with CAL and YD. CAL and YZ wrote and revised the manuscript. YJ, MPD and VGVV edited the manuscript.

    • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

    • Competing interests None declared.

    • Patient consent for publication Not required.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data availability statement Data are available on reasonable request with permission from the pharmacy.