Article Text
PDF
Abstract
Objective To assess whether direct to healthcare professional communications (DHPCs) are of sufficient quality to be applicable in clinical practice and study how the quality differs according to safety concerns and type of monitoring.
Design Retrospective cohort study.
Setting DHPCs containing monitoring instructions were identified among all DHPC issued in Denmark between 2007 and 2018.
Intervention Quality of information of monitoring instructions was assessed according to the Systematic Information for Monitoring (SIM) score. Associations between different characteristics of instructions and the SIM score were compared with analysis of variance and the post hoc test Tukey’s honestly significant difference if significant.
Results In total, 297 DHPCs were issued, of which 97 contained 134 monitoring instructions. For 95% of these DHPCs the European Medicines Agency was involved. The average SIM score was 2.6±1.6 (ranging 0–6) and only 47% were considered of sufficient quality (SIM score ≥3). In addition, even fewer (11%) instructions were considered a ‘adequate instruction’ which also reported about facts and risks. Differences between quality of information according to type of monitoring were observed, specifically between clinical monitoring (average SIM score 1.9) and biomarker monitoring (physical average SIM score 2.9, p=0.029 and laboratory average SIM score 3.4, p<0.0001).
Conclusions Monitoring instructions were found not to be of sufficient quality to be applicable in clinical practice according to the SIM score. Our study concludes the need for further research and regulatory steps to ensure improve quality of information in safety communications.
- health & safety
- public health
- adverse events
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
All direct to healthcare professional communications sent out in Denmark between 2007 and 2018 were included in this study, with 95% of these being send out in agreement with the European Medicines Agency.
Quality of information of all monitoring instructions were assessed by independent researches using the Systematic Information for Monitoring (SIM) score.
All items of the SIM score may not be equally relevant for all types of monitoring, thus underestimating the clinical relevance.
Introduction
Approximately 10% of all drugs require regulatory action to inform healthcare professionals about new safety concerns that appear after marketing approval.1 New important safety information can be delivered directly to healthcare professionals through direct healthcare professional communications (DHPCs) also known as ‘Dear Doctor Letters’.2–4 DHPCs are sent directly by the marketing authorisation holder or by a competent authority. When a DHPC pertains to a medical product authorised in more than one European Union (EU) member state European coordination is required to ensure that a consistent messages is send across all relevant EU countries.2 DHPCs are expected to support safe and effective use of medicines and it is the primary used additional risk minimisation measure.5 They are disseminated in situations where immediate action or a change of the practical use of a drug is required.2 The EU guideline on good pharmacovigilance practice (GVP) module XV specifies these situations to include: drug withdrawals, new contraindications, identification/re-identification of a new previous unknown or known risk and new recommendations or monitoring instructions for preventing adverse reactions.2 The GVP module XV also includes a template for DHPCs, stating that safety concerns provided in DHPCs should be presented in context along with the benefits of the drug. DHPCs should further include relevant information about the safety concern such as seriousness, severity and frequency and explain any recommendations to the healthcare professionals and/or patients on how to handle the safety concern, and evidence supporting the recommendation.2
DHPCs are regarded to be an important communication tool for regulatory authorities and pharmaceutical industry.6 7 However, the usefulness of DHPCs has been debated and previous studies found a lack of clarity of the information provided.8–10 Interview studies and surveys among healthcare professionals revealed the importance of clear and appropriate information provided in safety communications to enhance the usability of the content.8 10 11 Healthcare professionals also argued that the facts and risks behind the recommendations should be presented in order to understand the reasoning behind the monitoring and thereby enabling them to take action on the recommendations.8 11 Furthermore, the usability of DHPCs in clinical practice has been criticised.8 12 A study found that monitoring instructions were infrequently and irregularly performed despite repeated regulatory interventions.13
The quality and clinical applicability of monitoring instructions provided in other (routine) safety communications (ie, summary of product characteristics (SmPCs)), have previously been evaluated.14–16 These studies all came to similar results concluding a scarcity of the information describing the monitoring instructions. To our knowledge the applicability of the monitoring instructions provided in DHPCs has not yet been evaluated. Given the importance of DHPCs in communicating new emerging safety concerns, the objective of this study was to analyse the quality of information of monitoring instructions provided in DHPCs as well as reporting of facts and risks substantiating the instruction. Additionally, this study investigated how the quality differed according to safety concerns and type of monitoring.
Methods
Study design
This retrospective cohort study included all available DHPCs issued in Denmark between 1 January 2007 and 31 December 2018. The DHPCs were obtained through an informational request to the Danish Medicines Agency (DKMA) (2007–2012) and were retrieved from the DKMA website (2013–2018).17
DHPCs were excluded if they did not contain a minimum of one monitoring instruction for a drug. All instructions were collected by examining the complete DHPC and a monitoring instruction was defined as a statement in the DHPC to check or observe the patient in connection to the use of a drug. Screening only, was not considered as a monitoring instruction (eg, screening for a contraindication before initiating a treatment).
Data collection
Data collection was tiered; first high-level characteristics of the DHPC and instructions were collected for all DHPCs. This included, date of issuance, the drug (active ingredients) as defined by the WHO Anatomical Therapeutic Chemical (ATC) classification system, the adverse event (AE) as defined in the Medical Dictionary for Regulatory Activities (MedDRA version 20.1) using the preferred terms (PT) and aggregating at primary System Organ Class (SOC) level.18 19 In addition, the type of all instructions in each DHPC were categorised by subjects, which could include withdrawal, contraindication, quality issue, availability issue, screening/monitoring, general warning/precautions and dose reductions (not mutually exclusive). Second, all DHPCs containing recommendations regarding screening/monitoring were selected, from which all DHPCs with only screening instructions were excluded. For all included DHPC, each separate monitoring instruction was retrieved and categorised based on type of monitoring: clinical monitoring, biomarker monitoring with physical parameters or biomarker monitoring with laboratory parameters. Clinical monitoring included observational or behavioural signs and symptoms such as pain and suicidal behaviour and so on. Physical biomarker monitoring included physical measurements such as weight and blood pressure and so on, whereas laboratory biomarker monitoring included laboratory measurements such as potassium levels in blood or bone marrow biopsy and so on.
To evaluate the quality of information of the monitoring instructions the Systematic Information for Monitoring (SIM) score was used as applied in Nederlof et al.14 The SIM score was first developed by Ferner et al,16 and was later used in adapted versions as an instrument to evaluate quality of monitoring instructions provided in SmPCs.14 15 The SIM score used evaluates six information items as in Nederlof et al,14 omitting the item ‘why to monitor’ as it was considered too similar to the item ‘how to respond’, thus resulting in double counting. Each item was scored with either a 1 (yes) if it was specific or present or a 0 (no) if the information was lacking, resulting in a maximum score of 6 for each monitoring instruction. The information items were: ‘what to monitor’ (eg, heart rate instead of cardiovascular examination), ‘when to start monitoring’, ‘when to stop monitoring’, ‘how frequently to monitor’, ‘critical value’ (what to look for) and ‘how to respond’. In line with the study by Nederlof et al,14 monitoring instructions were considered of sufficient quality if the SIM score was 3 or above. Additionally, two of the information items (‘what to monitor’ and ‘how to respond’) were considered to be the most important SIM score information items applicable to all types of monitoring, in line with the study by Geerts et al.15
To capture whether facts and risks behind the instructions were reported, we coded whether the source of evidence supporting the instructions (eg, clinical trial or Periodic Safety Update Report) was provided (yes/no) and whether there was a quantified estimate of the risk mentioned in connection to the safety concern subject to monitoring (yes/no). As a combined measure of a ‘adequate instruction’, for each instruction it was assessed whether all of the following criteria were met: mentioning of the facts as well as risks behind the instructions, together with the two most important information items (‘what to monitor’ and ‘how to respond’) and a SIM score of at least 3.
All data were extracted by two researchers independently. In case of a discrepancy, consensus was sought and discussed with a third researcher.
Statistical analysis
The monitoring instructions from the DHPCs were used as the unit of analysis, meaning that DHPCs that contained more than one instruction, were counted as separate units (eg, monitor for rashes as well as blood pressure). Descriptive statistics were used to summarise the characteristics of the monitoring instructions and their quality of information. The assumption of normality was tested using D’Agostino normality test. The hypothesis, that the SIM score was different from 3, was tested with a one-sample t-test against the hypothetical value of 3.
To evaluate the association between type of safety concern as well as type of monitoring and the SIM score, analysis of variance (ANOVA) was performed and if significant, the Tukey’s honestly significant difference (HSD) was used as post hoc test to find the significant different means within the type of characteristic.
All statistical tests were two-sided and were considered statistically significant if the p value <0.05. All analyses were performed by using R (V.3.6.1).20
Patient and public involvement statement
Patients or the public were not involved in this research.
Results
A total of 297 DHPCs were issued in Denmark between 2007 and 2018.17 Ninety-seven of the 297 DHPCs (33%) included monitoring requirements of a patient’s treatment and/or AEs, with a majority of these being send out in agreement with European Medicines Agency (EMA) (95%). The 97 DHPCs contained 134 monitoring instructions, an average of 1.4 (range, 1–3) per DHPC. The number of DHPCs sent out per year fluctuated somewhat over the years and drug classes, with a peak in 2013 and most DHPCs sent out for antineoplastic and immunomodulating agents (ATC code L). This pattern was similar for the DHPCs with monitoring instructions (see table 1 and online supplementary figure 1).
Supplemental material
Overview of DHPC, DHPC with monitoring instructions and number of instructions send out per year, and per drug class
Monitoring instructions most frequently related to safety concerns categorised under infections and infestations (17%), hepatobiliary disorders (11%) and cardiac disorders (11%, table 2). A majority of the monitoring instructions concerned clinical monitoring (42%), where observational signs and symptoms often included examination of skin or eyes, as well as observations of a specific behaviour such as Parkinson’s symptoms. A combination of monitoring types was found in 14% of the instructions, where infusion reactions (PT: hypersensitivity, SOC: immune system disorders) had the highest contribution to this.
Characteristics of the Instructions for monitoring categorised by the most frequently identified safety concerns displayed by System Organ Class
The SIM score distribution is displayed in figure 1. The minimum SIM score of 0 was found for 13 (10%) instructions (see online supplementary table 1 for more details) and the maximum SIM score of 6 in 7 (5%) instructions.
Supplemental material
Distribution of the SIM score for all (n=134) monitoring instructions. SIM, Systematic Information for Monitoring.
The average SIM score (2.6, SD 1.6) of all monitoring instructions was found to be statistically different from 3 (t (133)=−3.305, p=0.001, table 3). The information items ‘what to monitor’ (68%) and ‘how to respond’ (69%) were the most commonly items fulfilled. The information item ‘critical value’ was the least present of the items (16%). There was no significant difference in the SIM scores between the safety concerns displayed as SOCs. A significant difference in SIM scores was observed between the types of monitoring (ANOVA: F (3 128)=9, p<0.0001, table 3). Specifically, between clinical monitoring and physical biomarker monitoring (Tukey’s HSD p=0.03), as well as between clinical monitoring and laboratory biomarker monitoring (Tukey’s HSD p<0.0001, table 3). Clinical monitoring had the lowest average SIM score (1.9, SD 1.1) and five out of six information items were less frequently described for clinical monitoring instructions than for the other types of monitoring (see table 3). Laboratory biomarker monitoring instructions had the highest average SIM score (3.4, SD 1.7, table 3).
Information items and SIM score by safety concern and monitoring type
Of all the monitoring instructions 47% had a SIM score of 3 or above, whereas the majority (69%) of laboratory monitoring instructions and only 27% of the clinical monitoring instructions had a SIM score of 3 or above (table 4 and online supplementary table 1 for other cut-off values). For 41% of the instructions a quantified estimation of risk was provided, and a bit more than half (54%) mentioned the source of evidence (see table 4). The most important information items (‘what to monitor’ and ‘how to respond’) were described together in 55% of all instructions and this was similar for all types of monitoring (table 4). Only 11% of the instructions were categorised as ‘adequate instruction’ according to the combined measure of the most important SIM score information items, mentioning of a source of evidence and quantified risk as well as a SIM score of at least 3.
Instructions fulfilling a SIM score of 3 or above, the two most important SIM items (‘what to monitor’ and ‘how to respond’), the mentioning of a source of evidence and a quantified risk
Discussion
Even though safety communications are sent when change of the practical use of a drug is needed and are one of the most widely used additional risk minimisation measures,2 5 this study indicates more than half of all monitoring instructions provided in DHPCs are not of sufficient quality to be clinically applicable. The average SIM score of all monitoring instructions in DHPCs sent out in Denmark in the period from 2007 to 2018 was below 3. In particular, instructions for monitoring of clinical signs and symptoms of AEs had a lower quality of information compared with instructions relating to monitoring of physical biomarkers or laboratory biomarkers. Overall, instructions most often lacked information about the critical value that should be monitored for (eg, a cut-off value for blood pressure). This was especially apparent for instructions about clinical monitoring, where only a small fraction of instructions provided information on critical values. For physical and laboratory biomarker monitoring the information about critical value was provided in about a quarter of the instructions. While ‘critical value’ could arguably be important for laboratory instructions (eg, potassium levels <2.5 mmol/L), it may be difficult to identify a specific critical value in clinical monitoring, such as a behaviour indicator or a sign/scale of a rash. For a majority of instructions, information on ‘what to monitor’, ‘when to start monitoring’ and ‘how to respond’, was present, with ‘what to monitor’ and ‘how to respond’ being the most common information items present in the instructions. ‘What to monitor’ and ‘how to respond’ have previously been indicated as the most important SIM score information items by others.15
In addition, to assess quality of information of monitoring instructions, our study also mapped the extent to which source of evidence supporting the instruction was cited in the DHPCs as well as how often the risks of concern were quantified. While the source of evidence was linked to instructions for a small majority, below half of the instructions had a quantified risk mentioned, even though this information is explicitly asked for in the DHPC template in the GVP guideline.21 A recent review by Møllebæk et al reported that lack of clinical applicability and insufficient information about evidence were found to be limiting factors for the usability of DHPCs.12
Other have used the SIM score to capture quality of monitoring instructions to healthcare professions although in SmPC.14 15 While all found that only a minority of instruction could be considered clinically applicable, differences in the quality can be observed. Instruction for DHPCs appear to fair a bit better compared with instructions in SmPCs as reported by Nederlof et al who found an average SIM score of 2.0±1.7 in 232 monitoring instructions in SmPCs of psychotropic drugs, with only 34% having a SIM score ≥3. Additionally, the information items ‘what to monitor’ (37%) and ‘how to respond’(34%) were found in fewer instructions than in this study.14 Geerts et al found only 17% of all laboratory biomarker instructions in SmPC from the 200 most frequently prescribed drugs in 2006/2007 in the Netherlands to be of sufficient quality. Our present study, however, found that 69% were of sufficient quality for this type of monitoring instructions. It is important to note that the SIM score was designed differently by Geerts et al, requiring information of at least three essential information items (‘critical value’, ‘what to monitor’ and ‘how to respond’) to be categorised as sufficient quality.15 In our study, the six SIM score items were weighed equally, with the reasoning that different items can be essential depending on the type of monitoring. Ferner et al used a wider scale for scoring of SIM items, with the minimal acceptable score 12 point of maximum 31 point considered as sufficient quality. Though, coming to similar results with more than 40% of instructions to monitor for haematological AEs in SmPC of 84 non-haematological drugs.16
Compared with previous work assessing quality of information of monitoring instructions, this study developed the aggregate measure of ‘adequate instruction’. This incorporated all three previous quality indicators: a SIM score of at least 3,14 the presence of the two of the information items (‘what to monitor’ and ‘how to respond’)15 and the reporting of facts and risks behind the instructions.8 11 12 Approximately 1 in 10 of the monitoring instructions included in our study were found to be an ‘adequate instruction’. Overall, the previous studies regarding monitoring instructions together with our study indicate inadequate guidance of monitoring in today’s safety communications. Yet, newly emerged risks could be associated with limited evidence, and therefore it may not be an easy task to create clear and informative instructions for monitoring. Correspondingly, a recent qualitative study investigated the pharmaceutical industry’s point of view the process of DHPC preparation, and found that company representatives were positive about the template provided in the GVP guideline, although they emphasised that the template is suggestive rather than law.22 The study further indicated that it was difficult to write clear and transparent information in DHPCs with very limited text.22
Strengths and limitations
The results of this study should be interpreted in perspective to the strengths and limitations. First, to minimise the subjectivity when applying the SIM score, two researchers evaluated the fulfilment of SIM score items for each monitoring instruction independently. Discrepancies were solved through discussion and when needed a third researcher was consulted. A conservative and strict approach was applied when the SIM score information items were evaluated. This may have led to an underestimation of the measure of quality of information of the instruction.
Although we collected DHPC disseminated in Denmark, these reflect the situation in the EU. European coordination is required to ensure that a consistent message is send across all relevant EU countries,2 which is reflected in the 95% EMA involvement in our sample.
Finally, the threshold for an instruction to be of sufficient quality was a SIM score at least 3, although some instructions with a SIM score below 3 might be applicable in clinical practice anyway. On the other hand, it is important to remember that necessary information about the monitoring could still be missing with a SIM score of 3 or above. Furthermore, depending on the type of monitoring, different information items could be essential and necessary. However, we are confident that a higher SIM score would give a better description of the quality of monitoring instructions and thereby higher applicability in the clinical practice.
Implications for clinical practice
It is important that healthcare professionals know how to monitor their patient’s treatment and/or AEs and thereby are able to protect their patients against potential harms. For this reason, it is a concern that the quality of information of monitoring instructions provided in a majority safety communication is inadequate and might not applicable in clinical practice according to the measure applied in this study. If healthcare professionals are expected to follow these monitoring instructions, clear guidance on monitoring and the data to support it should be provided. This is confirmed by Piening et al, who found that the healthcare professionals considered drug safety information very important for their work in clinical practice, but that they could not clearly identify the instructions to be followed in safety communications.23 The impact of safety communications according to new monitoring instructions, might reflect the poor quality of the instructions as well, when previous studies suggest that regulatory actions only had a modest effect and did not achieve meaningful or sustained improvement of the monitoring.13 24 25
Recommendations
Together with previous studies that investigated safety communications, this study indicates several concerns about a lack of clear and usable information provided in DHPCs. These results therefore offer valuable insights to improvements of the current risk and safety communication. Investigations into measures such as an expansion of more specific suggestions in the DHPC template might help to resolve these concerns.
The specificity of instructions might be improved by strengthening the DHPC template without making it law, and/or involving healthcare professionals in the drafting process. However, further research is needed to explore how best to ensure the quality of safety communications.
Conclusions
In conclusion, this study found a suboptimal quality of information of the monitoring instructions provided in DHPCs. Additionally, differences of the quality were observed, where laboratory and physical biomarker monitoring had a higher quality than clinical monitoring. The results of this study together with findings from previous research on safety communications show the need for further regulatory steps (eg, further improvement of the DHPC template) to ensure quality and applicability of safety communications in clinical practice. This study did not investigate how safety communications should guide the healthcare professionals extensively enough to suggest how guidelines can be updated and improved. However, our findings highly demand further research on this matter to further improve the regulatory guidelines.
Acknowledgments
We would like to thank Gudlaug Olafsdottir and Sif Maj Bukhave for assisting with the initial retrieval and classification of Danish DHPCs.
References
Footnotes
Contributors M-MGH led study conception and design, data collection, quality of information assessment of monitoring instructions, data analysis and interpretation of data. AB contributed to the quality of information assessment of monitoring instructions. MLDB and CEH were involved in study conception and design, data analysis and interpretation of data. All four authors participated in the manuscript preparation, editing and revision and agreed upon the final version of the paper.
Funding No funding was received for this study. AB’s employment at University of Copenhagen was funded by a grant from the National Health and Medical Research Council of Australia (APP1122332).
Competing interests MLDB and CEH are employed by the University of Copenhagen at the Copenhagen Centre for Regulatory Science (CORS). CORS is a cross-faculty university anchored institution involving various public (Danish Medicines Agency, Copenhagen University) and private stakeholders (Novo Nordisk, Lundbeck, Ferring pharmaceuticals, LEO pharma) as well as patient organisations (Rare Diseases Denmark). The centre is purely devoted to the scientific aspects of the regulatory field and with a patient-oriented focus and the research is not company-specific product or directly company related. In addition MLDB is part-time employed by Utrecht University as senior researcher conducting research under the umbrella of the Utrecht-WHO Collaborating Centre for Pharmaceutical Policy and Regulation. This Centre receives no direct funding or donations from private parties, including pharma industry. Research funding from public–private partnerships, for example, IMI, The Escher Project (http://escher.lygature.org/) is accepted under the condition that no company specific product or company related study is conducted. The centre has received unrestricted research funding from public sources, for example, WHO, Netherlands Organisation for Health Research and Development (ZonMW), the Dutch National Health Care Institute (ZIN), EC Horizon 2020, the Dutch Medicines Evaluation Board (MEB) and the Dutch Ministry of Health. M-MGH and AFB were affiliated to the University of Copenhagen while the research described in this paper was preformed, M-MGH is currently employed at Sandoz A/S and Arnela Boskovic is currently employed at Novo Nordisk A/S, neither of the companies were involved in any aspect of the described work.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available upon reasonable request. Direct Healthcare Professional Communications send out between 2007 and 2012 were retrieved through an informational request to the Danish Medicines Agency, and require approval to be shared with third parties.