Article Text
PDF
Abstract
Objectives To investigate the effect of an educational and feedback intervention to enhance consideration of sex differences in clinical guideline development.
Design Preintervention and postintervention questionnaires in intervention and control groups. Content analysis of intervention guidelines and former versions.
Setting Guideline consultants, working-group members and guideline documents of two Dutch guideline-developing organisations.
Main outcome measures Attitudes of guideline developers concerning the importance of considering sex differences and the number of the sex-specific statements in the contents of guideline documents.
Results The attitude of the intervention group did not change significantly relative to the control group. Consideration of sex-related factors within the guidelines increased relative to available previous versions.
Conclusion Education and expert feedback may increase consideration of sex differences in guidelines. Further efforts are needed to implement and test these interventions.
- Practice guideline
- sex characteristics
- attitude
Statistics from Altmetric.com
Clinical guidelines describe the best available evidence concerning the diagnosis and treatment of patients, and they play an important role in improving the quality of care.1 ,2 Since 1993, the National Institutes of Health and other research-funding agencies have required the equitable inclusion of men and women in clinical research.3–5 This has generated more evidence on sex differences in health outcomes.6–8 Women and men may need different preventive, diagnostic or therapeutic strat-egies.6 For example, the alcohol use disorders identification test (AUDIT) screening instrument for alcohol problems is less sensitive and more specific for women than it is for men, indicating different cut-off points.9 ,10
In a previous study, we showed that the two major Dutch guideline-developing organisations, the Dutch College of General Practitioners (NHG) and the Dutch Institute for Healthcare Improvement (CBO), paid no systematic attention to sex-related factors in either the development or content of their guidelines.11 Other research showed little attention to ethnic background of patients in guidelines.12 Both organisations therefore agreed to participate in a follow-up project (the Diversity Consultation Project) to improve consideration of diversity issues in guideline development.
Our previous study identified two barriers to considering gender issues in guideline development: lack of awareness that it might improve the quality of guidelines and lack of know-how for doing so.11 We designed an intervention to remove these barriers using education and expert feedback. Evidence from other settings suggests that such intervention strategies can effect change among professionals.13–17
This article describes the intervention and evaluates the extent to which it influenced the awareness and know-how of guideline developers regarding the importance of considering sex differences when developing and writing guidelines.
Methods
Setting
The CBO and the NHG have been developing clinical guidelines since the 1980s.18 ,19 Each uses a similar stepwise, internationally acknowledged, standard methodology of guideline development.20 Guidelines are produced by interdisciplinary working groups of experts. Consultants from the organisation provide technical and methodological support. Guidelines from the NHG focus on general practitioners, and the CBO guidelines focus on all medical (and paramedical) professions involved in care. The guidelines of the CBO are more extensive than those of the NHG.
The interventions
Two interventions were designed to enhance aware-ness among guideline working groups concerning the importance of considering sex differences in guideline development and to improve their ability to do so. Interventions in both organisations targeted consultants because they have the explicit task to provide methodological advice to guideline working groups. Also, as employees of the guideline organisations, they are a more steady factor in guideline development than members of working groups, who are sometimes involved in the development of only one guideline.
The first intervention involved a five-module interactive training course on incorporating the consideration of sex differences into the key steps of guideline development.21 The second intervention involved regular contacts between a gender expert and the consultants in the intervention groups to discuss how the working groups were considering sex differences in their work. The gender expert was a female epidemiologist. Four of the consultants in the intervention groups were women and two were men.
We expected that trained consultants would be able to transfer the methodology of sex-specific guideline development to their working group members, with the aid of occasional feedback from the gender expert.
We used control groups to evaluate the interventions.22
Outcome measures
Two outcome measures were assessed: the attitude of guideline developers towards the importance of considering sex differences in guidelines and the number of sex-specific statements in guidelines.
Selection of intervention and control groups
The project was carried out between 2004 and 2006.
In collaboration with senior staff members of the CBO and the NHG, for each organisation we selected three working groups as intervention groups. Selection criteria were as follows: the guideline development should start between January 2005 and June 2005; consideration of sex differences should be potentially relevant to the topic of the guideline; each group should have a different consultant.
The selected guidelines from the NHG groups involved eczema, thyroid gland disorders and chronic obstructive pulmonary disease; those from the CBO groups involved rheumatoid arthritis, alcohol abuse and dependence, and hereditary colon carcinoma.
For the measurement of the first outcome measure (attitude), the consultants and their groups were considered as the intervention group. The control group consisted of all other consultants within the CBO and the NHG, and members of nine working groups that were developing guidelines on other topics (eg, allergic rhinitis for the NHG and AIDS for the CBO) during the intervention period.
For the measurement of the second outcome measure—that is, the number of sex-specific statements in guidelines—we used the latest versions of the guideline documents that were produced by the CBO and NHG intervention groups as intervention guidelines. Because the three NHG guidelines were revisions of previous versions, we used the previous versions of those guideline documents as a control group. No previous versions of the guidelines from the CBO intervention groups were available.
Measurement of attitudes
To measure attitude, we included the following question on the consideration of sex differences within a questionnaire with statements on quality issues in guideline development: “I think that the current guidelines of the CBO/NHG should take sex differences between patients into account”. The other items in the questionnaire, which are available from the author, were derived from the AGREE instrument.23 They could be rated on a scale from 1 (not at all important) to 10 (very important). All consultants and working-group members from the intervention and control groups received the questionnaire, at the commencement of both interventions (T0), and 10–16 months later, after the last contact with the gender expert (T1). The questionnaires were distributed by email with an explanatory covering letter. Reminders were sent 2–3 weeks later.
Completed questionnaires were entered into SPSS Data Entry 4.0 and checked for errors using a random test. The above-mentioned question was used for analysis. Two-sample t tests were used to measure attitude change between the intervention and control groups. Data analysis was performed with SPSS V.14.0.
Content analysis of the guideline documents
To explore the consideration of sex differences in guidelines, content analysis was conducted on the most recent versions of the six selected guidelines. The content analysis screened the number of sex-specific statements and was based on a previous study.11 A statement was considered sex specific if it referred to men, women or both. We specified whether the content of sex-specific statements referred to reproductive health (pregnancy, breastfeeding and other aspects of reproduction) or to other aspects of health. We also specified whether statements were made in sections describing the underlying evidence and the considerations of the working group (hereafter, “underlying evidence”) or in a recommendation (hereafter, “recommendation”). DGK and a second researcher conducted independent content analyses of sex-specific statements. The second researcher was blinded as to which guideline development groups had participated in the interventions. Disagreements were resolved through discussion.
The content analyses of the three NHG guidelines were compared with the previous versions.
Implementation of interventions
The course was taught in March and April of 2005 to three NHG consultants and 11 CBO consultants. Because the CBO considered the course relevant for all consultants, control-group consultants were also invited to the course. Participants evaluated the content, working programme and training materials positively.21
The feedback intervention with the gender expert was offered only to the six consultants of the intervention groups. During the project, the expert contacted each of the five consultants three times, and one consultant twice. The gender expert initiated all but one of the contacts, in which the main topics included finding and interpreting relevant information on sex differences regarding the topic of the guideline. One guideline consultant asked the gender expert to comment on the most recent draft of the guideline document.
Results
Perceived importance of attention to sex differences in guideline development
The six intervention groups consisted of six consultants and 80 working-group members. The control groups consisted of 17 consultants and 112 working-group members. Response rates for the consultants of the intervention and control groups at T0 were 83% and 76%, respectively, and 100% and 76% at T1. Response was lower among working-group members: 46% and 42% at T0 and 36% and 39% at T1 for intervention groups and control groups, respectively (table 1).
Perceived importance of considering sex differences in clinical guidelines, before and after intervention
At T0, the responses of the control groups did not differ significantly from those of the intervention groups regarding whether sex differences should be considered in guideline development. Mean ratings varied between 7.38 and 8.00 (table 1). At T1, the consultants in the intervention group agreed more strongly with the statement than the control group did, but this difference was not significant. The working-group members in the intervention groups also responded more positively to the statement at T1 than those in the control groups, but the difference was not significant (table 1). Overall, no significant sex differences were observed in the responses of male or female consultants, or working-groups members in the control groups and the intervention groups (data not shown).
Content of the guidelines
Of the six guideline working groups selected for the intervention, two had finalised the guideline documents at the end of the study period and four had produced draft versions. These texts formed the basis of our content analysis. Table 2 reports results from the content analysis according to the typical structure of a guideline. All six guidelines provided sex-specific information in the introduction (table 2), although none mentioned gender when describing the target population. All but two guidelines (ECZ2 and RA) referred to sex differences in the section on diagnosis. Three (THYR2, COL and ALC) also provided sex-specific recommendations regarding diagnosis. All six guidelines provided underlying evidence on sex-specific issues in the section on therapy. Three guidelines (THYR2, COL and RA) contained 13, 4 and 13 sex-specific recommendations, respectively, with regard to therapy. Taken together, all six guidelines provided one or more sex-specific statements in sections reporting underlying evidence (see box 1 for examples). Many of the statements concerned reproductive health.
Consideration of sex differences in (current and previous) guidelines
Examples of sex-specific statements
| Introduction | On reproductive health | RA (in subsection containing key questions): what is the treatment policy for RA patients during lactation? |
| Other | THYR2 (in a description on hyperthyroidism and hypothyroidism): women acquire both conditions five times as frequently as men do. This ratio increases with age. | |
| Diagnosis underlying evidence | On reproductive health | THYR2: the results of T4 tests are lower for patients who have had haemodialysis or are in the third trimester of pregnancy |
| Other | ALC (in the description of a study examining various tests for primary care): the validity of the test was significantly different for men than it was for women. | |
| Diagnosis recommendation | On reproductive health | HYP2: if the condition began within a year of giving birth, it is probably a silent or painless lymphocytic hyperthyroidism and does not require referral for a thyroid gland scan. |
| Other | ALC: the recommended threshold for the AUDIT is 8 for men and 5 for women and older persons. | |
| Therapy underlying evidence | On reproductive health | RA: NSAIDs are likely to inhibit the contractions and cause haemorrhaging during labour. |
| Other | ALC: Poikolainen reports for women an average reduction of 51 g of alcohol intake per week as an effect of brief interventions. For men, an average could not be computed, but the reduction was more or less the same. | |
| Therapy recommendations | On reproductive health | RA: if there are clear medical grounds for pharmacotherapy before or during pregnancy, the possible risks of the pharmacotherapy must always be weighed against the possible risks of the active RA. |
| Other | COL: women with elevated risk for HNPCC–Lynch syndrome should be informed about the risk for endometrial carcinoma and the benefits and limitations of surveillance, and they should be educated about the early symptoms. |
AUDIT, alcohol use disorders identification test; NSAIDs, non-steroidal anti-inflammatory drugs; HNPCC, hereditary non-polyposis colorectal cancer.
All current NHG guidelines contained more sex-specific statements in underlying-evidence sections than the previous versions, and THYR2 provided more sex-specific recommendations (15) than THYR1 did (4).
Discussion
This article evaluates the impact of an intervention implemented by two Dutch guideline organisations to increase the consideration of sex differences in guideline development. The intervention consisted of a training course and feedback by a gender expert. The target group consisted of consultants of guideline working groups. The aim was to overcome previously identified barriers to the consideration of sex differences in guideline development.11
Questionnaire results revealed that the intervention and control groups deemed consideration of sex differences in guideline development relatively important, even before the intervention. The intervention groups considered it somewhat more important than the control groups did after the intervention, although the difference was not statistically significant. Therefore, there is no evidence that the interventions increased the gender awareness of the guideline developers, although these results should be generalised with caution. We intended to evaluate the effects of the intervention on the attitudes of the participants using a quasi-experimental design, contrasting intervention and control groups.24 In practice, however, the CBO did not fully comply with this research design. They were particularly enthusiastic about one of the interventions (the training), and they invited all consultants, not only the selected intervention group, to participate in it. This appears to be a more common obstacle in implementation studies: health organisations often find it difficult to delay the implementation of an intervention until it is properly tested.13 ,25 This may have influenced the responses of the control group, thereby affecting the measured difference between the intervention and control groups. Second, the validity of our data is limited by the low response rates (particularly among working-group members). Although financial incentives may have increased response,26 ethical requirements of Dutch guideline organisations prohibited their use. Third, social-acceptability bias cannot be ruled out, despite the inclusion of a question concerning sex differences in a longer questionnaire on quality issues in guideline development.
Content analysis revealed that the three NHG guidelines produced by the intervention groups included more sex-specific statements than the previous versions had. Although we were unable to compare the CBO guidelines with previous versions, comparison of our results with those from an earlier content analysis of CBO guidelines on other topics revealed that the new guidelines contain more sex-specific recommendations than the older guidelines had. The shares of sex-specific recommendations in the CBO guidelines for hypertension (2000), depression (draft 2002) and osteoporosis (2002) were 1%, 0% and 19%, respectively.11 The shares for the new guidelines COL, RA and ALC were 7%, 13% and 11%, respectively.
We cannot rule out that cultural and social developments have contributed to the somewhat greater attention to sex differences in the guidelines from the intervention groups. However, it also requires specific skills to be able to enhance the attention to sex differences.11 The identification, assessment and representation of sex-specific research evidence were important topics of the course and the conversations between the gender expert and the consultants. We found that, in the intervention guidelines, sex-specific statements often appeared in sections describing evidence underlying the recommendations. As consultants are key methodological advisors of guideline working groups, it is therefore likely that the interventions have enhanced their ability to incorporate the consideration of sex differences into the methodology of guideline development.
Several recommendations from the course and the gender expert were not evident in the guidelines. For instance, none of the guidelines refers to sex in describing the target population, and most of the sex-specific statements refer to reproductive health issues. Further research must determine whether this was attributable to the nature of the available research evidence27–30 or to a lack of attention among guideline developers to sex-specific evidence beyond reproductive health.
Despite its limitations, this study suggests that guideline developers find it important to consider sex differences in guideline development. This result is relevant, as attitudes are known to trigger behavioural change.16 ,31 The study also suggests that some changes had occurred in guideline documents produced by the intervention groups. We assume that the interventions contributed to these changes. Nonetheless, studies in other settings are needed to measure the overall and specific effects of the interventions. Given the experiences in this study, there are several methodological problems that future studies need to address.
In this study, we measured attitude changes with the answer on one single question. A broader range of questions and additional (qualitative) interviews with a sample of guideline developers may offer a better understanding of the nature of attitude changes that may occur as a result of the intervention. Given the complexities that are associated with the implementation of educational and organisational interventions in healthcare organisations, researchers must consider if evaluation of this intervention by means of a (randomised) controlled study is feasible. Other research designs may also be useful.32 Another question that requires future investigation is whether placing sex-specific evidence in the guidelines makes a difference in the decision making by health professionals in clinical practice.
Our interventions were limited to staff members of guideline organisations. Based on its experience with this project, the CBO has decided to provide information on sex-specific guideline development to the members of working groups. The organisation has incorporated aspects of the gender course into its regular training for guideline developers. The NHG and the CBO are currently using sex-specific search filters to locate sex-specific evidence. This filter has recently been tested and made available to a wider range of guideline organisations and researchers.33 Each organisation has appointed a staff member as the focal point for diversity issues.
Conclusion
This study suggests that guideline developers are becoming aware that the quality of clinical guidelines may be improved by considering this evidence. It also provides preliminary evidence that training on sex-specific evidence and feedback from a gender consultant are effective ways of increasing the ability of professionals to put this into practice. Our interventions are the first to address sex differences in guideline development. Because the modules of the training course and the methods used by the gender consultant are widely transferable, both interventions could be useful for many organisations that are active in guideline development.
Acknowledgments
We would like to thank the personnel at the NHG and the CBO for participating in this study and distributing the questionnaire and for their efforts to draw attention to this issue. We thank the working-group members for their cooperation with this study. We would also like to thank the gender consultant, Clara Moerman. In addition, we are grateful to Jolanda Habraken, Susanne de Kort and Clara Moerman for providing a second analysis of the guideline documents.
References
Footnotes
Funding We would like to express our appreciation to the Netherlands Organisation for Health Research and Development for making this study possible (ZonMw; grant no. 36100 028).
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.