Background and objective: Indicators to measure the quality of healthcare are increasingly used by healthcare professionals and policy makers. In the context of increasing antimicrobial resistance, this study aimed to develop valid drug-specific quality indicators for outpatient antibiotic use in Europe, derived from European Surveillance of Antimicrobial Consumption (ESAC) data.
Methods: 27 experts (15 countries), in a European Science Foundation workshop, built on the expertise within the European Drug Utilisation Research Group, the General Practice Respiratory Infections Network, the ESCMID Study Group on Primary Care Topics, the Belgian Antibiotic Policy Coordination Committee, the World Health Organization, ESAC, and other experts. A set of proposed indicators was developed using 1997–2003 ESAC data. Participants scored the relevance of each indicator to reducing antimicrobial resistance, patient health benefit, cost effectiveness and public health policy makers (scale: 1 (completely disagree) to 9 (completely agree)). The scores were processed according to the UCLA-RAND appropriateness method. Indicators were judged relevant if the median score was not in the 1–6 interval and if there was consensus (number of scores within the 1–3 interval was fewer than one third of the panel). From the relevant indicators providing overlapping information, the one with the highest scores was selected for the final set of quality indicators—values were updated with 2004 ESAC data.
Results: 22 participants (12 countries) completed scoring of a set of 22 proposed indicators. Nine were rated as relevant antibiotic prescribing indicators on all four dimensions; five were rated as relevant if only relevance to reducing antimicrobial resistance and public health policy makers was taken into account. A final set of 12 indicators was selected.
Conclusion: 12 of the proposed ESAC-based quality indicators for outpatient antibiotic use in Europe have face validity and are potentially applicable. These indicators could be used to better describe antibiotic use in ambulatory care and assess the quality of national antibiotic prescribing patterns in Europe.
Statistics from Altmetric.com
Quality assessment and improvement in healthcare is a major issue in many countries.12 Information on quality of healthcare is being demanded by policy makers, healthcare professionals and the general public.3 With most doctor–patient encounters in general practice resulting in a prescription for drug treatment, the quality of prescribing in general practice is an important issue.4 Prescribing also has a major influence on well-being and accounts for a substantial part of healthcare expenditure.5 The above statements hold true for antibiotics as well. In addition, antibiotic consumption is increasingly recognised as the main driver of resistance.6–8 Antibiotic resistance is a major European and global public health problem, and international efforts are needed to counteract its emergence.
There is a wealth of information on the prevalence of resistance in human pathogens, and these data show substantial geographical differences in the proportion of resistance to various classes of antibiotics in Europe.910 Whereas the rates of resistance remain low in northern European countries, they are reaching alarming levels in southern and central European countries. Studies have shown that differential selection pressure of antibiotic agents may be responsible for some of these observed differences.67 Consequently antimicrobial consumption was to be monitored to accompany analogous surveillance programmes on resistance.11
The European Surveillance of Antimicrobial Consumption (ESAC; www.ua.ac.be/ESAC) project, granted by DG SANCO of the European Commission, was launched in 2001. ESAC is an international network of surveillance systems, aiming to collect comparable and reliable data on antibiotic use in Europe. The methods and initial results have been published.712 The largest volumes of antibiotic prescriptions for systemic use are prescribed in primary care, with respiratory tract infections being the most common indication. More detailed analysis of antibiotic use in outpatients again showed considerable differences in overall antibiotic use, the use of the different kinds of antibiotics, and seasonal variation of antibiotic use, suggesting inappropriate antibiotic use in Europe,13–17 as these striking variations cannot be explained by differences in incidence of disease alone or by differences in aetiology or resistance rates between countries.
If we want to improve the use of antibiotics, we have to be able to measure the quality of antibiotic use in Europe. Therefore, our aim was to develop a set of valid drug-specific quality indicators for outpatient antibiotic use in Europe, which can be derived from ESAC data, taking into account the recommendations of DURQUIM (Drug Utilisation Research Quality Indicator Meeting).18
Developing a set of quality indicators
Quality indicators are defined as explicitly defined measurable items of antibiotic use giving a possible indication of the level of quality,1920 focusing on different aspects of quality (effectiveness, safety, appropriateness and costs,21 compliance, and persistence) and relevant to clinical practice.22 To produce a list of proposed quality indicators for outpatient antibiotic use in Europe based on ESAC data, a workshop was convened, funded by the European Science Foundation (ESF) (see fig 1 for a flowchart of the stages of development).18 This provided the unique opportunity to build on the interdisciplinary expertise within EuroDURG (European Drug Utilisation Research Group; www.eurodurg.com), GRIN (General Practice Respiratory Infections Network), ESPRIT (ESCMID Study Group on Primary Care Topics; www.escmid.org/esprit), BAPCOC (Belgian Antibiotic Policy Coordination Committee; www.bapcoc.be), WHO (World Health Organization; www.who.int), ESAC and other experts in this field. To prepare the workshop and to propose a draft list of indicators a scientific advisory board was set up with members of EuroDURG, GRIN/ESPRIT and of the ESAC Management Team. These clinicians and scientists, not policy makers, with expertise in general practice, microbiology, infectious diseases, pharmacoepidemiology, pharmacy and/or drug utilisation decided that the workshop should:
allow discussion on the development of quality indicators from the perspective of professionals and policy makers;
produce a proposed set of quality indicators for outpatient antibiotic;
produce a roadmap describing the next steps with these indicators.
The workshop included smaller work groups and plenary sessions. More details on the method of indicators development have been described elsewhere.23 After 2 days of relevant background presentations, constructive feedback and fruitful discussions, the participants produced an agreed set of proposed indicators related to the quality of outpatient antibiotic use (table 1 and S1 in online supplement at http://qshc.bmj.com/supplemental).
Assessing a set of quality indicators
To assess the relevance of the proposed quality indicators two consecutive cycles of scoring were conducted: one during the workshop and one after the workshop by email to finalise the process. All 27 participants (7 women; 15 countries) were asked to score the relevance of each of the proposed indicators to:
reducing antimicrobial resistance;
patient health benefit;
public health policy makers.
The scale for scoring ranged from 1 ( = completely disagree) through 5 ( = uncertain) to 9 ( = completely agree). The scores were processed according to the UCLA-RAND appropriateness method.2425 Proposed indicators were judged relevant if the median score was not within the 1–6 interval and if there was consensus—that is, if the number of scores within the 1–3 interval was fewer than one third of the panel. Along with the scores we asked for comments on the indicators. These were taken into account in earlier versions of this manuscript, which were reviewed and later approved by all participants.
Defining a final set of quality indicators
To define the final set only relevant indicators were selected. From relevant indicators providing overlapping information only the one with the highest scores for relevance was selected for the final set of quality indicators.
Applying a final set of quality indicators
The set of proposed quality indicators was developed using 1997–2003 ESAC data on outpatient antibiotic use in Europe.13–17 These ESAC data are use data of systemic antibiotics for ambulatory care, aggregated at the level of the active substance in accordance with the Anatomic Therapeutic Chemical (ATC) classification and defined daily dose (DDD) measurement unit (WHO, version 2005)—for example, amoxicillin, ATC code: J01CA04, DDD: 1 g.26 The DDD is the assumed average maintenance dose per day for a drug used for its main indication in adults.26 The antibiotic use is expressed in DDD per 1000 inhabitants per day (DID). The numbers of inhabitants in the European countries were the mid-year population of the countries involved.27 A complete description of the data providers, details of the methodology used and the associated problems, and indepth discussions of the validity of the collected data have been published previously.12
The values of the final set of relevant indicators were calculated using the most recent—that is, 2004 ESAC data. For each of the indicators we grouped the indicator values into four quartiles, and used this grouping in maps depicting the indicator values for all countries participating in ESAC.
A set of 22 proposed indicators was developed during the ESF workshop. The first of these indicators was the yearly use of antibiotics expressed in DID. The next 12 reflected the use of the six main subclasses of antibiotics expressed in DID and as percentage of the total antibiotic use. Four additional indicators reflected the use of four smaller therapeutic groups, again expressed as percentage of the total antibiotic use. There was one ratio (use of broad over narrow spectrum antibiotics), three indicators for seasonal variation of use (one for the total antibiotic use, and a similar and a more complex one for quinolone use). Finally, there was an indicator of longitudinal trends (table 1 and table S1 (see http://qshc.bmj.com/supplemental)).
We received scores from 22 participants (6 women; 12 countries). Nine indicators were rated as relevant antibiotic prescribing indicators on all four dimensions, and five additional indicators were rated as relevant if only relevance to reducing antimicrobial resistance and to public health policy makers was taken into account (table 1).
The information provided by “Consumption of quinolones (J01M) in percentage” and “Seasonal variation of quinolone consumption (J01M) multiplied by their use in DID” overlapped with the information provided by “Consumption of fluoroquinolones (J01MA) in percentage” and “Seasonal variation of quinolone consumption (J01M)”, respectively, but the latter indicators scored higher for relevance.
Table 2 shows the values for the final set of 12 quality indicators for 28 European countries in 2004. The indicator values for the period 1997–2003 are available online (table S2–8; see http://qshc.bmj.com/supplemental). For “Consumption of antibacterials for systemic use (J01) expressed in DID”, figure 2 depicts the grouping of 28 European countries based on the distribution of the respective indicator values for 2004 in four quartiles. Maps presenting the 2004 indicator values for the other relevant indicators are available online (figs S1–11; see http://qshc.bmj.com/supplemental).
Taking into account the scores from a relevant group of experts—professionals rather than policy makers—and the 2004 indicator values of 28 individual European countries, from a set of proposed ESAC based quality indicators for outpatient antibiotic use in Europe a final set of 12 indicators seems to be relevant—that is, has face validity and is potentially applicable. The indicators scored higher on the dimensions “resistance” and “public health policy” than on “patient health benefit” and “cost effectiveness”.
The most useful indicator probably is “Consumption of antibacterials for systemic use (J01) expressed in DID”, since overall this is likely to best indicate the size of the pressure driving antibiotic resistance, which in turn is highly relevant for public health.7 Other indicators should probably not be assessed individually—for example, a low “Consumption of penicillins (J01C) expressed in DID” apparently reflects good practice, but along with a high consumption of cephalosporins (J01D), macrolides, lincosamides and streptogramins (J01F) and quinolones (J01M) it suggests poor practice. All these other substances should be second-line drugs given the limited evidence of additional clinical benefit over penicillins for the most common indication, respiratory infections, and their increased costs. Similarly high or increasing trends in the percentage use of broad spectrum penicillins, cephalosporins and of fluoroquinolones would usually indicate poor practice, but not so much if overall consumption is coming down. Marked seasonal variation in antibiotic and quinolone use is likely to reflect poorer practice, since it represents higher use of antibiotics for respiratory infections, which has a poor evidence base.
The value of this set is associated with the self-imposed limitation to handle only those indicators that can be generated by the ESAC database. After all, one could argue that mere use data cannot indicate quality without being related to clinical information. This was, for instance, done in a Dutch study in which clinical information from individual patients was linked to antibiotic use and guidelines were used as a quality benchmark.28 This was even done on a national scale in the Netherlands,29 but in many European countries this is not feasible at present. Being able to use easily available national data that are valid and can be extracted from most European countries—34 European countries participate in ESAC in 2006—to produce the indicator values, can thus be considered as a strong side as well.
Nevertheless, more work will need to be done to improve the assessment of the quality of outpatient antibiotic use. Although we consider our data to be objective and available, and our indicators to have face validity and to be remediable, other attributes of the data, for example, the need for adjusting for contextual factors, and of the indicators, for example, concurrent and construct validity, and acceptability by those being assessed, need to be dealt with in future research activities. Conversely, we could argue that for some of the indicators content validity has been proven, since there is evidence for a (causal) link between the use of particular antibiotics with antimicrobial resistance for these antibiotics both on an ecological level7 and an individual level.8
One could consider limiting the final set—for example, to only the eight indicators from this set that were scored relevant for all dimensions. The latter set would probably be more useful also for feedback to individual prescribers, since the relevance to the dimensions “patient health benefit” and “cost effectiveness” was scored higher compared with our final set. Yet, we believe that, provided their limitations are taken into account, the indicators of the final set are valid as a signals ( = indicators) that call for more detailed analysis.
Benchmarking by comparisons between countries has proven to be important stimulus to quality improvement. This applies to antibiotic consumption as well.30 The set of indicators on outpatient antibiotic use presented here could allow individual countries to assess their position in relation to other countries, and trigger actions to improve antimicrobial prescribing. It could also allow identifying temporal trends and regional differences and therefore trigger investigation and action at the regional level. Finally, the set of indicators will be available to inform the process of development, implementation and evaluation of national and regional guidelines, and thus be useful for policy makers and medical professionals. However, defining benchmarks for this set of indicators on the basis of currently available ESAC data requires taking into account several contextual factors, such as local epidemiology of infectious diseases and local guidelines. Consequently this needs to be undertaken at a national level and might prove to be a difficult exercise. After all, some of the variations revealed by routine data may reflect real and important variations in actual healthcare quality, that is, inappropriate antibiotic use,713–17 that merit further investigation and action, but some apparent variation may also arise because of other misleading factors such as unadjusted casemix differences.1 Although measures of process may be less susceptible to spurious variations than measures of outcome, judgments about the quality of care should always be made with caution when using routine comparative data.
In conclusion, our work could be considered as a solid, but first step in the development of a set of valid quality indicators for outpatient antibiotic use in Europe. The challenge is to create indicators that are equally relevant across countries in Europe. In addition, more work is needed for indicators to be useful for individual prescribers. Until then even without correction for casemix, the patterns of resistance and other contextual factors, the reported set of ESAC-based indicator values provides valuable information about the appropriateness of antibiotic use in Europe in the context of the alarming levels of antimicrobial resistance.
We thank all participants of the ESF EMRC Workshop on Antibiotic Prescribing Quality Indicators for their valuable contribution to this work: Carl Cauwenbergh, Erik Hendrickx, Raf Mertens (Belgium), Morten Andersen (Denmark), Ulla-Maija Rautakorpi (Finland), Kathleen Holoway (Switserland), and the ESAC Project Group members Arjana Andrasevic (Croatia), Jiri Vlcek (Czech Republic), Arno Muller (Denmark), Philippe Cavalié (France), Sandra Berzina (Latvia), Rolanda Valenteliene (Lithuania), Robert Janknegt (the Netherlands), Hege Salvesen Blix (Norway), Pawel Grzesiowski (Poland), Milan Cizman (Slovenia), José Campos (Spain), Sigvard Mölstad (Sweden), Peter Davey (UK).
The other ESAC Project Group members are Helmut Mittermayer, Sigrid Metz (Austria); Boyka Markova (Bulgaria); Igor Francetic (Croatia); Despo Bagatzouni (Cyprus); Annemette Anker Nielsen (Denmark); Ly Rootslane (Estonia); Pentti Huovinen, Pirkko Paakkari (Finland); Didier Guillemot (France); Winfried Kern, Helmut Schroeder (Germany); Helen Giamarellou, Anastasia Antoniadou (Greece); Gabor Ternak, Ria Benko (Hungary); Karl Kristinsson (Iceland); Robert Cunney, Ajay Oza (Ireland); Raul Raz (Israel); Giuseppe Cornaglia (Italy); Robert Hemmer, Marcel Bruch (Luxembourg); Michael Borg, Peter Zarb (Malta); Margreet Filius (the Netherlands); Waleria Hryniewicz, (Poland); Luis Caldeira (Portugal); Irina Codita (Romania); Svetlana Ratchina (Russia); Viliam Foltan, Tomas Tesar (Slovakia); Edurne Lazaro, Francisco de Abajo (Spain); Otto Cars, Gunilla Skoog, (Sweden); Giuliano Masiero (Switzerland); Serhat Unal (Turkey).
Funding: The ESAC project was granted by DG/SANCO of the European Commission (2001/SID/136). The information contained in this publication does not necessarily reflect the opinion or the position of the European Commission. The workshop was co-financed by ESF, DG/SANCO, WHO Europe and the University of Antwerp. SC carried out this work as a postdoctoral fellow of the Research Foundation-Flanders.
Competing interests: None.
- Anatomic Therapeutic Chemical (classification)
- defined daily dose
- DDD per 1000 inhabitants per day
- European Surveillance of Antimicrobial Consumption
- European Science Foundation
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.