Qual Saf Health Care 17:318-323 doi:10.1136/qshc.2007.024224
  • Original research

Prescribing quality indicators of type 2 diabetes mellitus ambulatory care

  1. L Martirosyan1,
  2. J Braspenning2,
  3. P Denig1,
  4. W J C de Grauw3,
  5. M Bouma4,
  6. F Storms5,
  7. F M Haaijer-Ruskamp1
  1. 1
    Department of Clinical Pharmacology, University Medical Centre Groningen, University of Groningen, Groningen, The Netherlands
  2. 2
    Centre for Quality of Care Research (WOK), University Medical Centre St Radboud, Nijmegen, The Netherlands
  3. 3
    Department of General Practice, Academic Research Network CMR NMP, University Medical Centre St Radboud, Nijmegen, The Netherlands
  4. 4
    Dutch College of General Practitioners, Utrecht, The Netherlands
  5. 5
    Dutch Institute for Healthcare Improvement, Utrecht, The Netherlands
  1. Dr L Martirosyan, Department of Clinical Pharmacology, University Medical Centre Groningen, University of Groningen, Postbus 196, 9700AD, The Netherlands; l.martirosyan{at}
  • Accepted 28 February 2008


Background: Existing performance indicators for assessing quality of care in type 2 diabetes mellitus (T2DM) focus mostly on registration of measurements and clinical outcomes, and not on quality of prescribing.

Objective: To develop a set of valid prescribing quality indicators (PQI) for internal use in T2DM, and assess the operational validity of the PQI using electronic medical records.

Methods: Potential PQI for hypertension, hyperglycaemia, dyslipidaemia and antiplatelet treatment in T2DM were based on clinical guidelines, and assessed on face and content validity in an expert panel followed by a panel of GPs and diabetologists. Analysis of ratings was performed using the RAND/UCLA Appropriateness Method. The operational validity of selected indicators was assessed in a dataset of 3214 T2DM patients registered with 70 GPs.

Results: Out of 31 potential prescribing indicators, the expert panel considered 18 indicators as sufficiently valid, of which 14 indicators remained valid after assessment by the panel of GPs and diabetologists. Of these 14 indicators, one could not be calculated because of an absence of eligible patients. For the remaining indicators, outcomes varied from 10% for timely prescribing of insulin to 96% for prescribing of any antihyperglycemic medication in patients with elevated HbA1c levels.

Conclusions: This study provides a set of face- and content-valid PQI for pharmacological management of patients with T2DM. While outcomes of some PQI were limited to patients with registration of clinical values, the selected PQI had good operational validity to be used in practice for assessment of prescribing quality.

Efforts to measure and improve the quality of care in outpatient settings have focused especially on care for chronic medical conditions, such as type 2 diabetes mellitus (T2DM). This has resulted in the publication of clinical guidelines to assist doctors in management of diabetes, and the development of performance quality indicators. T2DM is a disease with a dramatically increasing prevalence throughout the world, and serious complications may occur if the disease is not adequately treated.12 Appropriate blood-pressure control, lipid-lowering therapy, angiotensin-converting enzyme (ACE) inhibition, and antiplatelet drugs significantly reduce the risk of cardiovascular and microvascular complications in patients with diabetes.34

Quality information has been demanded not only by policy makers, consumers and media, but also by healthcare providers themselves for internal use. In order to measure the quality of prescribing in T2DM patients, valid prescribing quality indicators (PQI) are needed. Such PQI can be used by healthcare providers as a “screening tool” to help flag potential problem areas that need further investigation, for giving feedback to individual doctors and to assess the impact of quality improvement initiatives. Although quality of care can be improved without explicit quality assessment, for example by peer review or educational programmes, measurements provide valuable information for monitoring and feedback.5 Quality-improvement initiatives using quality measurements and achievable standards have been shown to improve diabetes outcomes, such as long-term glucose control measurement.67

Several sets of quality indicators for diabetes care have been developed. Many of them include outcome indicators or focus on processes of care, such as registration of clinical characteristics, but do not include any PQI.810 PQI are process measures that can help to identify patients who may benefit from initiation or intensification of treatment. Such information is helpful for improving prescribing quality and dealing with so called “therapeutic inertia.”1112 Although some PQI for diabetes care are included in national sets of quality indicators,1314 and some detailed PQI for T2DM management have been described,1516 a comprehensive set of PQI for diabetes care is lacking.

We aimed to develop a set of PQI for pharmacological management in T2DM patients for internal use and to assess their operational validity using electronic medical records.


Development of indicators

A list of 30 potential PQI for pharmacological management in T2DM was developed based on the latest versions of English language and Dutch diabetes guidelines.1724 Key guideline recommendations regarding drug treatment were transformed into potential indicators on the basis of measurability. Indicators comprised the number of patients actually receiving the drug (numerator) over the number of patients for whom the drug was appropriate (denominator). Potential PQI were developed for the following areas: hypertension, hyperglycaemia, dyslipidaemia, antiplatelet treatment and secondary prevention of cardiovascular disease (CVD). Developed indicators focused on undertreatment, drug choice, dosage and safety.

Assessment of face and content validity

The face and content validity of potential indicators was assessed in a two-round expert panel followed by a panel of physicians from the field for whom the PQI were intended using the RAND/UCLA Appropriateness Method (RAM).25 The expert panel consisted of nationally recognised authorities from relevant specialties involved in ambulatory diabetes care: two GPs, two diabetologists and a professor of endocrinology. The panel members had considerable practice and scientific experience, and were members of the Dutch College of General Practitioners, Dutch Institute for Healthcare Improvement or Dutch Diabetes Association. For the field panel, 14 general practitioners (GPs) and six diabetologists were recruited from two regions in The Netherlands.

Both panels were asked to rate the PQI on a nine-point scale for two criteria: reflection of the key recommendations in the guidelines, and relevance for patient health gain. Before rating the indicators, participants of both panels received background information including the evidence-base and definitions used for PQIs, and it was made clear that the potential PQI were intended for internal use.

In the first round, potential PQI were mailed to the experts for individual rating. In addition, experts were asked to suggest new indicators if they believed that important drug treatment recommendations were insufficiently addressed. The ratings were analysed, and PQI rated with disagreement were identified. In the second round, panel members met to discuss PQI rated with disagreement. The intention of the discussion was to resolve misinterpretations, and improve definitions of PQI. In case of ambiguity, the experts were asked to introduce changes in the definitions or wording of indicators. Discussion was facilitated by a moderator experienced in chairing expert panels. After the discussion, the definitions of PQI were refined, and the panellists were asked to rate the indicators a second time. Based on these second ratings, indicators classified as having insufficient validity were discarded. PQI considered valid by the expert panel were mailed to GPs and diabetologists participating in the third round. After analysis of their ratings, the final PQI were selected.

Operational validity

To assess operational validity, the selected PQI were calculated in a dataset extracted for the GIANTT project from electronic records of 3214 T2DM patients registered with 70 GPs working in 37 practices in the north of The Netherlands.26 The dataset included information on demographics, prescribed medication, comorbidities, physical examination and laboratory measurements as documented in medical records of GP practices and a regional diabetes facility, which offers support to GPs by providing 3-monthly and yearly diabetes follow-up examinations of patients.


A PQI was considered to be valid if it met the following predefined criteria: both panels rated it with median score of seven or more and without disagreement for either criterion.25 Disagreement was analysed using the interpercentile range between the first and last tertials adjusted for symmetry (IPRAS) method developed in the RAM.25 The rationale behind this adjustment is that when ratings are symmetrical with respect to the middle (5 on the 1–9 scale), the interpercentile range (IPR) required to identify disagreement is smaller than when they are asymmetric. The detailed formulas and examples for calculation of IPR and IPRAS are available on the RAND WebPage.27

Operational validity was defined as the feasibility of calculation of PQI using electronic medical records. Indicators were calculated using SPSS for Windows version 11. For calculation of the PQI, we used values of blood pressure and HbA1c registered in the first half of 2004 and prescription data registered in the second half of 2004. In this way, we made sure that prescription occurred after observing elevated values of clinical measurements. For albuminuria and BMI, the last value in 2004 was used. Three PQI focusing on intensification of antihypertensive and antihyperglycaemic therapy were calculated in a longitudinal way by looking for patients who, in spite of a treatment, had two clinical values above target level in a period of any 4 months in 2004, and received treatment intensification in the following month. This approach has been selected, since sequential indicators have been shown to provide better estimates of treatment intensification.28 Detailed operational definitions for calculating all PQI are provided in Appendix A.


Selecting the face/content valid PQI

No indicator was discarded after the first round, but one new indicator was suggested. Therefore, in the second round, the experts rated 31 indicators (Appendix B). The panel of experts considered 18 indicators to be valid. The other 13 indicators were rated either with disagreement or with a median lower than seven for reflection of guidelines and patient health gain, and were discarded (table 1). Reasons for disagreement identified during the discussion included: too much dependence on case-mix, insufficient evidence, irrelevance for ambulatory care and disagreement on guideline recommendations (table 1). In particular, there was disagreement between GPs and diabetologists regarding the recommendation from Dutch diabetes guidelines to prescribe thiazides as a first-choice antihypertensive drug in T2DM patients without albuminuria. Diabetologists considered ACE inhibitors as first-choice drugs for patients with diabetes irrespective of albuminuria.

Table 1 Discarded prescribing quality indicators (PQI)

The field panel of GPs and diabetologists considered 14 indicators out of the 18 selected by the expert panel as being sufficiently valid. The four discarded PQI were rated with disagreement, because some members of the field panel gave low ratings to these PQI for reasons including lack of relevance for patient health gain and sensitivity to individual patient characteristics (table 1). The final PQI covered the main aspects of pharmacological treatment in T2DM patients (table 2).

Table 2 Outcome measures for the selected prescribing quality indicators (PQI)

Operational validity

It was feasible to calculate 13 PQI using data routinely documented in medical records. One indicator focusing on prescription of statins to T2DM patients younger than 40 years and with history of cardiovascular disease could not be calculated because of a lack of eligible patients (table 2).

Five PQI required information on BMI and albuminuria, which were not available for over a third of patients. The other eight indicators were calculated based on variables available at least for 70% of patients(table 3).

Table 3 Completeness of the dataset for variables used to calculate prescribing quality indicators

The best performance was observed for the indicators focusing on prescribing any antihypertensive or antihyperglycaemic drugs. The lowest PQI outcomes were observed for timely intensification of antihypertensive or antihyperglycaemic treatment.


Out of 31 potential prescribing indicators derived from diabetes guidelines, 14 were assessed by both expert and field panel as sufficiently valid for internal quality assessment. Thirteen of them were feasible to calculate using data available from electronic medical records.

To our knowledge, this is the first study that aimed to develop a set of PQI for diabetes care. We used the RAM methodology, which is considered the best method for systematically combining recommendations from clinical guidelines with the opinion of healthcare providers,29 to develop PQI that are face- and content-valid. Another strong point in the assessment of validity was the use of IPRAS to measure disagreement between participants in both rounds, as this method has shown excellent sensitivity and good specificity to measure the degree of dispersion among ratings.25 In addition, we followed a procedure of discussing reasons for disagreement, and improving definitions and wordings of the indicators before the final rating. This ensured that ambiguity was not the reason for disagreement or rejection of indicators.

It was our objective to develop and validate indicators for internal use by healthcare providers treating patients with T2DM. We selected the PQI using two different panels. The additional assessment of the prescribing indicators as selected by the experts in a field panel ensures acceptability of indicators for everyday practice by those for whom the indicators are actually intended. The majority of the indicators validated by the expert panel were also selected by the field panel.

A limitation of our method is that panel members from different groups may have different judgements which affects the ratings.30 Judgements made by any expert panel may not be representative for all healthcare professionals. However, in our study, we had two different panels, making the final selection of indicators more reliable and generalizable. Our results show the significance of combining evidence with expert and field opinion. In particular, we found that diabetologists and GPs disagreed on some recommendations in guidelines. Since our aim was to select indicators for which there was a consensus between both groups, indicators considered relevant by only some experts were not included in the final selection. In addition, we used a selection of seven diabetes guidelines for this PQI development. Therefore, it is possible that PQI based on relevant recommendations from other guidelines were not considered. There is, however, international consensus on the key clinical recommendations for diabetes care in different guidelines.31 Our PQI covered these central recommendations.

In all five areas of pharmacological management, several indicators of undertreatment and/or drug choice were considered valid. Except for aggressive management of hyperglycaemia, these were indicators with evidence grade A.23 None of the indicators focusing on dosage or safety reached sufficient face and content validity because of disagreement with the recommendations or expected influence of other patient characteristics which may not always be documented in records.

Two PQI selected in our study were also considered face- or content-valid in previous studies, and some are being used at national level, that is PQI focusing on prescribing ACE-inhibitors in T2DM hypertensive patients with albuminuria13143233 and prescribing β-blockers in T2DM patients with history of myocardial infarction.143436 The PQI focusing on prescribing ACE-inhibitors before prescribing ARBs did not reach face and content validity in our study, but was considered valid previously.37 PQI focusing on hypertension and hyperglycaemia undertreatment have been selected also by other panels, albeit with higher targets—for example 150 for systolic blood pressure and 9% for HbA1c.32 This can be explained by emerging evidence and by the differences in diabetes guidelines regarding specific recommendations in different countries.38 None of the proposed PQI was included in the set of diabetes quality measures owned by the National Committee for Quality Assurance36 or in the Australian set of indicators of quality prescribing in general practice.13 This implies that the proposed set of indicators can be seen as a welcome addition to the existing sets of indicators.

Operational validity for most of PQI was good. Only one PQI could not be measured because of an absence of young T2DM patients with history of cardiovascular disease in our dataset. We combined clinical information stored in two data sources to enhance completeness of data collection, but some variables were not available for all patients, possibly because they are not measured each year. Missing data are a problem of any clinical registry. However, it was shown that proportion-based PQI are robust to data loss up to 35% of an entire sample.39 Considering that our PQI are also proportion-based, and any change in denominator will cause a change in the numerator, the PQI based on variables available for at least 70% of patients can be considered sufficiently generalizable. The outcomes of PQI based on albuminuria and BMI with data missing for 43 and 57% of patients respectively may not reflect prescribing quality for whole population in this particular dataset. Nevertheless, they can be used by doctors to identify potential problems among patients with available clinical information. Some patients who are eligible for particular treatment may be missed, but those who meet the eligibility criteria for treatment (denominator) are expected to be prescribed an appropriate medication scheme (numerator).

Although we did not aim to assess quality of prescribing in this study, outcome measures for many PQI showed room for improvement. More problems were seen regarding prescribing of statins, acetyl-salicylic acid, and timely intensification of antihypertensive and antihyperglycaemic treatment. However, it should be noted that we used a dataset of 2004, and prescribing patterns may have changed since then.

In contrast, performance for some PQI was very good—for example, PQI focusing on prescription of any antihyperglycaemic agent showed a very high outcome (96%). If a PQI shows such a high performance over time for the same healthcare provider, it may be retired, since there is no potential for further improvement, as recently happened to one of the National Committee for Quality Assurance measures.40

The main restriction for the use of these disease-oriented PQI is availability of patient clinical information, which is not present in all administrative datasets. However, improvement of measurement and registration of clinical values as a part of quality improvement, and development of new data collection methods will provide databases for effective use of the PQI in the future. PQI are by definition proxy measures of prescribing quality. There will always be patient and clinical characteristics that will legitimate deviations from the recommended treatment.6 Finally, the recommendations in guidelines change over time, and PQI should be periodically updated to reflect the best evidence.

The study provides a set of face- and content-valid PQI for pharmacological management in T2DM that were tested for internal use by healthcare providers. This set can be used to make healthcare providers aware of specific areas of prescribing that may be suboptimal, including issues of undertreatment and drug selection.


The authors would like to thank H J Aanstoot, BHR Wolfenbuttel, and all GPs and diabetologists for participation in the project.


  • Funding: Graduate School for Health Research, University Medical Center Groningen, University of Groningen.

  • Competing interests: None.