You are here

  • Home
  • Archive
  • Volume 25, Issue 7
  • Perioperative diabetes care: development and validation of quality indicators throughout the entire hospital care pathway

Article Text

Article menu
Original research
Perioperative diabetes care: development and validation of quality indicators throughout the entire hospital care pathway
  1. Inge Hommel1,
  2. Petra J van Gurp2,
  3. Cees J Tack2,
  4. Hub Wollersheim1,
  5. Marlies EJL Hulscher1
  1. 1Radboud University Medical Center, Radboud Institute for Health Sciences, Scientific Institute for Quality of Healthcare, Nijmegen, The Netherlands
  2. 2Department of General Internal Medicine, Radboud University Medical Center, Radboud Institute for Health Sciences, Nijmegen, The Netherlands
  1. Correspondence to Inge Hommel, Radboud University Medical Center, Radboud Institute for Health Sciences, Scientific Institute for Quality of Healthcare, PO box 9101, Post 114, Nijmegen 6500 HB, The Netherlands; inge.hommel{at}radboudumc.nl

Abstract

Background In this study, we aim to develop a set of quality indicators for optimal perioperative diabetes care throughout the hospital care pathway and to gain insight into the feasibility of the indicator set in daily clinical practice by assessing the clinimetric properties of the indicators in a practice test.

Methods A literature-based modified Delphi method was used to develop a set of quality indicators. To assess clinimetric properties of each indicator (measurability, applicability, reliability, improvement potential and case-mix stability), a practice test was performed in six Dutch hospitals using a sample of 389 major surgery patients with diabetes who underwent abdominal, cardiac or large joint orthopaedic surgery.

Results We developed a set of 36 quality indicators for perioperative diabetes care. The practice test showed that one indicator was inapplicable, and nine indicators were unmeasurable. Interobserver reliability was good (0.61≤k≤0.8) for all indicators except for one with moderate (0.41≤k≤0.6) interobserver reliability. Improvement potential was low (<10%) for five indicators. Twenty-one indicators, including three outcome indicators, nine process indicators and nine structure indicators, could be used to assess the quality of care delivered in our six study hospitals.

Conclusion We developed a face and content valid set of quality indicators for optimal perioperative diabetes care throughout the hospital care pathway, using a rigorous and systematic approach. The results from our practice test show that it is essential to subject indicators to a practice test before applying them for quality improvement purposes.

Trial registration number ClinicalTrials.gov: NCT01610674.

  • Diabetes mellitus
  • Hospital medicine
  • Performance measures
  • Quality measurement
  • Surgery

http://dx.doi.org/10.1136/bmjqs-2015-004112

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Estimates of diabetes prevalence range from 20% in general surgery patients to nearly 35% in cardiac surgery patients.1 ,2 Surgery for patients with diabetes is associated with longer hospital stay, more use of healthcare resources and greater perioperative morbidity and mortality than surgery for patients without diabetes.2–5 The greater perioperative morbidity and mortality relates, in part, to the higher incidence of coronary heart disease, hypertension, renal insufficiency and postoperative complications associated with diabetes.4 Further, the relationship between perioperative hyperglycaemia and surgical complications is well established for a large range of surgical procedures.6–12

    Perioperative care is a complex and multidisciplinary process, involving multiple teams of caregivers and multiple transitions of care. To achieve optimal diabetes care in the perioperative period, all professionals involved should take the specific needs of the person with diabetes into account at all stages of the hospital care pathway that ranges from preoperative evaluation up to hospital discharge.13 Several large professional organisations, such as the American Diabetes Association and more recently the Joint British Diabetes Societies (JBDS), have developed recommendations for optimal diabetes care in the perioperative period, but it is unknown whether this has resulted in optimal or ‘good quality’ care delivery.13 ,14

    The development and use of quality indicators have been suggested to help define and assess the quality of the care delivered.15 ,16 Quality indicators are measurable elements of practice performance, for which there is evidence or consensus that they can be used to assess the quality, and hence change in the quality, of care provided.17 Three types of quality indicators can be distinguished. Outcome indicators specify the ultimate goal of the care given, process indicators refer to the actual care that is delivered to patients and structure indicators refer to the organisational structure of a healthcare system.18

    This study aims to develop a set of quality indicators for optimal perioperative diabetes care throughout the hospital care pathway and to gain insight into the feasibility of the indicator set in daily clinical practice by assessing the clinimetric properties (quality criteria) of the indicators in a practice test.

    Methods

    Development of quality indicators for perioperative diabetes care

    A modified Delphi method was used to develop a set of quality indicators.19–21 Recommendations for optimal perioperative diabetes care were extracted from the literature (international guidelines, websites and PubMed database), translated into potential quality indicators and presented to a multidisciplinary expert panel.

    Selection of recommendations

    The search for recommendations for optimal perioperative diabetes care was based on the guidelines for Quality Indicator Development of the Dutch Institute for Healthcare Improvement.22 In January 2009, we searched 33 websites of international healthcare organisations, for example, http://www.qualitymeasures.ahrq.gov and http://www.who.int/en/, to identify quality indicators for perioperative diabetes care.22 We continued by searching websites of national and international guideline organisations, and found seven guidelines that contained recommendations on perioperative diabetes care.14 ,23–28 Three reviewers (IH, PJvG and MEJLH) extracted all recommendations for optimal perioperative diabetes care from the American Diabetes Association guidelines. As agreement was very high, one reviewer continued to extract recommendations from the guidelines (IH).

    To complement the inventory of recommendations for optimal perioperative diabetes care, we used the PubMed database to review international literature for information about the development of quality indicators and recommendations for perioperative diabetes care. Online supplementary figure S1 depicts the search strategy, and online supplementary table S1 presents the exclusion criteria. Three reviewers checked the abstracts. Potentially relevant publications were checked in full text format.

    All recommendations were categorised as ‘patient outcome’, ‘process’ or ‘organisational structure’.18 Recommendations for process were categorised according to the various stages of the hospital care pathway: ‘preoperative evaluation’, ‘hospital admission’ (care that should be initiated on hospital admission), ‘preoperative management’ (care that should be initiated in preparation for the surgery), ‘perioperative management’, ‘postoperative management’ or ‘hospital discharge’.

    Rating procedure

    The rating procedure took place between May 2009 and November 2009. Our expert panel consisted of 35 Dutch experts who had experience with the care for patients with diabetes in the perioperative period (19 endocrinologists/internists with special interest in diabetes care, 6 anaesthesiologists, 5 intensive care specialists and 4 surgeons). The experts worked in 22 Dutch hospitals in the Netherlands. We selected the experts based on their experience with perioperative diabetes care (champions of diabetes care in the Netherlands), while aiming for sufficient diversity in hospital size (small-to-large size, academic and non-academic), the various regions in the Netherlands and disciplines of the panel members.

    First questionnaire round

    All recommendations for optimal perioperative diabetes care were translated into potential quality indicators by defining ‘who should do what to whom and when’, and were worked into a written questionnaire. The experts rated the quality indicators anonymously. We asked the experts to rate each quality indicator while considering the following criteria:

    • The recommended care leads to health gain for the patient (eg, fewer complications) or promotes efficiency of care.

    • The recommended care is generalisable to all patients with diabetes who undergo a major surgical procedure.

    • There is enough scientific evidence or expert consensus to justify the recommended care.

    A Likert scale rating ranging from 1 (not at all relevant for assessing the quality of care delivered) to 9 (very relevant) was added, and the answer category ‘cannot assess’ was made available. Online supplementary figure S2 shows an example of a rating scale for a quality indicator. Experts could also add quality indicators. Quality indicators were selected according to relevancy (median scores 7, 8 or 9) if there was no disagreement (30% or more of ratings in both the 1–3 and 7–9 tertiles). Quality indicators rated ≤6 were discarded.

    Expert panel meeting

    The analysis of the questionnaires was presented during a panel meeting to discuss the quality indicators with median scores 7, 8 or 9 and disagreement, and to rephrase quality indicators that were not defined specific enough. All panellists who had participated in the first questionnaire round were invited.

    Second questionnaire round

    Following the meeting, all accepted and added quality indicators, rephrased if necessary, were presented for final remarks and approval to the panel members who had completed the first questionnaire. We added a ranking procedure to single out key recommendations for each stage of the hospital care pathway (‘preoperative evaluation’, ‘hospital admission’, ‘preoperative management’, ‘perioperative management’, ‘postoperative management’ or ‘hospital discharge’). For each stage that contained more than four indicators, we asked the experts to select the top three of most relevant indicators. We awarded an indicator three points for each number-one expert ranking, two points for each number-two ranking and one point for each number-three ranking. Indicators that received >20% of the maximum possible ranking points were included in the final indicator set. The maximum possible ranking points were 57 points (19 experts times 3 points). The experts ranked the quality indicators anonymously.

    Practice test

    Setting

    The application of a systematic and rigorous modified Delphi method for developing quality indicators results in high face and content validity for the indicator set. To gain insight into the feasibility of the final indicator set in daily clinical practice, we performed a practice test, that is, a test to assess the clinimetric properties or quality criteria of indicators in daily clinical practice. We assessed the clinimetric properties of the final indicator set in six Dutch hospitals. There were two academic hospitals (953–1042 beds), two regional teaching hospitals (543–576 beds) and two tertiary teaching hospitals (730–846 beds). All six hospitals performed orthopaedic and abdominal surgery, and three of the six hospitals performed cardiac surgery. In one hospital, the department of abdominal surgery declined participation. The hospitals participated in a study to test the effectiveness of an implementation strategy to improve perioperative diabetes care in daily clinical practice (trial no. NCT01610674). The Regional Review Board for Human Research Arnhem-Nijmegen (CMO, no. 2008/333), approved the study for all hospitals.

    Adult patients with diabetes were included if they met our criteria of major surgery:29 (1) the person was under general anaesthesia during abdominal surgery or cardiac surgery, or under spinal or general anaesthesia during large-joint orthopaedic surgery and (2) the minimum operation time was 1 h. Patients who underwent outpatient surgery were excluded. If a participant underwent more than one surgical procedure, we included the first procedure. The surgery took place in the period between March 2009 and March 2010. We retrospectively identified the patients from data in the hospital prescription system, operating room management systems, preoperative screening records and anaesthesiology records. The patients were invited to participate in the study by mail and by phone.30 Informed consent was obtained from all patients. After informed consent had been obtained, information regarding the inclusion criteria was verified with the patient record. We aimed to include a minimum of 30 patients per surgical speciality (abdominal surgery, cardiac surgery and large-joint orthopaedic surgery) in each hospital, that is, 420 patients for a total of 14 departments. We invited 60 patients per surgical speciality if sufficient numbers were available.

    Data collection

    The patient data relevant to process and outcome indicators were manually abstracted from the patient records and entered into a computerised database. The data were collected from medical and nursing records, anaesthesiology records, medication records and laboratory reports. Four trained research assistants (EE, MG, ER and AV) and one researcher (IH) searched the patient records for the period between August 2010 and May 2011.

    We assessed the quality indicators for the recommended organisational structure of perioperative diabetes care with a questionnaire. A diabetes specialist in each of the six participating hospitals completed the questionnaire.

    Clinimetric properties assessed

    Outcome indicators and process indicators

    Individual patient indicator scores (dichotomous variable with the values 0 and 1) were computed for each patient and for each outcome and process indicator, and the scores described whether or not care was delivered as recommended. We constructed an algorithm for each indicator to calculate the indicator outcomes. Online supplementary figure S3 provides an example of an indicator algorithm. We computed hospital indicator scores (in percentages) by dividing the number of patients who received the recommended care as described by the indicator (numerator) by the number of patients to whom the recommended care applied (denominator). We used these data to assess the following clinimetric properties:

    1. Measurability31 ,32

      A specific indicator was considered ‘unmeasurable’ if >25% of the patients’ individual indicator scores could not be computed because of missing data.

    2. Applicability33

      Indicators that applied to <10% of the patients were considered ‘inapplicable’.

    3. Reliability31 ,34

      The interobserver reliability of the data collection was expressed in kappa coefficients, that is, the percentage of agreement between two data reviewers at the level of the patient indicator score, corrected for chance. Two independent data reviewers collected a sample of 30 patient records from three hospitals. Values of 0.41≤k≤0.6 were considered ‘moderate’, 0.61≤k≤0.8 were considered ‘good’ and values >0.8 were considered ‘very good’. Values <0.4 were considered ‘poor’.

    4. Improvement potential32

      The improvement potential of a specific indicator was considered ‘low’ if the median potential for hospital improvement was <10% (hospital indicator score ≥90%) in >80% of the hospitals.

    5. Case-mix stability31

      For indicators with good clinimetric properties, we explored whether the hospital indicator scores needed correction for case mix. We used multilevel analysis to study the relationship between the indicator results and the following patient characteristics: age, gender, insulin treatment and type of surgery. We corrected for data clustering at the hospital level. Significance for the multilevel analysis was set at p<0.05.

    Structure indicators

    Individual hospital indicator scores (dichotomous variable with the values 0 and 1) were computed for each hospital and for each structure indicator, and the scores described whether or not care was organised as recommended. The number of hospitals that were included in the practice test (six) was too small to determine all the clinimetric properties of the indicators for organisational structure. To get an impression of the ‘improvement potential’ of these indicators, we computed the fraction of hospitals with an organisational structure of perioperative diabetes care as described by the indicator.

    Results

    Quality indicators for perioperative diabetes care

    Selection of recommendations from guidelines and literature

    Figure 1 schematically presents the quality indicator development. No existing quality indicators for perioperative diabetes care were found on international websites. In total, 90 recommendations for perioperative diabetes care were derived from seven guidelines.14 ,23–28 The literature review yielded 1318 abstracts for 114 articles that contained recommendations for perioperative diabetes care. From these articles, we derived 22 recommendations that were complementary to the set of recommendations found in the guideline search. Further aggregation of the total of 112 recommendations by topic resulted in a total of 79 recommendations that were translated into potential quality indicators. The evidence level, as provided in the guidelines, was D (ie, a formal combination of expert views or other information, eg, Delphi study, expert opinion or informal consensus) for most quality indicators.35

    Figure 1
    Figure 1

    Developing and testing the indicator set. The flowchart shows the steps involved in the indicator development. Potential quality indicators and recommendations for optimal perioperative diabetes care were selected from websites, guidelines and the international literature, and were rated by a panel of experts. A practice test was performed to assess the clinimetric properties of the indicators.

    Rating procedure

    First questionnaire round

    Twenty-seven of 35 experts returned the first questionnaire (15 internists, 5 anaesthesiologists, 4 intensive care specialists and 3 surgeons; response rate 77%). The expert panel rejected 30 quality indicators, accepted 48 indicators as relevant for assessing the quality of perioperative diabetes care, disagreed on one indicator and suggested two new quality indicators.

    Expert panel meeting

    Six experts attended the panel meeting (three internists, one anaesthesiologist, one surgeon and one intensive care specialist). The panel discussed and rephrased 15 quality indicators, combined 2 quality indicators with similar contents and suggested 2 new quality indicators, resulting in a total of 52 quality indicators.

    Second questionnaire round

    Nineteen of 27 experts (10 internists, 4 anaesthesiologists, 3 intensive care specialists and 2 surgeons, response rate 70%) completed the second questionnaire. The panel approved all 52 quality indicators, including 4 outcome indicators, 39 process indicators and 9 structure indicators (see online supplementary table S2). The experts prioritised 15 out of the 39 process indicators. The 28 indicators in the prioritised set (4 outcome, 15 process and 9 structure) were systematically operationalised by defining numerators and denominators. Five indicators (see online supplementary table S2, indicator numbers 1, 2, 5, 15 and 27) were operationalised into two or more indicators. Tables 1⇓–3 present the final set of 36 indicators that were tested for feasibility in the study hospitals.

    View this table:
    Table 1

    Clinimetric properties for the outcome indicators in six practice test hospitals

    View this table:
    Table 2

    Clinimetric properties for the process indicators* in six practice test hospitals

    View this table:
    Table 3

    Fractions of practice test hospitals with organisational structure as described by the indicator*

    Practice test

    Setting

    We identified 1100 patients with diabetes who underwent major surgery. We aimed to include 30 patients per surgical speciality in each hospital, but these numbers were not available for the departments of orthopaedic and abdominal surgery in all hospitals. We performed a practice test with the final set of 36 indicators on a sample of 389 patients who underwent abdominal surgery (32%), orthopaedic surgery (39%) or cardiac surgery (29%). Figure 2 shows the patient inclusion. Of these patients, 62% were men and 38% were women. More than half of the patients (57%) had received oral treatment with hypoglycaemic agents only before admission to hospital. Twenty-one per cent of the patients used a combination of oral hypoglycaemic agents and insulin, 18% used insulin only and 4% took dietary measures only.

    Figure 2
    Figure 2

    Flow chart showing patient inclusion.

    Clinimetric properties assessed

    Outcome indicators and process indicators

    1. Measurability

      Tables 1 and 2 show the results of the practice test. Nine indicators (four outcome and five process) were ‘unmeasurable’ in our study hospitals because the relevant data for computing patient indicator scores were missing at the time of data extraction for >25% of the patients.

    2. Applicability

      As hypoglycaemia rarely occurred, we considered the indicator ‘intravenous glucose ordered for fasting patients with hypoglycaemia’ ‘inapplicable’ in our study hospitals.

    3. Reliability

      The indicator ‘blood glucose measurement every 4–6 h ordered for fasting patients’ had a kappa of 0.51, which indicates moderate interobserver reliability. All other indicators had kappa scores of >0.6.

    4. Improvement potential

      The potential for improvement was low for four process indicators and one outcome indicator.

    5. Case-mix stability

      Case-mix stability was assessed for 12 indicators that had good clinimetric properties (no. 1, 4, 8, 9, 11, 15, 16, 18, 21–23, 26). All these indicators needed correction for case mix. The most important determinants that influenced several indicator scores were diabetes medication and the type of surgery (cardiac, orthopaedic or abdominal). One indicator ‘blood glucose measured every 2 h during surgery’ was influenced by sex.

    Structure indicators

    Table 3 presents the fractions of hospitals with organisational structures as described by the indicators. Online supplementary table S3 presents the details for these results, which indicate that there is plenty of room for improving most structure indicators. For example, perioperative diabetes care protocols lacked clear descriptions of the tasks and responsibilities of the caregivers in five of the six hospitals. None of the multidisciplinary teams to coordinate perioperative diabetes care included all the professionals who should participate in such a team.

    Discussion

    In this study, we developed a set of 36 quality indicators describing optimal perioperative diabetes care throughout the hospital care pathway, based on international literature and a systematic modified Delphi procedure. The application of this systematic and rigorous method for indicator development resulted in indicators with high face and content validity. The results from the practice test showed that 21 indicators, including 3 outcome indicators, 9 process indicators and 9 structure indicators, could be used to assess the quality of care delivered in our six study hospitals.

    Our study has several strengths. First, to our knowledge, this is the first study to develop and appraise quality indicators for optimal perioperative diabetes care throughout the entire hospital care pathway using a rigorous and systematic approach. In the UK, the JBDS put audit standards in place as part of their guidelines for perioperative diabetes care.13 There is overlap between our quality indicator set and the audit standards in the JBDS document. However, the selection procedure for the audit standards and how they were appraised is unclear. We used a modified Delphi method to develop a set of quality indicators.18 This method has been used in other studies, and combines scientific evidence with the opinions of experts.15 ,16 ,32 ,36 ,37 Our procedure for quality indicator development follows the common steps of guideline-based quality indicator development that were recommended by Kötter et al.38 Boulkedid et al39 reviewed the use and reporting of the Delphi method for quality indicator selection. They noted considerable variability across studies in the characteristics of the Delphi method, and formulated practical guidelines for using this method for selecting healthcare quality indicators. Our procedure using the modified Delphi method is compatible with these guidelines.

    Second, our indicator set includes indicators for patient outcome, process and organisational structure of perioperative diabetes care—all three components of the Donabedian quality of care framework.18 To make inferences about quality, there needs to be an established relationship between these three components. To get an indication of the criterion validity of the quality indicators in our set (ie, were the outcomes more favourable for patients who received care as described with the quality indicators?), we analysed the relationship between the process indicator scores and the length of hospital stay. Preliminary results suggest a trend towards shorter hospital stay for patients whose blood glucose was measured according to the recommended frequency.

    A limitation of our study is that the indicator set was developed in a Dutch setting; so, the results do not automatically translate to other healthcare systems. As our set of quality indicators is based on international literature, we are convinced that the indicators are useful for other countries as well. For example, our set could be used as an extension to the requirements for inpatient diabetes care that were formulated by the Joint Commission in the USA, as part of their disease-specific programme on inpatient diabetes care.40 However, we strongly advise to test the clinimetric properties of the quality indicators to determine the feasibility of the indicator set in other settings. A practice test will also facilitate acceptance of the indicators. We provided an example of how to perform such a practice test. A second limitation is that nurses and people with diabetes were not represented in the procedure to develop indicators. We aimed to include experts that had both experience in the care of people with diabetes who have to undergo surgery and up-to-date knowledge of the international literature on perioperative diabetes care. The latter may apply less to nurses and people with diabetes. In a different substudy of a PhD study, we defined optimal perioperative diabetes care from the patients’ view.30

    In this study, we developed a set of 36 face and content valid quality indicators. The results from our practice test show that it is essential to subject indicators to a practice test before applying them for quality improvement purposes in a specific setting; only 21 could be used to assess the quality of perioperative diabetes care in our six hospitals. Nine indicators were unmeasurable, which is quite a high number compared with other studies.32 ,36 This was primarily caused by poor availability of medical data and the lack of adequate data resources. Poor availability of medical data and the lack of adequate data resources are common problems in quality assessment efforts.41 As these problems can be solved by changes in administrative policy and data registration, this should not lead to a rejection of these indicators. For example, the introduction of electronic patient records may facilitate the multidisciplinary sharing of relevant patient information and the collection of data to fill the indicators. Similarly, hospital improvement potential was low for five indicators in our six hospitals. These indicators are less sensitive to detecting differences in the quality of care. These indicators are probably not suitable to be used as quality measures in Dutch hospitals. However, again, this should not lead to a rejection of these indicators, as a practice test in a different setting may yield different outcomes. It should be monitored regularly whether adherence continues to be high.

    The indicator set consists mainly of indicators regarding the structures and processes of care, and only few regarding patient outcome. This reflects the ratios of these types of recommendations in guidelines. Insurers, policymakers and consumers may be more interested in outcome indicators; however, there are several disadvantages to the use of outcome indicators. Outcome indicators are sensitive to case mix and the methods of data collection.42 ,43 In addition, good outcomes do not necessarily indicate that care delivery is optimal. Indicators regarding the process and structure of care are more valuable in quality improvement initiatives because, unlike outcome measures, they offer a concrete starting point for improvement. The practice test in our six hospitals showed that there was ample room for improvement regarding many indicators. An improvement strategy was developed to address these problem areas.

    In conclusion, we developed a face and content valid set of quality indicators that defines optimal patient outcome, professional performance and organisational structure of perioperative diabetes care throughout the hospital care pathway. We showed that it is essential to subject indicators to a practice test before applying them for quality improvement purposes in a specific setting. Future studies must prove the value of the indicator set as a tool to guide and monitor an improvement strategy for perioperative diabetes care.

    Acknowledgments

    The authors thank the members of the expert panel for rating the quality indicators, the patients who contributed to this research and the clinical care providers and staff at the participating hospitals. The authors thank Janine Liefers (Radboud University Medical Center, Radboud Institute for Health Sciences, Scientific Institute for Quality of Healthcare) for her contributions to the cleaning and analysis of the data.

    References

    1. ↵
      1. Lauruschkat AH,
      2. Arnrich B,
      3. Albert AA, et al
      . Prevalence and risks of undiagnosed diabetes mellitus in patients undergoing coronary artery bypass grafting. Circulation 2005;112:2397–402. doi:10.1161/CIRCULATIONAHA.105.534545
      OpenUrlAbstract/FREE Full Text
    2. ↵
      1. Frisch A,
      2. Chandra P,
      3. Smiley D, et al
      . Prevalence and clinical outcome of hyperglycemia in the perioperative period in noncardiac surgery. Diabetes Care 2010;3:1783–8. doi:10.2337/dc10-0304
      OpenUrl
    3. ↵
      1. Clement S,
      2. Braithwaite SS,
      3. Magee MF, et al
      . Management of diabetes and hyperglycemia in hospitals. Diabetes Care 2004;27:553–91. doi:10.2337/diacare.27.2.553
      OpenUrlFREE Full Text
    4. ↵
      1. Smiley DD,
      2. Umpierrez GE
      . Perioperative glucose control in the diabetic or nondiabetic patient. South Med J 2006;99:580–9. doi:10.1097/01.smj.0000209366.91803.99
      OpenUrlCrossRefPubMed
    5. ↵
      1. Kwon S,
      2. Thompson R,
      3. Dellinger P, et al
      . Importance of perioperative glycemic control in general surgery: A report from the surgical care and outcomes assessment program. Ann Surg 2013;257:8–14. doi:10.1097/SLA.0b013e31827b6bbc
      OpenUrlCrossRefPubMed
    6. ↵
      1. Vriesendorp TM,
      2. Morelis QJ,
      3. De Vries JH, et al
      . Early post-operative glucose levels are an independent risk factor for infection after peripheral vascular surgery. A retrospective study. Eur J Vasc Endovasc Surg 2004;28:520–5. doi:10.1016/j.ejvs.2004.08.006
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      1. Olsen MA,
      2. Nepple JJ,
      3. Riew KD, et al
      . Risk factors for surgical site infection following orthopaedic spinal operations. J Bone Joint Surg Am 2008;90:62–9. doi:10.2106/JBJS.F.01515
      OpenUrlAbstract/FREE Full Text
    8. ↵
      1. McConnell YJ,
      2. Johnson PM,
      3. Porter GA
      . Surgical site infections following colorectal surgery in patients with diabetes: association with postoperative hyperglycemia. J Gastrointest Surg 2009;13:508–15. doi:10.1007/s11605-008-0734-1
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      1. Pomposelli JJ,
      2. Baxter JK III.,
      3. Babineau TJ, et al
      . Early postoperative glucose control predicts nosocomial infection rate in diabetic patients. JPEN J Parenter Enteral Nutr 1998;22:77–81. doi:10.1177/014860719802200277
      OpenUrlAbstract/FREE Full Text
    10. ↵
      1. Noordzij PG,
      2. Boersma E,
      3. Schreiner F, et al
      . Increased preoperative glucose levels are associated with perioperative mortality in patients undergoing noncardiac, nonvascular surgery. Eur J Endocrinol 2007;156:137–42. doi:10.1530/eje.1.02321
      OpenUrlAbstract/FREE Full Text
    11. ↵
      1. Ramos M,
      2. Khalpey Z,
      3. Lipsitz S, et al
      . Relationship of perioperative hyperglycemia and postoperative infections in patients who undergo general and vascular surgery. Ann Surg 2008;248:585–91. doi:10.1097/SLA.0b013e31818990d1
      OpenUrlPubMedWeb of Science
    12. ↵
      1. Dhatariya K
      . Should inpatient hyperglycaemia be treated? BMJ 2013;346:f134. doi:10.1136/bmj.f134
      OpenUrlFREE Full Text
    13. ↵
      1. Dhatariya K,
      2. Levy N,
      3. Kilvert A, et al
      . NHS Diabetes guideline for the perioperative management of the adult patient with diabetes. Diabet Med 2012;29:420–33. doi:10.1111/j.1464-5491.2012.03582.x
      OpenUrlCrossRefPubMed
    14. ↵
      American Diabetes Association. Standards of medical care in diabetes 2008. Diabetes Care 2008;31(Suppl 1):S12–54. doi:10.2337/dc08-S012
      OpenUrlFREE Full Text
    15. ↵
      1. van Hulst LT,
      2. Fransen J,
      3. den Broeder AA, et al
      . Development of quality indicators for monitoring of the disease course in rheumatoid arthritis. Ann Rheum Dis 2009;68:1805–10. doi:10.1136/ard.2009.108555
      OpenUrlAbstract/FREE Full Text
    16. ↵
      1. Mourad SM,
      2. Hermens RP,
      3. Nelen WL, et al
      . Guideline-based development of quality indicators for subfertility care. Hum Reprod 2007;22:2665–72. doi:10.1093/humrep/dem215
      OpenUrlAbstract/FREE Full Text
    17. ↵
      1. Lawrence M,
      2. Olesen F
      . Indicators of quality in health care. Eur J Gen Pract 1997;3:103–8. doi:10.3109/13814789709160336
      OpenUrlCrossRef
    18. ↵
      1. Donabedian A
      . The definition of quality and approaches to its assessment. Ann Arbor, MI: Health Administration Press, 1980:23–4.
    19. ↵
      1. Fitch K,
      2. Bernstein SJ,
      3. Aguilar MS, et al
      . RAND/UCLA Appropriateness method user's manual. Santa Monica: RAND, 2001. http://www.rand.org/pubs/monograph_reports/MR1269.html (accessed Jan 2014).
    20. ↵
      1. Dalkey NC
      . The Delphi method: an experimental study of group opinion. Santa Monica, CA: The Rand Cooperation, 1969.
    21. ↵
      1. Campbell SM,
      2. Braspenning J,
      3. Hutchinson A, et al
      . Research methods used in developing and applying quality indicators in primary care. BMJ 2003;326:816–19. doi:10.1136/bmj.326.7393.816
      OpenUrlFREE Full Text
    22. ↵
      1. Beersen N,
      2. Kallewaard M,
      3. van Croonenborg JJ, et al
      . Handleiding indicatorontwikkeling. Utrecht: CBO, 2007.
    23. ↵
      1. Rodbard HW,
      2. Blonde L,
      3. Braithwaite SS, et al
      . American Association of Clinical Endocrinologists medical guidelines for clinical practice for the management of diabetes mellitus. Endocr Pract 2007;13(Suppl 1):1–68. doi:10.4158/EP.13.S1.1
      OpenUrlCrossRefPubMed
    24. ↵
      Institute for Clinical Systems Improvement (ICSI). Diagnosis and management of type 2 diabetes mellitus in adults. Bloomington, MN: ICSI, 2008.
    25. ↵
      Dutch Diabetes Federation. NDF Care standard. Transparency and quality of diabetes care for people with type 2 diabetes. Amersfoort: NDF, 2007.
    26. ↵
      Association of Surgeons of the Netherlands (NVVH). Guideline preoperative evaluation. Utrecht: NVVH, 2007.
    27. ↵
      IDF Clinical Guidelines Task Force. IDF Clinical Guidelines Task Force: Global Guideline for Type 2 Diabetes: recommendations for standard, comprehensive, and minimal care. Diabet Med 2006;23:579–93. doi:10.1111/j.1464-5491.2006.01918.x
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      National Institute For Clinical Excellence UK (NICE). Type 1 diabetes: diagnosis and management of type 1 diabetes in children, young people and adults. NICE, 2004. http://www.nice.org.uk/CG015NICEguideline (accessed Jan 2014).
    29. ↵
      McGraw-Hill Concise Dictionary of Modern Medicine. Retrieved from http://medical-dictionary.thefreedictionary.com/major+surgery (last accessed 16 May 2010).
    30. ↵
      1. Hommel I,
      2. van Gurp PJ,
      3. Tack CJ, et al
      . Perioperative diabetes care: room for improving the person centredness. Diabet Med 2015;32:561–8. doi:10.1111/dme.12600
      OpenUrlCrossRefPubMed
    31. ↵
      1. Rubin HR,
      2. Pronovost P,
      3. Diette GB
      . From a process of care to a measure: the development and testing of a quality indicator. Int J Qual Health Care 2001a;13:489–96. doi:10.1093/intqhc/13.6.489
      OpenUrlAbstract/FREE Full Text
    32. ↵
      1. Schouten JA,
      2. Hulscher ME,
      3. Wollersheim H, et al
      . Quality of antibiotic use for lower respiratory tract infections at hospitals: (how) can we measure it? Clin Infect Dis 2005;41:450–60. doi:10.1086/431983
      OpenUrlAbstract/FREE Full Text
    33. ↵
      1. Mourad SM,
      2. Nelen WL,
      3. Hermens RP, et al
      . Variation in subfertility care measured by guideline-based performance indicators. Hum Reprod 2008;23:2493–500. doi:10.1093/humrep/den281
      OpenUrlAbstract/FREE Full Text
    34. ↵
      1. Landis JR,
      2. Koch GG
      . The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74. doi:10.2307/2529310
      OpenUrlCrossRefPubMedWeb of Science
    35. ↵
      1. Guyatt GH,
      2. Sackett DL,
      3. Sinclair JC, et al
      . Users’ guides to the medical literature. IX. A method for grading health care recommendations. Evidence-Based Medicine Working Group. JAMA 1995;274:1800–4. doi:10.1001/jama.1995.03530220066035
      OpenUrlCrossRefPubMedWeb of Science
    36. ↵
      1. Hermanides HS,
      2. Hulscher ME,
      3. Schouten JA, et al
      . Development of quality indicators for the antibiotic treatment of complicated urinary tract infections: a first step to measure and improve care. Clin Infect Dis 2008;46:703–11. doi:10.1086/527384
      OpenUrlAbstract/FREE Full Text
    37. ↵
      1. Hermens RP,
      2. Ouwens MM,
      3. Vonk-Okhuijsen SY, et al
      . Development of quality indicators for diagnosis and treatment of patients with non-small cell lung cancer: a first step toward implementing a multidisciplinary, evidence-based guideline. Lung Cancer 2006;54:117–24. doi:10.1016/j.lungcan.2006.07.001
      OpenUrlCrossRefPubMedWeb of Science
    38. ↵
      1. Kötter T,
      2. Blozik E,
      3. Scherer M
      . Methods for the guideline-based development of quality indicators—a systematic review. Implement Sci 2012;7:21. doi:10.1186/1748-5908-7-21
      OpenUrlCrossRefPubMed
    39. ↵
      1. Boulkedid R,
      2. Abdoul H,
      3. Loustau M, et al
      . Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One 2011;6:e20476. doi:10.1371/journal.pone.0020476
      OpenUrlCrossRefPubMed
    40. ↵
      Joint Commission: Advanced Certification in Inpatient Diabetes. http://www.jointcommission.org/certification/inpatient_diabetes.aspx
    41. ↵
      1. Pronovost PJ,
      2. Miller M,
      3. Wachter RM
      . The GAAP in quality measurement and reporting. JAMA 2007;298:1800–2. doi:10.1001/jama.298.15.1800
      OpenUrlCrossRefPubMedWeb of Science
    42. ↵
      1. Eddy DM
      . Performance measurement: problems and solutions. Health Aff (Millwood) 1998;17:7–25. doi:10.1377/hlthaff.17.4.7
      OpenUrlPubMed
    43. ↵
      1. Mant J
      . Process versus outcome indicators in the assessment of quality of healthcare. Int J Qual Health Care 2001;13:475–80. doi:10.1093/intqhc/13.6.475
      OpenUrlAbstract/FREE Full Text

    Supplementary materials

    Footnotes

    • Contributors IH designed the tools for collecting data, collected, cleaned and analysed the data and drafted and revised the manuscript. PJvG, HW, CJT and MEJLH contributed to the interpretation of the analysis and drafted and revised the manuscript. MEJLH designed the study, and is the guarantor for the study. All authors had full access to all of the data (including statistical reports and tables) in the study, and can take responsibility for the integrity of the data and the accuracy of the data analysis.

    • Funding The Netherlands Organization for Health Research and Development (ZonMw) funded this project (Grant No. 80-82315-98-09012).

    • Competing interests None declared.

    • Ethics approval Regional Review Board for Human Research (CMO), Arnhem-Nijmegen (CMO no. 2008/333).

    • Provenance and peer review Not commissioned; externally peer reviewed.