Article Text

Download PDFPDF

Estimating costs of quality improvement for outpatient healthcare organisations: a practical methodology
  1. Sydney E S Brown,
  2. Marshall H Chin,
  3. Elbert S Huang
  1. Section of General Internal Medicine and the Diabetes Research and Training Center, Pritzker School of Medicine, University of Chicago, Chicago, Illinois, USA
  1. Correspondence to:
 Dr Elbert S Huang
 University of Chicago, 5841 S. Maryland Avenue, MC 2007, Chicago, Illinois 60637, USA; ehuang{at}


Background: Outpatient healthcare organisations worldwide participate in quality improvement (QI) programmes. Despite the importance of understanding the financial impact of such programmes, there are no established standard methods for empirically assessing QI programme costs and their consequences for small outpatient healthcare organisations.

Objective and methods: The costs and cost consequences were evaluated for a diabetes QI programme implemented throughout the USA in federally qualified community health centres. For five case study centres, survey instruments and methods for data analysis were developed.

Results: Two types of cost/revenue were evaluated. Direct costs/revenues, such as personnel time, items purchased and grants received, were evaluated using self-administered surveys. Cost/revenue consequences, which were cost/revenue changes that may have occurred due to changes in patient utilisation or physician behaviour, were evaluated using electronic billing data. Other methods for evaluating cost/revenue consequences if electronic billing data are not available are also discussed.

Conclusion: This paper describes a practical taxonomy and method for assessing the costs and revenues of QI programmes for outpatient organisations. Results of such analyses will be useful for healthcare organisations implementing QI programmes and also for policy makers designing incentives for QI participation.

  • QI, quality improvement

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Deficiencies in the quality of care for chronic conditions are recognised as a major public health problem.1 To deal with these deficiencies, healthcare organisations throughout the world are being encouraged or required to take part in quality improvement (QI) programmes.2 Although QI programmes for chronic disease management have been found to be beneficial,3–5 their adoption has been much more limited than expected. One possible reason for this phenomenon is that outpatient healthcare organisations must expend considerable resources to implement QI programmes, but they may never financially benefit from them.6,7 To what extent these concerns are valid is uncertain. Given this uncertainty, outpatient organisations would benefit from indepth evaluations of how much these programmes cost to implement and run successfully.8 Such evaluations may also be valuable for policy makers designing and implementing financial incentives to encourage QI activities.2,9

Currently, little formal guidance exists for small organisations to empirically evaluate costs/revenues associated with QI programmes. There are several well-established methods for evaluating the societal value of healthcare programmes (table 1). Although methods such as cost-effectiveness analysis represent the gold standard in the medical literature,10 these analyses do not provide the information small organisations need to make informed business decisions. Apart from these methods, the National Institute for Health and Clinical Excellence in the UK provides cost accounting tools for use by local health authorities to inform decisions regarding the adoption of new healthcare technologies.11 These tools are appropriate for regional policy decisions; however, they may still need further adaptation for empirical evaluation of programmes from the perspective of small healthcare organisations.12

Table 1

 Application of standard health economic evaluation methods to evaluate quality improvement programmes

During a recent evaluation of a QI programme in US federally qualified community health centres, we developed an approach to empirically assessing costs that acknowledges formal health economic methods and accommodates common data limitations.


Between 2003 and 2005, we evaluated the financial impact of the Health Disparities Collaboratives using case studies13–15 of five health centres. The Health Disparities Collaboratives is an ongoing national programme to improve care for chronic conditions in federally qualified community health centres. Health centres are part of the US healthcare safety net and receive federal money to provide primary healthcare to the under/uninsured.16 The Collaboratives team members learn about chronic disease management and rapid QI,17–20 share best practices at learning sessions, and develop QI programmes specifically tailored to their organisations. We evaluated two types of costs/revenue:

  • direct costs/revenues, such as health professional time, items purchased, grants received;

  • cost/revenue consequences, which included financial effects due to changes in patient utilisation or physicians’ behaviour (figs 1 and 2).

Figure 1

 Direct costs/revenues.

Figure 2

 Cost/revenue consequences.


Direct costs/revenues

Key participants completed a retrospective survey on resources spent on the QI programme. Under prospective data collection, all individuals in the QI programme would track expenditures in real time. Costs included time spent on QI, and money spent on QI-related purchases. Revenues included donations and grants received for the QI programme. We also collected data about expenditures on training sessions. For this QI initiative, participating health centres received a subsidy from the US government to cover travel expenses for these sessions.

Next we converted the time spent by personnel on the QI programme into dollars (table 2) using the opportunity cost concept.21 We used the employee’s hourly wage for employees who did not generate clinical revenues through encounters. For medical care providers, we used the revenue lost to the health centre that the provider could have generated by seeing patients. To find the revenue generated from an average medical encounter, we divided the total revenue medical care providers earned from medical visits in a fiscal year by the total number of medical encounters at the health centre. For example, from one of our centres, a licensed practical nurse earned $29 120/year including fringe in the year 2001. Her hourly wage ($14.00) was determined by dividing her salary by the total number of hours of work per year (52 weeks/year, 40 h/week). For a medical care provider, an hourly wage equalled the average clinical revenue generated per hour. For the physician in our example, an average encounter generated $53.92 in 2001. Assuming four visits an hour, this physician could potentially earn $215.68/h on average in 2001. In settings where clinics function in a capitated healthcare environment, one can use the salary and fringe method of assigning opportunity costs described earlier.

Table 2

 Examples of calculating the costs of personnel time devoted to a quality improvement programme

We used surveys to collect information regarding QI revenues. During our study, healthcare organisations reported that their success in obtaining grants unrelated to the QI programme improved as a result of their participation in the QI programme; however, it was often unclear if the grants would have been received in the absence of the QI programme. For this analysis, we limited direct revenues to those grants/donations that were specifically obtained for the QI programme. When accounting for grants received for QI initiatives, it is critical that any time or usage restrictions on the grant money be taken into account. In the future, incentives such as pay-for-performance will also possibly be an important source of revenue generated from QI activities.2,22

Cost/revenue consequences

QI programmes may lead to changes in patient utilisation, test ordering, physician productivity, and insurance payor mix, which can affect a healthcare organisation’s financial performance. Ideally, electronic billing data should be used to evaluate these changes. In the absence of electronic data, paper data or time and motion studies can be used as substitutes for some analyses. Regardless of the type of data available for analysis, an important first step in evaluating cost consequences is to determine the patients who most probably will be affected by a QI programme, which may involve multiple chronic conditions. We encourage organisations to begin by accounting for the full breadth of costs and benefits that they perceive are related to their overall QI programme.

Electronic billing data available

When working with electronic billing data, first identify the patients most likely to be affected by the QI programme. For this diabetes programme evaluation, we identified all patients who had a diagnostic code (International Classification of Diseases (ICD)-9 code) for diabetes. We then extracted all billing information associated with these patients whether or not it was directly related to diabetes. This allowed us to look at all visits associated with these patients before and during the QI initiative.

In most billing systems, there will be a unique identifier for every charge, adjustment or receipt. Each identifier will be associated with a patient number and date, and transaction description. Other important fields include diagnostic and procedure codes, currency amount and who made the transaction. For our study, technical staff at the health centres extracted the data; if this is not possible, vendors of billing systems may be able to assist but at considerable cost. Multiple analyses of trends over time can be carried out with such data. We examined average number of encounters and revenue per patient, diagnostic test ordering and changes in number of patient encounters per provider. Patient–payor mix can also be evaluated, which may be especially relevant to clinics in the USA as QI programmes may attract sicker or uninsured patient populations to the clinic, which may be less profitable.

An important step in analysing costs is to assign the appropriate unit cost to each component of utilisation. Charges in the billing data are often unrelated to the actual cost of services provided. An organisation’s cost to charge ratio can be used to convert charges into costs.23 If such a ratio does not exist, the costs can be determined using commonly accepted tariff/price lists (eg, National tariff 2006–7 for the UK) for the country of operation. When such unit costs are not available, we recommend the use of costs from similar facilities or countries.24,25 In our study, we used the Medicare Physician Fee Schedule for our US context.26,27 Once these assumptions are settled, costs can then be summed to evaluate whether total costs for the year per patient changed since the initiation of the QI programme. These figures should be adjusted for inflation23 so that they are comparable across years. We used the first year of the QI programme as baseline, and inflated figures in subsequent years by the healthcare consumer price index. This index may vary according to country.

Electronic billing data are not available

If a healthcare organisation lacks billing data or the ability to extract this information, it may be difficult to find a reliable source to identify trends in cost/revenue consequences. Annual accounting summary reports can provide a global picture of an organisation’s finances; however, they cannot be used to determine whether observed trends are attributable to the QI programme. Furthermore, these reports may not be detailed enough to provide information on patient utilisation or physician productivity.

Without detailed electronic billing data, an organisation can still gain some insight into the clinical and cost consequences of a QI programme by analysing random samples of billing data and medical records of targeted QI patients. For instance, one can directly measure markers of clinical productivity such as number of daily patient visits on sampled days during the course of the QI intervention. To measure changes in service utilisation, insurance mix or test ordering, paper billing or medical records for a sample of patients can be analysed. In each case, define a pre-QI baseline and measure changes relative to the baseline.


Standard approaches to assessing the value and costs of healthcare programmes exist, but they are of limited value if healthcare organisations cannot use them in everyday practice to inform managerial decisions. To address this need, we have illustrated an approach to evaluating direct costs/revenues, and cost/revenue consequences that may help such organisations evaluate the financial effect of their QI programmes. These methods should be particularly relevant to small healthcare organisations working in resource-constrained settings such as the developing world where it is extremely important to manage costs effectively.

For a given organisation, the nature of the cost evaluation will depend in large part on what data are available. Our approach is based on QI programmes implemented during the Health Disparities Collaboratives, which focused on improving management of chronic disease in community health centres across the USA.6 We customised our surveys and methods to the specific needs of the organisations we worked with. Tools and methods that retain the concepts used by our survey and database extraction methods can be developed to evaluate the costs/revenues of QI programmes in other healthcare settings.

There are several limitations to estimating costs using our methods. First, retrospective personnel recollection may be inaccurate. Ideally, data collection should occur in real time. Second, it is difficult to determine if secular trends are responsible for the changes observed in the cost/revenue consequences, or whether they result from the QI programme. This can be partially dealt with by comparing institutional trends with those observed regionally and by comparing costs/revenues associated with targeted patients to the overall patient population. Third, when organisations support multiple, ongoing QI initiatives, it may be difficult to account for resources specific for an individual component of the QI programme. Fourth, this analysis does not measure intangible costs/benefits associated with QI. Participants in this QI initiative anecdotally reported increased employee knowledge and initiative, and the spread of QI techniques to other organisational functions, as well as increases in burnout and the reallocation of resources away from other programmes. These may not be captured with financial data.

Our recommended approach to evaluating costs/revenues is primarily intended for leaders of outpatient healthcare organisations to determine the resources required/spent operating a QI programme. It may also provide policy makers important information for determining the amount of incentive needed to encourage QI participation among small organisations. In addition, these cost evaluations generate useful data inputs for traditional cost-effectiveness analyses. The incorporation of empirically collected cost data in cost-effectiveness analyses may help policy makers compare the relative cost effectiveness of different QI programmes and identify those that should be widely disseminated in the future.



  • This study was supported by an Agency for Healthcare Research and Quality (AHRQ) R01 (MC, R01-HS010479), an AHRQ U01 (SB, MC and EH, U01-HS013635), a National Institute on Aging (NIA) Career Development Award (EH, K23-AG021963), a NIDDK Diabetes Research and Training Center (SB, MC and EH, P60 DK20595), the Chicago Center of Excellence in Health Promotion Economics (MC and EH, P30-CD000147), a Robert Wood Johnson Generalist Physician Faculty Scholar Award (MC), and a Midcareer Investigator Award in Patient-Oriented Research from the National Institute of Diabetes and Digestive and Kidney Diseases (MC, K24 DK071933-01).

  • Competing interests: None declared.