Article Text

Download PDFPDF
A framework for measuring quality and promoting accountability across silos: the case of mental disorders and co-occurring conditions
  1. A M Kilbourne1,
  2. C Fullerton2,
  3. D Dausey3,
  4. H A Pincus4,
  5. R C Hermann5
  1. 1VA Ann Arbor National Serious Mental Illness Treatment Research and Evaluation Center, VA Ann Arbor HSR&D Center of Excellence, and Department of Psychiatry, University of Michigan School of Medicine, Ann Arbor, Michigan, USA
  2. 2Department of Health Care Policy, Harvard Medical School, and Department of Psychiatry, Massachusetts General Hospital, Boston, Massachusetts, USA
  3. 3Department of Public Policy and Management, Carnegie Mellon University H. John Heinz III School of Public Policy, Pittsburgh, Pennsylvania, USA
  4. 4College of Physicians and Surgeons, Columbia University, and New York Presbyterian Hospital and Irving Institute for Clinical and Translational Research, New York, New York, USA
  5. 5Center for Quality Assessment and Improvement in Mental Health (CQAIMH) at Tufts-New England Medical Center's Institute for Clinical Research & Health Policy Studies, Boston, Massachusetts, USA
  1. Correspondence to Dr Amy M Kilbourne, VA Ann Arbor SMITREC (11H), 2215 Fuller Road, Ann Arbor, MI 48105, USA; amykilbo{at}


Background Quality measures can be effective tools for improving delivery of care and patient outcomes. Co-occurring conditions (COCs), including general medical conditions and substance use disorders, are the rule rather than the exception in patients with serious mental health disorders and lead to substantial morbidity and mortality burden. COCs among persons with mental health disorders are often treated by separate systems (“silos”) in the US healthcare system, making it difficult to establish expectations for performance, assign accountability for measure results and ultimately improve quality of care for this group.

Objectives A framework for measuring quality of care for COCs is proposed by reviewing the current state of quality for COCs and examples of quality measures based on the Donabedian model.

Methods and framework The framework will also be applied to better define which providers are accountable for quality improvement, to ultimately ensure that quality measures have an impact on improving care for COCs.

  • Quality of care
  • mental disorders
  • substance use disorders
  • comorbidity
  • primary care
  • outcome
  • healthcare quality
  • substance use disorders
View Full Text

Statistics from

Performance (quality) measures can be effective tools for improving quality of care for patients by identifying gaps between high-quality care and the care actually provided to patients.1 2 Throughout the US healthcare system, measurement results are gradually being incorporated into efforts to establish priorities for improvement, to provide oversight of providers and, most recently, to influence reimbursement practices.3 4

Serious mental health disorders (MHDs—eg, schizophrenia, bipolar disorder, major depression) are associated with substantial functional impairment, healthcare costs and premature mortality.5 Moreover, co-occurring conditions (COCs), notably general medical conditions (GMCs) and substance use disorders (SUDs) in persons with MHDs, are the rule rather than the exception. Rates of GMCs6 7 and SUDs in persons with MHDs exceed those found in non-psychiatric patient populations.8

A number of quality measures have recently been developed for MHDs and co-occurring SUDs9 10 and GMCs.11 12 Still, these existing measures primarily focus on treatment for one condition rather than the management of COCs. Persons with chronic MHDs and COCs are uniquely vulnerable to poor quality of care because many consider the mental health speciality setting their “home” site for care13 and many mental health treatment settings lack the resources to manage COCs.

In response, there is a call to better integrate services for COCs for persons with MHDs. This is partly due to the increased recognition of the medical consequences of psychiatric drugs (eg, the risk of diabetes associated with the use of new antipsychotic drugs)14 as well as efforts by national agencies such as the Substance Abuse and Mental Health Services Administration's “No Wrong Door Policy,”15 which is designed to reduce organisational barriers for dually diagnosed MHD/SUD individuals regardless of their point of entry.15 In addition, costs of COCs can transcend system boundaries, with up to 70% of costs associated with MHDs have been attributed to general medical services.16 Lack of integrated SUD care for persons with MHDs may be associated with increased inpatient and emergency services use and increased morbidity and mortality.17

Despite these concerns, there has been little effort to establish a set of measures focused on quality of care for patients with COCs that can be used to evaluate and improve care. Moreover, there have been fewer attempts towards establishing the accountability for these performance measures across different providers (eg, mental health, substance use treatment). With little attention to these issues, patients can fall through the cracks, with diagnoses missed and treatments delayed.

In light of these realisations, the purpose of this article is to propose a framework for measuring quality of care for COCs. In doing so, we review the current state of gaps in quality of care for COCs and walk through illustrative examples of quality measures based on the Donabedian model. We also apply the framework to better define which providers are accountable for quality improvement for COCs, to ensure that quality measures have an impact on improving care for COCs.

Gaps in quality for COCs

Individuals treated for MHDs are disproportionately affected by COCs. GMCs including diabetes (8–15%) and cardiovascular disease (19–31%) occur more frequently in patients with MHDs than the non-psychiatric patient population,6 7 and 51% of individuals with a MHD had at least one co-occurring SUD.8

Quality of care for GMCs is suboptimal in persons with MHDs. Increased mortality rates from cardiovascular disease among persons with chronic MHDs were explained by differences in the quality of treatment they received.18 Of particular concern is the failure to detect medical conditions caused or exacerbated by medications prescribed by mental health professionals, notably new types of antipsychotic medications. Fewer than half of patients with MHDs treated with atypical antipsychotics received recommended monitoring for common side effects increasing risk for cardiovascular disease (eg, lipids, fasting glucose).12 Moreover, co-occurring SUDs in patients with MHDs are frequently undocumented19 and fewer than a third receive treatment.20 21

Quality measures for COCS

Quality measures can be effective tools for improving care for patients with COCs.1 Donabedian's framework,22 describing the structures, processes and outcomes, provides a useful typology for distinguishing among different approaches to measurement. Specifically, structure measures evaluate characteristics of the treatment setting's services. Process measures examine interactions between consumers and the structural elements of the healthcare system. Outcome measures examine the results of these interactions for patients, including functioning, morbidity, mortality and patient satisfaction.

Using Donabedian's framework, we describe examples of measures that are relevant to COCs (table 1), and based on these examples, describe the advantages and disadvantages of each framework domain below. The measures were selected as illustrative examples relevant to COCs by the authors from the Center for Quality Assessment and Improvement in Mental Health's website clearinghouse of quality measures for COCs developed by government agencies, researchers, clinician/professional organisations, health systems/facilities, employer purchasers, consumer coalitions and commercial organisations (

Table 1

Illustrative measures of COCs for patients with MHDs


An example of a structure measure is the percentage of MHD providers within a mental health programme with competence in SUD care. To operationalise this measure, leaders within the mental health community would have to establish a standard for competence, such as a form of credentialing, testing or training. Another type of structure measure is fidelity to an evidence-based treatment model or to what extent the agency is implementing the model as it was originally developed and shown to improve outcomes. For example, the Dartmouth Assertive Community Treatment Scale contains 28 items, including patient–case manager ratio (caseload) and the range of services provided by the case manager (eg, referrals to medical providers, linkages to housing).23

The advantage of structural measures is that they can be collected without relying on incomplete claims data or expensive chart abstractions, which are required for processes and outcomes measures. However, it has proven difficult to correlate structural measures with variations in outcomes or processes of care. For example, training in SUD treatment in MHDs may not necessarily mean that providers impart adequate care for this group. Moreover, fidelity scales have been developed for a limited number of mostly psychosocial interventions.


Process measures for SUD care include the percentage of patients screened for SUD and, among those with SUD, the percentage “engaged in care” (eg, defined as the two or more outpatient visits within 2 weeks) (table 1). Tobacco use screening and receipt of counselling to quit smoking are also widely used SUD process measures. This latter measure has been linked to improved outcomes.24 For GMC, process measures include diabetes and hyperlipidaemia screening for patients on atypical antipsychotics.12

Nonetheless, process measures for COCs do not take into account patient preferences (eg, declining tobacco counselling or lab screenings for atypical antipsychotics). Two strategies that are often used to account for patient preferences in measuring quality include the acknowledgement of patient preferences that are documented in medical records by the provider or use of realistic benchmarks (eg, ≥80% as opposed to 100% of patients receiving process of care, to account for a subsample refusing a service). Whereas the former strategy can be more accurate in assessing patient preferences, chart reviews can be costly and time-consuming, and hence, the latter strategy of setting realistic benchmarks that take into consideration exceptions is preferred.

Still, these process measures are limited to a particular condition or procedures. Developing measures that address processes of care between general medical, mental health and SUD providers can be helpful in ensuring adequate quality of care for COCs. Conceptually, Shortell et al25 refer to these processes of care as “clinical integration” features, which constitute (1) coordination (proactive and shared understanding of goals and roles, and shared decision-making between different providers), (2) communication (degree of information flow across providers from different disciplines) and (3) continuity of care (timely and uninterrupted delivery of appropriate services across levels of care). However, to date, only continuity of care measures have been operationalised, and none of these clinical integration measures have been validated.25 For example, continuity of care has been defined as the percentage of patients who receive outpatient follow-up visits within 30 days after a medical hospitalisation discharge.26 Hence, further development and validation of these integration measures are crucial for understanding and addressing gaps in their quality of care.


Outcomes relevant to COCs include intermediate physiologic measures directly associated with the risk of cardiovascular disease (eg, high blood pressure) to measures of functioning directly impacted by SUD (eg, employment). For example, the Substance Abuse and Mental Health Services Administration has implemented the National Outcome Measures initiative, which includes outcome measures such as abstinence from alcohol or drug use.27 Patient satisfaction for care for COCs is also important and is linked to improved treatment adherence and fewer hospitalisations.28

Outcomes are desirable because they represent the “bottom line”—whether patients are doing better or worse. However, changes in outcomes are challenging to interpret because of the many factors such as health behaviours and socioeconomic status that are outside the healthcare system can influence health outcomes. Outcome measures are especially problematic if used to allocate payment, as this creates incentives for providers to “cherry-pick” patients. Although risk adjustment attempts to mitigate this concern, most risk adjustment algorithms are inadequate and explain <30% of the outcome variance.29 Some clinical outcome measures may also be difficult to define because they imply improvement across different conditions, yet in many cases, only one condition may improve. Moreover, one may observe improvement in one condition, but a decline in another, which may cancel each other out and lead to a distortion of quality.

Quality improvement: who is accountable?

Table 1 includes measures that are potentially useful for assessing the quality of care for COCs in persons with MHDs. However, few existing measures directly assess care for COCs specifically among persons with MHDs. A 2003 review of 300 process measures of MHD and SUD care found <3% assessed care for co-occurring MHDs and SUDs and <1% assessed medical care for individuals with MHDs.30 A unique concern in managing COCs is that quality is dependent on care received across different treatment providers.1 The primary site of care for persons with MHDs is often the mental health facility,13 in part because these patients feel more comfortable in this setting. Few incentives exist to promote quality of GMC and SUD care for MHDs. Available measures do not describe who is accountable for relevant care, especially for GMCs. The situation is further complicated given the metabolic risk factors emerging from the use of atypical antipsychotics in MHDs.

To improve the set of quality measures for COCs in MHDs, we propose as an initial step guidelines for which providers are accountable for what services or procedures. Accountability is defined as knowing who is responsible for the delivery of which service and thus where to target monitoring and accreditation.25 But what are the reasonable delineations of accountability for COCs?

Varying degrees of accountability for COCs

Figure 1 presents a framework for clarifying degrees of accountability for COCs between mental health, general medical and SUD providers. In developing this framework, we acknowledge that there are varying degrees of accountability. For example, to what extent are mental health providers accountable for general medical illnesses such as those derived from psychotropic medications? Monitoring the effects of psychotropic drugs should be the responsibility of the psychiatrist prescribing the medication, yet managing the ongoing glucose intolerance that may stem from atypical antipsychotic use might be a more appropriate role for the patient's primary care physician (table 2). Arguably, the decision to continue use of the antipsychotic agent despite the risk of glucose intolerance should be a shared decision between the psychiatrist and primary care physician.31

Figure 1

Spheres of provider accountability among general medical, mental health and substance use disorder specialists.

Table 2

Distribution of accountability for quality improvement between mental health and general medical providers

Standards for accountability for quality

The development and refinement of quality indicators that promote shared accountability is an important step towards improving integrated care for COCs. However, indicator development is dependent on the establishment of consensus-based standards of care. One example of where shared accountability has been operationalised using quality measures is in the Veteran's Health Administration (VHA). In 2005, the VHA External Peer Review Program (a national programme dedicated to monitoring the quality of VHA care via random samples of chart reviews) began measuring performance on key prevention measures (eg, flu vaccination, tobacco screening) among both mental health and primary care clinics.32 The goal is to hold mental health programme directors accountable for prevention for patients with MHDs and encourage mental health clinics to promote regular GMC visits. Standards for accountability may also in part be determined by the nature of the healthcare system (eg, carve-in behavioural health within HMOs) as well as by patient preferences (eg, persons with chronic mental illness may wish to have all of their care provided by the same mental health specialist). Needless to say, in these cases, establishing successful communication between speciality and primary healthcare sectors will be difficult but paramount for accountability standards to be successful.

Future directions

This paper highlighted the need to improve quality of care for persons with COCs and presented a framework for developing future quality measures relevant to COCs in MHDs based on definitions of accountability. Key challenges in operationalising the framework include refinement of quality measures for COCs, validation of measures for integrated services and defining accountability. Ultimately, this framework informs an agenda for further research in these areas, which would ensure the development of more appropriate quality measures for COCs that can be adopted by national organisations as a means to improve outcomes for this group.


View Abstract


  • Funding This work was supported by the Veterans Health Administration, Health Services Research and Development Service (IIR 02-283, IIR 07-115; AMK, PI), the Robert Wood Johnson Foundation Depression in Primary Care National Program (HAP, director), the Department of Veterans Affairs, the Robert Wood Johnson Foundation Substance Abuse Policy Research Program (DD) and the National Institute of Mental Health (CF). This work was also completed with the support of the Robert Wood Johnson Foundation Depression in Primary Care; Linking Clinical and System Strategies Program, the Irving Institute for Clinical and Translational Awards Grant Number UL1 RR024156 from the National Center for Research Resources, and the Mental Heath Therapeutics CERT at Rutgers, the State University of New Jersey, funded by the Agency for Healthcare Research and Quality (AHRQ) (5 U18 HS016097). The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.

  • Competing interests None.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.