Article Text

other Versions


Using balanced metrics and mixed methods to better understand QI interventions
  1. Peter J Kaboli1,2,
  2. Hilary J Mosher1,2
  1. 1The Comprehensive Access and Delivery Research and Evaluation (CADRE) Center at the Iowa City VA Healthcare System, Iowa City, Iowa, USA
  2. 2Department of Internal Medicine, University of Iowa Carver College of Medicine, Iowa City, Iowa, USA
  1. Correspondence to Dr Peter J Kaboli, Center for Comprehensive Access and Delivery Research and Evaluation (CADRE), Iowa City VAMC, 601 Highway 6 West, Iowa City, IA 52246, USA; peter.kaboli{at}

Statistics from

Improving quality while maintaining or reducing costs requires balancing competing demands to bring value to healthcare. High-value reporting of quality improvement (QI) initiatives similarly requires balancing descriptions of improvements achieved with assessments of potential costs and unintended consequences. Using balanced QI metrics allows simultaneous measurement of intended improvements (eg, reduced length of stay (LOS)) and of processes or outcomes that might worsen as a result of a given intervention (eg, mortality, hospital readmission). In their initiative to improve the efficiency of inpatient care without compromising safety at a large teaching hospital in Edmonton, Alberta in Canada, McAlister et al1 report balanced measures, use a methodologically evaluative QI design, and describe the local contextual factors that influenced their success, thus creating generalisable knowledge.

Their intervention bundles a number of plausible improvements on inpatient units: daily interdisciplinary care rounds, geographical cohorting of patients—that is, placing general medicine patients and their doctors at one place in the hospital, strategies to optimise care transitions (eg, medication reconciliation) and use of best practice through care maps, order sets and decision support tools. Many would regard these changes as components of high-quality inpatient care and appropriate to all patients. In reality, limited evidence supports these interventions individually and the magnitude of their benefits (at least on their own) is probably not large. Hence, the reason for a multifaceted or bundled intervention—we do not know which component will generate important improvements, and, it is possible all are needed. Some components may even have synergistic effects. Such bundled interventions will be increasingly important to deal with highly complex healthcare problems, which typically have no single ‘magic bullet’ solution.

On the one hand, a bundled approach aims at the QI target quickly (as opposed to testing each component in turn or trying different combinations of possibly synergistic components). On the other hand, the price often paid for the bundled approach (the proverbial kitchen sink approach—‘we threw everything at the problem except the kitchen sink’) is that the active ingredients remain unclear. Recognising this problem, McAlister et al incorporated a mixed-methods approach in their attempt to identify the most effective elements. Their focus groups attributed the benefits achieved largely to two of the changes. The first was geographical cohorting of patients: the intervention resulted in 97% of admissions going to general internal medicine wards from a baseline rate of 53%. The second was daily interdisciplinary discharge rounds occurring 5 days a week from a baseline rate of once weekly. Thus, the report by McAlister et al is noteworthy for the robust controlled evaluation that showed a reduced LOS attributable to their inpatient care transformation initiative, and also for isolating the likely ‘active ingredients’ in their intervention.2

Geographical cohorting of patients deserves further discussion. This innovation appeals to healthcare teams for many reasons, including improved communication and relationship building when working with a smaller group of individuals, and reducing the inefficiencies of walking to multiple physically separated units. Although there is little empirical evidence that geographical cohorting improves efficiency, the focus groups thought it did. However, this apparently simple change is far from straightforward to introduce owing to difficulty in managing bed flow. Administrators recognise that clinicians (including physicians and nurses) prefer to avoid having ‘off-service’ or ‘off-floor’ patients, but hospitals generally need to maximise bed occupancy to optimise fixed costs. If a medical patient is admitted to the emergency department and the first available bed is on a surgical floor, the financially optimal approach seems to be using the surgical bed. Not doing so means that nurses on the surgical ward are not working at capacity (a poor use of fixed staffing costs) and also means an admitted patient languishes in the emergency department—unpleasant for the patient and distracting for emergency staff.

To achieve the dramatic improvement in geographical cohorting reported by McAlister et al thus requires practical problem solving by clinicians and administrators. Some (perhaps many) hospitals may find the task of achieving the right balance between use of beds and geographical cohorting too daunting. But, the reported gains in reduced LOS, combined with the popularity of the model among clinicians, may provide sufficient incentive to solve the administrative and bed-flow problems involved. An additional incentive is that geographical cohorting probably is a necessary precondition for daily interdisciplinary discharge rounds, the only other component perceived as helpful in the intervention. Interdisciplinary rounds can achieve only limited effects if half the team's patients reside on other wards.

Targets for improvement will remain abundant and easily identifiable and QI work will become more complex. As hospitals and healthcare systems refine processes and improve quality—and see mixed results in the outcomes—the awareness of the need for new and valid metrics grows. Metrics and their reporting are needed to guide improvement efforts and ensure that hard-won results are maintained. Moreover, metrics need to be applicable in heterogeneous settings and account for local contextual factors. For example, LOS will become less important as we focus on excess bed-days of care—that is, those days of care provided in hospital that could be provided at another level of care (eg, skilled care) or with home care or in a hospice. Each patient should remain in hospital as long as needed to provide the care necessary and not an hour longer. Continually focusing on shortening LOS for the population may distract us from identifying and ensuring the optimal time of stay for each individual according to his or her healthcare needs. Systems lacking robust home care resources may find that optimal LOS, measured by the number of deaths or readmission, differs from that seen in systems with highly developed home care capacity.

As a new metric, excess bed-days of care need to be carefully monitored by management and discussion with emergency departments and inpatient care teams. This also applies to identification of admissions and readmissions as ‘appropriate’—a subjective description not easily agreed on. Our use and comfort with evolving metrics to determine excess bed-days of care, appropriate admissions, and preventable readmissions will allow us to continually improve inpatient care. By establishing benchmarks, we can develop ways to reduce unnecessary care. Pressures to prevent unnecessary care will continue to grow as hospitals run at, or near, capacity, as US healthcare moves away from traditional fee-for-service and switches to Accountable Care Organisations, and as the Canadian system responds to rising costs.

Selecting appropriate process and outcome measures to achieve balanced results requires careful attention to the way in which hospitals differ from each other and from other industries. Efficiencies, goals and processes drawn from analogous industries may not be applicable—for example, a metric that has been a focus of some is ‘discharge before noon.’ Although, this target may work in the hotel industry, it may not in hospitals (even as hospitals begin to look more and more like hotels).3 Patients should leave hospital at the best time for the patient and his or her caregivers. Our hospital systems are designed to allow patients to be admitted whenever care is needed. Surely we can design systems that offer patients their preferred time of discharge? Patients will self-select when they want to go home, resulting in a ‘smoothing’ of discharges that does not result in boluses of patients being discharged at any one time.

QI initiatives can be catalysts for change in a single hospital or across a healthcare system, as McAlister and colleagues’ study has demonstrated. Difficulties are numerous, yet the opportunities for learning and improving are far greater. Although not all QI initiatives achieve the intended improvements in processes or outcomes, they can result in the organisation and individuals learning from the experience and working together as a team with a common goal. Careful planning and balanced measurement are needed to ensure that even ‘failures’ can be used to make progress. QI failures are valuable if we can fail as fast and as harmlessly as possible, and learn from the experience to redirect our efforts.

The work by McAlister and colleagues is exemplary and motivating in a number of ways. First, producing scholarly work from QI is critical so that healthcare can learn from these successes and failures and appreciate both the generalisable knowledge gained and the local context of care. Second, as more QI work is published, the need to follow established guidelines such as the Standards for QUality Improvement Reporting and Excellence (SQUIRE)4 is even more important to guide the design, measurement and reporting of the work. Third, the incorporation of a mixed-method evaluation allows for both quantitative reporting of the improvement and qualitative evaluation of the parts of the intervention having the greatest perceived value. Finally, spread of innovation within or across healthcare systems will depend upon both experienced internal QI team members and external collaborators providing guidance in the mentored implementation model.5 Successful innovators ‘stand on the shoulders of giants;’ although the science of QI may be just transitioning from crawling to standing, we must aspire to create a stable foundation for others to climb. McAlister et al provide an exemplar of high-quality methods and reporting. Learning from the Alberta experience, much as the Alberta group learnt from previously published work and available toolkits and materials, is invaluable.

We look forward to reading about their future work, including the spread and sustainability of this successful intervention. Without spread and sustainability6 we are expending considerable resources with little net improvement in healthcare.


View Abstract


  • Competing interests None.

  • Provenance and peer review Not commissioned; internally peer reviewed.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles