Article Text

Download PDFPDF

Building an integrated methodology of learning that can optimally support improvements in healthcare
  1. Joanne Lynn
  1. Colorado Foundation for Medical Care, Maryland, USA
  1. Correspondence to Dr Joanne Lynn, Colorado Foundation for Medical Care, 2318 Ashboro Dr, Chevy Chase, MD 20815, USA; drjoannelynn{at}


The methods for healthcare reform are strikingly underdeveloped, with much reliance on political power. A methodology that combined methods from sources such as clinical trials, experience-based wisdom, and improvement science could be among the aims of the upcoming work in the USA on comparative effectiveness and on the agenda of the Center for Medicare and Medicaid Innovation in the Centers for Medicare and Medicaid Services. Those working in quality improvement have an unusual opportunity to generate substantial input into these processes through professional organisations such as the Academy for Healthcare Improvement and dominant leadership organisations such as the Institute for Healthcare Improvement.

  • Public policy
  • quality improvement
  • comparative effectiveness research
  • continuous quality improvement
  • health policy
  • leadership
  • randomised controlled trial
  • research

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: and

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Challenge: multiple poorly governed methods for reform

The USA spends too much and gets too little from its healthcare system—that is the current dogma, and there is little reason to doubt it. The logical follow-up is that the nation needs to reform its service delivery system, and that also is a readily justifiable claim. Indeed, the nation has taken substantial steps towards reforms in the recent Patient Protection and Affordable Care Act (PPACA).1 However, the methods for reform become less obvious as one sets out to fill in the details. The simple fact is that, other than developing breakthroughs in diagnosis and treatment of individuals or restructuring how we pay for services, we have strikingly little methodology of reform for medical care.

Exactly what do we aim to achieve, and exactly how would we achieve it? I contend that the answers that we offer for these two key questions rely excessively on philosophies of governance or philosophies of science, and far too little on deliberately learning how to accelerate the pace and guide the direction of improvements. I also contend that we could develop the needed insights quickly, if we set out to do so.

This paper explores some of the ideas and strategies that would lead to a broader and more integrated methodology for developing the insights needed to guide improvements in the delivery of healthcare services. Although the examples are mostly from the USA, one would expect that these methods and many of the insights would be useful globally.

Mix of methods

Within the recollection of older researchers and practitioners in medical care, society celebrated medicine as an ‘art,’ and most of its truths were distillations of experience from seasoned practitioners. In the past half century, the standard of proof in medical care has moved strongly towards a scientific model, seeking generalisable truths about the workings of the body and of the interventions proposed. This has led to a useful reification of the randomised controlled trial, with its ability to mask many effects of unmeasured variables, secular trends and random events; and the factual basis of medical practice is much stronger than accretion of experience by practitioners could have achieved. This work has been generally well supported—by government, academic institutions and businesses—and many people make their living in medical research governed by careful statistical proof.

However, the work has had its challenges. Some issues are just too expensive to study rigorously, some are too difficult to measure reliably, and some require so many exclusions before sampling that the scope of generalisability is uncertain or limited. Furthermore, thoroughly inappropriate use of human subjects in an array of medical research projects led to extensive review and regulation aimed to prevent abuse.

Within the last 20 years, a complementary set of process improvement methods has taken root, borrowed from management of manufacturing. Continuous quality improvement uses goal-setting, monitoring, standardisation, testing and strategies for sustainability, and spread in order to tackle potential dysfunctions in service delivery, mostly within one organisation or setting. The methods have often been successful in rapidly and sustainably achieving better outcomes and now form the backbone of major portions of professional and organisational credentialling. But the relationships between the methods of scientific research and of quality improvement have not yet been worked out well. For example, it not clear whether funding appropriated for ‘research’ could be used in ‘quality improvement,’ whether improvement projects deserve to be published or which ‘quality improvement’ projects must undergo review under the regulations governing ‘research on human subjects’. Perhaps most importantly, how to tackle important national priorities using a prudent mix of methods has received scant attention.

Furthermore, as the recent US legislation1 illustrates, sometimes major changes (which may yield improvements) arise from adoption of broad policies, such as coverage of new categories of services by public insurance, coverage of new beneficiaries or imposing performance requirements for practitioners or insurance companies. Those policies, of course, can also affect the funding and other resources available for research or quality improvement, and can dictate the resolution of some points of uncertainty as to methods. But again, how best to use public policy as part of the mix of methods to achieve improvement is often left mostly to power politics, informed only by scant information as to the wise course to follow and constrained to be ‘permanent’ because of the overwhelming practical difficulties in deliberately designing policy interventions as tests that will be monitored and adjusted on the basis of experience. So, when to use policy to achieve reforms currently has no better evidence-based methodology at its disposal (at least in the USA) than deciding how to draw insights from sources like conventional medical research and quality improvement. Other modes of gaining insight and making improvement also play their similarly happenstance roles—for example, epidemiological surveillance, public health interventions outside medical care (eg, housing, nutrition and environment) and programme implementation with evaluation.

Politics as usual

Society has not developed a methodology to guide use of the overall panoply of approaches that yield insights useful to efficient and effective reform. Instead, the developmental process often leaves the setting of goals and the implementation of methods to the political arena. Perhaps enlightened, informed and fair political agenda-setting is the best that complex democratic society can do in setting goals, but evidence and wisdom have strong roles to play in the selection of methods to ensure that political priorities reflect reliable insights.

At present, however, the strong allegiance of many powerful forces to the methods of orthodox medical research serves to denigrate most other methods of gaining insight. This is not merely an annoyance, since it serves to make some kinds of insight unavailable when costs, complexity or time pressures make conventional medical research unavailable. Thus, rather than relying on the experience of multiple sites in improving a clinical process or drawing from the experience of multiple jurisdictions in relieving inequities, leaders tend to discount those insights and thus to abandon the possibility of thoughtful and guided reform, either by not getting reforms under way or leaving them to political forces operating with inadequate information.

Opportunity: mixed methods in an experience-based methodology

Every nation has strong reasons to move towards a highly efficient, highly reliable medical care system quickly. Without that, healthcare costs will increasingly burden the economy generally and unacceptably diminish residents' opportunities. Some important steps are under way in the USA already. The Patient Protection and Affordable Care Act will bring financial access to medical services to almost all Americans, thus making the fate of the delivery system more strongly constructed as a social concern and less likely to be framed as subject to individual merit or good fortune. That statute also sets up a score of initiatives aiming for long-term improvement in quality and efficiency, including a Center for Innovation, funding for expanding quality improvement and funding for comparative effectiveness research. Under the American Recovery and Reinvestment Act,2 The Institute of Medicine reported a list of priorities for comparative effectiveness research.3

The ferment that the upcoming health reform will generate could provide an important and unusual opportunity to learn what we need to know about the methods of managing complex healthcare systems. What information can guide the work? What are the roles of conventional medical research, of quality improvement, policy changes and other elements? What information is reliable enough to work with, even if not well proven in multiple settings? What sorts of political pressures would help to sustain optimum improvement, defined by the public interest?

Example: the upcoming US investment in comparative effectiveness research

The example of comparative effectiveness could illustrate how to develop a methodology for mixed methods of reform. The US Congress appropriated about $10 billion for start-up and $500 million per year (from a fee assessed on health insurance) for this rather imprecisely defined work. The shortcomings that motivated this commitment centred on the recognition that much of medical research is not readily applicable to the usual situations of patients, for a number of reasons, and that deliberately expanding research to broader populations and to specified subsets could provide more authoritative answers to treatment questions that doctors and patients actually face. In some instances, this only requires repeating randomised clinical trials in broader populations to test whether effects documented in narrowly defined populations still are evident in more typical populations. However, half of the topics in the Institute of Medicine (IOM) list require attention to ‘system’ factors,4 meaning that the way that the delivery system operates is a key component affecting potential effectiveness.

For example, the third topic in the IOM priority list is ‘falls,’ which are a particular plague for older people. Specifically, the IOM Committee called for research to ‘Compare the effectiveness of primary prevention methods, such as exercise and balance training, versus clinical treatments in preventing falls in older adults at varying degrees of risk.’ Some of the issues in reducing the impact of falls are amenable to conventional clinical trials of a manageable size: for example, optimal treatment for reversible postural hypotension or optimal rehabilitation following elective hip replacement. However, reducing the impact of falls in a community will depend heavily upon ‘clinical treatments’ that rely upon local factors such as the ability to make home settings safer, the availability and services of monitoring agencies, and the supportive homemaker and rehabilitation services of the local Medicaid benefit. Indeed, perhaps the most important factor will be the prioritisation of mitigating this problem in many parts of the social system, from housing to transportation as well as healthcare services.

Thus, in devising a strategy to learn what we need to know in order to address the problem of falls, those who allocate funds and shape initiatives will need to develop a balanced strategy that tests medical treatments in broad populations and subsets—and that also tests the merits of various ways of forging the will to address the problem (generating motivating data and organising advocates, for example) and of various ways of managing a process of change (technical assistance, coalition-building and financial incentives, for example). The point would be to generate, within a reasonably short time, a substantially growing expertise in how to prevent and mitigate harm from falls—addressing, for example, individual clinicians and their patients, leaders of institutions providing services, public health officials, contributing social service and public infrastructure entities, and working groups focused upon a geographical locale. The products might include specific interactive models to assist physicians in decision-making about medication combinations and side effects, as well as specific recommendations on lighting, doorways and bathroom fixtures in new housing. In the USA, these opportunities rest not only in the comparative effectiveness initiative but also in the new funding and programmes for Accountable Care Organisations, Medical Homes, Health Homes, Dual Eligibles (beneficiaries with both Medicaid and Medicare) and Community-Based Care Transitions. I am sure that similar opportunities can be found in many other countries.

First steps for the improvement science field

Achieving important aims will require vision, leadership and rapid learning. What could work? Those now focused on organisational management and improvement activities will need to learn to address the health of populations, since most clinical priorities require working across an array of delivery system boundaries in order to benefit a population efficiently. The population of interest may well mostly be local, since sick and disabled people tend to have to use services within reach. Of course, we must also discern how to forge the will that will cause leadership to care to benefit the community—and in the USA, to learn how to stretch the limits of community-focused action by multiple self-interested businesses with no local governance.

Organising for responsible advocacy

We also need to learn how to keep, at least those involved in quality improvement, abreast of initiatives arising from healthcare reforms and how to enable us to take action to inform and shape these endeavours. For example, the US Assistant Secretary for Planning and Evaluation has recently asked the public for suggestions as to how to build an inventory of comparative effectiveness research5; quality improvement insights need to show up in such an inventory. The US Internal Revenue Service recently asked for comment as to how non-profit hospitals should have to use their community benefit funds6; quality improvement interests should have put their point of view. The PPACA1 established a Patient-Centred Outcomes Research Board which is required to support a Methodology Committee on comparative effectiveness research; the insights from quality improvement activities need to be part of that work. The opportunities are legion, but people involved in improved management of healthcare services delivery have no mature organisations monitoring and coordinating responses to these opportunities.

Some older issues will take on renewed importance in the months ahead. The persistently ambiguous scope of application of the US regulations concerning protection of human subjects in research will need to be revisited and clarified. The mandates of the National Institutes of Health, which mostly restrict their funding to generalisable research on prevention and cure, may need to be examined and revised to allow their participation in developing methods to draw insights from context-sensitive improvement work. US antitrust law raises new challenges for initiatives that focus upon local populations and require broad cooperation to succeed. Even copyright and trademark issues have a new layer of complications when multiple parties working in the public interest generate an instrument or protocol worthy of marketing to other localities. Having scholars and managers ready to handle these issues would position the field of improvement science well.

Some may believe that these issues have less urgency now that Dr Don Berwick is head of the Centers for Medicare and Medicaid Services in the USA. Dr Berwick has been a motivating force and guiding light for quality improvement, and some will think that he can lead the changes we need. However, Dr Berwick cannot do much on his own. Advocates are going to need to be developing insights, political strategies and action plans that make it possible for him to lead. In any reasonably democratic society, leaders cannot be much ahead of the flock—so people concerned for system performance improvement and other methods are going to need to move ahead briskly in order make it safe enough for Dr Berwick and political leaders in Congress and state legislatures to capitalise on the opportunities, rather than retreating to established patterns that accept delay and resist reform.

People interested in evidence-based improvement can find various strategies to gain the experience to have an effective voice in the current health reform efforts, but we do need to undertake to do them quickly. The Academy for Healthcare Improvement could take on an informing and coordinating function, including attention to a number of countries where its membership is active. The Institute for Healthcare Improvement could expand its scope to help around the world with monitoring, brainstorming and building consensus. Individual leaders and allied professional and trade organisations could invest in skills and contacts needed to participate effectively.


The time is upon us, and it may be a long time returning if the current opportunities end up doing the same old things and replicating the same familiar shortcomings. In policy, as in life generally, the time-honoured aphorism holds: ‘If we keep doing what we've been doing, we'll keep getting what we've been getting.’ Now is a very auspicious time to start doing things differently with regard to developing a methodology sufficient to guide healthcare services delivery reform with confidence and efficiency.


The author acknowledges with gratitude the opportunity to develop these ideas that The Health Foundation in the UK created by having an international meeting on epistemology of quality improvement at Cliveden House in April 2010.



  • Competing interests None.

  • Provenance and peer review Not commissioned; externally peer reviewed.