Article Text

PDF

A model to evaluate quality and effectiveness of disease management
  1. K M M Lemmens1,
  2. A P Nieboer1,
  3. C P van Schayck2,
  4. J D Asin3,
  5. R Huijsman1
  1. 1
    Erasmus University Medical Centre, Institute of Health Policy and Management, Rotterdam, The Netherlands
  2. 2
    Department of General Practice, Maastricht University, Research Institute CAPHRI, Maastricht, The Netherlands
  3. 3
    ACSION, Advance Care Solutions and Insights for Optimization, Amstelveen, The Netherlands
  1. Ms K Lemmens, Erasmus University Medical Centre, Department of Health Policy and Management, Room L4-062, PO Box 1738, 3000 DR Rotterdam, The Netherlands; k.lemmens{at}erasmusmc.nl

Abstract

Disease management has emerged as a new strategy to enhance quality of care for patients suffering from chronic conditions, and to control healthcare costs. So far, however, the effects of this strategy remain unclear. Although current models define the concept of disease management, they do not provide a systematic development or an explanatory theory of how disease management affects the outcomes of care. The objective of this paper is to present a framework for valid evaluation of disease-management initiatives. The evaluation model is built on two pillars of disease management: patient-related and professional-directed interventions. The effectiveness of these interventions is thought to be affected by the organisational design of the healthcare system. Disease management requires a multifaceted approach; hence disease-management programme evaluations should focus on the effects of multiple interventions, namely patient-related, professional-directed and organisational interventions. The framework has been built upon the conceptualisation of these disease-management interventions. Analysis of the underlying mechanisms of these interventions revealed that learning and behavioural theories support the core assumptions of disease management. The evaluation model can be used to identify the components of disease-management programmes and the mechanisms behind them, making valid comparison feasible. In addition, this model links the programme interventions to indicators that can be used to evaluate the disease-management programme. Consistent use of this framework will enable comparisons among disease-management programmes and outcomes in evaluation research.

Statistics from Altmetric.com

Healthcare systems throughout the world increasingly place primary emphasis on chronic conditions. And yet, traditional healthcare delivery systems have not fully evolved to meet the needs of patients and populations with chronic diseases: care is fragmented, duplicated and directed at acute disease.1 Recognition of this shortcoming has nevertheless led to new strategies for care delivery,2 including the concept of disease management. Over the past two decades, these strategies emerged from the overlapping approaches of integrated care (United States) and shared care (Western Europe). In The Netherlands, for example, shared care was a precursor to disease management, which was introduced here in 1997.3 Disease management appears to be a broad concept, however, and has been operationalised in very different ways.47 It is this heterogeneity that makes it hard to determine possible effects on quality and effectiveness of chronic care, for it not only hinders our understanding of what disease management actually is but also hinders uniform evaluation of programmes and, consequently, comparison of study results.89 This paper focuses on the programme theory of disease-management evaluation, asking the question: “How does a comprehensive, integrated model for disease management enable evaluation of chronic care?”

A consistent definition is indispensable to determine the pillars of disease management. The Disease Management Association of America (DMAA) was the first to state a clear definition.10 This definition, however, overexposes self-management, seeing that the DMAA’s primary aim is providing direct care to the patient. We therefore slightly adapted it to the purpose of this paper: “a system of coordinated healthcare interventions and communications for populations with chronic health conditions.” Disease management has three key features. First, it supports the professional/patient relationship and plan of care. Second, it uses evidence-based practice guidelines and patient empowerment strategies to prevent exacerbations and complications. Third, it continuously evaluates clinical, humanistic and economic outcomes so as to improve overall health.10 It would seem to follow that disease management involves the implementation and integration of various components to form a system of collaborative management that may accomplish better chronic care. It is therefore essential to define these pillars or core components and to understand how they operate and interact, since these are the mechanisms that actually make disease-management work.

The relevant literature proposes several components of disease management.451013 The categorisation of components and the delineation of specific interventions will help to provide a framework to develop and evaluate disease-management programmes.2 Such a framework could also help to define specific indicators for evaluation, dependent on the components of a disease-management programme.8 What this article therefore adds to the existing body of knowledge is a deeper understanding of the underlying mechanisms of disease management and a framework to guide development of proper indicators to evaluate disease-management programmes.

PILLARS OF DISEASE MANAGEMENT

There are two basic types of disease-management programmes: those based on primary care and integrated within organisation of physician practice (like the Chronic Care Model (CCM)), and commercial vendors to which employers and health plans may outsource their disease-management functions (carve-out).1416 By and large, these models share the same objectives, yet they largely differ in the processes applied to achieve the objectives. A major difference lies in integration of the provided services with patients’ other medical care.16 Carve-out programmes are not integrated with primary care; rather, they are for-profit ventures, and are marketed primarily as a cost-containment strategy.15 In contrast, primary-care-based programmes like the CCM attempt to enhance communication with patients while trying to change the organisation of physician practice.17 And even though the DMAA, the voice of commercial (carve-out) programmes, defines disease management as supporting the practitioner/patient relationship, this is not represented in its programmes. Therefore, the DMAA and CCM programmes do not really differ in the components defined but rather differ in the way these components are organised.18 The CCM’s focus on redesigning practices has the potential to improve care broadly.17 The CCM1922 thus provides the basis for the definitions of disease-management components. The CCM incorporates self-management interventions, decision support, delivery system design and clinical information systems, as well as relationships of the healthcare delivery system with the larger healthcare organisation and community. The interventions proposed should result in a well-prepared, pro-active practice team that interacts with an informed, activated patient. This interaction, in its turn, should improve outcomes.

From a review of disease-management components, we identified two key elements: patient-related and professional-directed interventions. These are strong pillars, seeing that effective control of most chronic diseases requires appropriate medical therapy as well as competent self-management.2325 These interventions are directed at equipping both the patient and the professional in managing the disease. Patient-related and professional-directed interventions are not possible, however, unless supported by the delivery system; they occur within the broader context of the social, political and physical environment.26 The interventions should therefore be supported by a third element, the organisational design of the healthcare system at different levels: delivery system design, healthcare organisation and community. It can be concluded that the concept of disease management supports the use of multiple interventions.

DEVELOPMENT OF THE EVALUATION MODEL

The programme theory defines the presenting problem and the target population for whom the programme is designed (input), specifies the causal processes underlying the programme effects (process) and identifies its expected outcomes (outcome) as well as factors that affect treatment processes.2728 The evaluation model, as derived from the programme theory, incorporates variables that reflect theoretical concepts and implementation issues addressed in disease-management evaluation. In the following sections, we will conceptualise the disease-management processes to indicate the mechanisms responsible for producing favourable, intended outcomes. In the programme theory, outcomes that are responsive to the evaluated processes of care are selected. These outcomes—clinical outcomes, quality of life, resource utilization and patient satisfaction—reflect the primary goals of disease management, to promote, maintain and enhance a patient’s health state.

Patient-related interventions

Patients with chronic conditions make decisions and engage in behaviours that affect their health. It would seem crucial, therefore, to involve them in their own care processes.1213 Patients who are empowered and effective in self-management are better prepared to positively influence disease control and health outcomes.2930 The continuum of self-management programmes ranges from the provision of written material only to more intensive programmes such as the Chronic Disease Self-Management Programme designed to enhance self-efficacy.31 Still, the question remains what mechanisms will make patient-related interventions really work.

Various meta-analyses, review articles, randomised trials and observational studies provide substantial evidence that programmes providing counselling, education, information and feedback to patients with common chronic conditions are associated with improved outcomes.11233237 In their meta-analysis of several disease-management components within various chronic diseases, Weingarten and colleagues38 found that programmes using patient strategies produced a slight but significant improvement in disease control. On the other hand, not all self-management education programmes for all diseases are effective, and possible publication bias is also evident.373941 There is still much to be learnt about the specifics of self-management programmes in chronic care. Nevertheless, some empirical evidence is available. For one, knowledge and skills are necessary but not sufficient conditions to bring about change in behaviour.4243 Also, education should have more bearing upon behaviour and health outcomes.44 Actively involving patients and integrating competencies in everyday life appear to be more effective, however, than didactic education.374445 Moreover, collaborative decision-making and goal setting appear to be crucial: self-management improves when a patient’s opinions and values are taken note of.4647 Finally, self-efficacy, the confidence that one can behave in a certain way to reach a desired goal, is associated with collaborative decision-making.4849

Relating these empirical findings to the theories on behavioural change can guide the programme theory. Linden and colleagues50 provided an overview of behavioural change models that intend to change patient self-management, much of which builds on the Behavioral Model developed by Andersen.51 Most of these models, however, are concerned only with predicting health habits but do not tell how to change health behaviour. Social learning theory offers both predictors and principles of behaviour change.5253 Social cognitive learning theory emphasises the interactions between patients’ cognitions, on the one hand, and their behaviour, on the other, through processes such as self-efficacy and outcome expectancies.5354 Belief in self-efficacy as a means to exercise control is a common pathway through which psychosocial influences affect health functioning. Knowledge of health risks and benefits creates the precondition for change. Those who are unaware of how lifestyle habits affect health have little reason to change. Stronger self-efficacy is bound to positively influence health behaviour and eventually health status, as borne out by the findings of many relevant studies.5255 As a result, the effects of a self-management programme on health status and healthcare utilization are hypothesised to result from behavioural change, which in turn is caused by enhanced self-efficacy, knowledge and skills.43535559

Professional-directed interventions

Effective chronic disease-management demands that professionals have a rich knowledge base.460 One of the knowledge elements is a set of clinical guidelines. These are systematically developed statements to assist practitioner decisions on appropriate healthcare for specific clinical circumstances.61 However, guidelines alone do not suffice. Continuing medical education, audit, feedback, reminders and critical pathways also represent a major thrust of these efforts.62

In the past decade, several meta-analyses have looked into the effects of guidelines6366 and other professional-directed interventions, such as education and feedback.6770 Improvement in healthcare outcomes was no more than modest, however, mainly as a result of implementation problems. The interventions that produced modest improvements tended to be multifaceted, with actively involved professionals.647173 Results suggest that effective strategies include educational outreach and reminders. However, conclusions about what works are still tentative.74

A number of techniques have been used to modify the behaviour of practising physicians. However, the relative effectiveness of each technique is largely dependent on the particular strategy employed in the implementation. As a result, there is a need to understand the underlying mechanisms strengthening physicians’ behaviour. Individual professionals need to be informed, motivated and trained to incorporate the latest evidence into their daily work. A review identified factors associated with physician barriers to change;75 all factors appeared to be associated with knowledge, attitude (behavioural intention) and behaviour.7677 Before professional-directed interventions can affect patient outcomes, they must first affect physicians’ knowledge, then behavioural intention and finally behaviour.75 Although behaviour can be modified without knowledge and intention being affected, behaviour change based on influencing knowledge and intentions is probably more sustainable than indirect manipulation of behaviour alone.75 As explained below, several theories also suggest similar constructs.

A variety of forces influence normative patterns of practice behaviour for the practising physician. Therefore, researchers have been unable to formulate a unifying theory of physician behavioural change, applicable and successfully proven among physicians in practice.78 However, social and behavioural science theories can further our understanding of the interplay of factors contributing to improve physician performance. Grol65 developed a general framework that integrates theoretical approaches and applications for changing clinical practice of physicians. Similarly, a number of researchers647980 have provided overviews of behavioural, social and educational theories underlying different approaches to implementing professional-directed interventions. We must conclude, however, that no single theory stands out as the most effective one; various theories may be valid and effective.65 In spite of this, a principal rationale underlying professional-directed interventions has been identified, based on various learning theories (adult learning, behavioural learning and social learning theory). It appears to combine intrinsic motivation, rational information seeking and decision-making, controlling performance by external stimuli, and social influence of significant peers or role models. For example, reminders are approaches that seek to control physicians’ performance by external stimuli. They stem from behavioural and learning theory.81 These theories suggest that professional-directed interventions will be effective in modifying physician behaviour by governing professional behavioural intention, a process which in its turn is influenced by knowledge and skills. Despite adequate knowledge and behavioural intention, organisational and contextual factors can affect the physician’s ability to execute recommended care.658283 The functioning of collaborative teams or broader organisational structures could therefore offer important explanatory mechanisms for quality improvement.84

The current empirical and theoretical knowledge leads to the following hypotheses. The common goal of professional-directed interventions is to increase knowledge and expertise necessary to care for patients. If professionals are better educated and are supported in decision-making, knowledge and expertise will improve. The necessary expertise, information, time and resources will affect behavioural intention (attitude), which leads to professional behavioural change. Eventually this should lead to improved health effects.8591

Interaction between the patient and the professional

In the previous sections on patient-related and professional-directed interventions, we hypothesised that these interventions lead to positive health effects. The hypothesis, however, concerned effects coming from single interventions. The current literature on disease management supports the assumption that combining these interventions should have more effect on health outcome and resource utilization.435992 Glasgow and colleagues44 emphasise that notwithstanding helpful recommendations, advice and counselling from healthcare professionals, patients themselves must decide which self-care actions to put into practice. Collaborative management is put in place when these two perspectives are combined, leading to shared goals, a sustained working relationship, and mutual understanding of roles and responsibilities.93 When informed patients take an active role in managing their health and professionals feel prepared and supported with time and resources, their interaction is likely to be much more productive.94

In the light of the empirical evidence about behavioural principles, Von Korff et al59 identified four critical elements that enhance collaborative management: (1) a collaborate definition of problems; (2) targeting, goal setting and planning; (3) creation of a continuum of self-management training and support services; and (4) active, sustained follow-up. Healthcare providers and patients have a better chance of seeing improvements in adherence and outcomes if they focus on a specific problem, establish realistic goals and develop an action plan for attaining those goals.119597 This supports the essence of the defined pillars of disease management; high-quality chronic illness care is characterised by productive interactions between professionals and patients.25 Enhancing communication between patient and professional is likely to result in more coordinated care. The evidence supporting sustained follow-up shows that patients benefit from knowing that they will have ongoing planned contact with healthcare providers in the absence of a medical crisis.3498 Professionals, on the other hand, benefit from obtaining information on their patients’ medical and functional status.

Healthcare organisation

Effective chronic disease management will often require changes in the delivery system.99 Patients with chronic conditions in particular will benefit most from coherence in care delivery and cooperation between care professionals.100 Theories of integrated care stress the radical or gradual redesign of the steps in providing care, focusing on improved organising and managing the care of specific categories of patients to more readily meet their needs.83 Effective chronic care programmes often include organising new collaborations of care providers, allocating tasks differently, transferring information more effectively, scheduling appointments and contacts more efficiently, case management, and using new types of health professionals.7101102 These concepts of organisational (re)design act as coordinating mechanisms,103 which in turn can be described by the organisational design104 and relational coordination frameworks.105 These coordinating mechanisms are all pitched under the umbrella of organisation of healthcare in Wagner’s model.103106 For example, delivery system re-design could reflect organisational design when telemedicine for rural patients is implemented or relational coordination when nurse educator is included in a planned COPD visit. Delivery system support and clinical information systems are described as separate components in the CCM.25 In the programme theory presented here, these are both considered organisational interventions supportive of the patient–professional interaction.

A clear description of the programme organisation and articulation of the underlying assumptions reveal many aspects that determine how effectively the programme performs its various functions. In addition, the organisational context of these interventions should be clearly described, since certain interventions are effective only in specific settings.6567 There is a growing evidence base of rigorous evaluations of organisational strategies. But the evidence underlying some strategies is limited, and the effects cannot be predicted with great certainty for a strategy.84107 However, a recent meta-regression analysis by Shojania and colleagues108 showed that team changes and case management produced more robust improvements in diabetes care. Evidence also supports benefits of case management for depression.109 Healthcare is also placed in a larger community context, and this should be taken into account when examining the organisational context. For example, a particular community’s healthcare systems and socio-demographics have been found to influence health outcomes.

Programme implementation

The implementation of quality improvement interventions such as disease management often appears to be a difficult affair. To understand why some interventions are successful whereas others fail to change practice, it is necessary to gain insight into the “black box,” that is the sequence of events through which targets contact the intervention.7 The programme must be organised such that the services intended to produce the desired outcome can and actually will be provided.110 The framework of assumptions and expectations about how to reach the target population is called the implementation plan. This plan lies at the basis of evaluation of the implementation process. Process evaluation checks whether the planned improvement activities have indeed been executed uniformly and whether the target population has actually been exposed to these activities as planned.7 Too often, however, interventions are evaluated without prior data regarding basic processes expected to mediate the target improvements.111 This practice is like conducting a drug trial lacking information on whether the patients actually took the medication.74 Implementation error is to blame if the intervention in its ultimate form differs considerably from the original plan.112

DISEASE-MANAGEMENT EVALUATION FRAMEWORK

The evaluation framework presented here is directed at the most aspects of the disease-management programme as described above, and also provides some indication of the level of intended performance (fig 1). The model provides a heuristic for understanding how interventions within the context of disease management affect outcomes of care. Two mechanisms are taken to be responsible for the effects of disease management on the final outcomes of care: (1) the patient’s learning and behavioural change mechanism; and (2) the professional support and professional behaviour change mechanism. Social learning theories clarify these mechanisms. The patient–professional partnership paradigm implies that while professionals are experts about diseases, patients are experts about their own lives.11 Learning is essential for being an expert; it gives professionals more knowledge and expertise, and it allows patients to identify their problems and cope with their conditions. As we have seen, knowledge, skills and expertise form the first step in patient-related and professional-directed interventions. For patients, the provision of knowledge should lead to enhanced self-efficacy, which in turn influences health behaviour and eventually health status. For professionals, access to decision support should affect professional behavioural intention, which in turn influences professional behaviour and eventually leads to improved health outcomes. Education alone, however, was found to have limited influence on physician behaviour;113 disease-management programmes should therefore incorporate more rigorous educational strategies. Approaches of learning that actively involve participants and focus on behaviour change seem most suitable for both professionals and patients. The model also shows that organisational support may facilitate learning; moreover it can act as a catalyst to behaviour change—for example in the case of a system that generates regular feedback for clinical teams on guideline compliance from registry data. The organisational interventions should thus be clearly described.

Figure 1 Evaluation model for disease-management programmes.

As we have seen, organisational structure and process variables are also crucial in comparing disease-management programmes. The evaluation framework provides the starting-point for the definition of structure, process as well as outcome indicators.114 Structure indicators can be based on healthcare organisation. Process indicators evaluate whether the implementation plan has indeed been executed accordingly.7 This information should be integrated with findings of the outcome evaluation. Consequently, the model suggests how to measure the various elements of the organisation of chronic disease management and how to resolve traditional controversies surrounding the evaluation of disease management. A detailed example is given in box 1.

Box 1 Example: evaluating a disease-management programme on chronic obstructive pulmonary disease

In a Dutch nationwide disease-management programme on chronic obstructive pulmonary disease (COPD), regional projects to improve care were supported by a multifaceted and multidisciplinary approach.

Input

The programme targets COPD patients diagnosed according to the guidelines. Patient characteristics such as age, sex, smoking habit, lung function and dyspnoea were gathered to address the question which patients, presenting with which characteristics, would benefit from the COPD programme.

Process

The approach comprised three facets: One, a patient-related intervention designed to promote increased understanding of the condition and teach specific prevention or treatment strategies (self-management) at every contact. Information booklets were also provided. Second, a professional-directed intervention designed to educate professionals on the guidelines and project. Continuing medical education meetings requiring active involvement were organised for GPs and practice assistants. Third, an organisational intervention in the shape of case management supported the two other interventions. A multidisciplinary team developed a system for coordinating diagnostic procedures, treatment, and ongoing patient management (arrangement for referrals and follow-up). The team consisted of a general practitioner, physician-assistant and a lung specialist, whose tasks and responsibilities were described in a protocol based on the guidelines. The physician-assistant independently carried out diagnostic and therapeutic activities, and taught patients on self-management.

  • Implementation: To check actual exposure to the disease-management programme, process indicators, linked to the specific interventions, were defined: adherence to evidence-based guidelines (an adherence score on four main guideline aspects) (professional), percentage of patients receiving information and instruction (patient), and percentages of planned routine visits and referral behaviour (organisational).

  • Mediating processes: The processes represent the series of changes for both patients and professionals occurring after receiving the interventions within the organisational setting. Therefore, intermediate outcomes were also defined. In the case of patients, these intermediate outcomes should reflect behavioural change as a result of enhanced self-efficacy, knowledge and skills, seeing that this is essential to achieve the intended effect of the COPD disease-management programme. To this aim, subjective and objective increase in knowledge on COPD, self-efficacy and self-care behaviour (medication and lifestyle) were measured. Likewise, intermediate outcomes were established for professional behaviour, behavioural intention, knowledge and skills.

  • Healthcare organisation: Information on specific characteristics of the programme organisation and organisational context, such as available resources and regional structure, was collected.

Outcome

The ultimate outcome of the disease-management programme is improved quality of life. In addition, positive effects were expected on health status, patient satisfaction, severity of dyspnoea, and direct costs.

DISCUSSION AND CONCLUSION

Disease management is assumed to be a systematic and comprehensive approach to improve the management of chronic healthcare. Regrettably, the supportive evidence base is not yet very powerful. As mentioned earlier, there is a need for a sound analytical method of evaluating the outcomes of disease management.115 The evaluation model introduced here provides a good starting-point. It puts the interaction between the patient and the professional first, and thus the interventions directed at these two key players. It follows that evaluation should focus on the combined effects of these interventions. In addition, the programme theory shows that organisational support is indispensable. This theoretical model supports the premise that the use of supplementary interventions strengthens the effects of disease management.

We feel that the current models on disease management, such as the Chronic Care Model (CCM), do not clarify the underlying mechanisms of the components of disease management. They rather focus on the definition and components of disease management and their empirical evidence. The evaluation model provided in this article indeed elucidates these mechanisms: on the one hand, the patient’s learning and behavioural change mechanism; on the other hand, the professional support and professional behaviour change mechanism. Having identified these mechanisms, which are nested in social learning theories, facilitated the construction of an evaluation framework. A valid set of indicators for the disease-management programme can be determined by linking indicators to the elements of this framework.

The evaluation model can be used to characterise a wide range of disease-management programmes. Placing a programme in the model can disentangle its components, thus allowing comparison between different programmes on comparable indicators. Consistent use of this framework will therefore enable valid comparisons of efficiency and outcomes of disease-management programmes. In addition, our framework corresponds to the three defined domains of care coordination programmes: baseline assessment of the specific patient care situation (input), coordination mechanisms (process) and outcomes of care (outcomes).103

This multifaceted perspective on disease management helps to frame the consequences of chronic disease (eg, COPD) in a theoretically fruitful model that may have important research and policy implications. An important research implication lies in the field of methodology. For example, as disease management is typically integrated with the organisational context, a randomised controlled trial (RCT) design will not always be applicable. The methodology therefore must be determined by the degree of control over the delivery of the intervention (real world feasibility) and the most rigorous research design possible. Quasi-experimental research designs or mixed methods are often more suitable. A major policy implication is the call for a consistent definition and framework for disease management. In this era plagued by ever-tightening healthcare resources, it is of utmost important to identify interventions that are of added value to quality and efficiency of care.

Acknowledgments

This article has been supported by a grant from PICASSO for COPD, an initiative of Pfizer BV, Boehringer Ingelheim BV and Caphri (Care and Public Health Research Institute) of Maastricht University.

REFERENCES

View Abstract

Footnotes

  • Competing interests: None.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.