Article Text


The effectiveness of quality systems in nursing homes: a review
  1. C Wagner1,
  2. G van der Wal2,
  3. P P Groenewegen1,
  4. D H de Bakker1
  1. 1NIVEL, Netherlands Institute of Health Services Research, P O Box 1568, 3500 BN Utrecht, The Netherlands
  2. 2EMGO, Institute for Research in Extramural Medicine, and Department of Social Medicine, Vrije Universiteit Amsterdam, 1081 BT Amsterdam, The Netherlands
  1. Dr C Wagner c.wagner{at}


Background—The need for quality improvement and increasing concern about the costs and appropriateness of health care has led to the implementation of quality systems in healthcare organisations. In addition, nursing homes have made significant investments in their development. The effects of the implementation of quality systems on health related outcomes are not yet clear.

Objective—To examine evidence in the literature on whether quality systems have an impact on the care process and the satisfaction and health outcomes of long term care residents.

Methods—Review of the literature.

Results—The 21 empirical studies identified concerned quality system activities such as the implementation of guidelines; providing feedback on outcomes; assessment of the needs of residents by means of care planning, internal audits and tuition; and an ombudsman for residents. Only four articles described controlled studies. The selected articles were grouped according to five focal areas of quality. The opinion of residents was seldom used to evaluate the effectiveness of quality systems. The effects on care processes and the health outcomes of long term care residents were inconsistent, but there was some evidence from the controlled studies that specific training and guidelines can influence the outcomes at the patient level.

Conclusions—The design of most of the studies meant that it was not possible to attribute the results entirely to the newly implemented quality system. As it is difficult in practice to design a randomised controlled study, future research into the effectiveness of quality systems should not only focus on selected correlates of quality, but should also include a qualitative and quantitative (multivariate and multilevel) approach. The methods used to measure quality need to be improved.

Statistics from

There is a need for improvement in the quality of long term care for the elderly provided in nursing homes.1–5 The increase in the number of elderly people and, especially, in those aged 80 years and over who generally have a greater level of disability secondary to multiple chronic diseases has resulted in a greater need for services and specialised care. A long term care facility or a nursing home is an institution which provides nursing care 24 hours a day, assistance with the activities of daily living and mobility, psychosocial and personal care, paramedical care, and also a room and board6 for people whose health has deteriorated to such an extent that they need constant nursing care.7 The aim of nursing home care is to maintain the limited physical, mental, and social capabilities of residents for as long as possible. To date, however, it is not clear which quality system activities should be used in nursing homes to assure or improve the quality of care. It is uncertain whether all processes should be standardised according to practice guidelines, whether peer review and clinical audit should be introduced in every department, and whether an integral quality system should be developed.

    Key messages

  • There is little scientific evidence that quality systems have an impact on the satisfaction and health outcomes of long term care residents.

  • Only four of 21 empirical studies identified were controlled.

  • Further research on the effectiveness of quality systems is needed and should include qualitative as well as quantitative methods.

The quality of care is the degree to which (a) nursing homes increase the likelihood of desired health outcomes for residents, and (b) the care process is consistent with current professional knowledge.8 Most quality system activities have been shown to be effective in some situations, yet no single activity is demonstrably superior in all, or even most, situations.9 Research on the effectiveness of quality systems has gradually become a new domain that needs to be further developed.10,11 This study is one of the first to examine evidence from the literature on whether quality systems or quality system activities have an impact on delivery of care, clinical health outcomes of long term care residents, and satisfaction of patients with delivery of care and health outcomes.

A quality system is broadly defined as “all the management activities explicitly designed to monitor, assess and improve the quality of care”. It is composed of different types of quality system activities that can be applied to certain focal areas.12,13 Quality system activities are, for example, peer review, practice guidelines, continuing education, and a quality policy. As a quality system is a means of improving the performance of an organisation, the focal areas are related to important organisational areas such as policy and strategy, people management, resources, and process management.13 Based on a national survey of a representative group of 1182 Dutch healthcare organisations, the relevant areas for quality systems in healthcare institutions are: process control based on standards, process improvement by implementing quality improvement procedures, human resources management, involvement of residents in quality system development, and quality assurance documents.14,15 The ultimate goal of healthcare organisations is optimisation of the health status of individuals and populations, so the “gold standard” for quality measurement will always be measures of health outcome.16 There has been little empirical evidence to date that nursing homes that introduce quality systems will achieve better health outcomes for residents. To evaluate these outcomes, quality indicators17,18 or tracers19 have been used to compare the quality of nursing homes. The question to be answered is which quality system activities have measurably improved delivery of care and resident related outcomes?



The search included all quality system activities in nursing homes but excluded research on the effectiveness of treatment interventions. The scope of the study was limited to publications in the English, Dutch and German languages. To answer the research question, only empirical studies of quality system activities in nursing homes were included. Furthermore, randomised controlled trials could not really be expected and it had been decided a priori that other research designs would also be included such as pre-test/post-test studies, time series, and post-test only studies. Studies which included no description of the impact of the quality system activity on process or outcome measures (resident satisfaction and health outcomes of residents) were excluded.


Literature pertaining to the effectiveness of quality systems in nursing homes was identified by several methods (table 1).

Table 1

Overview of methods of identification

The first searches in Medline, Cumulative Index to Nursing & Allied Health Literature (CINAHL) journal base, Cochrane Library controlled trials register, Cochrane EPOC reviews register, and the Netherlands Institute of Health Services Research (NIVEL) literature database for the years 1985 to the end of 1997 were based on a combination of the free text keywords “quality systems”, “quality assurance”, “quality improvement” or “quality control” and the free text keywords “long term care” or “nursing home”. In addition, specialist journals that could not been found in one of the above mentioned journal bases were searched by hand. Secondly, additional information was obtained from the references of identified articles and from colleagues (snowball method).

The selected literature was analysed by the authors on methodological characteristics (research design, sample size, quality management activity, and measurement instruments) and process and outcome measures resulting from implementation of the quality management. The selected articles were grouped by the authors according to the five focal areas of a quality system (table 2).15

Table 2

The five focal areas of a quality system in healthcare organisations



The first search resulted in 231 relevant publications but, after analysis, only six met the inclusion and exclusion criteria. The other reviewed articles were not selected for the following reasons: they only described the quality systems activity but not the effects (n=101), the setting appeared to be a hospital or home care agency (n=20), or the author(s) only expressed an opinion on quality improvement or long term care (n=99). Five references could not be traced. An additional 15 articles were identified from the references of articles and from colleagues, four of which were published shortly after the search period.

Of the 21 selected studies, two were based in the UK, one in Canada, and the remainder in the USA. Process and outcome measures were used to evaluate effectiveness and, in three studies, both measures had been used. Of the 13 studies which examined the effects on the process of care, as measured by the development of policies for care, all reported some improvements. All of the 11 studies which assessed resident outcome also reported some improvements. The results are shown in table 3 in which the controlled trials are indicated.

Table 3

Evidence of effectiveness of quality management and quality system activities in nursing homes


The research design and the data differed between the studies. Controlled studies described: (1) the effectiveness of specific educational programmes20,21; (2) the effectiveness of two different facility and resident assessment processes on the overall quality of care and the detection of problems22; and (3) the impact of working with quality assurance cycles.23

In 10 studies the researchers used a pre-test/post-test design with no control groups in which the interval between the pre-test and the post-test period was less than 1 year.5,24–32 There were two longitudinal studies in nursing homes with a follow up of several years,33,34 and four studies with a post-test design only.35–38 The number of participating nursing homes involved in the selected studies ranged from one to 268, with 13 studies of 1–16 nursing homes, two of 60 nursing homes, and six with more than 200 participating nursing homes.


The controlled trials were found in the focal areas “process improvement by quality improvement procedures” and “human resources management”. There were no studies in the “quality assurance documents” focal area. One study described the impact of a quality system.33 In the focal area “process control based on standards” five studies evaluated the effectiveness of the implementation of a new guideline—for example, for the prevention of pressure ulcers, the adequate use and reduction of indwelling catheters, and the reduction of psychotropic drug use—all of which reported a decline in the prevalence of adverse events.25,26,34,36,37

The studies related to the focal area “process improvement by quality improvement procedures” described three different activities: (1) feedback of information on patient related quality indicators, (2) a resident assessment instrument (RAI) to analyse the needs of residents and support the care planning process, and (3) clinical audits. Characteristically, all three activities made use of a quality cycle whereby caregivers assess, evaluate, and improve the quality of care when necessary. Information on the present situation was obtained and subsequently compared with the desired situation. If there was any discrepancy between the desired and the actual situation, changes were implemented. The results of one controlled study23 showed that there was a decrease in the occurrence of a number of adverse outcomes (constipation and hazardous mobility), but for other outcome measures (urinary incontinence and potential skin breakdown) the prevalence of adverse outcomes was the same as in the control group. Another controlled study22 showed that the assessment method helped to detect problems and that many process improvements would not occur without external review. In the other studies the authors admit that it is not entirely clear to what extent the results can be attributed to the intervention since there was no control group and various other changes were also made at the time. Three studies in this focal area made use of internal audits to improve the satisfaction and health status of residents. The effectiveness, for example, of the Continuous Assessment Review and Evaluation (CARE) scheme has been investigated in pilot studies.5,39 The audit process was applied to the procedures in nine mainly clinical fields—for example, decubitus, urine incontinence, drug consumption, and the autonomy of the resident. As a consequence of the audits, existing agreements and guidelines in institutions were improved or new ones developed. The improvement in processes was assessed on the basis of the number of institutions which had an explicit policy concerning the nine fields in question—for example, standard policies were made more comprehensive (missing components reduced from 38% to 19%).

The professionals in an institution determine the quality of the care that they provide, as well as the quality of their mutual collaboration. The improvement of expertise in the form of tuition, supervision of work, or peer review can have an influence on the quality of the care the residents receive. Of the 21 studies, four described and evaluated the implementation of training programmes for nurses and doctors. The object of the training was to reduce the number of protective measures,28 the prescription of psychoactive drugs (controlled studies),20,21 and the prevalence of incontinence in residents.24 The results of the two controlled studies show that, with the help of specific training, significant improvements can be achieved in comparison with the control group.

In many countries it has already been agreed that residents can play an important part in the execution and improvement of care. One study investigated whether an ombudsman for residents in nursing homes would lead to better outcomes and more compliance with standards.35 The researchers concluded that the presence of an ombudsman who can visit residents and resolve disputes on their behalf might improve the outcomes of care, but there were no indications of better compliance with standards. No control group was included in this study.


Systematic improvement in the quality of care provided for the elderly has been high on the agenda in the USA and Europe for a number of years. The research question addressed in this review was: which quality system activities have measurably improved delivery of care in nursing homes and resident related outcomes?

Over 200 publications were identified using keywords associated with quality assurance and long term care or nursing homes, but only 21 of these described the implementation of quality system activities and their effects on the quality of care provided for residents, and only four included a control group. From these studies it can be concluded that specific training, specific resident assessment procedures, and the use of quality assessment cycles with the assistance of a quality assessment consultant can be effective in improving the quality of care in specific aspects of the care process and certain health outcomes for residents. The results of the studies without a control group can only serve as an indication of what should be the subject of further research. The link between quality system activities at the process level and the effect on the health outcomes of residents has not yet been proved conclusively. In line with evidence-based medicine, it is important that more proof of the effectiveness of quality management is found in the near future or it might be debatable whether so much money should be spent on activities that have failed to demonstrate any improvement in the quality of care or quality of life of residents.

This considerable difference in the number of publications and the number of effect studies suggests that the effects of most initiatives are not reported or their effects are not evaluated. The fact that none of the identified studies reported predominantly negative results might imply that there is some publication bias. This means that valuable information and experience is not made available for other nursing homes, and it could therefore be beneficial to investigate and survey such unpublished activities. It is possible that, because of the search strategy, some studies evaluating quality system activities which do not mention “quality” in the title or the abstract have been missed.

In general, the studies which were identified concerned quality system activities such as the implementation of guidelines, providing feedback, assessment of the needs of residents by means of care planning, internal audits and tuition, and an ombudsman for residents. One study reported on the effects of a quality system. The opinion of the residents is apparently seldom used to evaluate the effectiveness of a quality system. In view of the high prevalence of demented people in nursing homes, this approach is difficult to implement. One possibility is to ask the opinion of relatives (proxies), but it is questionable whether relatives can really speak from the perspective of the residents. Participant observation of demented residents could therefore be an adequate though time consuming method of measuring satisfaction with the quality of care. However, there are studies which have measured “consumer satisfaction”,40 but not in relation to quality system activities.

A number of key issues emerge from critical appraisal of the selected studies. In most cases the design of the studies is such that it is not possible to attribute the results entirely to the newly implemented quality systems activity—for example, studies on the effects of the RAI.29–31 The results are often presented without detailed quantification. Moreover, the intervention is often not described in sufficient detail to allow an institution manager or carer to repeat it in order to achieve the same effects. There is also often no indication of which factors or elements were essential in achieving the (positive) effects.

It is concluded from the literature that, at present, there is no clear answer to the question as to which quality system activity should be used in nursing homes to improve the care provided for residents. From the controlled studies there is some proof that activities which are directly associated with the ability of the professional, such as training and guidelines, can influence the outcomes at patient level. It could be that carers in nursing homes lack training in the specific knowledge and abilities which are needed in caring for the increasing number of elderly residents who have more disabilities and greater comorbidity. Additional training and the use of guidelines can reduce the uncertainty of carers. In addition, there are indications that the RAI has some positive effects on the health outcomes of residents, but the study design does not allow further conclusions to be drawn. The RAI provides carers with a systematic approach that incorporates treatment suggestions and guidelines. In order to obtain more certainty about the effectiveness of quality system activities, future studies should at least include a pre-test/post-test design and a control group. The results show that, for some quality system activities, it is possible to set up a randomised controlled trial. It seems more difficult, in practice, to design a randomised controlled study which evaluates a more complex integral quality system. Future research into the effectiveness of quality systems could therefore also be based on a multivariate or multilevel approach—for example, to determine initially which resident related outcomes (corrected for case mix of residents) vary among institutions and subsequently to investigate in more detail what the differences between the institutions are with regard to structure and process characteristics. This would make it possible to identify influential organisational and environmental factors, or patterns of factors, and to determine which elements of a quality system are most effective and in which situations or circumstances. Another possibility is a combination of process and effect evaluation with randomisation of nursing homes instead of residents. Finally, it might also be necessary to evaluate quality systems with a combination of qualitative and quantitative research methods. Examples of qualitative methods include participant observations, site visits combined with interviews, or an accreditation procedure. The data gathered by these methods could be combined and analysed at a more conceptual level to help us understand the mechanisms between the organisation structure, the care process, and resident outcomes.


View Abstract

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles