Article Text

Systematic review of studies of quality of clinical care in general practice in the UK, Australia and New Zealand
  1. M E Seddon, head of quality1,
  2. M N Marshall, professor of general practice 2,
  3. S M Campbell, research fellow 2,
  4. M O Roland, director 2
  1. 1Department of Medicine, Middlemore Hospital, Auckland, New Zealand
  2. 2National Primary Care Research and Development Centre, University of Manchester, Manchester M13 PL, UK
  1. Dr M N Marshall martin.marshall{at}man.ac.uk

Abstract

Objectives—Little is known about the quality of clinical care provided outside the hospital sector, despite the increasingly important role of clinical generalists working in primary care. In this study we aimed to summarise published evaluations of the quality of clinical care provided in general practice in the UK, Australia, and New Zealand.

Design—A systematic review of published studies assessing the quality of clinical care in general practice for the period 1995–9.

Setting—General practice based care in the UK, Australia, and New Zealand.

Main outcome measures—Study design, sampling strategy and size, clinical conditions studied, quality of care attained for each condition (compared with explicit or implicit standards for the process of care), and country of origin for each study.

Results—Ninety papers fulfilled the entry criteria for the review, 80 from the UK, six from Australia, and four from New Zealand. Two thirds of the studies assessed care in self-selected practices and 20% of the studies were based in single practices. The majority (85.5%) examined the quality of care provided for chronic conditions including cardiovascular disease (22%), hypertension (14%), diabetes (14%), and asthma (13%). A further 12% and 2% examined preventive care and acute conditions, respectively. In almost all studies the processes of care did not attain the standards set out in national guidelines or those set by the researchers themselves. For example, in the highest achieving practices 49% of diabetic patients had had their fundii examined in the previous year and 47% of eligible patients had been prescribed beta blockers after an acute myocardial infarction.

Conclusions—This study adopts an overview of the magnitude and the nature of clinical quality problems in general practice in three countries. Most of the studies in the systematic review come from the UK and the small number of papers from Australia and New Zealand make it more difficult to draw conclusions about the quality of care in these two countries. The review helps to identify deficiencies in the research, clinical and policy agendas in a part of the health care system where quality of care has been largely ignored to date. Further work is required to evaluate the quality of clinical care in a representative sample of the population, to identify the reasons for substandard care, and to test strategies to improve the clinical care provided in general practice.

  • quality of care
  • clinical effectiveness
  • technical effectiveness
  • general practice
  • systematic review

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Key messages

  • Most of the care for common and chronic conditions is provided in general practice. This paper describes a systematic overview of the quality of that care.

  • The published research in the field presents an incomplete picture of the quality of clinical care in terms of its methodological rigor and comprehensiveness.

  • In almost all studies reviewed the quality of care did not attain acceptable standards of practice.

  • Attention should be paid by practitioners and policy makers to develop systematic ways of improving the quality of clinical care in general practice.

Improving the quality of medical care has become a major issue for all healthcare systems.12 This is in part a response to increasing evidence that the quality of care is suboptimal in terms of the standards attained, the degree of variability, and the level of accountability of health professionals.13 It also reflects an increasingly systematic and structured approach to the reform and development of health systems. The UK, for example, is currently implementing a 10 year plan of modernisation of the National Health Service4 and in the USA federal action is being taken to reduce the incidence and impact of medical errors.56

Until recently, evidence of deficiencies in quality have come largely from the hospital sector,78 encouraged by the ready availability of hospital performance data, the high cost of hospital care, and public interest in high technology medicine.9 Most health problems, however, are presented to and managed in primary care, and the quality agenda is starting to focus on this large heterogeneous area of healthcare delivery.10 Anecdotal reports of deficiencies and variation in the quality of primary health care abound, but to our knowledge there has been no previous systematic attempt to review published evaluations of the quality of clinical care and to summarise the evidence of the standards of care attained. Such an overview is an important component of a systematic approach to quality improvement.1112

The concept of quality in general practice has several components and these can be viewed from either an individual or population perspective (box 1).13 Firstly, patients or users need to be able to get to (access) a range of services; secondly, these services should be provided in a professionally competent (clinical or technical effectiveness) and humane way (interpersonal effectiveness); thirdly, all individuals and groups within a population should get a fair deal (equity); and fourthly, society should get value for money (efficiency).

Box 1. Dimensions of quality in general practice (adapted from Campbell et al13)

Components of quality for individual patients:

  • Access

  • Clinical effectiveness

  • Interpersonal effectiveness

Additional components of quality for populations:

  • Equity

  • Efficiency

All of these components are important, although the relative weight given to them is likely to vary between individuals or groups and they will be prioritised in different ways at different times. The purpose of this study was to evaluate just one of these components of quality—that of clinical or technical effectiveness. This paper attempts to present an overview of clinical care by describing a systematic review of published studies reporting on the quality of clinical care in three countries with relatively similar systems of general practice based primary care—UK, Australia, and New Zealand.

Methods

ELIGIBILITY OF STUDIES

Studies were included if they attempted to measure the quality of technical processes of care provided in general practice in the three countries. We specifically excluded studies of interpersonal care or those that only assessed structural aspects of care. We also excluded surveys of self-reported behaviour and studies without a clear denominator such as critical incident analyses and case reports. In most cases we only included studies that assessed the care provided against standards described explicitly by the authors of the papers. These standards were usually derived from external guidelines and most were evidence based. For some studies we accepted implicit standards—for example, for the excision of skin malignancies where it could be safely assumed that 100% complete excision was the standard. Only those studies that described deficiencies of care that could be held to be the responsibility of the general practitioner or the practice team were included. For example, in a study of warfarin use in atrial fibrillation we excluded those patients where the atrial fibrillation was only discovered as part of the screening procedures for the study14 and not previously known to the general practitioner.

We extracted data from three types of study: descriptive studies that specifically aimed to assess quality of care, evaluations of audit programmes from which we used pre-feedback data, and intervention studies from which we used baseline or control data. Where possible we obtained additional information from study authors to clarify published results (where conducted, this is highlighted in the table on the QHC website).

SEARCH STRATEGY

Three electronic databases (Medline, Embase and HealthStar) were searched by MS using the following Mesh headings: Family Practice or General Practice, and Quality of Health Care, Audit, Clinical Competence, Guideline Adherence, Healthcare Peer Review, Healthcare Quality Indicators, Total Quality Management, Healthcare Quality Assurance, Quality Control or Continuous Quality Improvement. The search was limited to English language journals and to the period from January 1995 to September 1999.

The titles and abstracts of all papers identified by the electronic search were inspected by MS. Papers were discarded which clearly failed to satisfy the inclusion criteria for the study—for example studies conducted in countries other than the UK, Australia or New Zealand, those studies evaluating components of quality other than the technical processes of clinical care, or those using critical incident or case study methodologies. The reliability of this screening procedure was assessed by MM who independently reviewed the titles and abstracts of a random sample of one in 10 of the electronically extracted references. There were no differences in the references selected, suggesting that the search strategy was reliable.

We then searched the reference lists of the retrieved articles and hand searched the main journals with family practice content from each of the three countries for the period 1995–9. This included the following journals: Audit Trends, Australian Family Physician, Australian Medical Journal, British Medical Journal, British Journal of General Practice, European Journal of General Practice, Family Practice, Lancet, New Zealand Medical Journal, and Quality in Health Care.

OUTCOME MEASURES AND DATA EXTRACTION

The following data were extracted from each paper: country, conditions studied, study design, description of the sample (both patients and general practitioners), date of data collection, criteria measured, standards used to judge quality, and the results. For each paper data were extracted independently by two of the authors (MS and one other author for each paper). Any disagreements were resolved by one to one discussions and by telephone conferences between the four authors.

QUALITY ASSESSMENT OF STUDIES

The selected studies were heterogeneous in terms of their design and scientific rigour. It was therefore inappropriate to aggregate or conduct a detailed quantitative analysis of the data. Instead, we assessed the studies using the following criteria:

  • Study design: simple descriptive; pre-feedback data from audit studies; control data from experimental studies.

  • Sampling strategy: all practices in the study locality; self-selected or volunteer practices; randomly sampled practices; sampling strategy not described.

  • Response rate of practices: percentage of practices approached that agreed to participate in study where random sampling methods were used.

  • Sample size: total number of patient records assessed in the study.

The above criteria were applied in a structured way to each of the selected studies but we did not use a formal scoring method as there is no agreed or validated instrument for such a purpose.15 As for the data extraction, the quality assessment of the study methodology was performed independently by two authors. The quality criteria were found to be easy to apply to the papers and no differences were found between assessors.

The following results describe the total number of studies identified at each of the stages of the search strategy, the distribution of the studies between countries of origin and condition, the methodological quality of the studies, and a summary of the quality of clinical care as described by the study authors. The full results are provided in a table available on the QHC website.

Results

The electronic search produced 5707 references and inspection of the titles and abstracts of these references reduced the number to 81 studies. A search of the references cited in these studies and a hand search of the main journals with family practice content led to the identification of an additional nine papers. A total of 90 studies were therefore included in the final review.

Most of the studies were from the UK (80), with six from Australia and four from New Zealand. The range of conditions studied is summarised in table 1. Of the 90 papers, 77 (85.5%) assessed the quality of care provided for chronic conditions including cardiovascular disease (n=17), hypertension (n=11), diabetes (n=11), and asthma (n=10). In addition, 11 (12%) studied the quality of preventive care and two (2%) studied acute conditions.

Table 1

Composition of papers by condition studied (total papers = 90)

The studies were highly variable in their scope and scientific rigour (table 2). Sixty of the studies (66.7%) assessed care in self-selected practices and 18 (20%) were based in single practices. In those practices where participation rates were recorded, the rate of participation was less than 75% in more than half (n=21) of all relevant studies.

Table 2

Quality assessment of studies

The most rigorous studies randomly sampled both the practices involved and the records to be reviewed.1627 Two papers stand out as examples of rigorously designed studies: the first20 studied epilepsy management in 31 randomly selected practices (86% response rate) in which all records were reviewed, and the second24 audited hypertensive care and again randomly sampled both general practitioners (n=58) and medical records, the data being extracted independently by two researchers. The least rigorous studies used self-selected practices and reviewed a non-random sample of the medical records.2831 Most of these studies failed to specify the response rate of the practices, the method of data extraction, or the percentage of records lost to follow up.

In almost all studies the processes of care did not attain the standards set out in national guidelines or those set by the researchers themselves. For example, in the highest achieving practices 49% of diabetic patients had had their fundii examined in the previous year and 47% of eligible patients had been prescribed beta blockers after an acute myocardial infarction. In the lowest achieving practices 3% of diabetic patients had had their feet examined in the previous year and 31% of treated hypertensive patients had their blood pressure controlled. We have summarised the key findings for the most common conditions in table 3.

Table 3

Summary of results for the four most common conditions identified in the review

Discussion

We believe that this is the first study to take a structured and systematic approach to assessing the quality of clinical care in general practice. The review shows that many of the published quality assessments are methodologically weak and therefore only tentative statements about the overall quality of care provided in general practice can be made. Nevertheless, despite the methodological limitations, it would appear from this study that clinical care in general practice consistently and sometimes dramatically fails to meet expected standards for all of the conditions studied. Most of the studies reported on quality of care for chronic conditions, and only a small number attempted to assess the management of acute conditions or preventive care. This is a significant gap, given that these modalities represent a major part of the work undertaken in general practice.32

There are limitations relating to the conduct and interpretation of a systematic review of this nature. Firstly, the heterogeneity of the studies limits aggregation of the data. If aggregation had been possible, a clearer picture of the standards across practices might have emerged. However, any aggregation of data across different countries would have had to take into account fundamental (and desirable) differences in the context and culture of the different countries. For example, the priority given to different conditions by service users or health professionals might vary or the scientific evidence might be interpreted in different ways.

Secondly, we have assessed quality for one very discrete and measurable part of what is a complex and integrated discipline; in particular, we have excluded from our assessment interpersonal communication, structural aspects of care, organisational culture, teamwork, and access. Thirdly, while the conditions listed in table 1 include those where primary care can potentially have a substantial impact on health,33 they represent only about 10% of all consultations in general practice. It would therefore be inappropriate to make global judgements about care in general practice from these data. Fourthly, the practicality and appropriateness of the standards set by the authors are open to debate and it is important to recognise that it is not always feasible or desirable to adhere tightly to a clinical guideline.3435 Fifthly, the studies may not be a true representation of the quality of care provided in general practice—for example, the sampling of volunteer practices could lead to an overestimate and publication bias to an underestimate of the actual quality of care provided. Finally, we are aware that quality might change over time. We chose to restrict our search to the last 5 years on the basis that a longer period would not reflect recent changes in care delivery. Nevertheless, some of the results are based on data that are several years older than this.

We decided to conduct the review in three countries which have similar systems, values, and expectations of general practice but contrasting approaches to quality improvement. Other comparisons would have been of interest—for example, with the Netherlands which has a very similar system of general practice to the UK. Likewise, we could have made comparisons with countries which have different health systems such as Germany or the USA. However, for these countries it would have been difficult to attribute aspects of quality specifically to primary care.

The paucity of studies from Australia and New Zealand investigating the quality of clinical primary care and the resulting imbalance in favour of the UK make it difficult to draw comparisons between the countries. The fact that most of the papers came from the UK could be explained in a number of ways. Firstly, in the last decade there has been greater investment and encouragement of audit and research activity in UK general practice than in Australia or New Zealand. This might explain the large number of publications critiquing the results. Secondly, it is easier to conduct population based studies in the UK. This is a consequence of the system of patient registration and payment by capitation of British general practitioners which clearly identifies a denominator for such studies and facilitates a practice or population based approach to clinical care and quality assessment. In contrast, in New Zealand most patients are not currently registered with a named general practitioner and the majority of general practitioners are paid by fee for service. Thirdly, the UK government has set specific targets for quality improvement as part of a national public health policy.36 This has stimulated an interest in collecting data to identify a baseline and to monitor changes in quality. Both the collection of data and the quality of the data are therefore being driven by government policy. Neither Australia nor New Zealand have such a clearly defined government policy on quality improvement in general practice.

Policy implications

The findings of this review have significant implications for the quality of care agenda in general practice. We believe that this study, which adopts a broad view of the magnitude and nature of studies of quality problems in general practice, creates an important perspective that a single condition based assessment of quality fails to create.1137 Judgements about quality of care tend to be based on fragmented information. General practice is an integrated and coordinated discipline and the provision of clinical care is not a series of unrelated procedures. Looking for patterns that facilitate high quality care or contribute to poor care has the potential to help create a picture of where problems lie and where solutions might be targeted. Such data could be used to stimulate improvements across the whole system using internal and external mechanisms by comparing performance over time, between providers, or against gold standards of practice.38

Several recommendations arise from this study. Most importantly, we need more and better information on the quality of care in general practice. This is particularly true for acute conditions and preventative care. For a more valid representation of quality, the evaluations should focus on randomly selected samples of records drawn from populations rather than from self-selected practices39 in order to reduce potential participation bias. In addition, there is a need to focus on non-technical aspects of care, particularly interpersonal care which is a fundamental component of general practice.

Further research and development activity should address the quality problems identified in this review—namely, the reasons why the quality of clinical care provided in general practice fails to meet expected standards, the implications of substandard care, and the possible solutions that could be considered. In an era where health professionals are increasingly expected to be accountable for the quality of care they provide, public trust will only be maintained where doctors and other health professionals are able to show that they are providing high quality care. A comprehensive and systematic approach to the assessment of quality of care in general practice will help practitioners, managers, policy makers, and researchers to start to address the problems identified in this paper.

Acknowledgments

The study was devised by Martin Marshall and Martin Roland. Mary Seddon carried out the primary literature review and data extraction. All authors contributed to the journal hand searching and to the data extraction and analysis. Mary Seddon wrote the first draft of the paper and all authors contributed to subsequent drafts. Martin Marshall and Martin Roland are the guarantors of the paper.

Mary Seddon was supported by the National Primary Care Research and Development Centre which is principally funded by the UK Department of Health. The views expressed in the paper are those of the authors and not necessarily those of the Department of Health.

References

Supplementary materials

  • Systematic review of studies of quality of clinical care in general practice in the UK, Australia and New Zealand

    M E Seddon, head of quality,
    M N Marshall, professor of general practice,
    S M Campbell, research fellow and M O Roland, director

    [Table summarising the results of 90 identified papers] is available for downloading here.

     

Linked Articles