Article Text


Statistics from

This journal scan is based on a hand search of the following seven journals focusing on health services research and policy for 1999 and up to March 2000: Journal of Health Services Research and Policy, Milbank Quarterly, Medical Care, Health Services Research, Health Affairs, Health Policy, and Medical Care Research and Review. A special issue of the Milbank Quarterly dedicated to the issue of quality is also included. This special issue contains seven papers and several commentaries which are based on an Institute of Medicine (USA) National Roundtable conference to discuss strategies for improving the quality of health care. The summaries below were written by the scanner, drawing on the published abstract where appropriate.

Schuster MA, McGlynn EA, Brook RH. How good is the quality of health care in the United States? Milbank Quarterly 1998;76(4):517–63.

A systematic review of the literature since 1993 which reported studies on the quality of care (other than in single hospitals or clinics) in the USA. The authors found surprisingly little systematic knowledge available and, what there was, highlighted large gaps between the care people should receive and the care actually received (both overuse and underuse). Summary tables are provided for each study.

Chassin MR. Is health care ready for six sigma quality? Milbank Quarterly 1998;76(4):565–91.

This paper explores the underlying causes of quality problems in health care, discusses some of the main obstacles to improvement, and suggests the components of effective strategies to increase the pace of quality improvement. “Six sigma quality” refers to the strategy devised by Motorola which sets the tolerance limits for defective products at six standard deviations from the mean (so that less than 3.4 defects occur per million produced).

Shortell SM, Benet CL, Byck GR. Assessing the impact of continuous quality improvement on clinical practice: what will it take to accelerate progress. Milbank Quarterly 1998;76(4):593–624.

A systematic review of research between 1991 and 1997 on the impact of the clinical application of CQI. 42 single site and 13 multisite studies were identified. Most of these were “before and after” studies so it is difficult to attribute reported change to the introduction of CQI. Most of the studies focused on misuse of health care. Overall it appears that CQI can improve quality and outcomes (though the results are less encouraging in studies using more rigorous research methods and there is likely to be publication bias). Important correlates of success include participation of physicians, feedback to individual practitioners, and a supportive organisational culture.

Blumenthal D, Kilo CM. A report card on continuous quality improvement. Milbank Quarterly 1998;76(4):625–48

An overview of the lessons of a decade of experience in the USA of using CQI in health care, based on interviews with thinkers, activists, and leaders of health care organisations, principally in hospitals. The authors report that the quality movement has not had the impact that many advocates had hoped for and that had been observed in manufacturing industries. No examples of major success stories were identified and quantitative evidence in support of CQI was not found. Early CQI was possibly too focused on administrators rather than clinicians and various obstacles to achieving aims included the lack of pressure for change, mistakes in the early application of CQI, the power structures, and the lack of an appropriate balance between measurement, external monitoring, and improvement. However, there was general support for maintaining CQI.

Other relevant papers from the special issue of Milbank Quarterly include:

Dudley RA, Miller RH, Korenbrot TY, Luft HS. The impact of financial incentives on quality of health care. Milbank Quarterly 1998;76(4):649–86.

Sisk JE. Increased competition and the quality of health care. Milbank Quarterly 1998;76(4):687–707.

Brennan TA. The role of regulation in quality improvement Milbank Quarterly 1998;76(4):709–31.

Marshall M, Shekelle P, Brook R, Leatherman S. Public reporting of performance: lessons from the USA. Journal of Health Services Research and Policy 2000;5:1–2.

A discussion of the experience of public disclosure as a method of improving accountability and drive quality improvement. The results have been mixed and there is little evidence to show that patients have been empowered. Several recommendations are made, such as clarity of purpose, focus on providing information for the provider organisations, and the need for sufficient validity (including risk adjustment) of the data. Professionals should be involved in this from the start and special education initiatives should accompany performance reports. The unintended consequences of public disclosure should be monitored as well as the benefits.

Goddard M, Ferguson B, Dawson D. Contracting for quality: does length matter? Journal of Health Services Research and Policy 1999;4:220–5.

This paper examines whether longer term contracts for health services facilitates increased attention on quality in contracts and away from finance and activity levels. The authors looked at 288 contracts in the British NHS and interviewed staff. No relationship was found between the duration of contract and attention to quality. Alternative approaches to quality improvement outside contracts are possibly better at dealing with quality.

Wray NP, Peterson NJ, Souchek J, et al. The hospital multistay rate as an indicator of quality of care. Health Services Research 1999;34:777–90.

Evaluation of the degree to which the hospital multistay rate (number of hospital stays per hospital user over a time period) could be useful as a performance indicator. Discharge data from the Veterans Affairs patient treatment file for 1994 were used to estimate a standardised multistay ratio which takes into account patient demographic characteristics and severity of illness. The maximum standardised ratio ranged from 1.12 to 1.39 depending on disease category. The authors suggest that this has the potential to be a useful measure, complementing standardised mortality data by measuring ongoing coordination of care.

Rosenthal GE, Baker DW, Norris DG, et al. Relationships between in-hospital and 30-day standardised mortality: implications for profiling hospitals. Health Services Research 2000;34:1449–68.

A study examining the relationship between in-hospital mortality rates, 30-day rates and hospital discharge policies in 30 hospitals in Ohio based on data for nearly 14 000 patients with congestive heart failure. The correlation between in-hospital and 30-day SMR was 0.78 and hospital outlier status varied for seven of the 30 hospitals according to the measure used. There was no significant relationship between discharge practices and post-discharge mortality.

Solomon DH, Schaffer JL, Katz JN, et al. Can history and physical examination be used as markers of quality? Medical Care 2000;38:383–91.

This paper assesses whether the thoroughness of documentation of the initial visit by patients presenting with acute musculoskeletal pain to specialists is associated with patient satisfaction, symptom relief, and functional improvement at three months follow up using data on 513 patients at a large academic medical centre in the USA. There was generally poor but variable levels of documentation. However, thoroughness of documentation was not significantly associated with patient outcomes and only very weakly associated with patient satisfaction.

Shortell SM, Jones RH, Rademaker AW, et al. Assessing the impact of total quality management and organizational culture on multiple outcomes of care for coronary artery bypass graft surgery patients. Medical Care 2000;38:207–17.

This study assesses the impact of total quality management (TQM) and organisational culture on outcomes of coronary artery bypass graft surgery using prospective data on 3045 patients from 16 hospitals in the USA using risk-adjusted clinical outcomes, functional health status, patient satisfaction, and cost measures. Implementation of TQM and organisational culture were measured using previously validated instruments. The observed 2–4-fold difference in all major outcomes was not associated with TQM or organisational culture.

Zaslavsky AM, Beaulieu ND, Landon BE, et al. Dimensions of consumer-assessed quality of Medicare managed-care health plans. Medical Care 2000;38:162–74.

Assessment of the relationship between member ratings of health plans and the report of plans using the Consumer Assessment of Health Plans Survey (CAHPS) on data from over 89 000 Medicare beneficiaries enrolled in 212 Medicare managed health plans who responded to CAHPS in 1998. Four factors explained 75% of the variance in the reports: interactions around the delivery of care in the doctor's office, consumer service from the plan, access to medical services such as specialist care, equipment, and therapy and advice on health promoting activities. CAHPS distinguished among dimensions of between plan variability of consumer assessed quality of care.

Thomas WJ, Hofer TP. Accuracy of risk-adjusted mortality as a measure of hospital quality of care. Medical Care 1999;37:83–92.

Determination of whether mortality rates, when perfectly adjusted for case-mix and severity differences, are valid indicators which are able to identify poor quality hospital providers accurately. This study is based on a model which simulates optimal conditions of perfect risk assessment and no variation in the patient volume among hospitals. Under virtually all realistic assumptions for parameter values, sensitivity remained less than 20% and predictive error was greater than 50%. The authors conclude that reports of hospital performance using risk adjusted mortality rates are misleading.

[Scanner's note: This conclusion confirms the findings of a recent review of the research which was published slightly too early to be included in this scan: Thomas JW, Hofer TP. Research evidence on the validity of risk adjusted mortality rate as a measure of hospital quality. Medical Care Research and Review 1998;55:371–404.]

Geraci JM, Ashton CM, Kuykendall DH, et al. The association of quality of care and occurrence of in-hospital, treatment-related complications. Medical Care 1999;37:140–8.

This study determined whether measurable differences in quality of care are associated with the occurrence of non-fatal, in-hospital and treatment-related complications. A retrospective cohort study was used including data from a chart review on over 2000 patients discharged alive from nine Veteran Affairs Medical Centres with congestive heart failure, chronic obstructive pulmonary disease (COPD), or diabetes. Process quality, based on assessment of admission work up and treatment, was compared with data on patient severity and complications during hospital stay. Better admission work up and treatment quality in COPD patients and treatment quality in patients with diabetes was associated with a lower risk of non-fatal treatment-related complications.

Harris-Kojetin LD, Fowler FJ, Brown JA, et al. The use of cognitive testing to develop and evaluate CAHPS 1.0 survey items. Medical Care 1999;37:MS10–21.

This paper describes the cognitive testing carried out as part of the Consumer Assessments of Health Plans Study (CAHPS) which aimed to develop an integrated set of tested and standardised surveys to obtain meaningful information from health plan enrollees about their experiences. 150 cognitive interviews were held across three organisations using multiple interview methods with different consumer populations. The authors discuss how the findings were used for improving the instrument to capture consumer views with less response error.

Schnelle JF, Ouslander JG, Buchanan J, et al. Objective and subjective measures of the quality of managed care in nursing homes. Medical Care 1999;37:375–83.

Comparison of the medical care received by nursing home residents enrolled in HMO and fee-for-service plans using both process of care and consumer perception measures of quality. On process of care measures, HMO residents received better quality care (e.g. more timely response to emergencies and more frequent routine visits by a primary care provider). However, there was no difference in consumer perceptions of quality which in turn differed from those of their families.

Iezzoni LI, Mackiernan YD, Cahalane MJ, et al. Screening inpatient quality using post-discharge events. Medical Care 1999;37:384–98.

This paper describes a computerised method developed to screen for hospital complications within 90 days of discharge using outpatient and non-acute care immediately following discharge using the Complications Screening Program for Outpatient data (CSP-O). The method of developing the programme is described and its testing on data from 739 248 discharges of Medicare beneficiaries (over 65 years) admitted to 515 hospitals in 1994. Complete post discharge data were available on about two thirds of patients and, although the CSP-O has some construct validity, it needs to be examined further before being used to screen hospital quality. As lengths of stay fall, ways of incorporating the immediate post-discharge experience in measures of quality is increasingly important.

Weissman JS, Ayanian JZ, Chasan-Taber S, et al. Hospital readmissions and quality of care. Medical Care 1999;37:490–501.

A study to test the association between readmission and quality and so the utility of using readmissions as a measure of quality. This is based on a case control study of 1758 Medicare patients hospitalised with pneumonia or congestive heart failure. Compared with non-readmitted patients, patients with related adverse readmissions had lower adjusted quality measured both by explicit and implicit methods for both conditions. However, this was not a useful tool for identifying patients who experience inferior care, or for comparing quality between patients.

Karon SL, Sainfort F, Zimmerman DR. Stability of nursing home quality indicators over time. Medical Care 1999;37:570–9.

Assessment of the degree to which the Quality Indicator (QI) system for measuring potentially poor practice and outcomes of nursing home care are stable over the short run. This system measures the facility level of conditions for residents. 512 nursing facilities in Kansas and South Dakota were measured for the first three quarters of 1996. The results indicate that the QIs are reasonably stable over time.

Dranove D, Reynolds KS, Gillies RR, et al. The cost of efforts to improve quality. Medical Care 1999;37:1084–7.

A study to assess the cost of efforts to improve quality and to determine whether quality improvement expenditures are correlated with outcomes or hospital overall costs. Detailed data from hospitals participating in a broad study of CQI activities in the USA were used. There was a wide range of expenditures on quality improvement activities which can be a significant percentage of total costs. Expenditure on quality improvement was not related to overall costs.

Harris LE, Swindle RW, Mungai SM, et al. Measuring patient satisfaction for quality improvement. Medical Care 1999;37:1207–13.

This paper describes the development and testing of a multidimensional, visit-specific, measure of satisfaction with primary care using a survey which included several existing measures (MOS Visit Specific Questionnaire and the American Board of Internal Medicine Patient Satisfaction Questionnaire) and also locally developed items. Satisfaction with the office and with the provider contributed independently to overall satisfaction and the former was associated with disenrolment from the health plan. Satisfaction with access was only associated with overall satisfaction for adults.

Lock P, McElroy B, Mackensie M. The hidden cost of clinical audit: a questionnaire study of NHS staff. Health Policy 2000;51:181–90.

This study aimed to assess the full costs of clinical audit in a Scottish Health Board and to extrapolate the results to Scotland. It was based on a survey of a sample of NHS staff to estimate time spent on clinical audit and audit budgets. Medical staff were more likely to be involved in audit than non-medical staff. In the Forth Valley the total spending was £1.72 million which aggregates to £36.3 million for Scotland, over 80% of which was due to staff time.

Lehman AF. Quality of care in mental health: the case of schizophrenia. Health Affairs 1999;18:52–65.

A summary of the results of the schizophrenia Patient Outcomes Research Team research and its implication for treating chronic mental illness. The paper provides an overview of the scientific knowledge about which treatments improve outcomes and then demonstrates that this evidence is not reflected in current practice in the USA. Proposals for improving care are made.

Miller T, Leatherman S. The national quality forum: a `me-too' or a breakthrough in quality measurement and reporting. Health Affairs 1999;18:233–7.

This paper discusses the role and tasks of the National Forum on Health Care Quality Measurement and Reporting, a new private sector organisation which has been given responsibility for monitoring and improving quality in health care in the USA in line with aims set by the Advisory Council for Health Care Quality (a public entity).

Scanlon DP, Chernew M. HEDIS measures and managed care enrolment. Medical Care Research and Review 1999;56(Supplement 2):60–84.

Examination of the relationship between 1996 health plan enrolment choices by employees of a large firm which disseminated plan performance ratings to its employees. Conditional logistic regression analysis was used to estimate the association between plan market share in the firm and the ratings of those plans by aggregate and individual measures ratings from HEDIS after controlling for price and type of plan. Overall, choice of plans was not strongly correlated with HEDIS scores. Indeed, where there was a relationship with ratings on individual measures, it was often the poorer performing plans that attracted more enrollees. The authors suggest that this may imply that performing well on some measured aspects of quality reflects poorer performance on aspects not measured but informally observed by employees already members of, or who know members of, those plans.

Trevor A Sheldon 
 Department Of Health Studies,University of York,Innovation Centre,York Science Park,YO10 5DG,UKemail:tas5{at}

View Abstract

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.