The NHS Plan said: “The NHS gets more and fairer health care for every pound invested than most other health care systems”. Is that true? ▸ The NHS is widely thought to face challenges in quality of care but, in the grand scheme of things, punches above its weight given the low funding levels. A BMJ paper caused a storm when it set out to expose this as a myth, concluding that an American HMO outperformed the NHS. It acknowledges that, in the US, health care overall “is more expensive” and “population outcomes are no better”, but Kaiser Permanente (a Californian HMO) “costs about the same as the NHS but performs considerably better”. As the paper states: “findings challenge the widely held view that the NHS is efficient and that its inadequacies are mainly due to underinvestment”. This flies in the face of the Wanless conclusions and the current “debate” on the future of the NHS—how to raise more money. It suggests the problems of the NHS are more to do with the way it works than spending levels. The paper says: “Kaiser and the NHS both provide comprehensive health services”. It has been pointed out that Kaiser does this for an insured population (93% through employee schemes) and the NHS for the whole population. Almost a quarter of California's population is uninsured. The paper acknowledges that “few Kaiser members are very poor” and the NHS “provides greater coverage for dental and long term psychiatric care services”. The NHS also covers long term medical and home health services which Kaiser does not. While the NHS spends 6% of its budget on education and training and research and development, technical services and associated grants, Kaiser spends 3.5% on the same services. Administrative costs account for 4% of the Kaiser budget.

Finance aside, the argument of the authors is that the speed of diagnosis and treatment is what accounts for better outcomes. Primary care doctors in America spend longer with patients and are able to perform more complicated procedures and patients do not suffer long waits for referrals. US doctors work in “multispecialty centres that employ between five and 40 doctors and are supported by physician assistants and nurse practitioners who have their own lists of patients and are able to conduct clinical examinations, make diagnoses, and prescribe some medications. These `physician extenders' increase the number of available clinical staff by almost two thirds.” (In the January issue of Social Policy and Administration Gavin and Esmail say it is time for the UK to consider physician assistants). “Laboratory, radiology, and pharmacy services are also available on site. Some centres also have physiotherapy and mental health services, while others include various specialist services in the same building. In addition, these facilities are open in the evenings and weekends for urgent visits.” There are four reasons for the better performance of Kaiser: (1) it achieves better integration between providers; (2) it treats patients at the most cost effective level (a third less are hospitalised than in the NHS); (3) it reaps the benefits of competition and choice and is responsive to the insured's decisions; and (4) it has more sophisticated information technology.

A few responses have been elicited. Michael White, the political editor of the Guardian, pointed out in the Health Service Journal that “coverage” of each system explains a lot. Jennifer Dixon of The King's Fund is more open to the criticisms the paper presents. She adds some other differences between the two systems which might be further explored. “Specific factors could include the form of organisation, the level and type of financial incentives operating, the extent that power and decision making concentrates at the top of the organisation, and the number and training of staff. Other possibly more important factors could include the type of leadership, the quality of management, the ethos of service in the organisation, how staff are valued and promoted, and the extent of party political involvement in management.” She suggests that the NHS should think seriously about what the paper says because “what counts is what works”. Don Berwick adds that the reasons Kaiser patients get more from their dollar is that “the systems differ in their capacity to configure care according to the needs of the patient throughout an episode of illness or, in the case of chronic illness, the patient's life. Kaiser integrates care much more reliably than the NHS does.” He suggests that a single SHA could experiment with an alternative model. Alain Enthoven says the study got it “`about right': British people ought to think about how and why Kaiser does it”. He says it is to do with competition and “one possible way forward would be to create a `wide open market' for hospital services in which private hospitals in Britain and European hospitals can compete to serve NHS patients. (This can be contrasted with the comparatively timid `internal market' that envisioned competition mainly among NHS hospitals).” The paper has caused quite a debate and the rapid response articles on www.bmj.com are worth looking at.

Ideology and the judgment of quality ▸ Cross system comparison is rare and, as the authors of the Kaiser Permanente study say, “have to be undertaken with great care”. They are “difficult because of the complexity of the systems and their contextual specificity”. An angry analysis of last summer's WHO performance tables, which rank international health systems, appeared in the January issue of the American Journal of Public Health, suggesting that the WHO's comparative studies have failed to heed this warning. The WHO report gave an overall score for international health systems based on three indicators: effectiveness, responsiveness and fairness. It is argued that the crude assumption that morbidity and mortality are a function of a nation's health system and the conclusion that greater investment in health care is the answer is the promotion of a medicalised concept of health “that is worrisome”. It ignores political, economic, and social factors. The emphasis on “responsiveness” is indicative of a market orientated bias toward “managed care” in its American and European manifestations. “Not surprisingly, therefore, the report lists the US as the most responsive health care system, even though the US population is the least satisfied”. The bias of the report “reaches vulgar proportions when it refers to the collapse of the Soviet Union as an indicator of unresponsiveness” and reveals its unscientific and value-laden methodology. The tables are described as “the reproduction of ideology through technocratic discourse”.

How can information on league tables become more sophisticated? ▸ It may be difficult to compare health systems across countries, but what about within systems? There have been plenty of complaints that a comparison of broad measures transferred into tables is not an effective way of comparing clinical quality and is not useful information for patients. One article discusses the merits and drawbacks of league tables and asks what other ways can quality be more effectively measured. League tables make providers anxious and they are not easy for “consumers” to interpret. Performance tables are more relevant for a competitive system where people have choice. In the NHS they do not, and tables can be alarming. “Furthermore, a systematic review of the literature on the effects of public release of performance data showed that individual consumers and purchasers don't search out, understand, or use available data”. Evidence also shows that the tables may lead to unintended consequences such as “gaming”—encouraging providers to focus on performance measures per se rather than improving the quality of care the measures supposedly indicate. Because of these problems, several alternatives have been suggested. The authors consider that control charts are most appropriate because they analyse performance without ranking and explore special circumstances. “This method, derived from statistical process control, is based on the recognition that the outputs of even the most perfectly tuned production system inevitably show some variation. This means that, even under ideal conditions, similar providers (doctors or trusts) will never match each other's performance exactly, or indeed their own performance from one month to the next. The major difference between control charts and league tables is that they are based on diametrically opposed premises. The reason for constructing league tables is the implicit assumption that there is a performance difference between providers. With control charts, however, there is an explicit assumption that all providers are part of a single system and have similar performance.” They are visually more straightforward and outliers easier to spot.

Subjectivity and the judgment of quality ▸ Measuring quality through outcomes is seen as an important means of changing behaviour in health systems. The reliability of measures based on medical records is increasingly questioned, and there is interest in incorporating the recording of measures into the process of care. There have been two approaches to improving reliability of the data recorded. One is to seek patients and carer perspectives on outcomes. The limit of this approach is that different perspectives have legitimately different ideas about the nature and severity of problems. “Parents have a different evaluative context than do their children”. The second approach is to develop audit instruments that are completed at the time of care and in retrospective audits. The effectiveness of such an instrument was evaluated with “crisis workers” assessing children in state homes presenting with behaviour problems. In the retrospective study staff accurately recorded the behaviour of the child and clinical assessment but were less accurate at recording external factors about, for example, the domestic situation. This sometimes led to unreliable assessment. The focus of staff is understandably on the child, but not necessarily on important contextual information. Providers are more likely to rate risk higher than reviewers. A possible explanation is that personal interaction made them more sensitive to risk and prone to hospitalise people. Another is that higher ratings were used to legitimise a culture of hospitalisation. “Recognition of the fact that providers have first hand experience of patients is crucial to understanding the usefulness and limitations of a measurement audit to evaluate outcomes such as clinical decisions. The temptation to label crisis workers as incorrectly rating clinical severity must be matched by the realisation that charts often lack the subjective impressions. This is all the more reason to make relevant information as clear and detailed as possible in documentation.”

Does working with clinicians on outcome assessment increase their adoption of assessment tools? ▸ “Outcome assessment has not been universally embraced by health care providers”. It is said to be time consuming, intrudes on the therapeutic relationship, and undermines professional autonomy and clinical judgement. A study set out to explore if attitudes towards the use of an outcome assessment instrument would improve if technical support was provided—namely, a system to coordinate and administer forms for the clinicians. The study involved 18 clinicians (average age 47.9, 53% women) with an outcome assessment developed by a research committee in the hospital. It aimed to collect assessment information on “well being”, “role functioning”, and “symptoms”. The study lasted 5 months. Attitude and behaviour were measured at four points: at the outset, during and after the 8 week period of support, and at the end of the project. In the immediate period following the technical support clinicians were more positive about using the instrument. This tailed off when support was removed and, by the end of the study, had returned to the same level as at the outset. The results suggest that “clinicians are not opposed to outcome assessment tools per se but rather are opposed and resistant to the additional administrative requirements” such protocols involve. The clinicians did not continue to collect the data as they thought it unimportant to care. The conclusion of the paper is that widespread adoption will require support systems (human or automated) to use outcome assessment.

Switching the focus on research to quality of care ▸ A paper in the Lancet set out to examine whether large ICU units have better outcomes than smaller ones. It found a link between the workload of the unit and risk adjusted mortality. “Infants committed at full capacity versus half capacity were 50% more likely to die”. Nursing workload is a particularly strong indicator. ICUs are very complex organisational structures. Healthcare professionals must communicate large amounts of information to each other while at the same time educating and consoling families. Comment in the Lancet says: “inevitably there are differences in quality between units influenced by, amongst others, unit size, number of admissions, structure, process issues, case mix, technological facilities, availability of specialists, teaching status” and so on. “Studies that have explored these features have been inconsistent” and “factors such as volume of patients and total staff numbers no longer adequately represent what is really happening at the bedside”. It suggests that an understanding of quality has moved on. “The delivery of care can be examined, and organisational and bedside management factors as well as clinical factors that are associated with clinical outcomes are starting to be understood”. It concludes that “the major challenge in research into quality of care in ICUs is to focus on the influence of bedside care rather than on broadly defined factors related to quality”.

How do consumers appraise information? ▸ A BMJ study describes techniques used by the public on the Internet in retrieving and assessing health information. This qualitative study observed 21 people searching for health information, then talked to them in semi-structured interviews immediately after the search and a subsequent group conversation in four focus groups. Although most search strategies are “suboptimal”, answers to questions typically take 5 minutes to find. “Surfers” tend to check the first few links returned and then search through general search engines; they do not use medical sites as a starting point. They are concerned with the source of information and look for validation in a professional design or authoritative source: “I consider it to be reliable if information is from public institutions or scientific publications”. Consumers want to know if information is opinion or evidence and get frustrated if they cannot tell the difference. Culture matters too. People try to find sites in their own language. When questioned, most remember the information but not its source; about 20% do. They would like information to be officially validated. People value the chance to learn more about health to enable them to have better informed discussions but they are also wary about the quality of some of the information. The authors suggest we should rethink rational assumptions about Internet searches.

Why are quality programmes decreasing as mainstream organisational activity in Fortune 1000 companies? ▸ Since 1987 a group of researchers at the University of California have surveyed the Fortune 1000 companies in America at 3 year intervals for their use of quality initiatives and employee involvement. Analysis of the five sets of data show that companies' involvement in quality initiatives is falling after rising and peaking in the 1990s. Why is quality activity tailing off? Some of it relates to declining use of certain approaches such as “quality circles”, which is less widespread now than when the survey began. Total quality management was used by 73% in 1990 and by 55% in 1999. It is suggested that companies have come to see quality programmes as creating initial successes that are difficult and costly to sustain once the initiative has been around for a while. It is also suggested that the activity associated with quality initiatives is now part of mainstay behaviour. Quality programmes are useful for reassessing behaviour. The survey finds a “levelling off” in employee involvement activities and a change in direction. Companies are facing up to the challenge of knowledge management and focusing efforts on ways of forging effective organisational memories of past experience. One result is the increasing use of IT as a form of employee interaction, in particular for surveying staff, which is one activity that has increased greatly. The authors suggest that there is a discernable productivity value in teamwork and asks: “if the research shows that teamwork pays, why isn't everyone doing it?” The answer is probably that the productivity is not discerned by senior managers.

Why do the quality advocates fail to persuade? ▸ A good example of the sort of research that fails to persuade is a study by the European Centre for Total Quality Management of NHS trusts to assess the influences of organisational effectiveness. The results are based on a self-assessed comprehensive questionnaire of 69 trusts in England and Wales and relate to management styles and structures. Accepting trusts' self-portraits, researchers looked more closely at best practice in the trusts that assessed themselves highly effective on a range of measures. It draws up a list of effective practices which it says are “a collection undertaken by highly performing NHS trusts” and “form the foundation” to “share and transfer best practice”. The material is pretty unconvincing. Best practices in the NHS are where “empowerment and ownership are key to the trust's success”. CEOs should adopt an “open door policy” and go to meet staff where they work, “producing a common bond”. Best of all, “the CEO leads the generation of new ideas within the trust by stating in the annual report: `there is nothing I like to hear more than a good idea'.” There is no attempt to fulfil its promise to explore the relationship between organisational effectiveness and management behaviour. Research that fails to probe or test concepts in reality do not enhance the cause of quality or make it likely that organisations will adopt associated practices.

The quality of NHS trust boards ▸ How different the picture of public sector management looks in a report from the Royal Society for the Encouragement of Arts, Manufacturers and Commerce (RSA). The picture it presents is far from that of the “increasingly proactive” and “quality orientated” approach of senior management in the previous paper. The authors find many boards in the public sector “guilty of tokenism, lack of focus and cohesion, and confusion about its roles and responsibilities and the legislative framework within which it operated”. Over 140 chairs, chief executives, non-executive directors, and trustees of public and voluntary sector organisations, including 62 from the NHS, were surveyed. Additional input came from a further 100 board members. Boards were inadequately skilled, particularly in information technology, and age biased, relying heavily on people who are retired or not employed. Vacancies were rarely advertised and appointments sometimes completely random. A large proportion of participants felt that they were sufficiently “trained and knowledgeable to take on the task”. The author thinks that “such complacency reflected the magnitude of the problem facing the public sector”. “The government doesn't understand corporate governance, and it is trying to change the NHS from the outside, but that's impossible without support and skills from the inside.”

A charter for fundamental principles for medicine ▸ High quality is principally the responsibility of medics rather than organisations, argues a commentary in the Lancet. Both the Lancet and Annals of Medicine published the results of an American-European project to develop a charter for medical professionalism. The Lancet's commentary cites Theodore Marmor's Rock Carling lecture 2001 in which he talks of the “linguistic muddle” and “conceptual confusion” that has marked the organisation of medicine during the past decade. “Business ideology infiltrated health care when costs spiralled and governments reconsidered their commitment to the welfare state. As professors were deposed by CEOs, and as the fog of HR jargon obscured an emptiness of serious thought, so mediocrity became the benchmark or running a health service. Priorities shifted. Quality was eroded by a concern for quantity, effectiveness gave way to efficiency and notions of professionalism were subsumed by mission statements.” The charter is seen as an opportunity for medicine to reclaim its central ethos. But medicine is governed by an ethos, not a balance sheet.

Fundamental principles

• Primacy of patients' welfare

• Principles of patients' autonomy

• Principle of social justice

Set of professional responsibilities

• Commitment to professional competence

• Commitment to honesty with patients

• Commitment to patients' confidentiality

• Commitment to maintaining appropriate relationships with patients

• Commitment to improving quality of care

• Commitment to improving access to care

• Commitment to scientific knowledge

• Commitment to maintaining trust by managing conflicts of interest

• Commitment to professional responsibilities