Article Text


From rhetoric to reality: the need for external quality initiatives to understand and better relate to organisational inner worlds
  1. T Smith
  1. Senior Policy Analyst, Health Policy & Economic Research Unit, British Medical Association, London WC1H 9JR, UK; tsmith{at}

    Statistics from

    Very often, central policy and locally managed initiatives fail to anticipate—or perhaps acknowledge—organisational culture and “the way things work around here”. The papers included in this JournalScan provide interesting insights into the disjunction between theoretical and actual views and suggest that one way to improve the success of initiatives is to contrast models of working with actual practise, leading to the improvement of both.

    The papers look at two issues: the development of initiatives to improve patient involvement and the use of guidelines and evidence to improve practise.

    Learning how external information is used in practice

    Importance of social interaction in the development of internal tacit guidelines ▸

    A paper from Family Practice begins by saying that “little is known about the types of decisions nurses make in routine practice and the sources of information that inform these decisions”. Research was undertaken in GP practices in the north of England through three case studies that involved interviews, observation, and documentary analysis. Interviews asked about “the use of research-based information in clinical decision making”. Observation (120 hours) sought to contrast the nurses’ accounts of their information seeking behaviour with observed “reality”.

    The types of decisions made by practice nurses and nurse practitioners can be divided into seven categories: (i) assessment; (ii) diagnosis; (iii) intervention (including (a) targeting, (b) timing and (c) prevention); (iv) communication (risks and benefits); (v) referral; (vi) service delivery and organisation (setting up a clinic, for example); and (vii) information seeking.

    Nurses used several sources of information in making clinical decisions. Personal experience was important, “particularly with regard to wound care, to the extent of their being dismissive of research evidence”. A nurse says she would ignore research evidence that contradicted her experience. Human sources of information are important where nurses experience clinical uncertainty. They preferred to seek information from a colleague rather than electronic or textual information. When information was sought using technology, nurses appeared to favour sources incorporating some form of human contact. Most often this was the GP but other colleagues were also viewed as reliable. “While the explicit use of clinical guidelines was rarely observed nurses saw them as useful … if they had been ‘endorsed’ by clinicians perceived as clinically credible and accessible”. Their use was only observed twice, both times in a practice that belonged to a larger medical group whose nurses worked with in a highly protocolized environment. Despite an awareness of specific guidelines, nurses thought it quicker and easier to ask a colleague.

    It was also reported that protocols did not supply nurses with clear messages for practice: “Some of the protocols you look at them and say, well, what do they mean? Where are you supposed to go from here?” Written sources of information were accessed during consultation only in relation to drug related enquiries which frequently acted as a trigger for information seeking behaviour. Participants reported they were much more likely to use the Internet at home than at work and primarily for CPD related study. During 224 patient consultations, the Internet was used on just two occasions.

    The authors note that “the study findings are consistent with other research on doctors’ use of information …which reveals that other professionals dominate as the prime source of information in uncertain clinical situations”. The paper concludes that “professional development in information retrieval and its clinical application should ideally be offered to all members of the primary care team in an integrated approach, based on the development and adoption of a coherent practice development plan”.

    Does information technology always have a positive impact? ▸

    A paper in the International Journal of Medical Informatics appraises empirical studies examining the impact of clinical information retrieval technology on physicians. The paper assesses impact using a four level scale: (a) high positive impact (practice improvement, learning and recall), (b) moderate positive impact (confirmation and reassurance), (c) no impact, and (d) negative impact (frustration).

    Of the papers that reported high impact, six reported self-assessed learning experiences, five reported changes in practice (again self-assessed), and two reported improved recall and considered that technology can increase use of evidence in clinical learning.

    Five studies found moderate impact—for example, that information retrieval technology may confirm decision making. A physician said: “Having quick access to technology is always reassuring”.

    One randomised controlled trial found that the use of computerised guidelines is not associated with guideline adherence. Three laboratory studies showed that the use of information retrieval technology does not increase the ability to solve clinical scenarios over other methods. Five observational studies also suggested this has no impact. Comments from physicians included criticism that information is incomplete, weak, or fails to provide “all information physicians are looking for”.

    Three observation studies suggest information retrieval technology may generate frustration or complete dissatisfaction.

    The paper argues that the evidence is mixed. Around a third of observational studies suggest a positive impact. But experimental and laboratory studies do not reach consensus. The authors conclude that “this contradictory evidence suggests that further research needs to examine the differential impact of information retrieval technology in everyday clinical practice”.

    Breaking down the wall between evidence based medicine (external control) and the art of medicine (individual autonomy) ▸

    A paper in the Journal of Medical Humanities argues that a false distinction has been drawn between the notions of individual autonomy in the art of medicine and the idea of being externally controlled by evidence based protocols.

    The author accepts some criticisms of evidence-based medicine (EBM). “Advocates of EBM often claim that it is only the best available evidence that needs to be used in relation to any clinical question, and so the areas that are less effectively researched using RCTs are not disadvantaged because whatever is the best available evidence can be used. This is somewhat disingenuous. If the best available evidence is of a lower status than what is possible in principle, it will still be regarded as less important and persuasive. What is needed is a revision of the evidentiary hierarchy which would serve two purposes. Firstly, it should not throw the RCT baby out with the bath water—statistically, RCTs are the best sort of evidence. It is also possible to establish evidence at conventionally high levels, even in some of the orphan areas—for example, evidence based clinical practice guidelines for the psychosocial care of adults with cancer have recently been developed and approved by Australia’s National Health and Medical Research Council. Secondly, the evidentiary hierarchy should be modified to account for the relative difficulties of obtaining different sorts of evidence in different fields, and for the range of factors beyond the biomedical, which are involved in illness causation and management. No easy task, but EMB should concern itself with developing multiple faces of evidentiary value to account for differences between fields of medical inquiry.”

    A key problem for some doctors in accepting EBM is the alleged “incommensurability” between a population and individual view. The author questions any “radical split” between the two perspectives. “Why would one conclude … that there is an implicit conflict between the ethics of individual care and the ethics of population health? Does the knowledge gained through population studies not serve the interests of individual patients? Not to ascribe such properties to individuals, when the evidence is good runs the risk of underestimating the importance of the risk, fails to respect patient autonomy, and may result in some individuals suffering harm.”

    Furthermore, “far from being irrelevant or marginally relevant to individuals, population data must be effectively presented to them in order for adequately informed decision making to occur”. The author argues that EBM is “falsely accused” of constraining patients’ and doctors’ freedom by limiting the expression of patient preferences and for “re-importing paternalism into the doctor/patient relationship”. But he says there is no reason to accept that the failure to incorporate patient values into clinical decisions is the fault of EBM.

    One way to avoid the problem of marginalising patient preferences in guidelines is to include the patient perspective in their preparation. The Netherlands had developed these models to the greatest extent, with patient organisations encouraged and funded to contribute to the development of guidelines, setting priorities for research and incorporating “non-clinical” values in the conclusions drawn from meta-analyses.

    In terms of following guidelines, which the author thinks should be more about prompting than directing, “it is important to distinguish between good and bad reasons for acting outside an accepted guideline … When the goal of treatment is not in dispute, departure from a clinically relevant guideline calls for justification”.

    “Clinicians resent EBM’s intrusion because they consider that it reduces their clinical discretion, but the better view is that they should use EBM to review and revise their clinical beliefs, since EBM is not remote from the clinicians’ world but essential to it. Clinical judgement should be viewed as the critical faculty, employed in the face of uncertainty, used to reduce conceptual biases, to disclose uncertainty to patients and to encourage the physicians’ own learning.”

    “Not altogether surprisingly perhaps, EBM advocates themselves sometimes get things the wrong way round by overcompensating for the backlash against EBM—for example, they often confer a high status on clinical expertise. The crucial point is that clinical expertise is not epistemically prior to evidence based knowledge, but is informed by it.”

    Moving beyond the rhetoric of patient involvement ▸

    New forms of patient involvement are increasingly being pursued, notably in Canada, America and latterly in England. A number of papers have come out of Canada over the last couple of years on patient participation. As policy develops, so researchers are becoming more interrogative of the process of involvement and what it means.

    The importance of establishing the “why” of patient participation ▸

    A paper in Health Expectations looks at the extent and manner of patient participation in the planning of regional care networks throughout the province of Ontario. Through a qualitative, multiple case study approach involving in-depth interviews and observation, the authors evaluate the process of involving patients.

    In the mid 1990s, Ontario established a new cancer care system based on eight separate regions with a new lead agency—Cancer Care Ontario (CCO). The regions were responsible for the development, implementation, and monitoring of standards and guidelines, and each was expected to pay close attention to the involvement of cancer patients and their families in planning processes. Each was mandated to establish a supportive care network by early 2000. Although most had appointed a coordinator by this time, network development proceeded slowly in most regions.

    The first problem encountered by the researchers was a lack of clear direction regarding networks and patient participation in them. “Despite the specific policy intent, the reality of patient involvement did not match the rhetoric”. It was difficult for networks to clarify what patient involvement meant and whether a representative of a patient group was the same as the unaffiliated patient and his/her unique perspective. A second problem was the dominance of regional cancer centres (RCCs). While the cancer centre was the largest cancer care provider in each region, only 50% of cancer patients in the province were cared for by RCCs. One of the regions, which the authors say were better at achieving participation than others, chose to involve providers from outside the centre as well as patients. Unlike in other regions, the coordinator of the network was not an employee of the RCC.

    A third problem was the emergence of competing of provincial priorities. “While even at the best of times it was difficult for patient participants to determine their place and role in network development activities, patient involvement was further sidelined by ongoing provincial reforms which pushed network development, supportive care, and patient perspectives further out of view”.

    The authors argue there is a “disconnect between the rhetoric and reality of patient involvement in network planning and coordination” and that there is a “significant gap between intentions to involve patients in health planning and their actual involvement”. They consider that their results “reinforce findings from a succession of public participation studies that have documented failed attempts at incorporating public and patient involvement in health planning, even when there appears to be a general policy mandate to do so”. However, this study points to “different underlying reasons” from those in other studies. “Our results point to benign neglect of the meaning and methods of public participation as the source of weakness rather than any efforts by dominant interests to actively exclude or prevail over public and/or patient interests. This neglect left efforts to engage patients in the building of networks vulnerable to policy instability and confusion at the central level.”

    The importance of developing relationships ▸

    A paper from Health Expectations investigates the characteristics and achievements of cancer partnership groups within cancer networks in England where patient involvement is said to be most advanced.

    The NHS Cancer Plan in 2000 set a target that, by 2001, cancer networks would be expected to take account of the views of patients and carers when planning services. In pursuing this target, the Department of Health—together with the charity McMillan Cancer Relief—proposed a 3 year programme to support the establishment of local partnership groups, similar to the Canadian initiative described in the paper above.

    The researchers mapped out partnership activity in the networks using survey and interview data. The majority of groups (83%) met at least once every 2 months. They typically had a “core” membership of 20 people with a larger mailing list of up to 100 people. On average, two-thirds of core members were service users and one-third were NHS staff. Of the service users in a group, characteristically 75% were cancer patients and 25% carers.

    The paper identifies stages of achievement from establishing a group (some had gone no further), acting as a “reference group for consultation” (to access service users views), networking with other groups, providing information, and influencing in a proactive way (such as getting people onto decision making groups). Activities progressed in scale and complexity as groups evolved. However, group members did not overstate the “impact” of their activities on patient care: when asked about the influence of the group, many respondents prefaced their comments by acknowledging that service user involvement is a huge and complex challenge which, if taken to its literal conclusion, would require substantial shifts in NHS culture, funding, and systems. The point was stressed repeatedly that “things take time”.

    Many groups felt they were at the periphery of corporate decision making. Stronger relationships between groups and providers are needed for patients to move to the next stage, being involved in developing services. Only a handful of groups demonstrated the capacity to undertake high level activity and these displayed certain characteristics: they had been established for 2 years or more; and they were well organised with systems in place to conduct communications and business in between meetings.

    Understanding and knowing how to work within the mainstream of the “system” was an important goal in itself, and learning who to contact and how to influence them were recognised as essential lessons. Influencing changes seemed effective when relationships were developed with key people.

    It is suggested that the most successful groups were those which understood the way things work in healthcare organisations, who to talk to, and the nature of change—complex and slow. Groups were beginning to treat organisations as “a messy political process” where influence can be brought to bear all the time and where learning and change are achieved in both information and informal (and often unpredictable) ways.

    The authors identify a challenge for future evaluation of public participation activities. They state that “at present there are no agreed criteria by which to judge the success or failure of these groups”.

    Different dimensions of patient involvement ▸

    A third paper looks at how criteria to assess public involvement might be developed. The authors recommend Arnstein’s schema (1969) as a way of exploring and evaluating patient involvement and present an eight-rung ladder of different levels of involvement:

    (8) Citizen control

    (7) Delegated power

    (6) Partnership (8–6 are “degrees of citizen power”)

    (5) Placation

    (4) Consultation

    (3) Informing (5–3 are “degrees of tokenism”)

    (2) Therapy

    (1) Manipulation (1–2 represent “non-participation”)

    Like the paper above, the authors are keen to involve a political dimension in patient involvement. They state: “Care must be taken not to underestimate the dynamic and interactive nature of decision making in the policy and implementation process, in which different interests may at times be in conflict and at times congruent, making a static classification of participation unhelpful or even misleading”.

    For example, although the UK Department of Health established a new body to promote involvement in January 2003, a date has since been announced for its dissolution. Before it is dissolved in 2007, the Commission for Patient and Public Involvement will consult on the best way forward to develop involvement.

    The authors say that activities thus far have been limited to “placation” on Arnstein’s ladder. “People involved need to be familiar with the problems and then entrusted with some responsibility within the decision process for participation within the decision process for participation to be deep, as opposed to shallow. Without such authority, patient involvement forums cannot have deep participation.”

    In order to incorporate greater complexity into the model, the authors develop Arnstein’s ladder by incorporating Cohen’s (1973) and Charles and De Maio’s (1993) multi-dimensional model. The resultant model focuses on (a) the domain of action (for example, influence on policy or on treatment); (b) role perspective (individual, group or social); (c) breadth (number of people involved); (d) depth (whether patients are involved in all or some of a review); and (e) range (scope of questions on which involvement is sought).

    The authors say: “A high level of involvement on each of these dimensions could be seen as a democratic ideal” and argue that “putting these different concepts together, we can delineate a multi-dimensional approach to analysis of involvement”.

    The authors conclude that “a change of culture is needed for patients to have genuine active participation in their own health care”. They call for healthcare professionals to receive training in ways to improve participation.

    The need for greater clarity and evaluation of objectives ▸

    One of the key messages from the papers on patient involvement is that there needs to be greater clarity about what involvement is intended to achieve, which in turn will aid the evaluation of objectives. A paper in the International Journal for Quality in Health Care suggests the same can be true of quality initiatives more generally.

    Making the business case for quality initiatives ▸

    The paper begins by noting that “policymakers, payers, and employers continue to express their frustration that quality interventions of demonstrated effectiveness are not being implemented on a broad basis” and argues that one of the principal reasons is that no “business case” is made for quality improving initiatives.

    The authors examined the peer reviewed literature between 1980 and 2004 and found only 15 papers that provided sufficient information on the cost of implementation. Among these, “no patterns emerged for setting or geographical area in which the quality enhancing intervention was studied. Four papers dealt with asthma but this likely reflects the significance of this disease … as well as a growing consensus regarding appropriate processes of clinical treatment and associated patient outcomes.”

    “More relevant to our ‘business case’ is that nine of the 15 … dealt with some aspect of patient self-management of chronic conditions. The finding does not mean that self-management interventions for chronic conditions necessarily yielded better business cases than other quality enhancing strategies. We suspect, rather, that researchers interested in the efficacy of self-management strategies may be more attuned to the need to establish an economic or business case for them, for instance to encourage health service mangers or policymakers to invest scarce resources in patient self-management initiatives. By contrast, the organization, financial, and other considerations inherent in implementing innovative quality-enhancing interventions may be more daunting than for more traditional quality-of-care steps.”

    The authors postulate that several factors may militate against reporting such information: (1) investigators are typically not trained to be concerned with the business case for the interventions; (2) internal sponsors may not be interested in implementation costs; (3) an external agency or foundation may have funded the intervention (in this situation, costs to the organization implementing the quality enhancing intervention may have been minimal); and (4) a series of technical or methods challenges may pose obstacles for this type of work (for example, better research methods to evaluate the financial impact of quality enhancing interventions are needed).

    The authors suggest that those funding research should “mandate that investigators routinely gather and report financial information that could be used by interested parties to analyse the business case for the interventions themselves”.

    View Abstract


    • Compiled by Tom Smith

    Request permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.