Introduction

Organisational sociology has long accepted that mistakes of all kinds are a normal part of work.¹ This paper focuses on characteristics of organisations that make errors, mistakes, and accidents almost inevitable. The context of the paper is the medical care of patients in healthcare settings, particularly within the UK's National Health Service where there has been a recent resurgence of concern about the frequency with which adverse events occur.² Adverse events occur in hospitals for many reasons. Some may be unavoidable. An uncommon allergic reaction to a properly prescribed and administered medication, for example, is unfortunate but is part of the risk associated with accepting any medical intervention. Some adverse events are the result of mistakes or malfeasance on the part of individuals. It is easy for an individual clinician to misread a prescription or administer the wrong dose of a drug to a patient as a result of stress or a momentary lapse of concentration. The list of mistakes that each clinician accumulates is probably a complex function of length of time in practice, situational factors, individual characteristics, and random chance. Stories about staff causing deliberate harm to patients are rare but memorable and undermine public trust in healthcare professionals. The names of Beverley Allitt and Harold Shipman are unlikely to be forgotten in a hurry. Although the chance of a nurse or doctor perpetrating similar crimes in the future is statistically infinitesimal, such dramatic and horrifying events form an important constituent of public discourse about risk and danger in healthcare settings.

The thesis of this paper is, however, that many adverse events that occur in hospital, as in other organisations, are due to failures of the system rather than individual failures. It reiterates the argument that many of the errors, accidents, and disasters that happen in hospitals are rooted in the organisation of health care.³ Starting from the premise that complex organisations have both the capacity to achieve goals that individuals cannot achieve and to introduce sources of error that are similarly not directly attributable to the behaviour of individuals, this paper argues against automatically ascribing errors in organisations to the ignorance, incompetence, or immorality of individuals. These arguments support efforts to move the culture of healthcare organisations away from blame and exhortation of individuals towards an analysis of systemic sources of error.⁴ Although there has been a growing realisation that the “blame culture” which tends to surround adverse events in hospital is self-defeating, there is still a tendency to locate the sources of accidents primarily in the behaviour of individual staff members rather than in the social organisation of work.

An additional aim is to contribute to the development of a more general theoretical approach on which future efforts to decrease risk can be based. The main problem with anecdotal or case study approaches to the analysis of adverse events is that it is all too easy to dismiss any lessons that might be learned by saying “it couldn't happen here”. Theories assist in the transfer of knowledge from one setting to another so that healthcare professionals can draw on the experiences of such catastrophes as the Challenger launch decision,^{5 6} aeroplane crashes,^{7 8} and chemical and nuclear spills such as occurred at Three Mile Island,⁹ Prince William Sound (Exxon Valdez),¹⁰ Bhopal,¹¹ and Chernobyl.¹² Although these high profile cases may at first seem to have little in common with adverse events in hospitals, they are linked by the fact that organisational structure, process, task and/or environmental pressure are almost always implicated in their production.¹³ At the same time the literature on “high reliability organisations” is also relevant. Studies of organisations where failures would lead to major disasters, such as aircraft carriers¹⁴ and the US air traffic system,¹⁵ show that organisations that manage to operate reliably and routinely without accidents share a number of features. Scott lists a number of organisational elements that appear to promote high reliability including “selection and training of personnel, redundancy of functions (equipment, procedures), reliance on collegiality and negotiation within a tight formal command structure, and a culture emphasising cooperation and commitment to high standards”.¹⁶ This paper is an attempt to access the wider sociological literature on the organisational sources of safety and danger to determine the extent to which knowledge from this field can be applied to the promotion of quality in healthcare settings.

Sociology of “the dark side of organisations”

There is a large literature in sociology on the social causes of errors, mistakes, and disasters. Until recently this literature has developed in diverse specialties with little cross fertilisation. Experts in disaster research had little contact with researchers focusing on accidents, for example, although they may have relied on many of the same explanatory concepts. This division of labour has retarded the development of sociological theory in this area and makes it difficult to generalise from the large number of case studies and anecdotal material that has been generated in the field. This has begun to change. In a comprehensive and judicious review of the literature concerned with organisational deviance Diane Vaughan has searched the sociological literature for “theories and concepts that explain, generically, how things go wrong in socially organised settings”.¹³ Her review has uncovered causal relationships between the environment in which organisations operate, organisational structures (including complexity, centralisation, and formalisation), processes (informal organisation, power, and learning), and tasks (level of skill, technology, and the role of knowledge). The goal of her work was to build the theoretical base for the study of “the dark side of organisations” as an integrated field. The present paper distils from this review, and from the sociology of organisations more generally, a number of organisational characteristics that can sometimes cast a dark and lingering shadow over attempts to care and cure in healthcare organisations.

We begin with the division of labour—the most fundamental characteristic of organisations—and to some extent their raison d'être, arguing that the compartmentalisation of work increases the likelihood of adverse events by introducing the need for communication and monitoring. The paper then discusses the homophily principle which, despite the fact that it is one of the few ideas in sociology for which there is overwhelming empirical support, is seldom discussed in medical sociology or in health services research. It is included in this paper because it is an important characteristic of the informal organisation of healthcare settings that could also have important ramifications for communication and monitoring. The focus then turns to the problem of “many hands” and the diffusion of responsibility that can occur in organisations. The literature on adverse events and, more generally, on governance in healthcare settings often conveys the impression that determining who is responsible for what in organisations is unproblematic. In fact, it can often be extremely difficult to determine an individual's contribution to patient care, whether good or bad, and this paper can only begin to suggest some of the ethical implications that ensue from the diffusion of responsibility in healthcare settings. The fourth section of the paper focuses attention on the environment in which healthcare organisations operate. The organisational literature is replete with examples of how organisations set up for one purpose come to strive for other, very different, goals. It seems important to remember that hospitals and other organised healthcare settings are extremely vulnerable to wider socioeconomic and political pressures that can divert their attention to goals that are not directly related to patient care. The purpose of this section is not to make a political point but to draw attention to the fact that the sources of danger to patients are often somewhat removed from the organisation itself.

Adverse events: definitions

In medicine, nursing, and allied professions, “adverse events” usually refer to incidents such as medication errors, surgical accidents or, in psychiatry, to harm inflicted by a patient on themselves or others. There are a number of useful definitions in the literature. The Harvard medical practice study defined adverse events as “an unintended injury caused by medical management rather than disease process. The injury is sufficiently serious to lead to prolongation of hospitalisation or temporary or permanent impairment or disability in the patient”.¹⁷ A recent article in this journal by Walshe¹⁸ listed several alternatives and proposed another: “an untoward or undesirable occurrence in the healthcare process, which has or potentially has some negative impact on a patient or patients and results or may result from some part of the heathcare process”. Walshe considers that any definition must incorporate three key elements—namely, the negative nature of the event; the fact that it impacts on the patient, and that its origins be traceable to some part of the healthcare process.¹⁸

Adverse events are particularly important in medical settings because they break the time honoured injunction that medicine should “first do no harm”. The Church of England Book of Common Prayer distinguishes between “sins of omission” and “sins of commission”. Adverse events related to the former would include neglect or negligence leading to malnutrition, decubitus ulcers, hospital acquired infections, or suicide. Events that are traceable to specific interventions, such as the maladministration of medication or poor surgical technique, would fall into the latter category. This distinction is potentially important because individuals and organisations may respond quite differently to the two types, whereas a focus on quality demands that both are included. There may be some room in the current literature for a more general definition than those that are currently used as a way of reinforcing the link between the study of adverse events and quality efforts more generally. The danger of a narrow definition is that we come to focus exclusively on the most visible, discrete, and easily measured events risking “a rather biased view of quality, focused on outlier events, technical quality, and patient safety issues.”¹⁸ It may be helpful therefore to think of an adverse event as any occurrence that diminishes quality of care or that is inconsistent with the stated goal of health care organisations, which is to cure or alleviate health problems and to promote health.

The division of labour and “structural secrecy” in complex organisations

Before the Industrial Revolution almost all aspects of social life were organised around a few organisations—the church, the manor, and the guild. Those organisations that did exist had a relatively simple structure with little elaboration or differentiation of social roles. Now organisations are not only ubiquitous in social life, they are complex in structure and proliferate social roles and positions. Modern health care, scientifically and technologically sophisticated, demands the existence of complex organisations to coordinate the activities of the many individuals required to contribute specialist expertise.

The division of labour that results from increased specialisation brings problems of coordination, communication, and cooperation. It is no longer possible for one person to hold all the specialist knowledge needed to treat patients. The members of the health care delivery team, who are often educated separately and have little informal communication,¹⁹ may have only a limited understanding of each other's role. This is not just because of the increasing scientific and technical sophistication of medical care, but also because of increasing specialisation of the occupations and professions involved in health care. The nursing profession, for example, has become increasingly specialised over time and is now recognised as encompassing a large number of groups, each of which is in possession of specialist expertise. Nurses who work in intensive care, psychiatry, or in the community are no longer interchangeable—a “nurse” is no longer just a “nurse”. At the same time, the role of the nurse in the healthcare team has become more specialised. Whereas, in the past, nurses were responsible for diverse aspects of patient care, their role is now more clearly defined. Nurses no longer have responsibility for ward cleaning and other ancillary services such as patient transport which could have led to a fragmentation of care and a decline in its quality. Thus, the more specialised occupations become, the more room there is for error unless systems for coordination, communication, and cooperation are functioning well. Vaughan, in a study of the Challenger launch decision, coined the phrase “structural secrecy” to describe the compartmentalisation of knowledge and information that follows from the division of labour in complex organisations.⁵ She identified the following implications of structural secrecy:

• information and knowledge will always be partial and incomplete;

• the potential for things to go wrong increases when tasks or information cross internal boundaries;

• segregated knowledge minimises the ability to detect and stave off activities that deviate from normative standards and expectations.

Structural secrecy emphasises the importance of communication in formal organisations. If no one individual or group is in possession of full knowledge about the way an organisation works or has complete information about its activities, then more emphasis needs to be placed on formal mechanisms of communication. Structural secrecy is also an argument for simplifying organisational structures and processes.²⁰ For example, the fewer people or departments involved in care the better because the process of transfer of care introduces a known source of danger, as noted above. If, for example, patient safety was the only consideration in decisions about the organisation of nursing care, primary nursing would be preferable to team nursing because it minimises the number of people involved and hence the need for communication.

In addition to simplifying the number of people and departments involved, standardisation and formalisation of tasks are ways of reducing the complexity of work in formal organisations. In hospitals standardisation means, for example, that the layout of each ward would be basically the same, procedures would be conducted in the same way across units, and there would be a minimal amount of variation in equipment throughout the organisation. Standardisation is particularly important if the organisation has a high turnover of staff or relies heavily on bank or agency nurses because it minimises the amount of time wasted on dealing with new or different ways of working. Studying the patterns of accidents and “near misses” will reveal further ways to simplify and systematise tasks that could be implemented easily and effectively.

Formalisation, which refers to the extent to which roles, rules, and procedures are codified and applied within an organisation, can also be seen as a way of simplifying tasks by minimising the amount of initiative required of individuals. However, formalisation can have negative as well as positive effects. Rules, standard operating procedures, guidelines, protocols, and role specifications cannot cover all eventualities and, unless deliberate steps are taken to review and revise, they will soon become out of date. Vaughan cites “… recency, perceived relevance, complexity, vagueness, and/or acceptability” as factors related to formalisation that have been implicated in the past with the systematic production of organisational deviance.¹³ A tendency towards formalisation is shown in the growth of the “guidelines movement” in the healthcare systems of many industrial nations where explicit protocols for treating particularly common diseases such as asthma, diabetes, and back pain have become increasingly visible components of “evidence-based care”. Guidelines and protocols are seen as important tools in quality improvement. If, as Vaughan suggests, formalisation also carries known dangers, it would seem wise to take these into account. The idea that there are risks associated with increasing formalisation is therefore an important caution and suggests that the impact of guidelines and other trends towards formalisation should be subject to evaluation and monitoring over time to ensure that the costs do not outweigh the benefits.

The homophily principle and social structural barriers to communication

In healthcare organisations barriers to communication are erected by the hierarchical nature of hospital organisations, by the importance of professional allegiances, and by the gendered nature of work in healthcare settings. A study of the informal networks of clinical directors and directors of nursing in acute NHS trusts showed that nurses and doctors rarely discussed important professional matters informally with each other.¹⁹ Most discussion partners were drawn from members of their own professional group and there was a marked preference for discussion partners of each person's own sex. The tendency to form relationships with other people who are like ourselves in salient dimensions is known as the homophily principle. Social network analysis has also revealed the tendency, less marked among nurses, for professionals' discussion partners to know each other.²¹ Doctors' social networks were particularly “dense”, which means that their informal social networks were not only dominated by other doctors, but by doctors who were “close” to each other and in frequent, often daily, contact. The density of these social networks suggests that there is still a very strong professional boundary around medicine. If these findings are generalisable, senior doctors in the NHS might be described as “socially isolated” because their pattern of informal relations is so confined to a male, middle class, professional region of social space. An alternative description is that they are “socially insulated” because they have so little contact with people who differ from themselves on important social dimensions such as race, age, income, education, and professional background. This means that when senior doctors in the NHS and, to a lesser extent, senior nurses discuss important professional matters in informal settings and with discussion partners of their own choosing they are unlikely to be confronted by ideas or interpretations of the world that differ markedly from their own. At least as far as informal communication goes, there appear to be cliques within hospitals among whom informal communication is frequent but there are also structural barriers based on hierarchy, profession, and sex which make some kinds of relationships unlikely. These boundaries, around medicine in particular, could be a barrier to communication with, and monitoring by, other professional groups.

One of the advantages of an organisation compared with individuals in promoting safety is the ability to design a system of “checks and balances” so that important actions are not the sole responsibility of a single individual. For example, powerful drugs are always checked by at least two people before they are administered to the patient. Again, in surgery someone has responsibility for checking that the right number of swabs and instruments are retrieved from the patients' body before the operation is completed. Such organisational arrangements only work, however, if there are few or no social structural barriers to communication between the parties involved. If a junior nurse feels unable to tell her senior partner that she has just drawn up the wrong dose of medication or is about to remove the wrong drain, then the mechanism for ensuring patient safety will break down.

Adverse events can happen simply because individuals of lower status experience difficulties in challenging the decision of a person of higher status. Perhaps the most powerful examples of this phenomenon come from flight recorders that reveal the vain attempts of a co-pilot to get the attention of a pilot or to divert them from a course of action that they realise will cause the death of everyone on the flight.⁷ One sociological study described and analysed the dialogue between pilots, controllers, and cockpit crews in two jumbo jets in fog over Tenerife where the barriers to communication of status, language, and task assignment were contributing factors in a crash in which hundreds of people died.⁸ Closer to home, the following quote from a paper on risks in obstetrics suggests that social dynamics may be implicated in medical accidents as well: “An experienced midwife may have to call an inexperienced doctor … a midwife may be frustrated by a doctor who does not respond appropriately to her concerns: yet she may be reluctant to ‘go over the head’ of a junior doctor to a more senior doctor”.²² Further analysis of the dynamics of interprofessional relations using concepts such as status, boundaries, and emotional labour might yield useful insights into the invisible barriers that divide members of the multidisciplinary team.

These examples suggest at least one testable hypothesis-—namely, that organisations in which status distinctions are marked will have a greater number of accidents because of the failure of junior members of a team to question or check the behaviour of senior members. Conversely, hospitals with flatter organisational structures, some blurring of professional boundaries, and where segregation of sexes is less marked are likely to report more “near misses” because lower status colleagues are better able to challenge the actions of their seniors.

Diffusion of responsibility

The story of Kitty Genovese who was murdered in New York in 1964 led to great public outrage and concern about changes in the nature of modern urban society that appear to discourage people from responding to cries for help. In social psychology, experiments devised by Latane and colleagues in the wake of this case showed that people are much more likely to intervene if they are alone rather than with someone else, especially if that person is a stranger. Social psychologists responsible for the “unresponsive bystander” experiments reasoned that the presence of a stranger weakened individual responses by diffusing their sense of responsibility.²³ In later laboratory studies Latane et al sought ways of intensifying individual responsibility and found that, if individuals have their efforts identified when they are part of a cohesive, high morale group, they will exert even more effort than they would if they were only working for their own personal benefit. However, if team members are not identified, individuals will tend to “loaf”—that is, they will not produce their best effort.^{24 25} This suggests that, while we are all capable of self-interested behaviour (in this case, lying back and letting others do the work), we are also capable of showing “team spirit” depending on slight but important modifications to the experimental conditions. The crucial variable seems to be the extent to which our contribution is visible to the rest of the team.

The recognition of individual contributions appears to be one way of motivating and encouraging individual clinicians and managers to contribute the most that they can to improving the quality of patient care. This research is also important because of its roots in and relevance to the problem of the “unresponsive bystander”. As the events at Bristol Royal Infirmary have demonstrated, it is crucial that individuals who become aware of problems relating to patient care take action as early as possible. Individuals may fail to act as “whistle blowers” because they fear the consequences, but also because they simply expect that someone else will take the appropriate steps to avert the potential tragedy. The idea that “if I don't do it somebody else will” shifts responsibility from the individual onto some unspecified “other”. The social psychological research noted above would suggest that the more we can disentangle the actions of individuals from the organisation in which they are embedded, the better able we will be to encourage their maximum contribution to the group, including alerting others when things are starting to go wrong.

However, this is not as simple as it sounds. The “problem of many hands” means that it can be extraordinarily difficult to discover who is responsible for what in a large organisation.²⁶ One philosopher notes that “With respect to complex organisations, the problem of many hands often turns the quest for responsibility into a quest for the Holy Grail”.²⁷ In many cases we simply cannot isolate individual contributions to organisational action. This suggests not only that we lack some of the basic incentives that could be used to increase individual effort in pursuit of quality, but that the ability to achieve justice in organisations is compromised. Research on decision making shows that some layers of the organisational hierarchy are responsible for decisions that are more visible, concrete, limited in time, and identifiable with specific individuals than are others.

Mistakes are often associated with a particular decision—for example, the decision to prescribe a certain medication, to remove a particular drain, or to decrease the intensity of observation of a psychiatric patient. If these decisions are followed by adverse events it usually seems that the decision maker was “at fault”. Greater understanding about the nature of decision making in complex organisations undermines such simple thinking. The legal literature has long been aware that people at the top and bottom of organisations tend not to be blamed when accidents happen.²⁸ Attention tends to focus on people in the middle—that is, those who are sufficiently senior to make important and visible decisions but insufficiently senior to be cloaked by the diffusion of responsibility that lies over senior managers.

Recent interviews with clinicians and managers in acute trusts suggests that decision making patterns in hospitals are similar to other organisations in this respect.²⁹ Whereas at operational levels decisions tended to be highly visible and to have a recognised beginning, middle and end, at higher levels decisions were much less clear cut. Doctors working as clinical directors, nurses in ward manager positions, and managers in divisions and directorates were well able to describe the kinds of decisions for which they were primarily responsible but senior managers were not. Senior managers described a decision making process which was fundamentally “evolutionary” and consensual, where negotiations continued with many individuals and groups over long periods of time.

The distinction between active and latent failures is relevant here.³⁰ The former are defined as “unsafe acts (errors and violations) committed by those at the ‘sharp end’ of the system … which can have immediate adverse consequences”. Latent failures, on the other hand, are “created as the result of decisions taken at the higher echelons of an organisation … their damaging consequences may lie dormant for a long time, only becoming evident when they combine with local triggering factors … to breach the system's defences.” These distinctions are important because they capture the temporal and status dimensions associated with different kinds of failures. The work on decision making draws out these distinctions and emphasises differences in the processes of decision making such as visibility and identifiability. These differences will become increasingly important as the policy of clinical governance is implemented in the NHS.³¹ In health care we are acutely aware of the behaviour of individual decision makers³² but we often fail to follow the causal chain back to the managers, civil servants, or politicians who may have failed in repeated decisions over many years to provide an environment conducive to patient safety.

Goal displacement

Following the famous study of the Tennessee Valley Authority by Selznick,³³ a stream of research has described how the original goals of an organisation can be changed, subverted, or undermined. These changes are often caused by environmental pressures to which the organisation has to respond, sometimes in order to survive. The legitimacy of healthcare organisations is at least partly dependent on their espousal of altruistic goals—to care for and, if possible, to cure patients of discomfort, disability, and disease. The greatest threat to these goals is the necessity for organisations in a capitalist system to remain economically viable. Financial goals vary for different healthcare organisations depending on the nature of the system to which they belong, as well as over time. The economic pressures on healthcare organisations are probably quite different in the USA and UK, for example, and we have some empirical evidence that the importance of economic goals has varied over time in the UK.

In qualitative research on clinical effectiveness and clinical management conducted in 1995³⁴ and 1996,³⁵ many participants voiced the opinion that economic goals had assumed too great an importance in the NHS. Some examples include:

“I don't feel that there is a top-down drive for clinical standards. The focus is on things like waiting times. Middle and upper management don't have a clinical background and don't care about standards.” (Clinical Director) “The public service ethic is still there—it is kept alive by the professions while the organisation has been diverted into business activity.” (Academic) “It's a confused culture at the moment. Dominant requirements of finance and quantitative performance—brute managerialism—but there is a growing recognition that this is demotivating.” (Chief Executive)

Although the current UK government has given clear messages that quality of care takes priority over other goals, at the same time this goal is subverted by the patent inadequacy of funding provided for the NHS. The most egregious example of the connection between resources and risks in the recent past was the use of junior doctors. A study conducted in 1992 stated that “… there is evidence that the existing system reduces quality of care, principally through mistakes associated with inadequate supervision and lowered humanity of care due to tiredness”.³⁶ These issues are now being tackled, but the argument that there is a connection between adequacy of resource and mistakes endures. Similar messages emerge from nursing research. For example, a recent study of over 500 acute hospitals in the USA found an inverse relationship between the number of full time registered nurses and adverse events such as infections following surgery.³⁷ More generally, the connection between resources and risk has been established in the literature on high reliability organisations. Among the many characteristics shared by high reliability organisations is the fact that they “enjoy abundant resources so that short term efficiencies can be neglected in favour of reliable operation”.³⁸ Although such abundance may never be possible in the context of the NHS, the difficulties involved in meeting all current demands on the system when UK spending on health care remains so low in comparison with other countries needs to be acknowledged. If top managers set goals that cannot be met within current resources, they are setting up individual clinicians, teams, and organisations for failure of all different kinds, including causing harm to patients.

Some tentative hypotheses

Some of the ideas mentioned above could be turned into testable hypotheses. A few suggestions, designed to stimulate thinking and to encourage empirical tests of the usefulness of sociological explanations of the causes of adverse events, are listed in box 1.

Conclusions

Donald Berwick²⁰ recently stated that “organisations need sound scientifically grounded theories about errors and safety”. A large number of disciplines can contribute to this goal. This paper draws on ideas, concepts, and mechanisms from the sociology of organisations where there is a long tradition of studying mistake, misconduct, and disaster—“the dark side of organisations”.¹³ I have outlined a number of ways that we could use some sociological concepts and mechanisms in the construction of empirical tests of strategies to decrease errors and accidents and to increase patient safety in healthcare settings. These ideas are meant to be suggestive rather than comprehensive because the literature in this area is rich and detailed. However, it seems clear that further empirical work on the effect of system simplicity, standardisation and formalisation—as well as status differences, role boundaries, job design, and environmental pressures—on rates of adverse events would be theoretically justified. Further study in this area will undoubtedly raise philosophical issues about the nature of responsibility and accountability in hospitals, particularly in the era of clinical governance. The study of adverse events is extremely important because it combines compassion for patients with concern for NHS employees, recognising that, at a deep level, these two are inextricably linked.