In the last decade there has been a growing awareness that errors and failures in the delivery of health care are far more common and serious than the recipients, providers, and funders of health care would like.^1–3 It seems that, on average, the most appropriate health care is delivered not much more than half the time,^4,5 and that at least one in 10 admissions to acute care hospitals is associated with some sort of iatrogenic harm.^{6,7,8,9,10,11,12} Although there has been debate about the accuracy of these estimates,¹³ observational studies have confirmed that things do go wrong with alarming frequency.^14,15 There is widespread confusion, with some health services having patient safety officers, risk managers, quality improvement practitioners, a nosocomial infection unit, epidemiologists, and more, with as many as 15 different reporting and/or management systems all doing essentially the same thing but using different paradigms.

There is no single source of information that can provide a comprehensive picture of the safety and quality of health care, and existing sources—many rich in information—are not systematically exploited to provide structured information that is useful for finding out about what is actually being done or how and why things go wrong.^16,17 Both are necessary if we are to move towards delivering “best practice” health care and devise effective preventive and corrective strategies for things that go wrong.

An integrated framework is needed that can operate across the entire spectrum of health care from local to national, and a full range of administrative arrangements. Such a framework, integrating safety, quality and risk management, is presented in fig 1. It includes the conventional medical record and ancillary information about patients, investigations and procedures, a system for logging, managing and monitoring progress when things go wrong, a data repository for collating information from all available sources, and a risk management framework underpinning both proactive and reactive responses. Central to this is a comprehensive universal classification supported by a system for eliciting, capturing, classifying, and analysing the information needed to improve the safety and quality of health care.

Safety is just one of the dimensions of the quality of health care, with access, timeliness, efficacy, efficiency, appropriateness and acceptability. Safety cannot be considered in isolation as resources spent on safety cannot be spent on other aspects of quality. Although some of the activities and information sources in fig 1 are useful for some of the other aspects of quality, the discussion in this paper will be directed towards safety and things that go wrong.

We have previously identified the need for “an international patient safety reference group to align terminology, tools and classification systems and to promote the rapid dissemination of strategies that prove to be successful”.¹⁶ Also needed is the ability to aggregate large amounts of information and compare patterns and trends over time and between individuals, organizations and countries, so that detailed pictures can be obtained of the individually rare but collectively important problems that make up the bulk of the things that go wrong.¹⁸

A platform from which to do this was established with the launch of the World Alliance for Patient Safety in October 2004. Two of its six early initiatives are relevant to this paper, “Developing a patient safety taxonomy” and “Reporting and learning to improve patient safety”.¹⁹ While it is essential for a classification and reporting and learning systems to be able to stand alone and function locally, we will propose here that the future lies in an integrated approach. Such an approach is shown in fig 1. The salient features of this figure are referred to below in bold text. Each of these represents an important activity but, as discussion about most is beyond the scope of this paper, a few key references have been provided.

At the apex of the pyramid in fig 1 an asterisk represents an intervention (diagnostic or therapeutic) or an incident (an event or circumstance that could have or did harm anyone, or cause loss or damage).²⁰ The first thing to do is to respond, and then record what happened and/or was done.

Records: The causes of the underlying diseases or injuries may be coded on discharge or death using the International Classification of Diseases (ICD),²¹ and procedures may be coded using the International Classification for Healthcare Interventions (ICHI).²² Subsets may be aggregated for special purposes. For example, identifying “diagnostic related groups” for reimbursement purposes or extracting information for registers (such as for cardiothoracic procedures or renal transplantation).²³ Information may also be extracted for indicators,²⁴audits,²⁵ and reviews²⁶ of activity, morbidity and mortality. Subsets of laboratory and other results can be studied for activities such as the surveillance and tracking of nosocomial infection,²⁷ and triggers may be used to identify certain types of problems.²⁸

If an incident has resulted in harm, those affected (patients, staff, and their families and friends) should be informed of the facts (disclose),²⁹ and arrangements made to look after them (support).³⁰

Basic data: For all significant or informative incidents, and especially when harm has occurred, basic data (what, who, when, where, risk, consequences) should be logged so that it can be passed on to the relevant people (notify), so that they can recommend and/or take the necessary local action. This process should also elicit sufficient information to generate a risk matrix or assessment to determine whether further investigation and/or remedial steps are needed.³¹ The kinds of people who should be notified are shown in table 1.

In most jurisdictions there are regulations which safeguard the privacy of patients, and a good general rule is that identifiers should not be passed on to anyone unless consent has been obtained or this is necessary for the immediate care of the person affected.³²

A process should be undertaken to determine if there is culpability and, if there is, an enquiry should be commissioned and appropriate action taken (including informing those affected about this process and its outcome).^33,34 Other processes may be invoked such as a complaint,³⁵medicolegal³⁶ or Coroners’ report,³⁷ or a report to a drug or device agency.^38–41 If the event has caused harm or a recurrence would pose a significant risk, details should be elicited, classified and stored, using a universal patient safety classification. Details may be elicited by a reporter going “online”, by a patient safety officer (PSO) or call centre; high risk incidents should be subjected to a root cause analysis.^42,43 Details of the universal classification will be discussed below. Knowledge from all available sources, including the literature,surveys and observation studies, and evidence from all levels including consensus meetings, guidelines and protocols make up the patient safety data repository. This information can then be used to understand the context, identify the risks, analyse the risks, and evaluate the risks in order to treat or managetherisks.³¹ Risks that can be dealt with should be subjected to a quality improvement cycle (plan-do-study-act),⁴⁴ and those that cannot should be placed on a risk register³¹ for future attention, and accepted and/or indemnified against. Socio-technical probabilistic risk analysis (ST-PRA)⁴⁵ and failure mode and effects analysis (FMEA)⁴⁶ represent proactive approaches to identifying problems and setting priorities. These quality and safety activities may all contribute to “quadruple-loop” learning shown on the side of the pyramid in fig 1(1, 2, 3, 4).⁴⁷ This will be discussed later; an example is given in box 1 at the end of the paper.

The shaded areas in fig 1 represent activities which are often managed in a piecemeal manner and which can currently be handled using an integrated system. We will outline the desirable attributes of such an integrated solution from various perspectives, and then describe the patient safety classification and Advanced Incident Management System (AIMS) we have developed as examples of components of this, with a brief account of how this has driven system-wide quadruple-loop learning.

DESIRABLE ATTRIBUTES OF AN INTEGRATED SYSTEM

The system should be accessible, useful and useable.^16,17 Experience over two decades has shown that the expectations of users increase rapidly and that simply collecting basic information is soon deemed inadequate. Some underlying principles are shown in table 2. The system should be built on experience from other high risk industries (including the underlying information model), and have broad applications with respect to use, users, and scope. It should interface with and complement existing systems, classifications, and sources of information.

The system should be able to be used for managing incidents at a local level and accommodate local terminology, language, legal, ethical and privacy requirements, but the information gathered should also be able to be used at state, national, and international levels (table 3). It should be able to interface with existing local systems and be applicable across all areas of health care.

There is also a set of requirements for those who have to run or administer the system (table 4). It should be able to be used reliably in a manner that is safe and secure by having security and data access controls, audit trails, and software prompts for complying with ethical, legal and privacy requirements, and have mechanisms for support and maintenance with robust arrangements for troubleshooting and updating the system.

Attributes important to users are listed in table 5. These include accessibility and ease of use, not being burdened with redundant views or information, being able to use the system to a level of detail appropriate to the incident in question, and the ability to track the progress of an incident report and any action taken.

Finally, there are attributes which are important for data extraction, analysis and reporting which include the secure but seamless exchange of de-identified information, statistical analyses, the ability to track trends and patterns, and standard as well as customised reports (table 6).

A COMPREHENSIVE UNIVERSAL PATIENT CLASSIFICATION

The need for a comprehensive universal patient safety classification with agreed definitions for the concepts which populate it, and preferred terms for these concepts, has been identified by ourselves and others.^16,17 The Australian Patient Safety Foundation has been developing such a classification since 1988.⁴⁸ From 1988 to 1993 several thousand “key words” were assembled and used in the analysis of the first 2000 incidents reported to an Australian anaesthesia based incident reporting system.⁴⁹ From 1993 the concepts of natural categories and natural mapping were incorporated into a hierarchical system called the Generic Occurrence Classification (GOC) which had some 12 400 “natural categories”.⁵⁰ This was developed to explain the apparent fivefold difference in the rate of adverse events in the US and Australia, when studies in both countries had used ostensibly identical methods.^6,7,18 It was successful in doing this by showing that the numbers and types of serious events were virtually identical in the two countries, with the discrepancy being made up by many minor or late complications which were apparently not counted by the US reviewers. However, the GOC had structural limitations and analysis was labour intensive.

An architecture was needed which could elicit patient safety information about any area of health care from a wide range of sources, and translate it into a common language so that electronic records could be created which could be compared with each other and analysed as part of a larger set of data. A classification was needed to “de-construct” the information in a way that would facilitate subsequent analysis and learning. An underlying information model for such a classification was developed, the Generic Reference Model (GRM), which is based on the model of complex system failure described by Rasmussen⁵¹ and Reason.⁵² It provides a structured approach to drawing out all the relevant information about an incident and underpins the overall process of collecting and classifying information.

The GRM is shown in fig 2. It was developed to provide a framework to define the relationships between components (classes) of the classification system and the terms which are used to describe the attributes of each of these components (the concepts).⁵³

There are five levels of information underlying the GRM:

• information sources;

• incident types;

• components (data elements, classes);

• attributes (concepts); and

• terms.

Information sources

The GRM can accommodate information that has been obtained from all available sources including incident reports, complaints, medicolegal cases, root cause analyses, Coroners’ investigations, enquiries, and any other account of an event or circumstance relevant to patient safety from any area of health care.¹⁶ Root cause analysis is particularly useful in that an important component of participant and organizational learning occurs just from the process of performing root cause analyses.⁵⁴

Healthcare incident types

Incidents are divided into types and the GRM is used to derive a “specific incident reference model” for each healthcare incident type (HIT) (see table 7 for a list of the basic and some of the specialty HITs that have been developed; more are planned).

Each incident may be classified using one or more HIT. For example, an overdose of morphine in an intensive care patient because of malfunction of an infusion pump would be classified using medication, therapeutic device and intensive care HITs. When an incident is classified using more than one incident type, the user needs to identify the “principal incident type” in order to allow the production of “enumerative” reports.

Each HIT has its own components, such as contributing factors, preventive factors and mitigating factors. For example, in the therapeutic device HIT indicated above (pump malfunction), contributing factors might have been a design fault in an infusion pump as well as a problem resulting from it being set up by an inexperienced nurse, whereas a poor outcome from the medication problem (morphine overdose) may have been due to a shortage of staff. If two or more incidents are associated with each other, they may be linked.

Components

Each specific incident reference model includes most, but not necessarily all, of the components of the GRM. Each component is represented by a box in fig 2. The specific incident reference models constitute the underlying information models for each incident type. Some components or concepts have attributes specific to each incident type, such as “what happened?”

Attributes

To characterize the attributes of the components of the reference model which are relevant to a particular incident type, each represented by a concept, the classification process prompts classifiers to select the attributes of an incident by responding to cascades of relevant and intuitively arranged “plain language” questions. The answers (concepts) are labelled by preferred “terms”. These can vary from area to area or from country to country. For example, the mobile bed-like device upon which patients are placed for transport around a hospital is variously called a “trolley”, a “stretcher”, a “barouche”, or a “gurney” in different jurisdictions. Further terms would obviously be used in different languages. Each concept is labelled by an alphanumeric code, making the system language independent as the underlying concepts remain the same.

Terms

The terms (representing concepts) which are presented to the classifier as possible answers to classification questions were collected from experience of classifying over 100 000 incidents, from the original GOC, and from searches of relevant medical literature and include certain “controlled vocabularies” such as the ECRI list of equipment and devices.⁵⁵ These terms may be reviewed, revised, and expanded as necessary. There are currently some 20 000 terms.

ADVANCED INCIDENT MANAGEMENT SYSTEM

The Advanced Incident Management System (AIMS) system is briefly presented here as an example of an attempt to meet the needs identified in tables 2–6 and is currently being redesigned to be a browser based system. It has evolved progressively in response to feedback from users, and is undergoing progression evaluation with respect to validity and reliability. The system provides software to facilitate the functions and activities shown in the shaded areas of fig 1; basic data may be entered and classified into a customisable electronic form, the relevant people notified (table 1), and the local actions recommended and taken can be logged. This may be by a person other than the person who entered the initial data. Risk assessment is presented as a score in a 5×5 matrix based on severity and likelihood of recurrence.^31,42 If the risk is low and the incident is of local relevance only, this may be all that needs to be done. However, if the risk is high or someone has been harmed, a patient safety officer or line manager may decide to elicit more information and classify the incident in detail. At this stage culpability should be ruled out and the necessary steps taken to ensure statutory immunity, if this is available.^33,34 Eliciting detailed information and classifying it using the classification may be done by a reporter directly interfacing with the AIMS software or, more usually, by communication with a patient safety officer or call centre operator who has access to the software at the time of the interview. The essence of eliciting and classifying the information is captured in using the HIT screens.

Using healthcare incident type screens

During basic data entry the relevant HITs would have been chosen, which may be accessed in turn. Each HIT screen elicits comprehensive information which is specifically related to the type of incident, and constitutes an enormous electronic form. The intuitive cascading question-answer format makes the classification process easy, reducing the amount of training required for classifiers and increasing the consistency of classification. For each HIT screen, classifiers are presented with a series of plain language questions. For each question, the classifier is offered a choice of plain language terms as answers. A definition of the concept underlying each term (with an example, if necessary) appears on the screen for each question. In this way the relevant information about the particular incident is drawn out or elicited. Some answers may trigger cascades of further questions to obtain more detailed information (fig 3). This process is designed to extract all the relevant features of an incident by eliciting all the attributes of each component shown in fig 2. Where there are insufficient choices to do this, the system always offers the option of using “free narrative”. An example of a small segment of some queries with a set of answers is shown in fig 3.

The AIMS suite of software tools

The current iteration of the system is made up of a suite of tools shown in fig 4. The six modules which make up the AIMS suite of software tools are:

• Data Manager

• Workflow

• Analyser

• Administrator

• Database Administrator

• Risk Register

Data Manager accommodates data entry, data management, and classification tools that allow records of incidents or events from any source to be created and managed. It elicits information by asking a series of questions which, when answered, is stored in the appropriate place in the patient safety classification, in a database from which it can subsequently be retrieved. It is also the portal to the workflow and risk register modules. Data manager can copy, edit, and link incidents and notify people via email about workflow tasks.

The Workflow module allows electronic notification of basic data to all relevant personnel within an organization and allows workflows (projects) to be designed and set up which consist of a series of tasks which can be assigned to staff to complete within an allocated timeframe. Root cause analyses can be managed using workflow. Progress can be monitored and escalating reminders can be set up so that, when a task has been overdue for a pre-determined time, additional people can be notified.

Analyser accommodates analysis and reporting tools. Standard records can be generated to compare information over time and to compare between units or organizations. Incident patterns which vary from the norm can be detected, which may trigger investigations and/or remedial action where appropriate. Changes in patterns can also be documented when interventions have been successfully implemented, together with a display of any new sets of problems which might have emerged as a result of implementation. Comparisons can be generated at a variety of levels from unit or ward up to national or international levels. It is possible to design reports to search on specific terms or on combinations of terms by organization(s) or by date(s). The capacity to “slice and dice” the data supports a thorough analysis. Each incident making up a set can be examined and/or printed (“drill down”).

Administrator is a tool for configuring and managing the AIMS system within a jurisdiction or organization. The first task is to build and maintain the organizational hierarchy (tree) for reporting by location of incident. It is necessary to determine who has permission to access the software suite and what each user can see and do (security and user access). It allows certain data fields to be set as “identified” or “de-identified”. An incident comparison can be entered and customised user defined fields can be created. It can also be used to set up notifications for AIMS users, and to set up special projects to allow staff involved in particular projects to have access to certain relevant information over and above the incidents their normal security permissions allow them to access. Audit trails can be viewed allowing user verification and reconstruction of how, when, and by whom the information was entered. Administrator can also be used to amend and update HITs.

The Database Administrator allows configuration and maintenance of the database and updates to be downloaded and applied to the AIMS software suite. De-identified classified data can also be uploaded (imported) to a central agent for incorporation on national or international databases, and data can be exported. Such data are de-identified and encrypted before transmission.

Risk Register: risks can be entered with their controls and treatments allowing managers to view the overall exposure to risks that have been identified. These can be assessed with respect to the impact they may have on the ability to deliver objectives, products, and services and allow regular audits to monitor current exposure to risk.

Current AIMS usage

AIMS is currently in use across the universal public health system in five of the eight states and territories of Australia (accounting for some 60% of the population of Australia), with additional sites in other states and in New Zealand, and a pilot site in the USA. AIMS is being used for research purposes in other countries and may be configured to capture data from a number of sources at a number of levels of functionality (fig 5). In Australia between 10 000 and 20 000 incidents are reported per million population each year; there are also speciality-specific collections of over 10 000 incidents in areas such as anaesthesia and intensive care. What matters, of course, is whether lessons are being learnt and applied as a result of its use.

QUADRUPLE-LOOP LEARNING

The notion of single, double and triple-loop learning has been described by Argyris and Schon⁵⁶ and by Parker,⁵⁷ and the concept extended to quadruple-loop learning by Braithwaite et al.⁴⁷ Single-loop learning occurs when clinicians or local managers learn from experience and take their own initiatives to follow up problems that have occurred, usually within existing paradigms and rules. This is represented in AIMS by the responses to individual incidents or problems. Double-loop learning occurs when the lessons learnt from such events are taken up by managers at a departmental or institutional level and change the rules or preferences; this may be prompted by a cluster of incidents with similar contributing factors. For example, many problems have been shown to occur because of the availability of a wide range of infusion pumps and many institutions have now put into place systematic processes for choosing, purchasing, and maintaining infusion pumps, with better protocols for their use and credentialling of users.⁵⁸ Triple-loop learning occurs when a regulator such as a health department, specialist college, or accreditation agency learns from double-loop experiences and disseminates guidelines or changes practice at a jurisdictional or national level. Quadruple-loop learning occurs when these lessons are disseminated internationally and taken up or incorporated into standards. The AIMS system has been set up so that there can be as many learning loops as the organizational structure allows. For example, in Western Australia there are 14 levels from sub-ward level to state level and data can be aggregated at each of these. Single-loop learning is documented by the AIMS system tens of thousands of times a year in Australia. Examples of double and triple-loop learning are provided by regular feedback, newsletters, and published articles; over 110 papers have been published using AIMS data.^59,60 An example of quadruple-loop learning driven by AIMS data over a decade ago is shown in box 1.

WHERE TO NOW?

It has now been recognized that systematic efforts will be required to “get it right” in health care¹⁹ and that information will need to be collected from many sources.^16,69 However, efforts are fragmented and are not underpinned by a coherent body of relevant information. Information currently has to be gathered “manually” from a wide range of sources.⁷⁰ Reporting systems which cannot capture detailed information or contribute usefully to a detailed database are of low value and have poor “buy in”.⁷¹ With the launch of the World Alliance for Patient Safety¹⁹ there is an opportunity to develop high value systems with the attributes listed in tables 2–6 within an integrated framework for safety, quality, and risk management. This will require appropriate attention to meeting requirements for interoperability and electronic messaging. It will also require the development of a universal patient safety classification which can be integrated into existing and new information systems. This must be accessible, usable, and useful, and should have the capacity to interactively elicit complete sets of information about all the components of the underlying information model. It must allow aggregation of information at many levels to support quadruple-loop learning from the bedside to international level. The internal structural properties and data relationships must be developed so that the complete package is future-proofed. Developments and enhancements to the classification must comply with best practice in information modelling, system design, and terminology development and management. The Australian Patient Safety Foundation has been commissioned to develop the underlying information model for an international patient safety classification, and a Delphi process seeking worldwide input from all WHO member countries for the concepts that populate such a model will have been initiated by the time this paper goes to press. There is no doubt that, with advances in health care, there will be always be new ways in which things will go wrong, but with appropriate integrated systems and mechanisms for disseminating information, a near miss in one part of the world should be able to be prevented from turning into a dreadful event in any other part of the world.