There is little information available on the proportions of types of patient safety event that are reported in general practice settings and no taxonomy that is widely used by general practitioners (GPs) to describe these events. Previous studies in primary care have provided descriptions of the relative proportions of different types of patient safety event that they have collected,1^–10 but none have been based on a representative sample of primary care clinicians contributing data, and all have used different classification methods.

In recent years, a small number of taxonomies of patient safety event related to a general practice setting have been proposed,711^–13 all based on patient safety event reports collected from small non-random samples of participants. These taxonomies have used both causative and descriptive elements of like themes to group events, sometimes referred to as “domain-specific” taxonomies.14 “Multiaxial” taxonomies capture additional elements of an event, such as harm levels, location, participants or preventability. One study has developed such a tool in a primary care setting in North America,1516 but its full details have not been published in the scientific literature. Another tool has been suggested as one that could allow comparison of safety events across disciplines, although no trial of it in a primary care setting has been described.17 There have also been some recent calls for classifications that address cognitive psychological processes,18 with one developed for general practice,14 also yet to be tested. Finally, one taxonomy based on patients’ perceptions of harms in primary care has been described, although it has a limited application in terms of categorising causes of events.19

The Threats to Australian Patient Safety (TAPS) study estimated the incidence of GP-reported patient safety events in the community, using a method based on a randomly selected representative sample of GPs.20 We found that limitations mainly relating to the internal validity of existing tools warranted further taxonomy development. We aimed to develop a taxonomy that would be comprehensible and practical for primary care clinicians to apply themselves. Here, we describe the first attempts to validate the taxonomy in terms of its reproducibility with GPs.

METHODS

The methods and definitions used in the TAPS study have been previously described. A total of 84 GPs from a random sample of 320 GPs, previously shown to be representative of 4666 GPs in New South Wales, Australia, anonymously reported “errors” (patient safety events) that they noted in their daily practice (including ambulatory clinics, hospital settings and residential aged care facilities) for a 12-month period via a secure online questionnaire.20

TAPS taxonomy development

Three GPs from the investigating team (MAM, SS and CBW) classified each report using an existing pilot taxonomy,12 and results were compared to obtain a baseline measure of reproducibility. Reports containing more than one event were given multiple codes in chronological order of events. A report required two out of three reviewers in agreement to be assigned a classification. Where three different codes had been assigned, the report was reviewed at a face to face meeting to determine its classification. The initial taxonomy was amended and retested using a quarter of the reports each time, over four sessions, to produce the TAPS taxonomy. A set of guidelines to improve coder consistency when using the taxonomy was also developed.

Agreement of investigators developing the taxonomy

Concordance amongst the coders was measured using the κ statistic, for both the initial taxonomy and the final TAPS taxonomy. Only one classification per report was included for the κ score calculations. If reports were assigned multiple classifications by one or more coders, the first classification most commonly assigned was included. The κ statistic can range from −1 (perfect disagreement) to +1 (perfect agreement). A κ score of between 0.40 and 0.75 indicates fair agreement.2122 All statistical analysis was performed using STATA version 8.0.

RESULTS

Patient safety event numbers

A total of 433 online submissions were received. Of these, 415 contained true reports, after discounting tests and reports with missing data (n = 15), and reports that investigators deemed not to describe any patient safety event (n = 3). Of the 415 reports, 320 contained one event, 82 contained two events, 11 contained three events, and 2 contained four events.

The TAPS taxonomy

The resulting taxonomy has three levels of classification. The first level (event type) relates to the underlying cause of the event, being either due to deficiencies in the process of delivering healthcare (type 1), or the knowledge and skills of health professionals (type 2). The second level has five groupings (themes) within type 1 errors, and two groupings within type 2 errors. The themes within type 1 are practice and healthcare systems, investigations, medications, non-medication treatments and communication. The type 2 themes are diagnosis and management of patient care. At the third level there are from three to nine descriptors per theme.

Table 1 shows the raw counts and proportions of total events for each category of the taxonomy. The guidelines for using the taxonomy are shown in box 1. Examples of reports describing type 1 and type 2 events are shown in boxes 2 and 3, respectively, with the codes used to describe them.

Coder agreement from pilot to TAPS taxonomy

The investigators coded the set of 433 reports using the pilot taxonomy,12 and the last 132 reports (approximately a quarter) were coded using the TAPS taxonomy. The agreement among the three coders at each level of both the pilot and TAPS taxonomies, with corresponding κ scores, are shown in table 2. At the third level of the code from the pilot to the TAPS taxonomy, the proportion of reports in which at least two of three coders agreed rose from 74% to 92%, and the κ score moved from 0.37 to 0.66.

DISCUSSION

Our proposed taxonomy builds on pilot work which has at its primary level a causative classification,1112 with subcategories based on grouping of like themes, and then addition of detail with descriptive categories in the style of a domain-specific taxonomy. Application of the taxonomy to the TAPS data shows that the majority of reports contained a single patient safety event, and the majority of events related to the processes of providing healthcare rather than deficiencies in knowledge and skills of health professionals, as postulated in previous work with non-representative samples.791112

The reporting GPs demonstrated a clear understanding of the definition of error used.20 Investigators found less than 1% of reports to contain no safety event. The proposed seven themes adequately described all reported events. The addition of “not otherwise specified” as a descriptor for each theme, unlike previous similar taxonomies,5^–7 allowed complete event coding, incorporating 13% of classified events. This compares with 2.2% of reports in one single level UK taxonomy,2 although it provided no other detail on the error type, and 20% of the “cause” of anonymous reports using a multiaxial taxonomy in one study from the USA.3

The largest proportion of events were classified as relating to “practice and healthcare systems” at the theme level (21%), consistent with American and UK studies,711 although our category is different from any in previously described taxonomies as it contains elements of the larger healthcare system rather than just administrative events. The 2002 pilot study had found a similar proportion of “office administration” events (20%).12

If the types 1 and 2 events are combined, medication errors represent the largest proportion (31%), similar to findings in earlier Australian work.34 The TAPS taxonomy allows the cause of medication errors to be considered, whether related to knowledge of their use (events 2.2.1) or to systems problems in their provision (events 1.3.1–1.3.5). This may assist in planning prevention measures, such as education for clinicians or systems changes that would reduce electronic prescription errors or dispensing mistakes.

Our purpose was to develop a tool that would be comprehensible to primary care clinicians reporting safety events. The use of investigators with a clinical background in general practice was important to produce language that would be acceptable for self-coding. One other study has asked the reporting clinicians to code the events using a simple descriptive taxonomy.7 However, the study period was brief, and the majority of reporters were reception staff, so some event types may not have been captured.

There are no studies with which we can compare our results on reproducibility of the taxonomy. The κ statistic and proportions in table 2 showed a marked improvement in agreement from the pilot to the TAPS taxonomy at all levels of the code. It is possible that a degree of improvement could have occurred through an unconscious learning of each others’ styles at the taxonomy development meetings, in addition to improved clarity of the code itself. As expected, there was less agreement among coders comparing level 1 with 2, and level 2 with 3, due to more coding choices.

At the most detailed third level of the taxonomies, complete disagreement among the coders fell from over a quarter to less than 10% of cases. In the cases where complete agreement was not reached, there was often a difficulty in interpreting the reporting language used, or a brevity of description provided in the report, requiring a degree of personal interpretation or assumption from the coding GPs. While further refinement of the taxonomy might lessen possible ambiguity, we believe that a system where the reporting clinician actually codes the event would reduce this type of error by eliminating a loss of detail in the process of describing the event and its cause to another clinician or analyst conducting the coding.

The taxonomy developed by the Applied Strategies for Improving Patient Safety (ASIPS) collaborative1516 is a multiaxial model developed in a primary care setting, using trained analysts to classify reported events. It has not been published in the peer-reviewed literature in full, however it can be viewed online.23 The TAPS electronic reporting system collects data of a similar nature to the additional axes and domains of ASIPS, including a harm scale, location check-box, event frequency scale, and details of patients such as age, gender and ethnicity.20 These elements are closed questions completed electronically by the reporter. They could be combined with reporters’ self-coding an event with the TAPS taxonomy to effectively produce a self-reported multiaxial taxonomy describing safety events.

An important limitation of our results is that they do not represent the underlying proportions of error types in the community, although the reporters were a representative sample. Some event types may have been under-reported in comparison with others, despite efforts to encourage reporting. Participants may have been unaware of deficits in their own knowledge or skills. The TAPS reports are a reflection of GPs’ experiences, and the taxonomy was created by GPs. As such, it may be limited in its application to other primary care groups. In other studies, a variety of clinicians or administrative staff have been required to report events,6714^–1624^–26 and it would be important to explore language differences that may exist. The TAPS reports and taxonomy may not have captured all error types that could be reported by a more diverse group. One other general practice taxonomy has recently been shown to be acceptable to opticians reporting safety events.27 The TAPS taxonomy may similarly have potential uses to other disciplines in a community setting. Our purpose in this paper did not include an appraisal of error types in relation to harm, preventability and demographics of patients affected by threats to safety, however this analysis is underway.

We believe that the TAPS taxonomy has the potential for use as part of an anonymous national electronic reporting system and offering guidance to policy makers in directing efforts to reduce patient safety threats in the community, particularly at a systems level. Further application of the TAPS taxonomy may also aid professional bodies developing educational tools aimed at improving the knowledge and skills of providers in primary care.