Article Text
Abstract
Objective: To develop a taxonomy describing patient safety events in general practice from reports submitted by a random representative sample of general practitioners (GPs), and to determine proportions of reported event types.
Design: 433 reports received by the Threats to Australian Patient Safety (TAPS) study were analysed by three investigating GPs, classifying event types contained. Agreement between investigators was recorded as the taxonomy developed.
Setting and participants: 84 volunteers from a random sample of 320 GPs, previously shown to be representative of 4666 GPs in New South Wales, Australia.
Main outcome measures: Taxonomy, agreement of investigators coding, proportions of error types.
Results: A three-level taxonomy resulted. At the first level, errors relating to the processes of healthcare (type 1; n = 365 (69.5%)) were more common than those relating to deficiencies in the knowledge and skills of health professionals (type 2; n = 160 (30.5%)). At the second level, five type 1 themes were identified: healthcare systems (n = 112 (21.3%)); investigations (n = 65 (12.4%)); medications (n = 107 (20.4%)); other treatments (n = 13 (2.5%)); and communication (n = 68 (12.9%)). Two type 2 themes were identified: diagnosis (n = 62 (11.8%)) and management (n = 98 (18.7%)). The third level comprised 35 descriptors of the themes. Good inter-coder agreement was demonstrated with an overall κ score of 0.66. A least two out of three investigators independently agreed on event classification in 92% of cases.
Conclusions: The proposed taxonomy for reported events in general practice provides a comprehensible tool for clinicians describing threats to patient safety, and could be built into reporting systems to remove difficulties arising from coder interpretation of events.
Statistics from Altmetric.com
There is little information available on the proportions of types of patient safety event that are reported in general practice settings and no taxonomy that is widely used by general practitioners (GPs) to describe these events. Previous studies in primary care have provided descriptions of the relative proportions of different types of patient safety event that they have collected,1–10 but none have been based on a representative sample of primary care clinicians contributing data, and all have used different classification methods.
In recent years, a small number of taxonomies of patient safety event related to a general practice setting have been proposed,711–13 all based on patient safety event reports collected from small non-random samples of participants. These taxonomies have used both causative and descriptive elements of like themes to group events, sometimes referred to as “domain-specific” taxonomies.14 “Multiaxial” taxonomies capture additional elements of an event, such as harm levels, location, participants or preventability. One study has developed such a tool in a primary care setting in North America,1516 but its full details have not been published in the scientific literature. Another tool has been suggested as one that could allow comparison of safety events across disciplines, although no trial of it in a primary care setting has been described.17 There have also been some recent calls for classifications that address cognitive psychological processes,18 with one developed for general practice,14 also yet to be tested. Finally, one taxonomy based on patients’ perceptions of harms in primary care has been described, although it has a limited application in terms of categorising causes of events.19
The Threats to Australian Patient Safety (TAPS) study estimated the incidence of GP-reported patient safety events in the community, using a method based on a randomly selected representative sample of GPs.20 We found that limitations mainly relating to the internal validity of existing tools warranted further taxonomy development. We aimed to develop a taxonomy that would be comprehensible and practical for primary care clinicians to apply themselves. Here, we describe the first attempts to validate the taxonomy in terms of its reproducibility with GPs.
METHODS
The methods and definitions used in the TAPS study have been previously described. A total of 84 GPs from a random sample of 320 GPs, previously shown to be representative of 4666 GPs in New South Wales, Australia, anonymously reported “errors” (patient safety events) that they noted in their daily practice (including ambulatory clinics, hospital settings and residential aged care facilities) for a 12-month period via a secure online questionnaire.20
TAPS taxonomy development
Three GPs from the investigating team (MAM, SS and CBW) classified each report using an existing pilot taxonomy,12 and results were compared to obtain a baseline measure of reproducibility. Reports containing more than one event were given multiple codes in chronological order of events. A report required two out of three reviewers in agreement to be assigned a classification. Where three different codes had been assigned, the report was reviewed at a face to face meeting to determine its classification. The initial taxonomy was amended and retested using a quarter of the reports each time, over four sessions, to produce the TAPS taxonomy. A set of guidelines to improve coder consistency when using the taxonomy was also developed.
Agreement of investigators developing the taxonomy
Concordance amongst the coders was measured using the κ statistic, for both the initial taxonomy and the final TAPS taxonomy. Only one classification per report was included for the κ score calculations. If reports were assigned multiple classifications by one or more coders, the first classification most commonly assigned was included. The κ statistic can range from −1 (perfect disagreement) to +1 (perfect agreement). A κ score of between 0.40 and 0.75 indicates fair agreement.2122 All statistical analysis was performed using STATA version 8.0.
RESULTS
Patient safety event numbers
Box 1: TAPS taxonomy guidelines for coding patient safety events
Consider the number of “patient safety events” or separate elements that have contributed to a report describing a threat to patient safety, and classify each distinct patient safety event separately if there are more than one.
Consider the underlying cause to first code the event type—whether the event has resulting from a breakdown in the “processes” around patient care (type 1) or due to the knowledge or skills base required for any person involved in the delivery of patient care (type 2).
Next assign the second level theme of the patient safety event, choosing the most specific option from those listed within the assigned event type (that is, 1.1–1.5 for type 1 events or 2.1 or 2.2 for type 2 events). For type 1 events, use theme 1.5 “Communication errors and process errors not otherwise specified” when a more specific theme is not suitable.
To complete, assign the most specific level 3 descriptor available from within the second level theme chosen. In general, these descriptors are listed from more specific to less specific when moving down the list.
Box 2: TAPS case study of a report containing a patient safety event relating to the processes of healthcare
A general practice patient with a background of developmental delay, epilepsy and schizophrenia attended regularly but used two different surnames on different occasions, both being the surnames of their divorced parents. The practice mistakenly held two electronic records for the patient under the two surnames, which contained different medication lists. The patient was prescribed a new medication and they became over-sedated, lethargic and depressed as a result of an interaction with a medication listed in the other chart, before the mistake was discovered.
TAPS code: 1.1.3
Type: Processes of healthcare
Theme: Errors in practice and healthcare systems
Descriptor: Patient record and filing system errors
Box 3: TAPS case study of a report containing patient safety events relating to the knowledge and skills of health professionals
A patient with severe depression was referred by their GP to the regional psychiatric hospital. A week later they returned to the GP for follow-up after discharge. The patient reported to the GP that they had complained of increasing pain in the chest after admission to the psychiatric unit. After some delay they were sent to the base hospital by the psychiatric unit for a chest x ray without actually being physically examined. The chest x ray showed normal findings. After another 3 days the patient was examined by a medical officer in the psychiatric unit and found to have a florid shingles rash. The patient was eventually sent home after being prescribed analgesics, but antiviral management had not been appropriately instituted.
TAPS codes: 2.1.2, 2.2.1 (2 events identified)
Type: Knowledge and skills of health professionals (both events)
Themes: Errors in diagnosis (event 1) and Errors in managing patient care (event 2)
Descriptors: Errors in patient physical examination (event 1) and Medication management errors (event 2)
A total of 433 online submissions were received. Of these, 415 contained true reports, after discounting tests and reports with missing data (n = 15), and reports that investigators deemed not to describe any patient safety event (n = 3). Of the 415 reports, 320 contained one event, 82 contained two events, 11 contained three events, and 2 contained four events.
The TAPS taxonomy
The resulting taxonomy has three levels of classification. The first level (event type) relates to the underlying cause of the event, being either due to deficiencies in the process of delivering healthcare (type 1), or the knowledge and skills of health professionals (type 2). The second level has five groupings (themes) within type 1 errors, and two groupings within type 2 errors. The themes within type 1 are practice and healthcare systems, investigations, medications, non-medication treatments and communication. The type 2 themes are diagnosis and management of patient care. At the third level there are from three to nine descriptors per theme.
Table 1 shows the raw counts and proportions of total events for each category of the taxonomy. The guidelines for using the taxonomy are shown in box 1. Examples of reports describing type 1 and type 2 events are shown in boxes 2 and 3, respectively, with the codes used to describe them.
Coder agreement from pilot to TAPS taxonomy
The investigators coded the set of 433 reports using the pilot taxonomy,12 and the last 132 reports (approximately a quarter) were coded using the TAPS taxonomy. The agreement among the three coders at each level of both the pilot and TAPS taxonomies, with corresponding κ scores, are shown in table 2. At the third level of the code from the pilot to the TAPS taxonomy, the proportion of reports in which at least two of three coders agreed rose from 74% to 92%, and the κ score moved from 0.37 to 0.66.
DISCUSSION
Our proposed taxonomy builds on pilot work which has at its primary level a causative classification,1112 with subcategories based on grouping of like themes, and then addition of detail with descriptive categories in the style of a domain-specific taxonomy. Application of the taxonomy to the TAPS data shows that the majority of reports contained a single patient safety event, and the majority of events related to the processes of providing healthcare rather than deficiencies in knowledge and skills of health professionals, as postulated in previous work with non-representative samples.791112
The reporting GPs demonstrated a clear understanding of the definition of error used.20 Investigators found less than 1% of reports to contain no safety event. The proposed seven themes adequately described all reported events. The addition of “not otherwise specified” as a descriptor for each theme, unlike previous similar taxonomies,5–7 allowed complete event coding, incorporating 13% of classified events. This compares with 2.2% of reports in one single level UK taxonomy,2 although it provided no other detail on the error type, and 20% of the “cause” of anonymous reports using a multiaxial taxonomy in one study from the USA.3
The largest proportion of events were classified as relating to “practice and healthcare systems” at the theme level (21%), consistent with American and UK studies,711 although our category is different from any in previously described taxonomies as it contains elements of the larger healthcare system rather than just administrative events. The 2002 pilot study had found a similar proportion of “office administration” events (20%).12
If the types 1 and 2 events are combined, medication errors represent the largest proportion (31%), similar to findings in earlier Australian work.34 The TAPS taxonomy allows the cause of medication errors to be considered, whether related to knowledge of their use (events 2.2.1) or to systems problems in their provision (events 1.3.1–1.3.5). This may assist in planning prevention measures, such as education for clinicians or systems changes that would reduce electronic prescription errors or dispensing mistakes.
Our purpose was to develop a tool that would be comprehensible to primary care clinicians reporting safety events. The use of investigators with a clinical background in general practice was important to produce language that would be acceptable for self-coding. One other study has asked the reporting clinicians to code the events using a simple descriptive taxonomy.7 However, the study period was brief, and the majority of reporters were reception staff, so some event types may not have been captured.
There are no studies with which we can compare our results on reproducibility of the taxonomy. The κ statistic and proportions in table 2 showed a marked improvement in agreement from the pilot to the TAPS taxonomy at all levels of the code. It is possible that a degree of improvement could have occurred through an unconscious learning of each others’ styles at the taxonomy development meetings, in addition to improved clarity of the code itself. As expected, there was less agreement among coders comparing level 1 with 2, and level 2 with 3, due to more coding choices.
At the most detailed third level of the taxonomies, complete disagreement among the coders fell from over a quarter to less than 10% of cases. In the cases where complete agreement was not reached, there was often a difficulty in interpreting the reporting language used, or a brevity of description provided in the report, requiring a degree of personal interpretation or assumption from the coding GPs. While further refinement of the taxonomy might lessen possible ambiguity, we believe that a system where the reporting clinician actually codes the event would reduce this type of error by eliminating a loss of detail in the process of describing the event and its cause to another clinician or analyst conducting the coding.
The taxonomy developed by the Applied Strategies for Improving Patient Safety (ASIPS) collaborative1516 is a multiaxial model developed in a primary care setting, using trained analysts to classify reported events. It has not been published in the peer-reviewed literature in full, however it can be viewed online.23 The TAPS electronic reporting system collects data of a similar nature to the additional axes and domains of ASIPS, including a harm scale, location check-box, event frequency scale, and details of patients such as age, gender and ethnicity.20 These elements are closed questions completed electronically by the reporter. They could be combined with reporters’ self-coding an event with the TAPS taxonomy to effectively produce a self-reported multiaxial taxonomy describing safety events.
An important limitation of our results is that they do not represent the underlying proportions of error types in the community, although the reporters were a representative sample. Some event types may have been under-reported in comparison with others, despite efforts to encourage reporting. Participants may have been unaware of deficits in their own knowledge or skills. The TAPS reports are a reflection of GPs’ experiences, and the taxonomy was created by GPs. As such, it may be limited in its application to other primary care groups. In other studies, a variety of clinicians or administrative staff have been required to report events,6714–1624–26 and it would be important to explore language differences that may exist. The TAPS reports and taxonomy may not have captured all error types that could be reported by a more diverse group. One other general practice taxonomy has recently been shown to be acceptable to opticians reporting safety events.27 The TAPS taxonomy may similarly have potential uses to other disciplines in a community setting. Our purpose in this paper did not include an appraisal of error types in relation to harm, preventability and demographics of patients affected by threats to safety, however this analysis is underway.
We believe that the TAPS taxonomy has the potential for use as part of an anonymous national electronic reporting system and offering guidance to policy makers in directing efforts to reduce patient safety threats in the community, particularly at a systems level. Further application of the TAPS taxonomy may also aid professional bodies developing educational tools aimed at improving the knowledge and skills of providers in primary care.
Acknowledgments
We gratefully acknowledge the contribution of the 84 New South Wales general practitioners who provided the data for this study, and thank Ms Geraldine Card for her valuable contribution in managing the study and collating results.
REFERENCES
Footnotes
Funding: The National Health and Medical Research Council (NHMRC) provided project grant funding for the direct research costs of the study, and MM was awarded an NHMRC Scholarship. The Primary Health Care Research Evaluation and Development (PHC RED) Program of the Department of Health and Ageing, Commonwealth of Australia, provided Researcher Development awards to MM and SS.
Competing interests: None identified.
Ethics approval: The University of Sydney Human Research and Evaluation Ethics Committee approved the study.