METHODS

Definition of a significant event

A broad definition of a significant event was provided on the front page of the questionnaire to assist those GPs who may have been unfamiliar with both the term and its meaning (box 1).

Data collection

A postal questionnaire was sent to all 617 principals in 222 general practices in Greater Glasgow primary care trust. Non-respondents were sent two additional reminders and the survey was completed in January 2002. The questionnaire was devised by the authors and pre-tested on five departmental colleagues who are part time GPs. A pilot survey involving 12 GPs based in the eastern Glasgow Local Health Care Cooperative (LHCC) was undertaken in October 2001. Minor amendments were made to the design of the questionnaire based on the feedback received from pilot participants.

Data were collected on GPs’ professional and practice characteristics, reported awareness of a recent significant event, whether a structured analysis of the event was undertaken, and the perceived risk of this event recurring. Additional data on the discussion forums for significant events and the composition of the meetings were also collected.

Structured analysis of a significant event

Respondents were asked to indicate if certain activities were carried out during their analysis of the recently identified significant event. We adapted the format used in previous work to determine if a structured analysis of this significant event had taken place.^15,16 If all of the relevant stages of analysis outlined in box 2 were reportedly carried out by respondents, then we agreed a structured analysis of the significant event in question had been achieved. Because of the work involved this could not be independently verified, nor could the overall effectiveness of the analysis being measured.

Statistical methods

The denominator in this study was the principal in general practice. Although more than one principal from the same general practice could have returned the questionnaire, with the exception of training practice status, number of partners and list size, no other data linking individual GPs with their surgery were recorded.

Characteristics of the GPs including sex, time as a principal, and professional status were summarised. The characteristics of the practices to which the GPs were attached were also summarised but were not linked; thus, for a practice returning the experience of, for example, three GPs, the same data on practice characteristics would appear identically three times.

Statistical associations between the proportion of GP respondents from training practices, multi-partner and single handed non-training practices with 95% confidence intervals (CI) were calculated. The frequency of the occurrence of the different stages of the previously defined “structured analysis of a significant event” was tabulated and the outcome “Did the GP undertake a structured analysis?” derived. Univariate logistic regression was used to explore the relationship between a number of covariates which it was felt might be important predictors for this outcome—for example, time as GP, training practice status, number of practice partners, and perceived risk of significant event recurring. The odds ratios (ORs) for the stated unit of comparison or change—for example, time as a GP ⩽5 years or training practice versus non-training practice—were calculated (with 95% CI) and the associated p value for the significance test of unity of the OR. A multivariate logistic regression model was then fitted to adjust for all of these factors simultaneously.

Fisher’s exact tests were calculated to determine if there was an association between the types of significant event meetings held, levels of primary care staff involvement, and training practice status. Caution should be exercised when interpreting the p values for these comparisons because the analyses make no adjustment for the possible clustering of responses from within a practice, nor do they adjust for the other practice characteristics.

All analyses were performed using SAS 8.2 for Windows NT. No adjustment has been made for multiple comparisons.

RESULTS

Response rate and characteristics of respondents

Of the 617 GPs in 222 surgeries, 466 (76%) from 188 surgeries (85%) completed and returned the questionnaire. A higher response rate was achieved from training practices (162/186, 87%) than from multi-partner non-training practices (273/376 (73%); difference 14.5%, 95% CI 7.9 to 21.1) or single handed non-training practices (31/55 (56%); difference 30.7%, 95% CI 16.8 to 44.7). Table 1 summarises the characteristics of respondents and their practices.

Awareness of a recent significant event

401 of the 466 principals who responded (86%) reported being aware of a recent significant event associated with their practice, while 65 (14%) indicated that they were unaware. GPs who reported being unaware of a recent significant event were more likely to be based in non-training practices (adjusted OR 4.00 (95% CI 1.71 to 9.35), p<0.001).

Structured analysis of the recent significant event and perceived risk of recurrence

Table 2 shows that most GPs who were aware of a recent significant event indicated to varying degrees that the individual stages involved in the analysis of a significant event were undertaken; 219 (55%) reported participating in all of the stages of a structured analysis as previously described (box 2). Both univariate and multivariate logistic regressions (table 3) show that GPs from training practices were much more likely to undertake a structured analysis than those based in non-training practices (adjusted OR 0.25 (95% CI 0.15 to 0.41), p<0.0001). In addition, GPs who indicated that the risk of recurrence of a significant event was either nil or very low were more likely to have undertaken a structured analysis than those who perceived a higher risk of recurrence (adjusted OR 2.19 (95% CI 1.18 to 4.07), overall p = 0.014).

Team involvement in significant event analyses and the forums used

A few respondents (5%), almost exclusively from non-training practices, reported that significant events were never discussed in their practices (p<0.001). GPs from training practices were much more likely to report that the discussion of significant events took place at partners’ meetings (p<0.001), practice team meetings (p<0.001), or dedicated audit/significant event meetings (p = 0.013) than colleagues from non-training practices (table 4).

The majority of GPs (94%) reported always being involved in the discussion of significant events associated with the practice compared with 58% of practice managers and 31% of practice nurses. Fewer than 10% of respondents reported that other health professionals such as health visitors and district nurses were always involved in the discussion of significant events within the practice. However, much greater levels of involvement in significant event discussions were reported for these staff groups only if required by the practice partners.

DISCUSSION

The main findings from this study provide a measure of the variation in the awareness and analyses of significant events among GPs. Most were aware of a recent significant event associated with their practice, although it may be of concern that a few were not. Of those who recalled a recent significant event, a full structured analysis was reportedly undertaken by just over half. Analysis of this significant event in a structured manner was associated with a decrease in the perceived risk of the event recurring. Most GPs formally discussed significant events at various types of meetings in the practice, although only a small number have dedicated SEA meetings. Members of the primary care team only tend to be involved in these discussions when considered appropriate by GPs, although the practice manager has much greater involvement than other staff.

Our survey generated a good response rate from the GPs involved. However, a number of limitations are associated with this type of descriptive cross sectional study. There may have been response bias because GPs from training practices were more likely to respond and single handed GPs less so. This may reflect the knowledge and experience of the subject matter of those from the training environment. We also rely on respondents to self-report, which potentially limits the reliability of the data because there is no independent means of verifying responses. Caution should therefore be exercised when extrapolating these data for general purposes. Traditional SEA allows for significant events that highlight good practices to be identified for discussion;⁷ however, respondents taking this approach may have had difficulty undertaking our version of a “structured analysis”. We also provided one definition of a significant event but acknowledge that respondents might interpret this and their own concept of “event analysis” differently, which may again influence the reported data. If we compare the time and resources put into the root cause analysis of individual events in other industries, it may be that SEA in general practice is applied superficially, particularly as there is as yet no standardised method of event investigation or data collection in general practice.¹² This may also reflect the complexity and uncertainty that characterises much of primary health care.²²

A small number of GPs reported being unaware of a recent significant event, although if extrapolated nationally these figures would be quite substantial. It is possible that some of these doctors had genuinely not experienced a recent significant event. However, it seems more likely that inexperience and lack of knowledge are more plausible explanations for them being unaware. Another possibility is that some of these doctors lack insight in this area and may find it difficult to imagine they are ever involved in significant events. The introduction of appraisal for all doctors may provide an opportunity to address this particular concern.

Most GPs reported undertaking a recent SEA. Previous informal estimates of SEA participation rates among GPs and practices have ranged from 6% to 20%.^23,24 However, it is also clear that a substantial minority indicated that a full structured analysis of the recent event did not take place. Worryingly, one fifth of respondents who attempted an analysis perceived the risk of the event recurring as moderate to very high. Merely identifying and discussing the event may be insufficient if lessons are to be learned and changes considered or introduced. The main purpose of SEA is that it goes further than simple discussion and should show that insight into the event has been established and action taken to minimise recurrence. The formal documentation of the analysis highlights that the significant event is probably more likely to be taken seriously by the practice and that a permanent record of it exists—especially if it is linked to a complaint or potential litigation.

The necessary stages of a structured analysis we outline are open to further debate and scrutiny. We also recognise that not every SEA will involve the implementation of change. However, the limited research available indicates that all of these factors are normally present when an effective analysis of a significant event is performed.^15,16 Further education for GPs has already been highlighted as an issue if SEA is to be applied effectively and is to have an impact on practitioner behaviour and improving patient care.¹⁶ Our findings also appear to confirm that a number of GPs may not be applying SEA as robustly as other colleagues.

Most practices appear to discuss significant events at a specified internal forum, with a minority favouring informal discussions only. Informal or unstructured discussion of significant events may be ineffective for that very reason.⁶ GPs are present at most significant event discussions, but there appears to be wider team involvement where it is felt appropriate, with the practice manager being most involved. This is not surprising given the administrative nature of many significant events.¹⁵ It is interesting that SEA is promoted as a key team activity with the underlying assumption that non-medical staff should be involved in most significant event discussions.^7,10,13 However, our evidence appears to suggest that GPs are selective about which staff members attend these meetings.

The local training practice environment appears to be an important factor in the participation and analyses of significant events. This finding is not unexpected given that participation in SEA is a training practice requirement in Greater Glasgow. It is also known that GP trainers participating in an educational peer assessment model of SEA are more likely to undertake satisfactory analyses than non-trainers.¹⁶ The personal ability of the trainer in this area together with the compulsory focus on quality improvement required of a training practice may directly influence the development of SEA within the practice and the involvement of the primary care team.

A potential difficulty is the small proportion of GPs (and, conceivably, practices) not engaging in the discussion or analyses of significant events, even on an informal basis. The reasons for this merit further exploration but, given that the finding is almost exclusively related to non-training practices, it may point to an educational issue for the practitioners involved, especially with the imminent introduction of appraisal and revalidation. A clinical governance issue may also have been identified for those local primary care organisations wishing seriously to address potential risk management and patient safety concerns.

We have shown that the SEA technique is reportedly practised by a small majority, but with mixed results in terms of application and potential impact. There is a potential concern that GPs who fail to address significant events or who are ineffective with their analyses might be missing opportunities to improve patient care and safety. Thus, an important aspect of the clinical governance agenda may not be adequately addressed. If confirmed by further research, an educational strategy may be required to maximise the application and effectiveness of the technique.

Setting aside the need for research into the underlying causes of significant events, more evidence is required into GPs’ understanding and interpretation of these events and the effectiveness of their analyses. In addition, more research into the impact of SEA on practitioner behaviour, the healthcare quality agenda, and how to optimise the sharing of best practice in this area is needed. Ultimately, this will inform the robustness or otherwise of SEA as a tool to enhance patient care and safety.