Key measures for improvement

Three measures were used for the evaluation of the intervention:

1. Patient satisfaction, experience and awareness

2. Information about general practice services

3. Experiences of a mystery patient.

Process of gathering information

Design and participants

We asked 129 general practices in the south of The Netherlands to participate in our project. Sixty-six practices responded of which 36 agreed to participate voluntarily (61%). In each practice, 40 patients registered with the general practice were asked to participate. These patients were randomly selected. We collected data about accessibility and availability using: (1) questionnaires completed by adult patients (>18 years), (2) questionnaires about general practice services completed by general practitioners and (3) feedback from one mystery patient who made 15 calls for ordinary consultations and five emergency calls. When the phone was busy or not answered, the phone numbers were called three more times. When the phone was picked up and the patient was asked to hold, the service was classed as accessible. Data were collected before the intervention and 5 months after the intervention.

Outcome measures

The measures used were waiting time on the phone for emergencies and ordinary consultations, waiting time in the waiting room before consultation, waiting time for an appointment (both acute and chronic illnesses) and the quality of information service. Patients were asked to report their satisfaction and experience with the phone accessibility for emergencies and ordinary consultations, waiting time for an appointment and the waiting time in the waiting room before consultation. A specific part of an internationally validated questionnaire covering access of care (Europep, Visitation Instrument Practice Management) was used to collect the data.15 16 Practice information was measured using the same instrument.16 The patients' awareness of an information leaflet, a telephone number for emergencies and information about waiting time in the waiting room were also asked for. The general practitioner was asked to provide practice information concerning the presence of an information leaflet, a practice website and a specific telephone number for emergencies. The mystery patient was asked to make an appointment for an ordinary consultation and another for an emergency. Good phone accessibility for an ordinary consultation was defined as receiving personal contact within 60 s or when the answering machine gave an alternative telephone number within 60 s. For emergencies, good accessibility was defined as receiving personal contact within 30 s or when the answering machine gave an alternative telephone number within 30 s. When no contact had been established after three attempts, the service was classed as inaccessible at that time.

Data collection

The patients completed the questionnaires after their consultations. A practice assistant handed out the questionnaires during one week. Patients that filled in a questionnaire could deposit this questionnaire in a closed box. After one week, all questionnaires that were filled in were sent to the research team. One general practitioner in each practice filled in the questionnaire about practice services. The mystery patient called for an ordinary consultation three times a day for 5 days and for emergency services called once a day for 5 days. The measurement of patient satisfaction, experience and awareness; practice information; and the mystery patient's investigation were repeated after 5 months during which the practice could have improved their performance using the individual feedback and information on best practices. At the end of the study, the process was evaluated by means of a questionnaire to investigate the experiences of the practices with this study. The questions covered the distribution of the patient questionnaires, the mystery patient's investigation, the quality of the feedback, and the information on best practices.

Strategy for change: feedback and encouraging design of practice-based improvement plans

The practices received individual feedback from the research team based on their accessibility and they received information about three best practices. These best practices were selected based on the most positive patient satisfaction with waiting times in the waiting room, accessibility by phone, and information service. The recommendations for practice-based improvement plans were suggested to the general practitioner and concerned all practice members. Recommendations were made when practice performance was 5% lower than the mean of all participating practices or when the individual performance was relatively low (patient satisfaction, experience or awareness <75% for one specific subject). Dutch literature shows that the average perceived accessibility of patients is 80%.6 We accepted a deviation of 5%.

Two and 5 months after designing the improvement plans, the practices were called to report on their progress.

Analysis and interpretation

We grouped the improvement plans into four categories: completed, ongoing, not started and cancelled. The focus of the improvement plans included:

1. Information service

2. Accessibility by phone

3. Waiting time in the waiting room

4. Organisation of consulting hours

5. Consulting hours by phone.

We examined the overall impact of the quality-improvement intervention on patient satisfaction and experience, including those of the mystery patient. An independent sample t test was used to analyse the data regarding the impact of the quality-improvement strategy. Only quality-improvement plans implemented in more than two practices were selected for this analysis.

Study population

At baseline, 1256 patients from the 36 general practices filled in the questionnaire. The response rate was 87.2%. Only one of the 36 practices originally recruited dropped out because we did not succeed to make an appointment with the general practice for discussion about their performance during the study period. After 5 months, 1071 (76.5%) patients from 35 practices filled in the patient questionnaire. The practices included in the study were representative for all Dutch general practices (table 1).17 There was a small over-representation of group practices and practices in rural areas in comparison with the overall Dutch general practices.

Patients aged 44–64 years were over-represented in the study population. Women in the study population were over-represented in comparison with Dutch general practices.

Effects of change

The 36 participating practices developed 123 practice-based improvement plans using the practice feedback results and practice information. The practices perceived the feedback about their performance as very useful. After 5 months, 26 practices filled in a questionnaire concerning the progress of their improvement plans. Table 2 shows that almost half (53/123) of the improvement plans were related to information service. One fourth (31/123) were related to phone accessibility, and another one fourth (33/123) to waiting time in the waiting room. Six plans were related to the consulting hours. After 5 months, 51 (42%) improvement plans were completed.

Table 3 presents the impact of four specific practice-based improvement plans. The plans “distribution of information leaflet”, “publicity of the telephone number for emergencies” and “phone accessibility for an ordinary consultation within 2 min” improved significantly.

Table 4 shows that there were significant changes in patient experience of phone accessibility for ordinary consultation and for making an appointment within two working days. There was also a significant change in patient satisfaction with the waiting time in the waiting room.

Lessons learned

All participating general practices were motivated to improve their accessibility and availability before the study started. They found the feedback about their performance very useful. Most of the practice-based improvement plans concerned information service, waiting time in the waiting room and phone accessibility. After 5 months, 80% of the plans were completed or almost completed. The improvement plans concerning process improvements would likely be completed after a longer period. The type of improvements concerned structural changes in daily practice and routines, which are expected to be sustainable.

The experience of the mystery patient and patient satisfaction showed that the accessibility in general practice was good. These results are consistent with other studies that show high scores on patient satisfaction in The Netherlands.7 8 It can be argued that our results are an over-estimation as we defined accessibility as “good” when an alternative phone number was provided and did not restrict accessibility to speaking to a person since we assumed that a patient knows how to handle according to this information. Over-estimation of patient satisfaction could also be due to the patient population, which included relatively more patients with chronic disease. These patients visit the practice more often and know the preferred times to call the practice and may be, therefore, more satisfied than other patients.

Adding a mystery patient for data collection could contribute to more objective measurements of practice accessibility than patient questionnaires alone. In our study, the mystery patient was more satisfied with the accessibility than the real patients. It may be argued that we only included one mystery patient in our study. We gave clear instructions to this “patient”, which could explain more patient satisfaction. Another explanation is related to our concept of accessibility, which was defined as acceptable if the mystery patient contacted the practice after three attempts at most. In contrast, patients might expect that there should be personal contact directly after the first call. If more information is available on patient expectations, specific interventions could improve satisfaction with services subsequently.18 Finally, practices might have anticipated on a call from a mystery patient. However, they had no idea when the mystery patient would call during the study period, so anticipation is unlikely.

A substantial proportion (43%) of the improvement plans dealt with practice information services. This can be very effective in enhancing a patient's knowledge on when and how the general practice is accessible. Increasingly, practices use websites for information services, which can be easily updated with the latest information on access and availability. Future research on accessibility could address the use of websites and other alternative information services. Future research should also take into account that there is a gap between the perceptions of a mystery patient and an actual patient concerning accessibility.