Objectives—To develop and evaluate an information service in which a “clinical informaticist” (a GP with training in evidence-based medicine) provided evidence-based answers to questions posed by GPs and nurse practitioners.
Design—Descriptive pilot study with systematic recording of the process involved in searching for and critically appraising literature. Evaluation by questionnaire and semi-structured interview.
Participants—34 clinicians from two London primary care groups (Fulham and Hammersmith).
Main outcome measures—Number and origin of questions; process and time involved in producing summaries; satisfaction with the service.
Results—All 100 clinicians in two primary care groups were approached. Thirty four agreed to participate, of whom 22 asked 60 questions over 10 months. Participants were highly satisfied with the summaries they received. For one third of questions the clinicians stated they would change practice in the index patient, and for 55% the participants stated they would change practice in other patients. Answering questions thoroughly was time consuming (median 130 minutes). The median turnaround time was 9 days; 82% of questions were answered within the timeframe specified by the questioner. Without the informaticist, one third of questions would not have been pursued.
Conclusion—The clinical informaticist service increased access to evidence for busy clinicians. Satisfaction was high among users and clinicians stated that changes in practice would occur. However, uptake of the service was lower than expected (22% of those offered the service). Further research is needed into how this method of increasing access to evidence compares with other strategies, and whether it results in improved health outcomes for patients.
- clinical informaticist
- evidence-based medicine
- information services
Statistics from Altmetric.com
Doctors frequently have questions arising in their day to day work but often these remain unanswered.1–6 Although most doctors are enthusiastic about evidence-based medicine,7 few wish to develop the necessary skills7,8 and awareness and use of evidence-based resources by GPs is low.7,9 Only 5% of GPs think that a move towards evidence-based medicine should be by “identifying and appraising the primary literature or systematic reviews” themselves.7
The use of evidence centres has been advocated10 and a hospital centre providing such services to specialists has been described.11 A recent editorial suggested that professionals offering clinical information services should have a clear understanding of both information science and the essentials of clinical work.12
We developed a service based in primary care in which a “clinical informaticist,” a GP with further training in evidence-based medicine, provided evidence-based answers to questions, addressing the barriers of lack of time and skills.7,13 We envisaged that the service could be used in a way analogous to a pathology laboratory, the clinician asking for evidence to answer a question and acting on the results in the context of the individual patient. The informaticist did not offer specific advice on patient management.
A 1 month pilot study of a similar service in Australia has been reported,14 and the ATTRACT project recently evaluated an answering service provided by a non-clinician based on a rapid appraisal process.15 We are unaware of any published reports of a GP led answering service in the UK. The reasons for choosing a GP as informaticist were threefold16:
a GP is likely to understand the precise nature and context of the question;
a GP is ideally placed to help a colleague “frame” a question.13
Here we describe the service, including the search methods and resources used, and the time taken to answer questions. We also describe the evaluation of the service by means of questionnaires and semi-structured interviews.
This is the first study to evaluate the role of a clinical informaticist (a GP with training in evidence-based medicine) in the UK.
Clinicians who used the service expressed very high levels of satisfaction and reported that they would change their practice as a result of using it.
Without the informaticist, one third of questions would not have been pursued.
Uptake of the service was lower than anticipated (22% of those approached).
TRAINING THE INFORMATICIST
The informaticist was a vocationally trained GP interested in evidence-based medicine but with no previous experience of providing a service of this type. Her training involved a series of eight 90-minute tutorials with a senior medical librarian during which she learned how to do rigorous searches of important medical databases. She also attended the 4th London Workshop on Teaching Evidence-Based Health Care and had three tutorials on critical appraisal with MP, a senior lecturer in primary care. Her learning was supplemented by background reading of textbooks on evidence-based medicine.17,18
The informaticist continued to work half-time as a GP for the duration of the project.
RECRUITMENT OF PARTICIPANTS
All 100 GPs and nurse practitioners (“clinicians”) of the Fulham and Hammersmith primary care groups (box 1) were offered the service free of charge. Two mailings were sent and interested clinicians were offered a practice visit by one of the investigators. Thirty four clinicians (32 GPs and two nurse practitioners) registered from 14 practices.
Since 1999 GPs in England have been members of local Primary Care Groups (PCGs), along with other health professionals. The responsibilities of PCGs include:
assessing local health needs;
planning and commissioning health services for their local community;
developing primary care;
improving and maintaining the quality of local services.
DELIVERY OF THE SERVICE
Participants submitted questions on a short request form adapted from a form developed by Hayward et al.14 If additional information was required to formulate the question, this was discussed with the questioner (n=8).
The search cascade shown in box 2 was used, starting with a search of pre-existing databases of evidence. If questions could be answered fully at one level, lower levels of the cascade were not searched routinely. A primary literature search (step 8) was conducted if insufficient evidence was found earlier. Some questions could be answered partially at one level in the cascade, but also required use of resources further down the cascade. Medline was used for primary literature searching as there are well developed “quality filters” for this database. Unpublished evidence was not sought and, when good evidence was unavailable, this was made explicit.
The informaticist critically appraised the literature using recognised critical appraisal checklists and returned an evidence-based summary to the questioner. A standard format was used, a modification of the CAT (“critically appraised topic”),18 consisting of:
The Question (asked by the questioner)
The Modified Question (if question reformulated)
The “Clinical Bottom Line” (the main message, in bold type)
The Evidence (summary of the critical appraisal)
Participants were invited to request original references, although none did. A database of questions and answers was available to participants on the project website19 and in project newsletters.
Twenty two of the 34 participants used the service (20 GPs and two nurse practitioners), generating 60 questions of which 57 could be answered. Examples of questions asked are shown in box 3. Twelve GPs failed to use the service, seven asked one question each, while the two highest users asked 13 of the 60 questions (22%). Fourteen of the 60 questions originated from a single group practice.
In a fit immunocompetent 26 year old man with early chicken pox, what is the evidence that acyclovir is effective?
In a 56 year old patient with ischaemic heart disease, is there any evidence that folic acid supplementation reduces cardiovascular events?
Is there any evidence that breast self-examination reduces mortality from breast cancer?
What is the evidence that quinine sulphate is effective for nocturnal leg cramps?
In a 73 year old woman with long standing moderately severe osteoarthritis, is there any evidence that oral glucosamine reduces pain or slows the course of disease?
The classification and origin of the questions is shown in table 1 with information on how the questions would otherwise have been pursued.
Answering the questions was time consuming, taking a median of 130 minutes (range 25–450). Median turnaround time, defined as the time between the date on the request form and the answer date, was 9 days (range 0–34); 72% of questions were answered within two weeks. Table 2 shows the distribution of resources used in answering questions.
EVALUATION OF THE SERVICE
A questionnaire was sent with each evidence-based summary to assess satisfaction with the answers provided and to find out how the information was used. A 95% response rate was achieved, details of which are shown in table 3.
At the end of the 10 month period 17 (77%) of the 22 participants who had asked questions agreed to participate in a final evaluation of the service based on a semi-structured questionnaire. Of these, 12 were interviewed (two face to face, 10 by telephone) and five submitted the completed questionnaire by fax. We explored the participants' overall impressions of the service, how they had used the information, how their behaviour had changed as a result of the service, and how they saw the service fitting into general practice.
The comments added further support to the positive evaluation derived from the individual questionnaires. High levels of satisfaction were expressed and most commented on the expertise of the informaticist. Six said they would have like to make more use of the service, reporting “time” and “lack of organisation” (in the practice) as barriers. Eleven respondents said that access to the service had increased their likelihood of asking questions and 10 said it had increased their likelihood of pursuing answers. Seven respondents said it had increased their likelihood of searching for answers themselves. Most were enthusiastic about the potential role for this kind of service in the future of general practice, although for some this was tempered with concerns about the potential costs involved. Some direct quotes from this part of the evaluation are shown in box 4.
“It made us talk on not only that query but also other things.”
“I've heard of people needing evidence, and for GPs this seems like a big hassle. On reflection, it is worth having evidence.”
“It definitely made me question more . . .questioning about various things.”
“... saves time—may not have had time to search for the answers myself.”
“ . . . great plus that you don't have to do the work yourself.”
“... useful resource for general practice . . . depends on funding I suppose.”
We have shown that a clinical informaticist service was a valuable method of providing relevant, high quality information to primary health care workers. Satisfaction with the service by those who used it was very high, in line with the suggestion that doctors value summarised evidence.7,8 Users reported that changes in practice would occur, and reported other positive spin-offs such as increased understanding and promotion of discussion with colleagues. Although it is a small study based on 34 interested participants in a small geographical area, we have collected useful data on the processes involved in providing such a service, including the types of resources accessed in the answering process. Despite a growing body of systematic reviews and evidence databases, primary research literature was accessed to answer 61% of questions. This was similar to the findings of Hayward et al who used primary literature to answer 58% of questions.14 Rigorous critical appraisal of primary literature is a time consuming process, requiring access to full text articles and frequent requests for interlibrary loans. This (and the fact the informaticist was part time) accounted for the longer than desirable turnaround time (median 9 days). However, although slower than the recently reported ATTRACT service,13 satisfaction with the timescales was high and requested timescales were met for 82% of the questions. We also collected more data on the process involved in providing the service and obtained a more thorough ongoing evaluation by requesting feedback on every question handled.
Appropriate implementation of evidence requires incorporation of the information in the individual clinical context. This developmental study was not designed to assess objective measures of change, relying solely on outcomes reported by users of the service. Further research is needed to establish whether providing an informaticist service results in better health outcomes, and how this strategy compares with other possible methods of increasing access to evidence.
Uptake of the service was lower than anticipated. There are various reasons why uptake may have been poor and this should be the focus of further research. It may be related to the time required to submit questions, although the service was designed to ensure this was minimal. It takes time for clinicians to incorporate novel services into clinical routines; 10 months may have been insufficient. Some GPs may not perceive the evidence-based medicine approach as helpful to their day to day work.20
There is still much to be learned about the best search strategy to employ. The cascade we used was systematic and a practical way of using different resources. The order in which the resources were used (especially at steps 2–7 of box 2) does not necessarily equate with “quality” of evidence retrieved at each level and, as such, this should not be regarded as a search “hierarchy”. Neither is it fully comprehensive. However, it allowed a consistent logical approach. Step 7 (other evidence-based medicine web sites) was usually carried out only when the informaticist had prior awareness of a relevant website for a particular question. It is unknown whether using a longer (or, indeed, shorter) cascade would significantly alter the search results or the answers provided. It has been suggested that a pragmatic approach to searching is effective,21 but we are unaware of any research on the relative benefits of different approaches in the context of providing evidence-based information for clinicians. It is possible that a flexible approach could be adopted in which users of informaticist services could determine their preferred approach in advance. It is likely that the “stakes” attached to different questions vary, and it may be that different search strategies could be used in different circumstances.
The clinical informaticist service seems to represent a useful additional method of acquiring evidence-based information in primary care. It has the added advantage that the informaticist responds directly to the specific information needs of clinicians and that communication is from clinician to clinician. There are still aspects that require further research, but we have shown that a service like this is highly valued by users and leads to reported changes in practice.
The authors would like to thank all the GPs and nurses who took part in this project and in the pilot study, and Mr R Wentz for his contribution to training the clinical informaticist.
Contributors: MP was the principal investigator and took the lead in designing the study. DS contributed to the design of the study and data collection, acted as clinical informaticist and wrote the first draft of the paper. JF contributed to the design of the evaluation, collected quantitative data, conducted the interviews, and analysed the qualitative data. MP and JF commented on drafts of the paper.
Funding: NHS Executive (North Thames).
Competing interests: None
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.