Interventions in primary care to reduce medication related adverse events and hospital admissions: systematic review and meta-analysis
======================================================================================================================================

* S Royal
* L Smeaton
* A J Avery
* B Hurwitz
* A Sheikh

## Abstract

**Objective:** To identify and evaluate studies of interventions in primary care aimed at reducing medication related adverse events that result in morbidity, hospital admission, and/or mortality.

**Methods:** Fourteen electronic databases were systematically searched for published and unpublished data. Bibliographies of retrieved papers were searched and experts and first authors contacted in an attempt to locate additional studies. There were no restrictions on language of publication. All interventions applied in primary care settings which aimed to improve patient safety by reducing adverse events resulting from medication overuse or misuse were considered. Randomised controlled trials, controlled trials, controlled before and after studies, and interrupted time series studies were eligible for inclusion. Study quality assessment and data extraction were undertaken using the Cochrane Effective Practice and Organisation of Care data collection checklist and template. Meta-analysis was performed using a random effects model.

**Results:** 159 studies were initially identified, of which 38 satisfied our inclusion criteria. These were categorised as follows: 17 pharmacist-led interventions (of which 15 reported hospital admissions as an outcome); eight interventions led by other primary healthcare professionals that reported preventable drug related morbidity as an outcome; and 13 complex interventions that included a component of medication review aimed at reducing falls in the elderly (the outcome being falls). Meta-analysis found that pharmacist-led interventions are effective at reducing hospital admissions (OR 0.64 (95% CI 0.43 to 0.96)), but restricting analysis to the randomised controlled trials failed to demonstrate significant benefit (OR 0.92 (95% CI 0.81 to 1.05)). Pooling the results of studies in the other categories did not demonstrate any significant effect.

**Conclusions:** There is relatively weak evidence to indicate that pharmacist-led medication reviews are effective in reducing hospital admissions. There is currently no evidence for the effectiveness of other interventions which aim at reducing admissions or preventable drug related morbidity. More randomised controlled trials of primary care based pharmacist-led interventions are needed to decide whether or not this intervention is effective in reducing hospital admissions.

*   medication error
*   patient safety
*   primary care

Medication related adverse events in primary care represent an important common cause of morbidity.1 A recent prospective cohort study has shown that, within 4 weeks of receiving a primary care prescription, 25% of patients experience an adverse drug event, 11% of which are judged preventable.2 A systematic review and meta-analysis reported that a median 7.1% (interquartile range 5.7–16.2) of hospital admissions result from drug related problems, of which 59% were considered preventable (that is, attributable to error).3

Clinical errors, escalating costs of negligence claims, and continuing public debate about the prevalence of drug related morbidities have raised the profile of safety considerations in delivery of health care. Improving patient safety is therefore now a government priority in many economically developed countries including the UK and USA.4,5 Reduction of prescribing errors is of particular interest, both as a result of the disease burden posed and the likelihood of finding effective interventions.

To date, however, there has been no systematic review to help inform the development of interventions aimed at reducing the incidence of preventable drug related morbidity. Furthermore, there has been little research seeking to evaluate interventions that might lead to safer prescribing. We therefore sought systematically to identify and evaluate studies of interventions delivered in primary care settings which aimed to reduce preventable drug related morbidity.

## METHODS

### Searching

A systematic search for published material was performed, initially for the period 1981–2001 and then extended for the main biomedical databases to 2005. Medical subject headings and text words were used in 10 electronic databases: Cochrane Database of Systematic Reviews (Issue 1, 2005), Cochrane Effective Practice and Organisation of Care (EPOC) specialised register, Cochrane Controlled Trials Register (CCTR) (Issue 1, 2005), MEDLINE (1966–Feb 2005), EMBASE (1980–Feb 2005), CINAHL (1982–Feb 2005), Psychinfo (1966–2001), Pharmline (1978–2001), Science Citation Index (1981–2001), and International Pharmaceutical Abstracts (1970–2001).

A further four databases were searched to identify dissertations and unpublished work including: the UK National Research Register (Issue 4, 2001), Dissertation Abstracts (1994–2001), Index to Thesis (1970–2001), and the System for Information on the Grey Literature (SIGLE). Bibliographies of key background papers and studies included in the review were also searched to identify additional published studies. In an attempt to identify other relevant unpublished studies, we wrote to subject experts and the first authors of included studies.

Search strategies, customised for each database, did not employ any language restriction and comprised four key concepts: study design, primary care setting, medication, and error. Search strategies were designed for each concept and then combined. Full details of the search strategy used are available from the first author.

### Selection

In keeping with the Cochrane EPOC guidelines, we accepted data from randomised controlled trials and high quality controlled clinical trials, controlled before and after studies, and interrupted time series studies. Table 1 describes the quality criteria used to assess each study design.

View this table:
[ Table 1 ](http://qualitysafety.bmj.com/content/15/1/23/T1)

 **Table 1**  
 EPOC inclusion criteria for study design

Studies were eligible for inclusion if they involved health care professionals providing community based family medical services. Community settings included family and general practice, community pharmacies, and nursing and residential homes. Studies of interventions in clinics attached to a hospital were excluded unless they were described as a primary care clinic.

We included interventions applied in primary care which aimed to reduce drug-related morbidity, hospitalisation or death resulting from medication overuse or misuse. We did not include studies that contained data solely relating to errors of underuse.6

Two reviewers independently screened the titles and abstracts retrieved to assess studies against the inclusion criteria. Full text copies of all papers considered to be of potential relevance were obtained and first authors of studies were contacted for clarification where necessary. Any disagreement about relevance was resolved by discussion between the reviewers.

### Validity assessment

The quality of all included studies was assessed independently by two reviewers, using the criteria developed by the EPOC group.7 Parameters including baseline measurements, concealment of allocation, blinding of outcome assessors, and losses to follow up were assessed.

### Data abstraction and synthesis

Data extraction was completed by one reviewer and checked by a co-reviewer using a data collection template. Discrepancies were resolved by discussion between reviewers. Studies were grouped together according to similarity of interventions and common outcomes. STATA 8 software was used to pool data; random effects models were used to allow for the anticipated statistical heterogeneity between studies. Unadjusted data from studies in which participants were recruited in clusters were adjusted for the clustering effect assuming an intraclass correlation coefficient (ICC) of 0.02.8

## RESULTS

### Description of studies

159 studies were identified, of which 38 satisfied our inclusion criteria. The main reasons for excluding studies are summarised in the QUOROM flow diagram (fig 1).9 Our searches also identified 10 systematic reviews in related areas10–19 that provided additional references.

![ Figure 1 ](http://qualitysafety.bmj.com/https://qualitysafety.bmj.com/content/qhc/15/1/23/F1.medium.gif)

[ Figure 1 ](http://qualitysafety.bmj.com/content/15/1/23/F1)

 **Figure 1**  
 QUOROM flow diagram.

The characteristics of included studies are described in table 2. Eighteen studies were set in the USA, 16 in Europe, three in Australia, and one in New Zealand. Most studies examined a number of patient outcomes (for example, mortality rates, morbidity assessments and quality of life scores), while others examined data on processes of care (for example, completed medication reviews and drug utilisation data). Few studies, however, used patient outcomes as an a priori defined primary end point and none were designed to link patient outcomes causally to drug related adverse events.

View this table:
[ Table 2 ](http://qualitysafety.bmj.com/content/15/1/23/T2)

 **Table 2**  
 Characteristics of included studies

### Methodological quality of included studies

Comments on the important methodological features of each study are presented in table 2. None of the studies made any adjustment for a clustering effect in the data presented, and none that used randomisation described this in sufficient detail for us to comment on the adequacy of concealment. We were, through discussion, able to classify studies according to the main features of the intervention.

### Pharmacist-led interventions

Seventeen studies included a medication review component in the intervention arm that was performed by a pharmacist.20–36 Thirteen of these studies20–32 included hospital admission data in a form that allowed the calculation of an odds ratio to summarise the findings; the remaining four did not, however, present data in this form and were excluded from the meta-analysis.33–36 We found significant heterogeneity between studies (χ2 = 126.71, df = 12, p<0.001). Random effects meta-analysis showed a significant positive effect of these interventions on hospital admissions (OR 0.64 (95% CI 0.43 to 0.96), fig 2).

![ Figure 2 ](http://qualitysafety.bmj.com/https://qualitysafety.bmj.com/content/qhc/15/1/23/F2.medium.gif)

[ Figure 2 ](http://qualitysafety.bmj.com/content/15/1/23/F2)

 **Figure 2**  
 Forest plot of pharmacist-led intervention studies.

A sensitivity analysis restricting the included studies to randomised controlled trials removed the heterogeneity (χ2 = 5.62, df = 7, p = 0.58) but no longer found a positive effect (OR 0.92 (95% CI 0.81 to 1.05), fig 3). A sensitivity analysis using an ICC of 0.01 when adjusting the results of clustered studies did not affect the above results.

![ Figure 3 ](http://qualitysafety.bmj.com/https://qualitysafety.bmj.com/content/qhc/15/1/23/F3.medium.gif)

[ Figure 3 ](http://qualitysafety.bmj.com/content/15/1/23/F3)

 **Figure 3**  
 Forest plot of pharmacist-led intervention randomised controlled trials.

A funnel plot was prepared and this suggested the presence of publication bias (fig 4). This was supported by Begg’s rank correlation p value for bias of 0.04, but not by Egger’s weighted regression method (p value for bias 0.88).

![ Figure 4 ](http://qualitysafety.bmj.com/https://qualitysafety.bmj.com/content/qhc/15/1/23/F4.medium.gif)

[ Figure 4 ](http://qualitysafety.bmj.com/content/15/1/23/F4)

 **Figure 4**  
 Funnel plot of all pharmacist-led interventions.

### Interventions led by other primary healthcare professionals

Eight studies reported interventions led by other primary healthcare professionals. Nurses used protocols to manage diabetes, heart failure, depression, and asthma in six of these37–42 and the remaining two involved education programmes for primary care physicians.43,44 Four of the nurse led interventions reported the incidence of adverse drug events which satisfied our inclusion criteria and allowed the calculation of an odds ratio.37,39–41 These were combined in a meta-analysis but no significant effect was found (OR 1.05 (95% CI 0.57 to 1.94)); there was no significant heterogeneity (χ2 = 1.95, df = 3, p = 0.58).

### Complex interventions to reduce falls in the elderly

Thirteen studies described interventions with a number of components that aimed to reduce the incidence of falls in the elderly.45–57 To be included in this review, one of the components had to be a medication review undertaken by a primary healthcare professional, the presumption being made that any reduction in the incidence of falls was at least in part a reduction in drug related morbidity. Nine of the studies presented data in a way which allowed the calculation of an odds ratio and these were pooled in a meta-analysis.45–53 No significant effect was demonstrated (OR 0.91 (95% CI 0.68 to 1.21)) and there was no significant heterogeneity (χ2 = 14.59, df = 8, p = 0.07).

### Studies not included in the meta-analysis

Table 2 presents the key features of the design and the principal findings of all studies that satisfied our inclusion criteria, including those that could not be included in the meta-analysis.

## DISCUSSION

We have shown that there is some evidence that pharmacist-led interventions incorporating a medication review component are effective in reducing hospital admissions. However, when restricted to randomised controlled trials (which are less susceptible to bias than controlled before–after studies and interrupted time series analysis), the pooled odds ratio became non-significant. We found no evidence of any significant effect of primary care medication reviews aimed at reducing falls in the elderly on the primary outcome, or of nurse-led chronic disease management programmes in reducing drug related morbidity.

### Strengths of review

We searched a very broad range of published and unpublished sources of information and coupled this with rigorous quality assessment and appraisal of studies. We deliberately narrowed the focus of the review to those studies which attempted to address errors resulting in actual patient harm as opposed to process outcomes only.

### Limitations of review

Publication bias is an important potential source of bias in systematic reviews.58 Considerable effort was therefore made to locate unpublished studies. However, a small number may have been omitted from the review, as is suggested by the borderline assessment of evidence of publication bias.

The setting for this review was primary care and our findings are unlikely to be applicable to all healthcare systems. For example, studies undertaken in ambulatory patients based in general medical clinics in the USA met our inclusion criteria but their relevance to the primary care systems of Western Europe can be questioned. We deliberately chose “bottom line” patient outcome measures as the focus of this review in order to maximise its usefulness to healthcare policy makers and service commissioners. Some studies that were included showed significant improvements in upstream outcomes and their value in this respect is not acknowledged by our criteria.

### Implications for health policy, clinical care, and future research

This systematic review has shown a paucity of high quality evaluations of interventions aimed specifically at preventing medication related adverse events in primary care. The clinical implications of these studies are therefore at present limited.

Given the high disease burden associated with prescribing errors in primary care, there is a pressing need for further studies in this field. In developing future interventions, researchers should focus on patient safety and should endeavour to select outcome measures that allow for ready comparisons with other studies. For example, criteria exist to classify hospital admissions as “medication related”, yet none of the studies identified in the review used these criteria.4 Future studies need to be powered adequately to be able to detect clinically important reductions in prescribing errors, and they should consider building in a cost-effectiveness analysis.

In the USA and several other countries, the use of information technology to support medication safety is well developed. We were therefore surprised not to find more evaluations of the role of computers in improving patient safety in primary care, given the benefits that have been shown to accrue from its use in hospital facilities.59 There is therefore a need to assess the effectiveness of these system interventions in preventing medication related adverse events, and to evaluate future developments in these systems.

## CONCLUSIONS

There is some evidence that pharmacist-led interventions aimed at optimising medication regimens are effective in reducing hospital admissions from primary care. Larger, rigorously designed intervention studies are now needed to evaluate whether the significantly increased body of understanding of the causes of medication errors can be translated into meaningful improvements in patient outcomes.

### **Key messages**

*   Medication related adverse events originating in primary care are an important cause of morbidity and mortality.

*   There has been limited formal evaluation, using randomised controlled study designs, of interventions aiming to reduce medication related adverse events in primary care.

*   Relatively weak evidence was found that pharmacist-led medication reviews are effective in reducing hospital admissions.

*   There was no evidence for the effectiveness of other interventions aimed at reducing admissions or preventable drug related morbidity.

*   More work is needed in the development and rigorous evaluation of interventions in this field.

## Footnotes

*   AA, AS and BH conceived, designed and secured funding for the study with SR assisting in study design. SR and LS undertook the literature searches and selected studies for inclusion and extracted and analysed data under the supervision of AS. All authors contributed to the analysis of the results with SR performing the meta-analysis. SR and LS led the drafting of this report, a process to which all the other authors contributed. AA and AS are guarantors.

*   Funding: BUPA Foundation.

*   Competing interests: None declared.

## REFERENCES

1.  **Avery AJ**, Sheikh A, Hurwitz B, *et al.* Safer medicines management in primary care. Br J Gen Pract 2002;52 (Suppl) :S17–22.
    
    

2.  **Gandhi TK**, Weingart SN, Borus J, *et al.* Adverse drug events in ambulatory care. N Engl J Med 2003;348:1556–64.
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1056/NEJMsa020703&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=12700376&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000182248900007&link_type=ISI) 

3.  **Winterstein AG**, Sauer BC, Hepler CD, *et al.* Preventable drug-related hospital admissions. Ann Pharmacother 2002;36:1238–48.
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3Bhb3AiO3M6NToicmVzaWQiO3M6MTE6IjM2LzctOC8xMjM4IjtzOjQ6ImF0b20iO3M6MTc6Ii9xaGMvMTUvMS8yMy5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

4.  **Department of Health**. An organisation with a memory. Report of an expert group on learning from adverse events in the NHS. London: The Stationery Office, 2000.
    
    

5.  In: **Kohn LT**, Corrigan JM. Donaldson MS, eds. To err is human: building a safer health system. Washington, DC: National Academy Press, 2000.
    
    

6.  **Chassin MR**, Galvin RW. The urgent need to improve health care quality. Institute of Medicine National Roundtable on Health Care Quality. JAMA1998;280:1000–5.
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1001/jama.280.11.1000&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=9749483&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000075891400034&link_type=ISI) 

7.  **Cochrane Effective Practice and Organisation of Care Group**. Data collection checklist. 2002. Available at [http://www.epoc.uottawa.ca/tools.htm](http://www.epoc.uottawa.ca/tools.htm) (accessed 23 November 2003).
    
    

8.  **Adams G**, Gulliford MC, Ukoumunne OC, *et al.* Patterns of intra-cluster correlation from primary care research to inform study design and analysis. J Clin Epidemiol 2004;57:785–94.
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1016/j.jclinepi.2003.12.013&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=15485730&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000224719400005&link_type=ISI) 

9.  **Moher D**, Cook DJ, Eastwood S, *et al.* Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of reporting of meta-analyses. Lancet 1999;354:1896–900.
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1016/S0140-6736(99)04149-5&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10584742&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000083909100044&link_type=ISI) 

10. **Thomson O’Brien MA**, Oxman AD, Davis DA, *et al.* Educational outreach visits: effects on professional practice and health care outcomes (Cochrane Review). In: The Cochrane Library, Issue 4. Chichester, UK: John Wiley, 2003.
    
    

11. **Jamtvedt G**, Young JM, Kristoffersen DT, *et al.* Audit and feedback: effects on professional practice and health care outcomes (Cochrane Review). In: The Cochrane Library, Issue 4. Chichester, UK: John Wiley, 2003.
    
    

12. **Thomson O’Brien MA**, Freemantle N, Oxman AD, *et al.* Continuing education meetings and workshops: effects on professional practice and health care outcomes (Cochrane Review). In: The Cochrane Library, Issue 4. Chichester, UK: John Wiley, 2003.
    
    

13. **Thomson O’Brien MA**, Oxman AD, Haynes RB, *et al.* Local opinion leaders: effects on professional practice and health care outcomes (Cochrane Review). In: The Cochrane Library, Issue 4. Chichester, UK: John Wiley, 2003.
    
    

14. **Beney J**, Bero LA, Bond C. Expanding the roles of outpatient pharmacists: effects on health services utilisation, costs, and patient outcomes (Cochrane Review). In: The Cochrane Library, Issue 4. Chichester, UK: John Wiley, 2003.
    
    

15. **Gillespie LD**, Gillespie WJ, Robertson MC, *et al.* Interventions for preventing falls in elderly people (Cochrane Review). In: The Cochrane Library, Issue 4. Chichester, UK: John Wiley, 2003.
    
    

16. **Haynes RB**, McDonald H, Garg AX, *et al.* Interventions for helping patients to follow prescriptions for medications (Cochrane Review). In: The Cochrane Library, Issue 4. Chichester, UK: John Wiley, 2003.
    
    

17. **Renders CM**, Valk GD, Griffin S, *et al.* Interventions to improve the management of diabetes mellitus in primary care, outpatient and community settings (Cochrane Review). In: The Cochrane Library, Issue 4. Chichester, UK: John Wiley, 2003.
    
    

18. **McGhan WF**, Einarson TR, Sabers DL, *et al.* A meta-analysis of the impact of pharmacist drug regimen reviews in long term care facilities. J Geriatr Drug Therapy 1987;1:23–34.
    
    

19. In: **Shojania K**, Duncan B. McDonald K, Wachter RM, eds. Making health care safer: a critical analysis of patient safety practice. Rockville, MD: Agency for Healthcare Research and Quality, 2001.
    
    

20. **Bernsten C**, Bjorkman I, Caramona M, *et al.* Improving the well-being of elderly patients via community pharmacy-based provision of pharmaceutical care: a multicentre study in seven European countries. Drugs Aging 2001;18:63–77.
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.2165/00002512-200118010-00005&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=11232739&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000167131800005&link_type=ISI) 

21. **Bond C**, Matheson C, Williams S, *et al.* Repeat prescribing: a role for community pharmacists in controlling and monitoring repeat prescriptions. Br J Gen Pract 2000;50:271–5.
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYmpncCI7czo1OiJyZXNpZCI7czoxMDoiNTAvNDUzLzI3MSI7czo0OiJhdG9tIjtzOjE3OiIvcWhjLzE1LzEvMjMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

22. **Bouvy ML**, Heerdink ER, Urquhart J, *et al.* Effect of a pharmacist-led intervention on diuretic compliance in heart failure patients: a randomized controlled study. J Cardiac Failure 2003;9:404–11.
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1054/S1071-9164(03)00130-1&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=14583903&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000186326500011&link_type=ISI) 

23. **Cummings DM**, Corson M, Seaman JJ. The effect of clinical pharmacy services provided to ambulatory patients on hospitalization. Am J Pharmacy1984;156:44–50.
    
    

24. **Hawkins DW**, Fiedler FP, Douglas HL, *et al.* Evaluation of a clinical pharmacist in caring for hypertensive and diabetic patients. Am J Hosp Pharmacy 1979;36:1321–5.
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=507073&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 

25. **Herborg H**, Soendergaard B, Froekjaer B, *et al.* Improving drug therapy for patients with asthma—Part 1: Patient outcomes. J Am Pharm Assoc 2001;41:539–50.
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=11486980&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 

26. **Krska J**, Cromarty JA, Arris F, *et al.* Pharmacist-led medication review in patients over 65: a randomized, controlled trial in primary care. Age Ageing 2001;30:205–11.
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoiYWdlaW5nIjtzOjU6InJlc2lkIjtzOjg6IjMwLzMvMjA1IjtzOjQ6ImF0b20iO3M6MTc6Ii9xaGMvMTUvMS8yMy5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

27. **Lai LL**. Effects of a pharmaceutical care intervention in primary care ambulatory settings among Medicaid population. J Pharm Care1998;2:1–13.
    
    

28. **Malone DC**, Carter BL, Billups SJ, *et al.* An economic analysis of a randomized, controlled, multicenter study of clinical pharmacist interventions for high-risk veterans: the IMPROVE study. Impact of Managed Pharmaceutical Care Resource Utilization and Outcomes in Veterans Affairs Medical Centers. Pharmacotherapy 2000;20:1149–58.
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1592/phco.20.15.1149.34590&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=11034037&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000089681100001&link_type=ISI) 

29. **McCombs JS**, Liu G, Shi J, *et al.* The Kaiser Permanente/USC Patient Consultation Study: change in use and cost of health care services. Am J Health Syst Pharm 1998;55:2485–99.
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpocCI7czo1OiJyZXNpZCI7czoxMDoiNTUvMjMvMjQ4NSI7czo0OiJhdG9tIjtzOjE3OiIvcWhjLzE1LzEvMjMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

30. **Roberts MS**, Stokes JA, King MA, *et al.* Outcomes of a randomized controlled trial of a clinical pharmacy intervention in 52 nursing homes. Br J Clin Pharmacol 2001;51:257–65.
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1046/j.1365-2125.2001.00347.x&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=11298072&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000167732500010&link_type=ISI) 

31. **Thompson JF**, McGhan WF, Ruffalo RL, *et al.* Clinical pharmacists prescribing drug therapy in a geriatric setting: outcome of a trial. J Am Geriatr Soc 1984;32:154–9.
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=6693703&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=A1984SE03900014&link_type=ISI) 

32. **Zermansky AG**, Petty DR, Raynor DK, *et al.* Randomised controlled trial of clinical medication review by a pharmacist of elderly patients receiving repeat prescriptions in general practice. BMJ 2001;323:1340–3.
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEzOiIzMjMvNzMyNS8xMzQwIjtzOjQ6ImF0b20iO3M6MTc6Ii9xaGMvMTUvMS8yMy5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

33. **Furniss L**, Burns A, Craig SK, *et al.* Effects of a pharmacist’s medication review in nursing homes. Randomised controlled trial. Br J Psychiatry 2000;176:563–7.
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6ImJqcHJjcHN5Y2giO3M6NToicmVzaWQiO3M6OToiMTc2LzYvNTYzIjtzOjQ6ImF0b20iO3M6MTc6Ii9xaGMvMTUvMS8yMy5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

34. **Hanlon JT**, Weinberger M, Samsa GP, *et al.* A randomized, controlled trial of a clinical pharmacist intervention to improve inappropriate prescribing in elderly outpatients with polypharmacy. Am J Med 1996;100:428–37.
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1016/S0002-9343(97)89519-8&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=8610730&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=A1996UD08000010&link_type=ISI) 

35. **Kimberlin CL**, Berardo DH, Pendergast JF, *et al.* Effects of an education program for community pharmacists on detecting drug-related problems in elderly patients. Med Care 1993;31:451–68.
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=8501992&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=A1993LC37900007&link_type=ISI) 

36. **Knowlton CH**, Knapp DA. Community pharmacists help HMO cut drug costs. Am Pharmacy1994;NS34:36–42.
    
    

37. **Aubert RE**, Herman WH, Waters J, *et al.* Nurse case management to improve glycemic control in diabetic patients in a health maintenance organization. A randomized, controlled trial. Ann Intern Med 1998;129:605–12.
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.7326/0003-4819-129-8-199810150-00004&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=9786807&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000076431800002&link_type=ISI) 

38. **Kane RL**, Garrard J, Skay CL, *et al.* Effects of a geriatric nurse practitioner on process and outcome of nursing home care. Am J Public Health 1989;79:1271–7.
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.2105/AJPH.79.9.1271&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=2504064&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=A1989AR62700013&link_type=ISI) 

39. **Piette JD**, Weinberger M, McPhee SJ, *et al.* Do automated calls with nurse follow-up improve self-care and glycemic control among vulnerable patients with diabetes? Am J Med 2000;108:20–7.
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=11059437&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000084710800005&link_type=ISI) 

40. **Olivarius NF**, Beck-Nielsen H, Andreasen AH, *et al.* Randomised controlled trial of structured personal care of type 2 diabetes mellitus. BMJ 2001;323:970–5.
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMjMvNzMxOS85NzAiO3M6NDoiYXRvbSI7czoxNzoiL3FoYy8xNS8xLzIzLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

41. **de Sonnaville JJ**, Bouma M, Colly LP, *et al.* Sustained good glycaemic control in NIDDM patients by implementation of structured care in general practice: 2-year follow-up study. Diabetologia 1997;40:1334–40.
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1007/s001250050829&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=9389427&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 

42. **Wilkinson G**, Allen P, Marshall E, *et al.* The role of the practice nurse in the management of depression in general practice: treatment adherence to antidepressant medication. Psychol Med 1993;23:229–37.
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=8475211&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=A1993KT52100025&link_type=ISI) 

43. **Avorn J**, Soumerai SB, Everitt DE, *et al.* A randomized trial of a program to reduce the use of psychoactive drugs in nursing homes. N Engl J Med 1992;327:168–73.
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1056/NEJM199207163270306&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=1608408&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=A1992JD31900006&link_type=ISI) 

44. **Kendrick T**, Burns T, Freeling P. Randomised controlled trial of teaching general practitioners to carry out structured assessments of their long term mentally ill patients. BMJ1995;311:93–8.
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjExOiIzMTEvNjk5Ny85MyI7czo0OiJhdG9tIjtzOjE3OiIvcWhjLzE1LzEvMjMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

45. **Carter S**, Campbell E, Sanson-Fisher R, *et al.* A randomised controlled trial of two strategies aimed at reducing falls and other unintentional events through home modification and review. Cited in Gillespie LD, Gillespie WJ, Robertson MC, *et al*, eds. Interventions for preventing falls in elderly people (Cochrane Review). In: The Cochrane Library. Issue 4. Chichester, UK: John Wiley, 2003.
    
    

46. **Coleman EA**, Grothaus LC, Sandhu N, *et al.* Chronic care clinics: a randomized controlled trial of a new model of primary care for frail older adults. J Am Geriatr Soc 1999;47:775–83.
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10404919&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000081323400001&link_type=ISI) 

47. **Jensen J**, Lundin-Olsson L, Nyberg L, *et al.* Fall and injury prevention in older people living in residential facilities. Ann Intern Med 2002;136:733–741.
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=12020141&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000175664600003&link_type=ISI) 

48. **McMurdo ME**, Millar AM, Daly F. A randomized controlled trial of fall prevention strategies in old peoples’ homes. Gerontology 2000;46:83–7.
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1159/000022139&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10671804&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000085600900005&link_type=ISI) 

49. **Ray WA**, Taylor JA, Meador KG, *et al.* A randomized trial of a consultation service to reduce falls in nursing homes. JAMA 1997;278:557–62.
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1001/jama.1997.03550070049037&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=9268276&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=A1997XQ98600029&link_type=ISI) 

50. **Tinetti ME**, Baker DI, McAvay G, *et al.* A multifactorial intervention to reduce the risk of falling among elderly people living in the community. N Engl J Med 1994;331:821–7.
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1056/NEJM199409293311301&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=8078528&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=A1994PH53000001&link_type=ISI) 

51. **van Haastregt JC**, Diederiks JP, van Rossum E, *et al.* Effects of a programme of multifactorial home visits on falls and mobility impairments in elderly people at risk: randomised controlled trial. BMJ 2000;321:994–8.
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMjEvNzI2Ny85OTQiO3M6NDoiYXRvbSI7czoxNzoiL3FoYy8xNS8xLzIzLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

52. **Vetter NJ**, Lewis PA, Ford D. Can health visitors prevent fractures in elderly people? BMJ1992;304:888–90.
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMDQvNjgzMS84ODgiO3M6NDoiYXRvbSI7czoxNzoiL3FoYy8xNS8xLzIzLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

53. **Wagner EH**, LaCroix AZ, Grothaus L, *et al.* Preventing disability and falls in older adults: a population-based randomized trial. Am J Public Health 1994;84:1800–6.
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=7977921&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=A1994PR44800018&link_type=ISI) 

54. **Campbell AJ**, Robertson MC, Gardner MM, *et al.* Psychotropic medication withdrawal and a home-based exercise program to prevent falls: a randomized, controlled trial. J Am Geriatr Soc 1999;47:850–3.
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10404930&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000081323400012&link_type=ISI) 

55. **Kempton A**, Van Beurden E, Sladden T, *et al.* Older people can stay on their feet: final results of a community-based falls prevention programme. Health Promotion Int 2000;15:27–33.
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoiaGVhcHJvIjtzOjU6InJlc2lkIjtzOjc6IjE1LzEvMjciO3M6NDoiYXRvbSI7czoxNzoiL3FoYy8xNS8xLzIzLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

56. **Poulstrup A**, Jeune B. Prevention of fall injuries requiring hospital treatment among community-dwelling elderly. Eur J Public Health2000;10:45–50.
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoiZXVycHViIjtzOjU6InJlc2lkIjtzOjc6IjEwLzEvNDUiO3M6NDoiYXRvbSI7czoxNzoiL3FoYy8xNS8xLzIzLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

57. **Zullich SG**, Grasela TH Jr, Fiedler-Kelly JB, *et al.* Impact of triplicate prescription program on psychotropic prescribing patterns in long-term care facilities. Ann Pharmacother 1992;26:539–46.
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=1576393&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=A1992HN28200019&link_type=ISI) 

58. **Eggar M**, Dickerson K, Davey Smith G. Problems and limitations in conducting systematic reviews. In: Eggar M, Davey Smith G, Altman DG, eds. Systematic reviews in health care. 2nd ed. London: BMJ Books, 2001:43–68.
    
    

59. **Bates DW**, Gawande AA. Improving safety with information technology. N Engl J Med2003;348:2526–34.
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1056/NEJMsa020847&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=12815139&link_type=MED&atom=%2Fqhc%2F15%2F1%2F23.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000183577200008&link_type=ISI)