You are here

Article Text

Article menu
Original research
Incorporating evidence review into quality improvement: meeting the needs of innovators
  1. Margie Sherwood Danz1,2,
  2. Susanne Hempel1,
  3. Yee-Wei Lim1,3,
  4. Roberta Shanman1,
  5. Aneesa Motala1,
  6. Susan Stockdale2,
  7. Paul Shekelle1,2,
  8. Lisa Rubenstein1,2
  1. 1RAND Corporation, Santa Monica, California, USA
  2. 2Veterans Affairs Greater Los Angeles Healthcare System, North Hills, California, USA
  3. 3Department of Epidemiology and Public Health, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Republic of Singapore
  1. Correspondence to Dr Margie Sherwood Danz, RAND Corporation, 1776 Main Street, Santa Monica, CA 90401, USA; mjsdanz{at}gmail.com

Abstract

Background Achieving quality improvement (QI) aims often requires local innovation. Without objective evidence review, innovators may miss previously tested approaches, rely on biased information, or use personal preferences in designing and implementing local QI programmes.

Aim To develop a practical, responsive approach to evidence review for QI innovations aimed at both achieving the goals of the Patient Centered Medical Home (PCMH) and developing an evidence-based QI culture.

Design Descriptive organisational case report.

Methods As part of a QI initiative to develop and spread innovations for achieving the Veterans Affairs (VA) PCMH (termed Patient Aligned Care Team, or PACT), we involved a professional evidence review team (consisting of review experts, an experienced librarian, and administrative support) in responding to the evidence needs of front-line primary care innovators. The review team developed a systematic approach to responsive innovation evidence review (RIER) that focused on innovator needs in terms of time frame, type of evidence and method of communicating results. To assess uptake and usefulness of the RIERs, and to learn how the content and process could be improved, we surveyed innovation leaders.

Results In the first 16 months of the QI initiative, we produced 13 RIERs on a variety of topics. These were presented as 6–15-page summaries and as slides at a QI collaborative. The RIERs focused on innovator needs (eg, topic overviews, how innovations are carried out, or contextual factors relevant to implementation). All 17 innovators who responded to the survey had read at least one RIER; 50% rated the reviews as very useful and 31%, as probably useful.

Conclusions These responsive evidence reviews appear to be a promising approach to integrating evidence review into QI processes.

  • Quality improvement
  • Evidence-based medicine
  • Medical homes

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/

https://doi.org/10.1136/bmjqs-2012-001722

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Context-responsive local innovation is often required for achieving successful organisational and provider behaviour change. Such innovation can provide the tools and locally appropriate policies and procedures that enable national policies or guidelines to succeed. However, these QI innovation efforts, typically carried out through quality improvement (QI) initiatives, are often expensive and may not be successful.1 ,2 Access to relevant prior evidence could potentially improve QI effectiveness and success. There is little documentation, however, showing that systematic approaches to assessing prior evidence are routinely integrated into QI efforts.3–5 One reason may be a mismatch between traditional evidence reviews and QI innovator needs.

    Traditional evidence reviews, such as those published in journals, often do not meet the specific needs of, and consequently may be underused by, QI innovators. Undertaking new reviews of this type is expensive and time-consuming; and use of existing reviews is limited by their goals. Traditional evidence reviews are primarily directed at establishing whether an innovation is effective. The yes/maybe/no answers provided by these reviews are extremely useful for a wide variety of purposes, such as policy making, guideline development, and large-scale organisational decision making. These reviews, however, include few of the intervention, evaluation, or contextual details needed by local QI teams (who generally work within established organisation guidelines and policy) to determine whether and how to apply the interventions to their own situations. Some teams are at the earliest stages of project conception and require a topic overview, while others may have a specific area of uncertainty for a project they have been working on for some time. Additionally, even if relevant reviews are available, innovation teams may lack the resources and expertise to find or assess them. Teams may have little prior experience with using evidence review, formulating QI questions, planning and implementing multidisciplinary interventions and evaluating outcomes. Finally, the limited duration typical of QI projects demands a more rapid review result than can be achieved using traditional evidence review methods. The Responsive Innovation Evidence Review (RIER) project aims to provide a reasonably objective and rapid approach to evidence review to help innovators access evidence that is relevant and tailored to their needs.

    The use of rapid reviews in healthcare is increasing and audiences for these reviews are diverse. They include policy makers, healthcare providers and managers, and patients and their representative organisations; all of whom desire evidence-based recommendations on emerging issues provided in a timely manner.6–10 Methods for rapid review are evolving and are not yet standardised.6–9 One approach being used to support policy makers involves the use of a database of policy-relevant systematic reviews with or without additional services.11 ,12 Rapid reviews may limit literature searches by years, databases, or language; and may use sources beyond electronic searches.8 Rapid reviews also frequently emphasise user-friendliness and end-user focus.9 Although brief reports have been found to be a useful component of a health technology assessment programme,13 there is no agreement as to the impact of the strategies used to expedite evidence reviews on quality and risk of bias.6–9 Additionally, few rapid review efforts have sought to address the unique needs of local QI innovators directly. The RIER project adds to the evolving literature on rapid review methods by emphasising the specific needs of QI innovators.

    This paper describes the process and tools used to introduce responsive evidence review into innovation projects; discusses the approaches considered and the search strategies used; and reports on a survey of innovation team members regarding uptake and usefulness of the evidence review programme with suggestions for improvement. Our evaluation questions are: (1) are review results perceived as useful by QI innovators? and (2) does the new rapid review approach show promise as a feasible method for delivering timely and relevant reviews?

    Methods

    Project

    The RIER project is a component of the Veterans Affairs (VA) VAIL-PACT initiative. PACT (Patient Aligned Care Team) represents the VA effort to implement the Patient Centered Medical Home (PCMH).14–17 VAIL (VISN 22 Veterans Assessment and Improvement Laboratory) is a large VA research/clinical effort in the Southern California region that aims to promote evidence-based QI innovation and to develop an evidence-based QI primary care culture. VAIL involves (1) six primary care QI demonstration practices in three different VA medical systems and (2) four interdisciplinary, cross-site workgroups on specific topics (eg, homelessness). Each demonstration practice is overseen by an interdisciplinary quality council that reviews and monitors local QI efforts. A regional steering committee prioritises innovations submitted by quality councils or workgroups.

    Evidence review workgroup

    The Evidence Review Workgroup was created to explore and test a way of introducing evidence review into ongoing and future innovation projects that are part of the VAIL-PACT project. The workgroup includes Southern California Evidence-based Practice Center (EPC) and VA researchers, the EPC reference librarian, and the EPC project assistant. The Southern California EPC is part of the EPC Program established by the Agency for Healthcare Research and Quality (AHRQ) to produce the AHRQ Evidence Reports.

    Innovator–evidence review workgroup interaction

    To help focus and formulate the evidence review requests, the Evidence Review Workgroup developed a request form (see online supplementary appendix A). The request form has undergone several revisions over time to maximise its effectiveness in practice. The current version requires the innovation teams to specify which problem, area of concern, or outcome measure the team aims to improve. The form also asks teams to formulate specific questions they want to have answered. Finally, the form requires the team to specify which innovations are being considered.

    Often, innovation teams have already identified a particular area or intervention category, or are limited by system constraints. In some cases, this narrows the list of possible QI interventions reviewed. In other cases, based on clinical judgment, Evidence Review Workgroup staff may determine it is necessary to broaden the review beyond the identified interventions.

    Search strategy

    The overviews employ a number of search strategies selected for their effectiveness in identifying relevant information. The approach and features that are now in RIERs were developed over the course of the project. Searches are performed by the EPC reference librarian, and studies are typically selected by one reviewer; data are abstracted by a graduate student and checked by an EPC systematic reviewer; and the literature flow is managed by the EPC project assistant.

    Every RIER includes a search for systematic reviews on the topic of interest. For this, we use the Systematic Review Clinical Query function in the PubMed database. We also search the Database of Abstracts of Reviews of Effects (DARE) which covers published systematic reviews, Cochrane reviews and AHRQ Technology Assessment (TA) reports. If exploratory searches reveal key articles, we use these as seed articles for additional searches. For this, we employ the PubMed Related Citations function for automated relevance ranking, and we conduct Forward Searches in the Web of Science by identifying articles that cited the seed articles.

    Since innovators are often interested in research information stemming from comparable systems, we may restrict searches to the VA setting using the author affiliation field coding in PubMed. Additionally, some of our searches target research in settings with established PCMH models. We, therefore, developed a PCMH search filter using an article set maintained by the VAIL-PACT project. This article set contains publications considered relevant for the development of medical home-based QI innovations. We used it as the reference standard and iteratively tested and improved the filter. The PCMH search strings for the PubMed, CINAHL and EMBASE databases are presented in box 1.

    Box 1

    PCMH Search filter

    • Database PubMed:

    • “patient centered medical home” OR “patient centered medical homes” OR “medical home” OR “medical homes” OR “advanced medical home” OR: “advanced medical homes” OR “patient-aligned care team” OR “patient aligned care team” OR “patient-aligned care teams” OR “patient aligned care teams” OR pcmh

    • NOT

    • letter[pt] OR editorial[pt] OR editorial* OR letter*

    • Database CINAHL:

    • “patient centered medical home” OR “patient centered medical homes” OR “medical home” OR “medical homes” OR “advanced medical home” OR “advanced medical homes” OR “patient-aligned care team” OR “patient aligned care team” OR “patient-aligned care teams” OR “patient aligned care teams” OR pcmh

    • NOT

    • letter[pt] OR editorial[pt]

    • Search modes—Phrase Searching (Boolean)

    • Database EMBASE:

    • “patient centered medical home” OR “patient centered medical homes” OR “medical home” OR “medical homes” OR “advanced medical home” OR “advanced medical homes” OR “patient-aligned care team” OR “patient aligned care team” OR “patient-aligned care teams” OR “patient aligned care teams” OR pcmh

    To identify the ‘How to’ information for design and implementation of the innovations, we employ Google searches, consult content experts, or identify resources referenced in research publications (reference mining).

    In deciding which articles or ‘How to’ sources to include in the evidence overviews, we typically use the following priority criteria: (1) systematic reviews on relevant topics, (2) articles published by established content experts in peer-reviewed journals and (3) manuals and tools found on websites created by recognised organisations conducting relevant QI studies.

    Format for RIERs overviews

    RIERs use a standard format aimed at enabling a diverse innovator group to access and interpret the information. Overviews include the review questions or objectives, the methods employed to answer the questions, the results of the review, the references cited, and an evidence table summarising the key articles. The use of evidence tables is an established method in systematic reviews to enable comprehensive literature overviews. The RIERs provide sufficient detailed information to allow innovators to obtain more details if desired. The three primary components of the overviews are: (1) an effectiveness assessment of the likelihood that the general approach proposed for the innovation could result in the desired outcomes; (2) ‘how to’ information on designing and implementing the innovations with access to online sources; (3) a summary of key articles in an evidence table. The focus and scope of the reviews determines the weight given to each of the components.

    Dissemination

    To introduce the evidence review programme, we presented 11 RIERs orally at a project collaborative (a meeting of leaders from the quality councils, workgroups and innovation teams) at which progress for each innovation project was presented. The pertinent evidence overviews were presented together with the progress reports. Written copies of the RIERs were also available. Later, RIERs were emailed or distributed in written form, and all RIERs were placed on a SharePoint site available to members of the quality councils, workgroups and innovation teams.

    Evaluation survey

    To test the response of members of the quality councils, workgroups and innovation teams to the evidence overviews, we conducted an online survey. The survey asked respondents to assess and comment on the usefulness of RIERs and on how both the content and the process of obtaining the RIERS could be improved. The survey was anonymous and was designed to capture participant experiences over the first 16 months of the programme.

    Results

    Evidence reviews

    In the first 16 months of the project, the team provided 13 RIERs covering the following topics: Advanced Access, Homelessness, Interactive Communication, Motivational Interviewing, PCMH Evaluation Measures, Patient Registries, PCMH and Mental Health, Pharmacists, Primary Care Team Functioning, Readmissions, Relational Coordination, Secure Messaging, and Self-Management (table 1).

    View this table:
    Table 1

    Description of responsive innovation evidence reviews (RIERs)

    All but two of the topics were selected based on innovation projects already approved by the VAIL Steering Committee and at various stages of planning, implementation and evaluation at the time the RIERs were initiated. The reviews varied based on the goals and questions of those requesting the reviews. Some of the reviews were primarily topic overviews, some were ‘how to’ reviews, and some addressed specific items. For example, the Homelessness review focused on homelessness only in the context of the PCMH for which the literature available was very limited.

    The 11 reviews presented at the collaborative were initiated to support innovation projects, introduce the evidence review resource to the innovation teams, and encourage development of an evidence-based QI culture among the team members. These reviews were requested by the VAIL leadership based on discussions with the innovation teams, quality councils and workgroups. The quality councils oversee innovation activities within the participating demonstration sites and are composed primarily of clinicians with patient care and/or administrative responsibilities; workgroups include members from one or more demonstration sites and are generally composed of both clinicians with direct patient care responsibilities and academic researchers. The other two topics were requested directly by innovators after the collaborative meeting. Reviews were produced in 2–6 weeks depending on the project timeframe and requirements.

    RIERs ranged in length from six to 15 pages with a mean of nine pages. The text was purposefully kept short with about 1000 words per topic (500–2100 words, mean 1171). The number of citations varied between three and 60 (mean 16, median 11) according to the review questions and purpose. Evidence tables described 3–31 key articles (mean 11 articles, median eight). Citations and key articles were included to provide sources that the innovation team members could refer to for additional information or guidance. In some reviews, citations and key articles were also used to illustrate the conclusions. We included systematic reviews where possible. For example, in the Self-Management review with six items in the evidence table, one article was a review of 83 reviews and meta-analyses; another, a review of 30 Cochrane systematic reviews; a third, a Cochrane review of 17 studies; and a fourth, a Cochrane review of 14 studies.

    Two examples of RIERs are shown in online supplementary appendix B. The evidence review on Advanced Access was requested by the VAIL leadership to support a VISN system redesign project. The objective of the review was to provide an overview for innovators on what is being done and to generate ideas for new interventions. The evidence review on Interactive Communication was requested by the Primary Care/Mental Health Provider Communication Workgroup to provide information on effective and efficient ways of enhancing two-way communication between mental health and primary care providers and on the associated barriers and facilitators. The workgroup itself had 11 members—nine clinicians, two of whom had research experience, and two academic researchers. The clinicians consisted of primary care physicians, psychiatrists, nurse practitioners and a nurse manager. The workgroup also had an advisory group consisting of one primary care physician, three clinical social workers, and one researcher. The workgroup received additional input from the Sepulveda Primary Care/Mental Health Integration Workgroup.

    Evaluation survey

    Respondents

    Seventeen out of 28 invited innovators (61%) responded to the rapid review survey. Seventy-five per cent of respondents were members of the site-specific management teams (quality councils), 31% were members of the functionally based workgroups, and 50% were members of individual innovation teams. At the time the RIERs were first introduced, 63% of respondents were involved with innovations in the implementation stage; 31% in the planning stage; only 6% in the evaluation stage; and no respondent was working on spread or dissemination of an innovation.

    Over 80% of respondents had some experience with performing literature or information searches to support project planning or activities: of these, 71% had used PubMed or other academic databases, and 57% had performed Google or other internet searches. Close to 60% of respondents had requested or received help with literature reviews from project staff prior to the creation of the Evidence Review Workgroup.

    Uptake

    Most (88%) of the respondents were present for the RIER presentations at the project collaborative meeting when they were first introduced; and all respondents had read at least one of the write-ups. About 24% indicated that they had requested a rapid review following the May 2011 meeting. In all, 53% of respondents indicated that they would request a rapid review from the Evidence Review Workgroup in the future.

    Satisfaction

    Overall, 50% of respondents rated the RIERs as very useful, and 31% as probably useful. About 50% of respondents thought the RIERs were very useful in helping them to think more clearly and broadly about their areas of concern, and 44% of respondents stated that they were very helpful in helping them to identify next steps in the innovation process. Finally, 56% thought the RIERs were helping them to gain confidence in how the innovation project fits into established evidence from the literature. Examples of the qualitative comments regarding the RIERs are shown in box 2.

    Box 2

    Satisfaction with the service

    • I was very impressed by the scope of the reviews and the excellent summaries

    • I think the presentations and summaries are fine. I would like to start using them during the planning stage because I really think they could be helpful in further developing our innovations.

    • The evidence review is good already. I didn't fully understand what they provided until I saw the product at the May 2011 VAIL Collaborative in Costa Mesa. Now that I see what they do, I am more inclined to communicate with them and bounce ideas back and forth starting in the beginning phases of an innovation.

    • The rapid reviews are very helpful. The ability to request a rapid literature review is an important asset in expediting project or programme development.

    • Thank you for this wonderful resource. We will try to use it more effectively

    Although respondents were generally positive about the content of the reviews, 26% indicated that they did not know how the process of requesting and receiving evidence overviews worked or felt it was not working very well. The only negative comments that we received stated that the process of requesting the reviews was confusing.

    Suggestions for improvement

    The respondents had a number of suggestions for improving the rapid review process. Most respondents felt that they would like to have more contact with the Evidence Review Workgroup staff to discuss the problem (87%) before the RIER is started. A substantial number would like to have contact both during the RIER production process (67%) and after the RIER is completed (60%). Examples of improvement suggestions are shown in box 3. Several respondents indicated that reminders about the service would be helpful since the resource may be forgotten in the day-to-day activities of clinical practice and working on the project.

    Box 3

    Example suggestions for improvement

    • It would be helpful to have a conversation with a VAIL Researcher prior to putting in a request to ensure that the review request is narrowly and adequately focused.

    • Speak with the people developing the innovation and ask them what they are struggling with. As projects develop, it would be nice to have direct contact with the Evidence Review Workgroup, since other issues (especially related to evaluation) come up.

    • Have the innovation team talk directly to those conducting the rapid review in order to get a sense of context for the overall innovation project. It would eliminate time wasted researching areas that are not within the scope of the innovation.

    • Require the requestors to narrow the focus of the request and articulate clearly what they desire to know. Provide a mechanism for communication between the requestors and those conducting the review.

    Discussion

    Currently, routine local QI methods do not emphasise the integration of systematic evidence review and innovation design and evaluation. Efforts to bring evidence into practice, however, could be strengthened and magnified if evidence reviews were used by local QI teams. Additionally, the success of the QI projects themselves might be enhanced. We propose that further efforts to tailor evidence review approaches to the needs and constraints of QI innovators are needed. This paper reports on our ongoing efforts to develop such an approach to enable innovators to effectively use available information to guide innovation activities.

    Preliminary results, based on experience with 13 RIERs, produced during the project's first 16 months, appear promising in terms of both user acceptability and the time frame within which the reviews were generated. Our innovator survey revealed that all respondents had read at least one of the RIERs. Fifty per cent rated the RIERs as very useful, and 31% as probably useful. Innovator process improvement comments focused on requests for more interaction with the Evidence Review Workgroup, suggesting that future efforts should explore how to facilitate this type of communication. The review team also met its goal of generating each review in a 1–2-month time frame.

    The RIER approach also appears promising in terms of the strategy used to provide relevant and timely information while minimising bias. Our strategy first involves a search for systematic reviews, and then uses key seed articles to identify additional articles looking backward and forward in time. By using terms such as ‘VA’, and predeveloped search strategies such as ‘PCMH’, we can maximise the proportion of articles with relevant contextual information in the search results. The RIER approach may provide a useful foundation for additional investigation and methodological development.

    As the programme evolved, we found there was a need to support the innovators in formulating review questions and deciding what kind of information would be most useful to obtain. We addressed this by developing a RIER request form. The RIER request form helps to focus the scope of the review and facilitate interaction between the review group and the innovators to ensure that the review meets innovator needs. In providing evidence summaries for regional managers and stakeholders, one study9 reported on an approach that also required collaborative development of a clear and effective research question and proposal.

    Support for QI requires information about advances in evidence-based medicine as well as information about how to implement advances in routine practice.18 ,19 Traditional reviews typically emphasise the former. We found that the implementation information often desired by members of the innovation teams belongs to the latter. Frequently sought information included (1) whether an innovation had been implemented in a similar context20; (2) how much designing and implementing the innovation would cost and (3) exactly how the innovation was carried out. Although the Standards for Quality Improvement Reporting Excellence (SQUIRE) reporting guidelines recommend including enough information about an intervention that it can be reproduced,21 most journal articles do not include or reference this type of information. We, therefore, searched resources such as Google Scholar and other relevant web sites to identify manuals, tools and other information from tested innovations that we identified in our searches. Links to these types of information are included in the reviews. Since innovators often need to extrapolate design and implementation strategies from one context to another, we facilitated contact with content experts by including relevant links in the reviews.10 ,13 ,22

    Questions to be explored differ among various types of evidence review stakeholders, including innovators, policy makers, managers and individual clinicians.14 ,15 Different needs among stakeholders requiring different evidence review methodologies suggest that rather than attempting to formalise the components of rapid reviews, emphasis should be placed on clearly describing what was done and on discussing potential bias and impact on validity of results.10 ,13 ,14 In this paper, we attempt to explore and document our experience in meeting the needs of one stakeholder group, rather than in developing a generic approach to rapid evidence review.

    A limitation of our evaluation is the lack of formal cost assessment. By leveraging the resources of an evidence synthesis centre, we were able to carry out the reviews using limited amounts of experienced reviewer and librarian time. We do not know to what extent this would be feasible under other circumstances. Even with an experienced EPC librarian and experienced reviewers, we initially tried a variety of approaches in developing our search strategies, and ‘a learning period’ should be expected for organisations trying this type of review for the first time. Searches related to QI are often harder to capture due to the diverse nature of projects.23 It should be noted that we did not keep track of the literature flow (eg, number of articles screened), and did not record the reasons for exclusion/inclusion of articles in order to adhere to rapid turnaround times for reviews. Another limitation of our approach is that we only assessed yield, uptake and satisfaction with the service; the accuracy of the RIERs was not formally tested. The Evidence Review Workgroup staff relied on their systematic review experience to produce valid and unbiased reports. Broad overviews, however, may miss issues that comprehensive systematic reviews will uncover. In a published comparison between rapid and full reviews, substantial differences were found; however, the authors also highlighted that the essential conclusions of the reviews did not differ extensively.10 ,13 Finally, we have not measured the impact of the RIERs on innovator design and implementation activities. Generally, literature on the impact of systematic evidence review and the interventions that encourage healthcare policy makers and managers to use evidence review is sparse.24

    In summary, based on the first 16 months of an ongoing VA evidence-based QI project aimed at primary care redesign, RIERs showed promise for increasing the impact of evidence review on QI initiatives.14 Additional research and development of systematic approaches for integrating prior evidence into QI are critical for maximising the sophistication and impact of QI efforts in healthcare organisations.

    Acknowledgments

    We would like to thank John Ovretveit and Miriam de Kleijn for helpful comments on earlier drafts of this paper.

    References

    1. ↵
      1. Liu CF,
      2. Hedrick SC,
      3. Chaney EF,
      4. et al
      . cost-effectiveness of collaborative care for depression in a primary care veteran population. Psychiatr Serv 2003;54:698–704.
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. Liu CF,
      2. Rubenstein LV,
      3. Kirchner JE,
      4. et al
      . Organizational Cost of quality improvement for depression care. Health Serv Res 2009;44:225–44.
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Goldberg HI,
      2. Wagner EH,
      3. Fihn SD,
      4. et al
      . A randomized controlled trial of CQI teams and academic detailing: can they alter compliance with guidelines?. Jt Comm J Qual Improv 1998;24:130–42.
      OpenUrlPubMed
    4. ↵
      1. Solberg LI,
      2. Fischer LR,
      3. Wei F,
      4. et al
      . A CQI intervention to change the care of depression: a controlled study. Eff Clin Pract 2001;4:239–49.
      OpenUrlPubMed
    5. ↵
      1. McLaughlin C,
      2. Kaluzny A
      . Continuous quality improvement in health care: theory, implementation, and applications. London, UK: Jones and Bartlett Publishers International, 2004.
    6. ↵
      1. Watt A,
      2. Cameron A,
      3. Sturm L,
      4. et al
      . Rapid reviews versus full systematic reviews: an inventory of current methods and practice in health technology assessment. Intl J Technol Assess Health Care 2008;24:133–9.
      OpenUrl
    7. ↵
      1. Watt A,
      2. Cameron A,
      3. Sturm L,
      4. et al
      . Rapid versus full systematic reviews: validity in clinical practice? ANZ J Surg 2008;78:1037–40.
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      1. Gannan R,
      2. Ciliska D,
      3. Thomas H
      . Expediting systematic reviews: methods and implications of rapid reviews. Implement Sci 2010;5:56–66.
      OpenUrlCrossRefPubMed
    9. ↵
      1. Khangura S,
      2. Konnyu K,
      3. Cushman R,
      4. et al
      . Evidence summaries: the evolution of a rapid review approach. Syst Rev 2012;1:10–19.
      OpenUrlCrossRefPubMed
    10. ↵
      1. Wyer PC
      . The critically appraised topic: closing the transfer gap. Ann Emerg Med 1997;30:639–40.
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      1. Moat KA,
      2. Lavis JN
      . Supporting the use of Cochrane reviews in health policy and management decision-making: health systems evidence [editorial]. Cochrane Database Syst Rev 2011;(8):ED000019. http://www.thecochranelibrary.com/details/editorial/1018237/Supporting-the-use-of-Cochrane-Reviews-in-health-policy-and-management-decision-.html
    12. ↵
      1. Lavis JN,
      2. Wilson MG,
      3. Grimshaw JM,
      4. et al
      . Effects of an evidence service on health-system policy makers’ use of research evidence: A protocol for a randomised controlled trial. Implement Sci 2011;6:51–8.
      OpenUrlCrossRefPubMed
    13. ↵
      1. Hailey D,
      2. Corabian P,
      3. Harstall C,
      4. et al
      . The use and impact of rapid health technology assessments. Intl J Technol Assess Health Care 2000;16:651–6.
      OpenUrl
    14. ↵
      1. Rubenstein LV,
      2. Chaney EF,
      3. Ober S,
      4. et al
      . Using evidence-based quality improvement methods for translating depression collaborative care research into practice. Fam Syst Health 2010;28:91–113.
      OpenUrlCrossRefPubMedWeb of Science
    15. ↵
      1. Yano EM,
      2. Rubenstein LV,
      3. Farmer MM,
      4. et al
      . Targeting primary care referrals to smoking cessation clinics does not improve quit rates: implementing evidence-based interventions into practice. Health Serv Res 2008;43:1637–61.
      OpenUrlCrossRefPubMedWeb of Science
    16. ↵
      1. Fortney JC,
      2. Pyne JM,
      3. Smith JL,
      4. et al
      . Steps for implementing collaborative care programs for depression. Popul Health Manag 2009;12:69–79.
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      1. Yano EM,
      2. Green LW,
      3. Glanz K,
      4. et al
      . Implementation and spread of interventions into the multilevel context of routine practice and policy: implications for the cancer care continuum. J Natl Cancer Inst Monogr 2012;44:86–99..
      OpenUrlAbstract/FREE Full Text
    18. ↵
      1. Shortell SM,
      2. Rundall TG,
      3. Hsu J
      . Improving patient care by linking evidence-based medicine and evidence-based management. JAMA 2007;298:673–6.
      OpenUrlCrossRefPubMedWeb of Science
    19. ↵
      1. Glasziou P,
      2. Ogrinc G,
      3. Goodman S
      . Can evidence-based medicine and clinical quality improvement learn from each other? BMJ Qual Saf 2011;20:i13–17.
      OpenUrlAbstract/FREE Full Text
    20. ↵
      1. Shekelle PG,
      2. Pronovost PJ,
      3. Wachter RM,
      4. et al
      . Advancing the science of patient safety. Ann Intern Med 2011;154:693–6.
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      1. Ogrinc G,
      2. Mooney SE,
      3. Estrada C,
      4. et al
      . The SQUIRE (Standards for Quality Improvement Reporting Excellence) guidelines for quality improvement reporting: Explanation and elaboration. Qual Saf Health Care 2008;17(Suppl I):i13–32.
      OpenUrlAbstract/FREE Full Text
    22. ↵
      1. Bambra C,
      2. Joyce KE,
      3. Bellis MA,
      4. et al
      . Reducing health inequalities in priority public health conditions: using rapid review to develop proposals for evidence-based policy. J Public Health 2010;32:496–505.
      OpenUrlAbstract/FREE Full Text
    23. ↵
      1. Hempel S,
      2. Rubenstein LV,
      3. Shanman RM,
      4. et al
      . Identifying quality improvement intervention publications—a comparison of electronic search strategies. Implement Sci 2011;6:85–94.
      OpenUrlCrossRefPubMed
    24. ↵
      1. Perrier L,
      2. Mrklas K,
      3. Lavis JN,
      4. et al
      . Interventions encouraging the use of systematic reviews by health policymakers and managers: a systematic review. Implement Sci 2011;6:43–50.
      OpenUrlCrossRefPubMed

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    Footnotes

    • Contributors MSD, SH, YWL, RS, AM, SS, PS and LR made substantial contributions to the conception and design of the current project, and also to the acquisition of data and/or the analysis and interpretation of data. MSD, SH, YWL, RS, AM, SS, PS and LR were involved in the drafting and/or critical revision of the article, and approved the final version.

    • Funding This work was undertaken as part of the Veterans Affairs PACT Demonstration Laboratory initiative, supporting and evaluating VA transition to a patient-centred medical home. Funding for the PACT Demonstration Laboratory initiative is provided by the VA Office of Patient Care Services.

      Study# 2011-0521: Patient Centered Medical Home Innovation Evidence-Based Support and Evaluation.

    • Competing interests None.

    • Ethics approval HSPC: The HSPC is RAND's Institutional Review Board to review research involving human subjects, as required by federal regulations. Study# 2011-0521: Patient Centered Medical Home Innovation Evidence-Based Support and Evaluation is considered exempt.

    • Provenance and peer review Not commissioned; externally peer reviewed.