You are here

Article Text

Article menu

Download PDFPDF

Original research
Defining quality outcomes for complex-care patients transitioning across the continuum using a structured panel process
  1. Lianne Jeffs1,2,3,
  2. Madelyn P Law4,
  3. Sharon Straus5,6,7,8,
  4. Roberta Cardoso1,
  5. Renee F Lyons9,10,
  6. Chaim Bell11,12,13,14
  1. 1St. Michael's Hospital, Toronto, Ontario, Canada
  2. 2Keenan Research Centre of the Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Ontario, Canada
  3. 3Lawrence S. Bloomberg Faculty of Nursing, University of Toronto, Toronto, Ontario, Canada
  4. 4Department of Community Health Sciences, Brock University, St. Catharines, Ontario, Canada
  5. 5Knowledge Translation Program, Li Ka Shing Institute, St Michael's Hospital, Toronto, Ontario, Canada
  6. 6Department of Medicine, University of Calgary,
  7. 7Department of Medicine, University of Toronto, Toronto, Ontario, Canada
  8. 8Department of Geriatric Medicine, University of Toronto, Toronto, Ontario, Canada
  9. 9Complex Chronic Disease Research, Bridgepoint Collaboratory for Research and Innovation, Toronto, Ontario, Canada
  10. 10Professor Dalla Lana School of Public Health and Institute of Health Policy, Management and Evaluation, University of Toronto, Bridgepoint Health, Toronto, Ontario, Canada
  11. 11Department of Medicine, Mount Sinai Hospital, Toronto, Ontario, Canada
  12. 12Department of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
  13. 13Institute for Clinical Evaluative Sciences (ICES) of Ontario, Toronto, Ontario, Canada
  14. 14Department of Medicine, Division of General Internal Medicine, University of Toronto, Toronto, Ontario, Canada
  1. Correspondence to Dr Lianne Jeffs, St. Michael's Hospital, 30 Bond Street, Toronto, Ontario, Canada M5B 1W8; jeffsl{at}smh.ca

Abstract

Background No standardised set of quality measures associated with transitioning complex-care patients across the various healthcare settings and home exists. In this context, a structured panel process was used to define quality measures for care transitions involving complex-care patients across healthcare settings.

Methods A modified Delphi consensus technique based on the RAND method was used to develop measures of quality care transitions across the continuum of care. Specific stages included a literature review, individual rating of each measure by each of the panelists (n=11), a face-to-face consensus meeting, and final ranking by the panelists.

Results The literature review produced an initial set of 119 measures. To advance to rounds 1 and 2, an aggregate rating of >75% of the measure was required. This analysis yielded 30/119 measures in round 1 and 11/30 measures in round 2. The final round of scoring yielded the following top five measures: (1) readmission rates within 30 days, (2) primary care visit within 7 days postdischarge for high-risk patients, (3) medication reconciliation completed at admission and prior to discharge, (4) readmission rates within 72 h and (5) time from discharge to homecare nursing visit for high-risk patients.

Conclusions The five measures identified through this research may be useful as indicators of overall care quality related to care transitions involving complex-care patients across different healthcare settings. Further research efforts are called for to explore the applicability and feasibility of using the quality measures to drive quality improvement across the healthcare system.

  • Adverse events, epidemiology and detection
  • Transitions in care
  • Quality measurement
  • Performance measures
  • Health services research

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/

https://doi.org/10.1136/bmjqs-2012-001473

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Patients with complex needs associated with their multiple diseases, comorbidities and conditions, frequently require care in multiple settings, and are particularly vulnerable to poorly executed transitions in care.1 ,2 Complex-care patients are managing multiple chronic diseases, taking several medications, may have limited social supports or suffer from dementia, mental illness or addiction. This complex-care patient cohort is predicted to grow substantially as the population ages and chronic diseases continue to be a major expenditure of healthcare costs.3 Poor transitions often result in unnecessary adverse patient outcomes and additional healthcare spending.4

    Care transitions refer to a set of actions designed to ensure the coordination and continuation of appropriate healthcare provision as a patient transfers between different levels of care in the same clinical setting or different locations.5 Current literature examining transitions in care has mainly focused on transfers within acute-care organisations and to the home setting.6–8 However, complex patients, at any age, may experience transitions across the continuum of health services from acute care to multiple healthcare settings, such as complex continuing care rehabilitation centres, long-term care, home, or possibly from acute care teaching hospitals to community-based hospitals.

    The importance of gaining insight into transfers of complex-care patients across the lifespan and between settings is a critical and timely policy issue. At present, there is no standardised set of quality measures associated with transitioning complex-care patients across the various healthcare settings and discharged and/or readmitted to/from home.9 This is an important gap to address, as quality indicators are increasingly being employed to assess and improve quality of care in many healthcare settings.10 Therefore, the purpose of this study was to define quality measures for care transitions involving complex-care patients across the continuum of care through a structured panel process. Our focus was to develop indicators that would be useful for accountability purposes as well as ones that had the potential for broader quality improvement (QI) efforts.

    Methods

    Overview of modified Delphi panel process

    A modified Delphi consensus technique based on the RAND method was used to develop measures of quality care transitions across the continuum of care.11 The panel process was built on a technique that was shown to be effective for developing quality indicator measures in healthcare.12–18 For this study, multiple stages of the modified Delphi technique (literature review, individual rating, face-to-face consensus meeting, and final ranking) were used (see figure 1). Key to this methodology is engaging in multiple rounds of feedback from the expert panel using both anonymous surveys and round table meeting techniques to allow for optimal, unbiased expression of opinions.

    Figure 1
    Figure 1

    Illustration of modified-Delphi process.

    Initial work: literature review and selection of initial set of measures

    The first step in the Delphi technique is to engage in a structured review of the literature. The intent of this review was to produce a list of measures that had been reported in the literature that would then be presented to a panel of experts to review and rate. This literature search was undertaken in September 2011 using OVID Medline with the following key words with the number of abstracts reported: Quality Assurance, Health Care or Quality of Health Care; Health Transition (4 217 740); Patient Transfer, Patient Discharge, Continuity of Patient Care, Health Services for the Aged, Delivery of Health Care, (714 391) Home Care Services, Chronic Disease, Aged, Long Term Care (2 280 704); Primary Health Care, Delivery of Health Care, Patient Transfer, Continuity of Patient Care (723 017); Health Transition (650); Outcome and Process Assessment (Health Care) and Outcome Assessment (Health Care) (571 586). These search terms were used as they were identified by the research team and key stakeholders as key terms for care transitions.

    Other databases beyond OVID Medline were not included as the initial search yielded a significant number of abstracts. The search was limited to those written in the English language (1675) and from 2000 to 2011, as any previous studies would have been included in literature syntheses during this time frame. Other limits included articles that were clinical conferences, lectures and legal cases, newspaper articles, commentaries and bibliographies/biographies (1658). The final limit was to include meta-analysis and reviews only (76). These 76 abstracts were reviewed by three members of the research team to determine which ones reported on outcome measures associated with care transitions. This review yielded 23 articles including two meta-analyses 19 ,20 and 21 literature reviews,21–41 This review was augmented with other literature known to the research team, or were references included in the reference lists of the literature review of primary studies.5 ,41–51

    Additionally, a search of targeted journals and grey literature was conducted in early January 2012 that yielded 14 other documents that included evidence for transitions of care outcome measures.9 ,52–64 This second search was undertaken for several reasons. First, some of the documents reviewed referred to other references that were not picked up in the initial literature search.9 ,51–58 61–64 Second, shortly after the literature search was conducted, a report was released on avoidable hospitalisations.52 Third, the research team became aware of additional interventions studies published in early 2012.59 ,60 Of the literature that was identified, all the studies were included in the next phase of the project measure development. Our research team developed an initial survey of potential measures from all measures of quality of transitions of care for complex-care patients listed in the final set of documents reviewed. The list of measures (n=119) were categorised under the following headings: health service usage; patient outcome; patient satisfaction and experience; provider satisfaction. Once the initial set of measures was determined, the research team identified potential panel experts to be engaged in this project.

    First-round ratings

    Panel member selection. The research team developed criteria for panel member selection in partnership with key stakeholders (Canadian Patient Safety Institute, Accreditation Canada and Patients for Patient Safety). Criteria for selecting panel members included having an interest in care transitions, patient safety, QI, patient care experience and related research initiatives. Additionally, key stakeholders (listed above) were asked to provide names of experts and organisations to include as potential panel members. From this, an initial list of panel members was created by the research team which had expertise in care transitions and measure development. There was originally a panel of 16 who committed to being part of the research project, with the final panel of 11 participants representing a broad scope of expertise in care transitions. The withdrawal of the five participants was due to scheduling conflicts. The final panel consisted of clinicians (medicine, pharmacy and nursing) including a general internist with patient safety expertise, researchers with measure development expertise and one who had extensive research background in care transitions. Both a national and provincial level policy maker were part of the panel as were decision makers from a variety of sectors, including acute care, home care, long-term care, complex continuing care/rehabilitation care and primary care. Stakeholder organisations and patient advocacy group were also represented on the final panel.

    First round review survey. In the first round of the Delphi approach, the panel was sent the questionnaire electronically 4 weeks prior to the face-to-face panel meeting and asked for input to identify measures worthy of consideration. A reminder email was sent to panelists 2 days before, and on the day of the deadline. We chose to include all relevant measures (n=119) in the survey (see online supplementary file for list). The survey package included an overview of the literature review process, the list of measures with definitions for calculations as described in the literature (where provided), and literature summaries of each of the articles and reports. Additionally, an overview of the rating scale was provided and outlined in table 1⇓.

    View this table:
    Table 1

    Rating matrix domains Desirable Attributes of a Quality Measure—Definitions: The research team has selected the Quality Measures Attributes developed by the Agency for Healthcare Research and Quality (AHRQ) to guide the rating matrix of the Transitions of Care Outcomes/Measures. Table 1 provides a description of each of the desirable attributes of a Quality Measure. For further information of AHRQ's Desirable Attributes of a Quality Measure visit: http://www.qualitymeasures.ahrq.gov.

    View this table:

    Specifically, panel members were asked in the survey to evaluate the proposed measures using the Quality Measures Attributes developed by the Agency for Healthcare Research and Quality (AHRQ).65 This framework groups measures according to four broad conceptual areas: (1) importance of the measure; (2) scientific soundness of the measure—clinical logic; (3) scientific soundness of the measure—measure properties; (4) feasibility of the measure. Given the focus of our study was to identify quality measures that may in the future be useful for accountability purposes and to make system improvements for complex-care patient transitions, we also included two additional criteria for evaluation including the usefulness of the measure: (1) to drive QI efforts and (2) for accountability purposes in public reporting (PR). We included the means for the two latter measures, in addition to the overall mean average of the six evaluation criteria, due to our interest in developing measures for accountability purposes and QI efforts. The panel used a nine-point Likert scale which considered scores of 1–3 as strong disagreement, 7–9 as strong agreement and 4–6 as uncertain agreement. An overall total score that averaged all six evaluation criteria was calculated alongside the score for the usefulness of the measure to drive QI efforts and the usefulness of the measure for accountability purposes in PR. Panel members were also provided with the opportunity to suggest additional measures prior to the in-person meeting. To ensure measures developed through the panel are useful and relevant over time, panelists were encouraged to consider the extent to which identified measures can be linked to currently employed quality of care processes as well as what they view as future areas that necessitate attention related to improving transitions of care for complex-care patients across the continuum. Consensus on these surveys was determined to be reached upon a 75% agreement (standard percentage in other studies)16 ,17 among panel members on the three scores (mean total score for all six rating criteria; mean score for usefulness to drive QI; and mean score for usefulness for PR).

    Second-round panel meeting

    For the second round, panel members met in person for a 1-day facilitated structured discussion of the results compiled from the first round of the questionnaire. At the conclusion of the discussion, members were asked to complete a second round of the questionnaire using the same evaluation criteria and rating scale. After completion of all discussion and questionnaires, panellists were also asked to anonymously comment upon their experience regarding the overall consensus process as a mechanism to ensure the process was fair and unbiased.

    The final results were tabulated using the same consensus agreement score (≥75%). The panel was then contacted once more to provide a priority ranking (from 1–11) among the final selected measures. Again, individual responses remained anonymous to all other panel members. The mean frequency of the rankings of the set of measures was calculated to yield the final set of measures. Additionally, panellists were asked for their personal views of where the priority for measure usage and development should be regardless of current data availability and ease in feasibility.

    Results

    First round ratings

    From the initial set of 119 measures, 30 measures received an aggregate rating of >75% (7/9 or more) across one or more of the total score of the six domains, useful to drive QI, and useful for the PR domain (see table 2).

    View this table:
    Table 2

    Results from prepanel questionnaire of measures achieving at least 75% consensus

    Second-round ratings—panel

    Analysis from the second round of panel member ratings revealed a set of 11 measures (see table 3). This included seven measures that received an aggregate rating of >75% (7/9 or more) in all three scores—total score of the six domains, useful to drive QI and useful for the PR domain; three measures received an aggregate rating of >75% (7/9 or more) total score of the six domains, and useful to drive QI score and between 6 and 6.99 on the useful for PR domain; and one measure received an aggregate rating of >75% (7/9 or more) in the useful to drive QI. Totally, 11 out of the original measures were selected to go to the final round of rating, with wording changes to four of the measures (medication reconciliation, time from discharge to homecare nursing visit for high-risk patients, mortality rate and discharge summary), and readmission rate stratified by three time frames—within 72 h, 30 days and 90 days.

    Totally, 21 out of the original 30 measures were eliminated in this round. Analysis of the key discussion points at the panel meeting and hand written notes from the participants revealed that although these measures were important quality measures of care, they were not perceived collectively to be a quality measure of a care transition. For example, several patient clinical outcome measures (eg, nosocomial infections, pressure ulcers, pain, functional status and medical errors) were viewed as not being an outcome of an ineffective care transition. Other key issues identified with those measures that were eliminated included the perception that the measures were unclear or too broad to be an effective indicator of the transition (eg, potentially avoidable hospital admissions and hospital discharge process). Other key discussion points included a focus on patients and family perspectives for what constitutes quality care transitions for complex-care patients across the care continuum. The panel members recommended further work be undertaken to develop more system-level measures that would capture complex-care patient and family members’ satisfaction and experiences along the care continuum.

    A postmeeting survey found unanimous agreement from the participants that their opinions were valued, the process was fair, and that no one altered their responses or opinions as a result of feeling intimidated.

    View this table:
    Table 3

    Results from postpanel questionnaire of measures achieving at least 75% consensus

    Final review ratings

    When asked to rank the most important final indicators with respect to overall score, QI and PR purposes, with the exception of readmission rates within 30 days as the number one priority and multiple psychiatric readmissions in 30 days as the last priority, there were mixed responses regarding where the priority focus should be (see table 4). Ranking the measures using the overall total score yielded the following top five measures are readmission rates within 30 days, primary care visit within 7 days postdischarge for high-risk patients, medication reconciliation completed at admission and prior to discharge, readmission rates within 72 h, and time from discharge to homecare nursing visit for high-risk patients.

    View this table:
    Table 4

    Final rankings

    Discussion

    Through a structured, modified Delphi panel process, a series of quality measures for care transitions involving complex-care patients across the continuum of care were delineated. Our inclusion of health services researchers, decision-making organisations, frontline clinicians, (physicians, nurses and pharmacists) and a patient representative increased the likelihood that the selected quality measures are scientifically sound and have broad applicability and feasibility.10 ,18 Our final list of quality indicators included outcomes from the three domains of focus including nine health services measures (readmission rates within 30 days, primary care visit within 7 days postdischarge for high-risk patients, medication reconciliation completed at admission and prior to discharge, readmission rates within 72 h, time from discharge to homecare nursing visit for high-risk patients, discharge summary all medications and follow-up appointments given to primary healthcare provider, discharge summary all medications and follow-up appointments given to patient, readmission rates within 90 days, multiple psychiatric readmissions in 30 days); one patient outcome measure (mortality/death/cause of death/risk of death within 72 h); and one patient satisfaction and perceptions measure (patient and caregiver satisfaction follow-up during interventions and/or after). These measures were all associated with posthospital discharge and warrant further attention in future research to determine what measures should be included and/or developed to capture measures that hospitals could use to assess the quality of care transitions with complex-care patients.

    Implications for practice and policy

    To our knowledge, there has been no previous research in the area of quality measures for care transitions involving complex-care patients across the continuum of care. As a next step, further validation of these measures is recommended with other key stakeholders. This work will require: (1) testing of whether these measures actually measure the quality of care (vs differences in case mix); (2) determining if there are sufficient numbers to enable stable estimates when measuring at the provider/hospital level; (3) determining the need for risk adjustment and (4) ascertaining the level of measurement for the indicators. There is potential for these measures to be used as indicators of overall care quality related to care transitions involving complex-care patients across the continuum of care.

    There is an increasing focus on moving a patient out of the hospital system to alleviate wait times and in an effort to provide patients with more appropriate care in the right settings, such as home-based care and community-based programmes and services. However, in order to gain the confidence of patients and their caregivers in these movements across settings, it is essential that organisations be accountable for providing and monitoring high-quality transitions. Having measures such as these can allow organisations to ascertain how well they are doing, make improvements where necessary and be accountable to the patients who they are transitioning. We acknowledge that this study has been conducted in the Canadian healthcare system, therefore, generalisablity to other countries and contexts may not apply exactly. However, the Delphi technique described here can be replicated in other countries in order to understand whether these measures are consistent in other contexts.

    Moreover, there is broad applicability in the majority of the final quality measures selected as they use data that is already commonly collected by hospitals and other healthcare organisations and could feasibly be collected and reported. Some specific points made by panel members during the discussion merit emphasis. For example, the recommendation to further develop measures that are reflective of complex-care patient and family members’ satisfaction and experiences along the care continuum warrants further attention. This finding is consistent with the view that patients and their families play an integral role in ensuring they receive safe care, as they are the one constant in care transitions processes.5 This work can draw from existing measures such as the Care Transitions Measure used to measure satisfaction with discharge from hospital to home.1 ,5 ,56

    Strengths and limitations

    Engaging key stakeholders in quality measure development is paramount. Important attributes of high-quality measures are their relevance to the selected problem and field of application, their feasibility and their reliability.10 It is important to note that the inclusion of a patient representative on the panel is important, as a recent systematic review revealed that the participation of patients in quality indicator development is extremely uncommon.10 Our final panel consisted of individuals (researchers, front-line clinicians and a patient) with broad professional representation as well as consultants from stakeholder groups (national accreditation body and provincial quality council). Additionally, our panel included individuals from different healthcare systems and sectors increasing the potential for generalisability of our findings.

    The literature review that was included as a summary in the background information and anonymous survey used in our process ensured that all members were entering the discussion with balanced background and content knowledge. We also included open-ended questions in the presurvey to provide an opportunity for panel members to comment on the list of indicators and submit additional indicators. Additionally, an experienced health services researcher facilitated the discussion and maintained equality among all panel members. The anonymous end-of-day survey verified that all panelists were comfortable with the process.

    Furthermore, panel discussion was also focused on finding important and relevant measures that could act as quality indicators in addition to the ones initially presented by the research team. This open-ended discussion was presented to the panel prior to the face-to-face meeting to provide opportunity for panelists to review and reflect on the measures presented and potential missing measures reflective of quality care transitions for complex-care patients across the care continuum. Upon review of the measures premeeting and postmeeting, it was evident that the face-to-face discussion was a vital component to our process as the measures and parameters were redefined through the modified Delphi process. This is an important finding that should guide future quality performance measures—research and development.

    There are limitations associated with our modified Delphi panel process. First, the review of literature was not a systematic review, rather it was a targeted review that was designed by researchers with expertise in performance measures and care transitions. We drew on the expertise of other researchers, clinicians and researchers that were part of the panel to ensure that key measures were not omitted. Second, there was the potential for selection bias inherent in panel member selection. To minimise selection bias, the research team used a similar process used by one of the researchers17 ,18 and solicited suggestions from unbiased third party individuals, stakeholder groups as well as national patient safety organisations. However, our initial international panels were not able to participate. Thus, some important perspectives may not have been identified. Another measure to reduce the risk of selection bias was the use of predesigned forms containing rating and selection criteria during systematic consensus processes.10 Third, the complex nature around the validity of the measures and whether they are measuring quality of care and the feasibility of collecting data on the identified measures is another limitation. In our study, panelists were not asked to limit their selection to quality measures that could be applied immediately with commonly available data. The use of administrative data for any quality measures always has the limitation of clinical application, as it is only a part of a patient's clinical story. Fourth, our panel had only one patient representative, and although they were a leader of a national patient safety advocacy organisation, their views may not be completely representative of patients and caregivers.

    Conclusion

    Quality measures are important tools used for accountability and QI in healthcare systems. The modified Delphi consensus panel approach applied in this study enabled us to garner a consensus on measures in an important area—care transitions. Although a number of these measures are not new and are used for the evaluation of other healthcare areas, this study provides evidence and support that these measures may also be effective for evaluating the quality of care transitions. This is the first study focused on care transitions and performance measures, thus, more research and testing of these measures using a rigorous research design are needed to determine their actual usefulness and feasibility. Further work is required to create standardised operational definitions, for example, the patient and caregiver satisfaction follow-up during interventions and/or after measure.

    Collectively, future research efforts will produce findings that can be used by healthcare administrators, clinicians and policy makers to monitor care transitions and to work together to improve the quality of transitions of care with complex-care patients. Moreover, individual institutions can evaluate interventions aimed at enhancing care transitions for complex-care patents’ efforts, and provide healthcare systems with a consistent means to compare programme effectiveness across institutions. A longer-term goal is that this work will inform the development of a reporting standard of measures that demonstrate quality outcomes associated with transitions of care with complex-care patients.

    Acknowledgments

    This consensus process was funded by a Canadian Institutes of Health Research (CIHR) Meetings, Planning and Dissemination Grant—PA: Health Services and Policy Research 201102 PLH (#248538). We wish to thank the panel members: Dr Brian Hutchison, Senior Advisor for Primary Care—Health Quality Ontario; Carol Kushner, Co-Chair Patients for Patient Safety Canada; Dr Carole Estabrooks, Professor & Canada Research Chair in Knowledge Translation, Faculty of Nursing, University of Alberta; Dr Chris Hayes, Medical Officer, Canadian Patient Safety Institute; Dr Dan Cass, Chair, Patient Safety Review Committee, Office of the Chief Coroner for Ontario, Jennifer Kodis, Clinical Director, Regional Rehabilitation Centre, Complex Continuing Care & Seniors Health, Hamilton Health Sciences; Jonathan Mitchell, Manager, Policy and Research, Accreditation Canada; Dr Karen Chien, Palliative Care, St Michael's Hospital & Staff Physician Bridgepoint Health; Dr Olavo Fernandes, Pharmacy Clinical Site Leader, Toronto General Hospital, UHN; Dr Onil Bhattacharyya, Assistant Professor and Clinician Scientist, University of Toronto, Department of Family and Community Medicine, Li Ka Shing Knowledge Institute of St Michael's Hospital; Dr Walter Wodchis, Associate Professor, University of Toronto, Research Scientist, Toronto Rehabilitation Institute, Adjunct Scientist, Institute for Clinical Evaluative Sciences. The authors would like to thank Stacey Brener for her contribution to the search strategy and literature review.

    References

    1. ↵
      1. Coleman EA,
      2. Boult C
      . Improving the quality of transitional care for persons with complex care needs. J Am Geriatr Soc 2003;51:556–7.
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. Levine C
      . Rough crossing: family caregivers odysseys through the health care system. New York United Hospital Fund of New York, 1998.
    3. ↵
      1. Bloom DE,
      2. Cafiero ET,
      3. Jané-Llopis E,
      4. et al
      . The global economic burden of non-communicable diseases. Geneva: World Economic Forum, 2011.
    4. ↵
      1. Burgers JS,
      2. Voerman GE,
      3. Grol R,
      4. et al
      . Quality and coordination of care for patients with multiple conditions: results from an international survey of patient experience. Eval Health Prof 2010;33:343–64.
      OpenUrlAbstract/FREE Full Text
    5. ↵
      1. Coleman EA,
      2. Parry C,
      3. Chalmers S,
      4. et al
      . The care transitions intervention: results of a randomized controlled trial. Arch Intern Med 2006;166:1822–8.
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      1. Cohen MD,
      2. Hilligoss PB
      . Handoffs in Hospitals: a review of the literature on information exchange while transferring patient responsibility or control. 2001. http://hdl.handle.net/2027.42/61498
    7. ↵
      1. Behara R,
      2. Wears RL,
      3. Perry SJ,
      4. et al
      . A conceptual framework for studying the safety of transitions in emergency care. A conceptual framework for studying the safety of transitions in emergency care [Internet]. 2005;2:309–21. http://www.ahcpr.gov/downloads/pub/advances/vol2/Behara.pdf
    8. ↵
      1. Forster AJ,
      2. Murff HJ,
      3. Peterson JF,
      4. et al
      . The incidence and severity of adverse events affecting patients after discharge from the hospital. Ann Intern Med 2003;138:161–7.
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      1. Naylor MD,
      2. Aiken LH,
      3. Kurtzman ET,
      4. et al
      . The care span: the importance of transitional care in achieving health reform. Health Aff (Millwood) 2011;30:746–54.
      OpenUrlAbstract/FREE Full Text
    10. ↵
      1. Kotter T,
      2. Blozik E,
      3. Scherer M
      . Methods for the guideline-based development of quality indicators-a systematic review. Implement Sci 2012;7:21.
      OpenUrlCrossRefPubMed
    11. ↵
      1. Fink A,
      2. Kosecoff J,
      3. Chassin M,
      4. et al
      . Consensus methods: characteristics and guidelines for use. Am J Public Health 1984;74:979–83.
      OpenUrlCrossRefPubMedWeb of Science
    12. ↵
      1. Guttmann A,
      2. Razzaq A,
      3. Lindsay P,
      4. et al
      . Development of measures of the quality of emergency department care for children using a structured panel process. Pediatrics 2006;118:114–23.
      OpenUrlAbstract/FREE Full Text
    13. ↵
      1. Kroger E,
      2. Tourigny A,
      3. Morin D,
      4. et al
      . Selecting process quality indicators for the integrated care of vulnerable older adults affected by cognitive impairment or dementia. BMC Health Serv Res 2007;7:195.
      OpenUrlCrossRefPubMed
    14. ↵
      1. Lindsay P,
      2. Schull M,
      3. Bronskill S,
      4. et al
      . The development of indicators to measure the quality of clinical care in emergency departments following a modified-delphi approach. Acad Emerg Med 2002;9:1131–9.
      OpenUrlCrossRefPubMedWeb of Science
    15. ↵
      1. Minkman MM,
      2. Ahaus KT,
      3. Huijsman R
      . A four phase development model for integrated care services in the Netherlands. BMC Health Serv Res 2009;9:42.
      OpenUrlCrossRefPubMed
    16. ↵
      1. Greenberg A,
      2. Angus H,
      3. Sullivan T,
      4. et al
      . Development of a set of strategy-based system-level cancer care performance indicators in Ontario, Canada. Int J Qual Health Care 2005;17:107–14.
      OpenUrlAbstract/FREE Full Text
    17. ↵
      1. Bell CM,
      2. Brener SS,
      3. Comrie R,
      4. et al
      . Quality measures for medication continuity in long-term care facilities, using a structured panel process. Drugs Aging 2012;29:319–27.
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      1. Morris A,
      2. Brener S,
      3. Dresser L,
      4. et al
      . Use of a structured panel process to define quality metrics for antimicrobial stewardship programs. Infect Control Hosp Epidemiol 2012;33:500–6.
      OpenUrlCrossRefPubMed
    19. ↵
      1. Preyde M,
      2. Macaulay C,
      3. Dingwall T
      . Discharge planning from hospital to home for elderly patients: a meta-analysis. J Evid Based Soc Work 2009;6:198–216.
      OpenUrlCrossRefPubMed
    20. ↵
      1. Phillips CO,
      2. Wright SM,
      3. Kern DE,
      4. et al
      . Comprehensive discharge planning with postdischarge support for older patients with congestive heart failure: a meta-analysis. JAMA 2004;291:1358–67.
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      1. Stolee P,
      2. Steeves B,
      3. Glenny C,
      4. et al
      . The use of electronic health information systems in home care: facilitators and barriers. Home Healthc Nurse 2010;28:167–79.
      OpenUrlCrossRefPubMed
    22. ↵
      1. Chiu WK,
      2. Newcomer R
      . A systematic review of nurse-assisted case management to improve hospital discharge transition outcomes for the elderly. Prof Case Manag 2007;12:330–6; quiz 7–8.
      OpenUrlPubMed
    23. ↵
      1. Langhorne P,
      2. Holmqvist LW
      . Early supported discharge after stroke. J Rehabil Med 2007;39:103–8.
      OpenUrlCrossRefPubMedWeb of Science
    24. ↵
      1. Larsen T,
      2. Olsen TS,
      3. Sorensen J
      . Early home-supported discharge of stroke patients: a health technology assessment. Int J Technol Assess Health Care 2006;22:313–20.
      OpenUrlPubMed
    25. ↵
      1. Hastings SN,
      2. Heflin MT
      . A systematic review of interventions to improve outcomes for elders discharged from the emergency department. Acad Emerg Med 2005;12:978–86.
      OpenUrlCrossRefPubMedWeb of Science
    26. ↵
      1. Teasell RW,
      2. Foley NC,
      3. Bhogal SK,
      4. et al
      . Early supported discharge in stroke rehabilitation. Top Stroke Rehabil 2003;10:19–33.
      OpenUrlCrossRefPubMed
    27. ↵
      1. Richards S,
      2. Coast J
      . Interventions to improve access to health and social care after discharge from hospital: a systematic review. J Health Serv Res Policy 2003;8:171–9.
      OpenUrlAbstract/FREE Full Text
    28. ↵
      1. Anderson C,
      2. Ni Mhurchu C,
      3. Brown PM,
      4. et al
      . Stroke rehabilitation services to accelerate hospital discharge and provide home-based care: an overview and cost analysis. Pharmacoeconomics 2002;20:537–52.
      OpenUrlCrossRefPubMedWeb of Science
    29. ↵
      1. Parker SG,
      2. Peet SM,
      3. McPherson A,
      4. et al
      . A systematic review of discharge arrangements for older people. Health Technol Assess 2002;6:1–183.
      OpenUrlPubMed
    30. ↵
      1. Marineau M
      . Health/Illness transition and telehealth: a concept analysis using the evolutionary method. Nurs Forum 2005;40:96–106.
      OpenUrlPubMed
    31. ↵
      1. Albert N
      . Improving medication adherence in chronic cardiovascular disease. Crit Care Nurse 2008;28:54–64.
      OpenUrlFREE Full Text
    32. ↵
      1. Boling PA,
      2. Parsons P
      . A research and policy agenda for transitions from nursing homes to home. Home Health Care Serv Q 2007;26:121–31.
      OpenUrlCrossRefPubMed
    33. ↵
      1. Wenger NS,
      2. Young RT
      . Quality indicators for continuity and coordination of care in vulnerable elders. J Am Geriatr Soc 2007;55(Suppl 2):S285–92.
      OpenUrlCrossRefPubMedWeb of Science
    34. ↵
      1. Popejoy LL,
      2. Moylan K,
      3. Galambos C
      . A review of discharge planning research of older adults 1990–2008. West J Nurs Res 2009;31:923–47.
      OpenUrlAbstract/FREE Full Text
    35. ↵
      1. Marek KD,
      2. Baker CD
      . Nurse home visit programs for the elderly. Annu Rev Nurs Res 2006;24:157–78.
      OpenUrlPubMed
    36. ↵
      1. Halcomb E,
      2. Daly J,
      3. Davidson P,
      4. et al
      . Life beyond severe traumatic injury: an integrative review of the literature. Aust Crit Care 2005;18:17–18, 20–24.
      OpenUrlCrossRefPubMed
    37. ↵
      1. Flick CL
      . Stroke rehabilitation. 4. Stroke outcome and psychosocial consequences. Arch Phys Med Rehabil 1999;80(5 Suppl 1):S21–26.
      OpenUrlCrossRefPubMedWeb of Science
    38. ↵
      1. O'Hare PA,
      2. Yost LS,
      3. McCorkle R
      . Strategies to improve continuity of care and decrease rehospitalization of cancer patients: a review. Cancer Invest 1993;11:140–58.
      OpenUrlCrossRefPubMedWeb of Science
    39. ↵
      1. Gilmartin M
      . Transition from the intensive care unit to home: patient selection and discharge planning. Respir Care 1994;39:456–77; discussion 77–80.
      OpenUrlPubMed
    40. ↵
      1. Ecklund MM
      . Successful outcomes for the ventilator-dependent patient. Crit Care Nurs Clin North Am 1999;11:249–60.
      OpenUrlPubMed
    41. ↵
      1. Steiner MM,
      2. Brainin M
      . The quality of acute stroke units on a nation-wide level: the Austrian Stroke Registry for acute stroke units. Eur J Neurol 2003;10:353–60.
      OpenUrlCrossRefPubMed
    42. ↵
      1. Khouli H,
      2. Astua A,
      3. Dombrowski W,
      4. et al
      . Changes in health-related quality of life and factors predicting long-term outcomes in older adults admitted to intensive care units. Crit Care Med 2011;39:731–7.
      OpenUrlCrossRefPubMedWeb of Science
    43. ↵
      1. Lee DS,
      2. Stukel TA,
      3. Austin PC,
      4. et al
      . Improved outcomes with early collaborative care of ambulatory heart failure patients discharged from the emergency department. Circulation 2010;122:1806–14.
      OpenUrlAbstract/FREE Full Text
    44. ↵
      1. Wang TY,
      2. Nallamothu BK,
      3. Harlan MK
      . Early supported discharge after stroke. JAMA 2011;305:2540–7.
      OpenUrlCrossRefPubMedWeb of Science
    45. ↵
      1. Mallinson TR,
      2. Bateman J,
      3. Tseng HY,
      4. et al
      . A comparison of discharge functional status after rehabilitation in skilled nursing, home health, and medical rehabilitation settings for patients after lower-extremity joint replacement surgery. Arch Phys Med Rehabil 2011;92:712–20.
      OpenUrlCrossRefPubMed
    46. ↵
      1. Schnipper JL,
      2. Linder JA,
      3. Palchuk MB,
      4. et al
      . Effects of documentation-based decision support on chronic disease management. Am J Manag Care 2010;16(12 Suppl HIT):SP72–81.
      OpenUrlPubMedWeb of Science
    47. ↵
      1. Snyder CF,
      2. Jensen RE,
      3. Geller G,
      4. et al
      . Relevant content for a patient-reported outcomes questionnaire for use in oncology clinical practice: Putting doctors and patients on the same page. Qual Life Res 2010;19:1045–55.
      OpenUrlCrossRefPubMedWeb of Science
    48. ↵
      1. Utzolino S,
      2. Kaffarnik M,
      3. Keck T,
      4. et al
      . Unplanned discharges from a surgical intensive care unit: readmissions and mortality. J Crit Care 2010;25:375–81.
      OpenUrlCrossRefPubMedWeb of Science
    49. ↵
      1. Arnetz JE,
      2. Winblad U,
      3. Höglund AT,
      4. et al
      . Is patient involvement during hospitalization for acute myocardial infarction associated with post-discharge treatment outcome? An exploratory study. Health Expect 2010;13:298–311.
      OpenUrlPubMedWeb of Science
    50. ↵
      1. Alonso-Babarro A,
      2. Bruera E,
      3. Varela-Cerdeira M,
      4. et al
      . Can this patient be discharged home? Factors associated with at-home death among patients with cancer. J Clin Oncol 2011;29:1159–67.
      OpenUrlAbstract/FREE Full Text
    51. ↵
      1. Berges IM,
      2. Seale G,
      3. Ostir GV
      . Positive affect and pain ratings in persons with stroke. Rehabil Psychol 2011;56:52–7.
      OpenUrlCrossRefPubMed
    52. ↵
      Enhancing the Continuum of Care Report of the Avoidable Hospitalization Advisory Panel. 2011.
    53. ↵
      1. Bell CM,
      2. Fischer HD,
      3. Gill SS,
      4. et al
      . Initiation of benzodiazepines in the elderly after hospitalization. J Gen Intern Med 2007;22:1024–9.
      OpenUrlCrossRefPubMed
    54. ↵
      1. van Walraven C,
      2. Bennett C,
      3. Jennings A,
      4. et al
      . Proportion of hospital readmissions deemed avoidable: a systematic review. CMAJ 2011;183:E391–402.
      OpenUrlAbstract/FREE Full Text
    55. ↵
      1. Howrey BT,
      2. Kuo YF,
      3. Goodwin JS
      . Association of care by hospitalists on discharge destination and 30-day outcomes after acute ischemic stroke. Med Care 2011;49:701–7.
      OpenUrlCrossRefPubMedWeb of Science
    56. ↵
      1. Ong MS,
      2. Coiera E
      . A systematic review of failures in handoff communication during intrahospital transfers. Jt Comm J Qual Patient Saf 2011;37:274–84.
      OpenUrlPubMed
    57. ↵
      1. Hysong SJ,
      2. Khan MM,
      3. Petersen LA
      . Passive monitoring versus active assessment of clinical performance: impact on measured quality of care. Med Care 2011;49:883–90.
      OpenUrlCrossRefPubMedWeb of Science
    58. ↵
      1. Coleman EA,
      2. Smith JD,
      3. Frank JC,
      4. et al
      . Development and testing of a measure designed to assess the quality of care transitions. Int J Integr Care 2002;2:e02.
      OpenUrlPubMed
    59. ↵
      1. Lin WC,
      2. Chien HL,
      3. Willis G,
      4. et al
      . The effect of a telephone-based health coaching disease management program on Medicaid members with chronic conditions. Med Care 2012;50:91–8.
      OpenUrlCrossRefPubMedWeb of Science
    60. ↵
      1. Graham J,
      2. Tomcavage J,
      3. Salek D,
      4. et al
      . Postdischarge monitoring using interactive voice response system reduces 30-day readmission rates in a case-managed Medicare population. Med Care 2012;50:50–7.
      OpenUrlCrossRefPubMedWeb of Science
    61. ↵
      1. Arling G,
      2. Abrahamson KA,
      3. Cooke V,
      4. et al
      . Facility and market factors affecting transitions from nursing home to community. Med Care 2011;49:790–6.
      OpenUrlCrossRefPubMedWeb of Science
    62. ↵
      1. LaMantia MA,
      2. Scheunemann LP,
      3. Viera AJ,
      4. et al
      . Interventions to improve transitional care between nursing homes and hospitals: a systematic review. J Am Geriatr Soc 2010;58:777–82.
      OpenUrlCrossRefPubMed
    63. ↵
      1. Boult C,
      2. Green AF,
      3. Boult LB,
      4. et al
      . Successful models of comprehensive care for older adults with chronic conditions: evidence for the Institute of Medicine's "retooling for an aging America" report. J Am Geriatr Soc 2009;57:2328–37.
      OpenUrlCrossRefPubMedWeb of Science
    64. ↵
      1. Peikes D,
      2. Chen A,
      3. Schore J,
      4. et al
      . Effects of care coordination on hospitalization, quality of care, and health care expenditures among Medicare beneficiaries: 15 randomized trials. JAMA 2009;301:603–18.
      OpenUrlCrossRefPubMedWeb of Science
    65. ↵
      Agency for Healthcare Research and Quality (AHRQ). Quality Measures Attributes. 2012.

    Footnotes

    • Contributors The primary authors (LJ, MPL and CB) (1) made substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; (2) drafted the article and revised it critically for important intellectual content; and (3) provided final approval of the version to be published. The other three authors (SS, RC and RFL) also contributed to the (1) design and analysis and interpretation of data; (2) revision of the article critically for important intellectual content and (3) provided final approval of the version to be published.

    • Funding This study was funded by a Canadian Institutes of Health Research (CIHR) Meetings, Planning and Dissemination Grant—PA: Health Services and Policy Research 201102 PLH (#248538).

    • Competing interests None.

    • Provenance and peer review Not commissioned; externally peer reviewed.