Skip to main content
SpringerLink
Log in

The Relationship Between Change in Therapeutic Alliance Ratings and Improvement in Youth Symptom Severity: Whose Ratings Matter the Most?

  • Original Paper
  • Published:
Administration and Policy in Mental Health and Mental Health Services Research Aims and scope Submit manuscript

Abstract

This paper presents the psychometric evaluation of brief measures of therapeutic alliance (TA) for youths, clinicians and caregivers and a longitudinal analysis of relationships between changes in TA and changes in youth symptom and functioning severity. Psychometric analyses using methods from Classical Test Theory, Item Response Theory, and Factor Analysis indicate that the measures of TA used in this study offer something new for both practice and research. The measures have variability, sensitivity to change over time, brevity and can be used with multiple parties through parallel forms. The longitudinal analyses, employing hierarchical linear modeling with time-varying covariates, found that TA ratings of the clinician correlated with symptom improvement as rated by the clinician, caregiver and youth. Additional analyses showed that decreases in clinician-rated youth TA was most important in predicting a lower rate of youth improvement. Implications for future research and clinical practice are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Anker, M. G., Owen, J., Duncan, B. L., & Sparks, J. A. (2010). The alliance in couple therapy: Partner influence, early change, and alliance patterns in a naturalistic sample. Journal of Consulting and Clinical Psychology, 78(5), 635–645. doi:10.1037/a0020051.

    Article  PubMed  Google Scholar 

  • Armbruster, P., & Kazdin, A. E. (1994). Attrition in child-psychotherapy. Advances in Clinical Child Psychology, 16(16), 81–108.

    Google Scholar 

  • Athay, M. M., Riemer, M., & Bickman, L. (2012). The Symptoms and Functioning Severity Scale (SFSS): Psychometric evaluation and discrepancies among youth, caregiver, and clinician ratings over time. Administration and Policy in Mental Health and Mental Health Services Research. doi:10.1007/s10488-012-0403-2.

  • Barber, J. P., Connoly, M. B., Crits-Christoph, P., Gladis, L., & Siqueland, L. (2009). Alliance predicts patients’ outcome beyond in-treatment change in symptoms. Personality Disorders: Theory, Research, and Treatment, S(1), 80–89. doi:10.1037/1949-2715.S.1.80.

    Article  Google Scholar 

  • Bickman, L. (2008). A measurement feedback system (MFS) is necessary to improve mental health outcomes. Journal of the American Academy of Child and Adolescent Psychiatry, 47(10), 1114–1119. doi:10.1097/CHI.0b013e3181825af8.

    Article  PubMed  Google Scholar 

  • Bickman, L., Rosof-Williams, J., Salzer, M. S., Summerfelt, W. T., Noser, K., Wilson, S. J., et al. (2000). What information do clinicians value for monitoring adolescent client progress and outcomes? Professional Psychology: Research and Practice, 31(1), 70–74.

    Article  Google Scholar 

  • Bickman, L., Andrade, A. R., Lambert, E. W., Doucette, A., Sapyta, J., Boyd, A. S., et al. (2004). Youth therapeutic alliance in intensive treatment settings. Journal of Behavioral Health Services and Research, 31(2), 134–148.

    PubMed  Google Scholar 

  • Bickman, L., Riemer, M., Lambert, E. W., Kelley, S. D., Breda, C., Dew, S., et al. (2007). Manual of the Peabody Treatment Progress Battery (Electronic version). Nashville, TN: Vanderbilt University. http://peabody.vanderbilt.edu/ptpb/.

  • Bickman, L., Athay, M. M., Riemer, M., Lambert, E. W., Kelley, S. D., Breda, C., Vides de Andrade, A. R. (2010). Manual of the Peabody Treatment Progress Battery (2nd ed.). Nashville, TN: Vanderbilt University. Retrieved from http://peabody.vanderbilt.edu/ptpb/.

  • Bickman, L., Kelley, S., Breda, C., Vides de Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to clinicians on youth mental health outcomes: A randomized cluster design. Psychiatric Services, 62(12), 1423–1429.

    Article  PubMed  Google Scholar 

  • Bordin, E. S. (1979). The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory, Research and Practice, 16(3), 252–260. doi:10.1037/h0085885.

    Article  Google Scholar 

  • Chiu, A. W., McLeod, B. D., Har, K., & Wood, J. J. (2009). Child-therapist alliance and clinical outcomes in cognitive behavioral therapy for child anxiety disorders. Journal of Child Psychology and Psychiatry and Allied Disciplines, 50(6), 751–758. doi:10.1111/j.1469-7610.2008.01996.x.

    Article  Google Scholar 

  • Chu, B. C., & Kendall, P. C. (2004). Positive association of child involvement and treatment outcome within a manual-based cognitive-behavioral treatment for children with anxiety. Journal of Consulting and Clinical Psychology, 72(5), 821–829.

    Article  PubMed  Google Scholar 

  • Crits-Christoph, P., Gibbons, M. B. C., Hamilton, J., Ring-Kurtz, S., & Gallop, R. (2011). The dependability of alliance assessments: The alliance-outcome correlation is larger than you might think. Journal of Consulting and Clinical Psychology, 79(3), 267–278. doi:10.1037/a0023668.

    Article  PubMed  Google Scholar 

  • Crocker, L., & Algina, J. (2006). Introduction to classical and modern test theory. Mason, OH: Wadsworth.

    Google Scholar 

  • De Bolle, M., Johnson, J. G., & De Fruyt, F. (2010). Patient and clinician predictors of therapeutic alliance as predictors of improvement in depression. Psychotherapy Psychosomatics, 79(6), 378–385. doi:10.1159/000320895.

    Article  Google Scholar 

  • DeRubeis, R. J., & Feeley, M. (1990). Determinants of change in cognitive therapy for depression. Cognitive Therapy and Research, 14(5), 469–482. doi:10.1007/bf01172968.

    Article  Google Scholar 

  • DeRubeis, R. J., Brotman, M. A., & Gibbons, C. J. (2005). A conceptual and methodological analysis of the nonspecifics argument. Clinical Psychology: Science and Practice, 12(2), 174–183. doi:10.1093/clipsy.bpi022.

    Article  Google Scholar 

  • Eltz, M. J., Shirk, S. R., & Sarlin, N. (1995). Alliance formation and treatment outcome among maltreated adolescents. Child Abuse and Neglect, 19(4), 419–431. doi:10.1016/0145-2134(95)00008-v.

    Article  PubMed  CAS  Google Scholar 

  • Faw, L., Hogue, A., Johnson, S., Diamond, G. M., & Liddle, H. A. (2005). The Adolescent Therapeutic Alliance Scale (ATAS): Initial psychometrics and prediction of outcome in family-based substance abuse prevention counseling. Psychotherapy Research, 15(1–2), 141–154.

    Article  Google Scholar 

  • Feeley, M., DeRubeis, R. J., & Gelfand, L. A. (1999). The temporal relation of adherence and alliance to symptom change in cognitive therapy for depression. Journal of Consulting and Clinical Psychology, 67(4), 578–582.

    Article  PubMed  CAS  Google Scholar 

  • Fernandez, M., & Eyberg, S. (2009). Predicting treatment and follow-up attrition in parent–child interaction therapy. Journal of Abnormal Child Psychology, 37(3), 431–441. doi:10.1007/s10802-008-9281-1.

    Article  PubMed  Google Scholar 

  • Friedlander, M. L., Lambert, J. E., & de la Peņa, C. M. (2008). A step toward disentangling the alliance/improvement cycle in family therapy. Journal of Counseling Psychology, 55(1), 118–124.

    Article  Google Scholar 

  • Gonzalez, A., Weersing, V., Warnick, E., Scahill, L., & Woolston, J. (2011). Predictors of treatment attrition among an outpatient clinic sample of youths with clinically significant anxiety. Administration and Policy in Mental Health and Mental Health Services Research, 38(5), 356–367. doi:10.1007/s10488-010-0323-y.

    Article  PubMed  Google Scholar 

  • Green, J. (2009). The therapeutic alliance. Child: Care, Health and Development, 35(3), 298–301.

    Article  CAS  Google Scholar 

  • Hawley, K. M., & Garland, A. F. (2008). Working alliance in adolescent outpatient therapy: Youth, parent and therapist reports and associations with therapy outcomes. Child and Youth Care Forum, 37, 59–74.

    Article  Google Scholar 

  • Hawley, K. M., & Weisz, J. R. (2005). Youth versus parent working alliance in usual clinical care: Distinctive associations with retention, satisfaction, and treatment outcome. Journal of Clinical Child and Adolescent Psychology, 34(1), 117–128. doi:10.1207/s15374424jccp3401_11.

    Article  PubMed  Google Scholar 

  • Hedeker, D., & Gibbons, R. D. (1997). Application of random-effects pattern-mixture models for missing data in longitudinal studies. Psychological Methods, 2(1), 64–78.

    Article  Google Scholar 

  • Hogue, A., Dauber, S., Stambaugh, L. F., Cecero, J. J., & Liddle, H. A. (2006). Early therapeutic alliance and treatment outcome in individual and family therapy for adolescent behavior problems. Journal of Consulting and Clinical Psychology, 74(1), 121–129.

    Article  PubMed  Google Scholar 

  • Horvath, A. O. (2001). The alliance. Psychotherapy: Theory Research and Practice, 38(4), 365–372.

    Article  Google Scholar 

  • Horvath, A. O., Del Re, A. C., Flückiger, C., & Symonds, D. (2011). Alliance in individual psychotherapy. Psychotherapy: Theory/Research/Practice/Training, 48(1), 9–16. doi:10.1037/a0022186.

    Article  Google Scholar 

  • Hughes, A. A., & Kendall, P. C. (2007). Prediction of cognitive behavior treatment outcome for children with anxiety disorders: Therapeutic relationship and homework compliance. Behavioural and Cognitive Psychotherapy, 35(4), 487–494. doi:10.1017/s1352465807003761.

    Article  Google Scholar 

  • Karver, M. S., Handelsman, J. B., Fields, S., & Bickman, L. (2006). Meta-analysis of therapeutic relationship variables in youth and family therapy: The evidence for different relationship variables in the child and adolescent treatment outcome literature. Clinical Psychology Review, 26(1), 50–65. doi:10.1016/j.cpr.2005.09.001.

    Article  PubMed  Google Scholar 

  • Karver, M. S., Shirk, S., Handelsman, J. B., Fields, S., Crisp, H., Gudmundsen, G., et al. (2008). Relationship processes in youth psychotherapy. Journal of Emotional and Behavioral Disorders, 16(1), 15–28. doi:10.1177/1063426607312536.

    Article  Google Scholar 

  • Kazdin, A. E. (1996). Dropping out of child psychotherapy: Issues for research and implications for practice. Clinical Child Psychology and Psychiatry, 1(1), 133–156. doi:10.1177/1359104596011012.

    Article  Google Scholar 

  • Kazdin, A. E., & Whitley, M. K. (2006). Pretreatment social relations, therapeutic alliance, and improvements in parenting practices in parent management training. Journal of Consulting and Clinical Psychology, 74, 346–355.

    Article  PubMed  Google Scholar 

  • Kazdin, A. E., Marciano, P. L., & Whitley, M. (2005). The therapeutic alliance in cognitive-behavioral treatment of children referred for oppositional, aggressive, and antisocial behavior. Journal of Consulting and Clinical Psychology, 73, 726–730.

    Article  PubMed  Google Scholar 

  • Kazdin, A. E., Whitley, M., & Marciano, P. L. (2006). Child-therapist and parent-therapist alliance and therapeutic change in the treatment of children referred for oppositional, aggressive, and antisocial behavior. Journal of Child Psychology and Psychiatry, 47, 436–445.

    Article  PubMed  Google Scholar 

  • Kendall, P. C., Flannery-Schroeder, E., Panichelli-Mindel, S. M., Southam-Gerow, M., Henin, A., & Warman, M. (1997). Therapy for youths with anxiety disorders: A second randomized clinical trial. Journal of Consulting and Clinical Psychology, 65(3), 366–380.

    Article  PubMed  CAS  Google Scholar 

  • Kendall, P. C., Comer, J. S., Marker, C. D., Creed, T. A., Puliafico, A. C., Hughes, A. A., et al. (2009). In-session exposure tasks and therapeutic alliance across the treatment of childhood anxiety disorders. Journal of Consulting and Clinical Psychology, 77(3), 517–525. doi:10.1037/a0013686.

    Article  PubMed  Google Scholar 

  • Kivlighan, D. M., & Shaughnessy, P. (1995). Analysis of the development of the working alliance using hierarchical linear modeling. Journal of Counseling Psychology, 42(3), 338–349.

    Article  Google Scholar 

  • Kivlighan, D. M., & Shaughnessy, P. (2000). Patterns of working alliance development: A typology of client’s working alliance ratings. Journal of Counseling Psychology, 47(3), 362–371. doi:10.1037//0022-0167.47.3.362.

    Article  Google Scholar 

  • Kramer, U., de Roten, Y., Beretta, V., Michel, L., & Despland, J. N. (2009). Alliance patterns over the course of short-term dynamic psychotherapy: The shape of productive relationships. Psychotherapy Research, 19(6), 699–706. doi:10.1080/10503300902956742.

    Article  PubMed  Google Scholar 

  • Lambert, M. J., Hansen, N. B., & Finch, A. E. (2001a). Patient-focused research: Using patient outcome data to enhance treatment effects. Journal of Consulting and Clinical Psychology, 69(2), 159–172.

    Article  PubMed  CAS  Google Scholar 

  • Lambert, M. J., Whipple, J. L., Smart, D. W., Vermeersch, D. A., Nielsen, S. L., & Hawkins, E. J. (2001b). The effects of providing therapists with feedback on patient progress during psychotherapy: Are outcomes enhanced? Psychotherapy Research, 11(1), 49–68. doi:10.1093/ptr/11.1.49.

    Article  Google Scholar 

  • McLeod, B. D. (2011). Relation of the alliance with outcomes in youth psychotherapy: A meta-analysis. Clinical Psychology Review, 31(4), 603–616. doi:10.1016/j.cpr.2011.02.001.

    Article  PubMed  Google Scholar 

  • Miller, L., Southam-Gerow, M., & Allin, R. (2008). Who stays in treatment? Child and family predictors of youth client retention in a public mental health agency. Child and Youth Care Forum, 37(4), 153–170. doi:10.1007/s10566-008-9058-2.

    Article  PubMed  Google Scholar 

  • National Institute of Mental Health. (2001). Blueprint for change: Research on child and adolescent mental health’s council’s workgroup on child and adolescent mental health intervention development and deployment. (DHHS Publication No. CG 031–591). Washington, DC: US Government Printing Office.

    Google Scholar 

  • Nich, C., & Carroll, K. (1997). Now you see it, now you don’t: A comparison of traditional versus random-effects regression models in the analysis of longitudinal follow-up data from a clinical trial. Journal of Consulting and Clinical Psychology, 65(2), 252–261.

    Article  PubMed  CAS  Google Scholar 

  • Rector, N. A., Zuroff, D. C., & Segal, Z. V. (1999). Cognitive change and the therapeutic alliance: The role of technical and nontechnical factors in cognitive therapy. Psychotherapy, 36(4), 320–328.

    Article  Google Scholar 

  • Riemer, M., Athay, M. M., Bickman, L., Breda, C., Kelley, S. D., & Vides de Andrade, A. R. (2012). The Peabody Treatment Progress Battery: History and methods for developing a comprehensive measurement battery for youth mental health. Administration and Policy in Mental Health and Mental Health Services Research. doi:10.1007/s10488-012-0404-1.

  • Shelef, K., Diamond, G. M., Diamond, G. S., & Liddle, H. A. (2005). Adolescent and parent alliance and treatment outcome in multidimensional family therapy. Journal of Consulting and Clinical Psychology, 73(4), 689–698. doi:10.1037/0022-006x.73.4.689.

    Article  PubMed  Google Scholar 

  • Shirk, S. R., & Karver, M. (2003). Prediction of treatment outcome from relationship variables in child and adolescent therapy: A meta-analytic review. Journal of Consulting and Clinical Psychology, 71(3), 452–464.

    Article  PubMed  Google Scholar 

  • Shirk, S. R., Caporino, N., & Karver, M. (2010). The alliance in adolescent therapy: Conceptual, operational, and predictive issues. In D. Castro-Blanco & M. Karver (Eds.), Elusive alliance: Treatment engagement strategies with high-risk adolescents. Washington, DC: American Psychological Association Press.

    Google Scholar 

  • Shirk, S. R., Karver, M. S., & Brown, R. (2011). The alliance in child and adolescent psychotherapy. Psychotherapy: Theory/Research/Practice/Training, 48(1), 17–24. doi:10.1037/a0022181.

    Article  Google Scholar 

  • Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis. New York: Oxford University Press.

    Book  Google Scholar 

  • Stiles, W. B., Glick, M. J., Osatuke, K., Hardy, G. E., Shapiro, D. A., Agnew-Davies, R., et al. (2004). Patterns of alliance development and the rupture-repair hypothesis: Are productive relationships U-shaped or V-shaped? Journal of Counseling Psychology, 51(1), 81–92. doi:10.1037/0022-0167.51.1.81.

    Article  Google Scholar 

  • Topham, G. L., Wampler, K. S., Titus, G., & Rolling, E. (2011). Predicting parent and child outcomes of a filial therapy program. International Journal of Play Therapy, 20(2), 79–93. doi:10.1037/a0023261.

    Article  Google Scholar 

  • Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8, 370. Retrieved 10 March 2011 from http://www.rasch.org/rmt/rmt83b.htm.

Download references

Author information

Authors and Affiliations

  1. Vanderbilt University, Nashville, TN, USA

    Leonard Bickman, Ana Regina Vides de Andrade & M. Michele Athay

  2. Department of Psychology, University of South Florida, Tampa, FL, USA

    Jason I. Chen, Alessandro S. De Nadai, Brittany L. Jordan-Arthur & Marc S. Karver

Authors
  1. Leonard Bickman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ana Regina Vides de Andrade
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Michele Athay
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jason I. Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alessandro S. De Nadai
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Brittany L. Jordan-Arthur
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Marc S. Karver
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Leonard Bickman.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bickman, L., de Andrade, A.R.V., Athay, M.M. et al. The Relationship Between Change in Therapeutic Alliance Ratings and Improvement in Youth Symptom Severity: Whose Ratings Matter the Most?. Adm Policy Ment Health 39, 78–89 (2012). https://doi.org/10.1007/s10488-011-0398-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10488-011-0398-0

Keywords

Search

Navigation