What to Measure in Client Behavior

Transcription

1 What to Measure in Client Behavior Alec Boros, Ph.D. Research Manager, Oriana House, Inc. Rebecca Fischbein, Ph.D. Research Specialist, Oriana House, Inc. Asli Buldum, M.A. Research Specialist, Oriana House, Inc.

2 Using Assessments to Measure Offender Change Measuring Change Responsivity Principle Focus on Three Assessments Criminal Thinking Scale Adverse Childhood Experiences Short Inventory of Problems Issues to Consider with using an Assessment Implementation Interpretation Application Tracking Discussion

3 Where does our presentation fit in? Fidelity (2-C) Quality Assessments: Overview of ODRC s ORAS Quality Assurance Tracking Behavioral Change (2-C)Behavior Management System for Residential Programs Risk Assessments for Specialized Populations (3-D)Specialized Populations, When Is the ORAS Not Enough? Reponsivity Assessments (2-B) Using Assessments to Measure Offender Change

4 Change We could measure all kinds of change in community corrections Staff competence in service delivery Clients in knowledge of cognitive and skill development Risk Need scores Criminal and anti-social behavior Drug/alcohol use Barriers

5 Why is it important to measure change during programming? Maximizes rare treatment resources Allows for timely adjustment in client programming Provides feedback to offender Program effectiveness Have better outcomes One of the neglected areas is in Responsivity, and also one of the most confusing

6 Relationship of Targeted Interventions: Neglected Areas Desired Outcome Fidelity Responsivity Treatment Plan Risk Dosage Need Treatment

7 Responsivity: General vs. Specific Responsivity are individual factors that interfere with or facilitate learning Addressing responsivity can have a major impact on the effectiveness of other programming

8 Responsivity: General vs. Specific General Responsivity- is associated the use of the most effective correctional programming to change the criminogenic needs of offenders Specific Responsivity Use cognitive behavioral interventions that take into account characteristics of the individual. Failure to address can hinder treatment efforts

9 Specific Responsivity: The changing focus Not only most ignored but also the most difficult to define Definitions have changed over time Three approaches of responsivity: 1. Match treatment approach with learning style of offender 2. Match characteristics of staff member with that of the offender 3. Match skills of the therapist with the type of program Catering intervention to fit the offender Barriers All of the above

10 Responsivity: Internal vs. External Internal Responsivity Factors Characteristics of the individual offender Demographic More difficult to assess and accommodate factors such as personality and intelligence They can contribute to the engagement of offenders into treatment and the development of therapeutic alliance

11 Responsivity: Internal vs. External External Responsivity Factors The interaction between Facility, Staff and Client characteristics Facility Staff Client

12 How can we categorize these responsivity areas? Gender Race Age Ethnicity Religion Demographic Characteristics Peers Motivation Trauma Literacy Attitude/Thinking Style Family relations Personality Dynamic Characteristics Programming Intelligence Communication style Learning Style Static or Near-Static Characteristics

13 Selecting an Appropriate Responsivity Assessment

14 Sample of Assessments Used at OHI Assessment Construct Our Recommendation Adverse Childhood Experiences (ACE) TCU Criminal Thinking Scales (TCU CTS) TCU Family and Friends Psychological Inventory of Criminal Thinking Style (PICTS) Assesses exposure to childhood emotional, physical or sexual abuse and household dysfunction Measures 6 criminal thinking styles Family relationships, family drug use, peer socialization, peer criminality Assesses 8 criminal thinking styles Under review. Currently developing norms to examine how we can use information regarding adverse childhood experiences to guide programming. Yes. Good assessment for determining client thinking errors prior to programming. Can be used to guide programming as well as one-on-one client-staff interactions. Also, an effective measure of client change when pre and post test scores are compared Under review. Currently developing norms to examine how we can use information regarding social functioning (family/friends) to guide programming. It depends. An excellent, well-validated assessment of preintervention thinking as well as pre and post test change, but maybe not be practical due to length and difficulty in scoring

15 Sample of Assessments Used at OHI Assessment Construct Our Recommendation PTSD Diagnostic Scale (PDS) Short Inventory of Problems (SIP) University of Rhode Island Change Assessment Score (URICA) Assesses severity of Post Traumatic Stress Disorder (PTSD) Symptoms stemming from a traumatic incident Measure 5 life areas that could be affected by drug/alcohol use in past 3 months Motivation to change as assessed by Transtheoretical Model of Change Yes. Good for assessment of PTSD symptoms to determine whether client requires a referral for PTSD treatment. Also can be used as a post-test to determine changes in PTSD symptoms. Please note: self report assessment that is administered and scored by a clinician. Yes. Good assessment for determining problems related to drug and alcohol use. Can be used to provide insight into areas where client requires assistance. Also, an effective measure of client change and improvement in client functioning when pre and post test scores are compared. Yes. Good assessment for determining the client s initial motivation to change. Can be adapted to address any problem. We found it is not effective as a post-test and cannot effectively be used to determine change in motivation.

16 Texas Christian University Criminal Thinking Scale (TCU CTS) What is criminal thinking? Psychological Inventory of Criminal Thinking Walters, 1995 Criminal Thinking Scale Knight et al., 2006 TCU CTS Scales Personal Irresponsibility Entitlement Power Orientation Justification Cold Heartedness Criminal Rationalization Definition Blaming others/external factors for criminal behavior. Feeling of privilege Need for power/ control over others Minimalization of seriousness of antisocial acts Callousness Negative attitude toward law and authority figures

17 TCU CTS Sample Items Knight, K., Garner, B. R., Simpson, D. D., Morey, J. T., & Flynn, P. M. (2006). An assessment for criminal thinking. Crime and Delinquency, 52(1),

18 TCU CTS - Scoring Scoring Assessment and scoring guide can be found online at: trt.html Higher scores indicate greater criminal thinking

19 CTS: Scoring Breakdown

20 CTS: Tracking Individual Scores

21 TCU CTS - Interpretation How can CTS scores be used? How we use at OHI: Intake Sample Personal Irresponsibility homework Exit Evaluate programming success

22 CTS: Tracking Many Client Scores

23 TCU CTS Evaluation Results 546 Cuyahoga County probation clients who underwent Thinking for a Change programming completed the TCU CTS intake and exit assessment Pre-Test Post-Test T-Test Results Average Intake Score for 546 clients who completed post-test Average post-test score p-value Entitlement Justification * Power Orientation Cold Heartedness Criminal Rationalization * Personal Irresponsibility * Note. *indicates a statistically significant difference between scores at Intake and Post-Test; significance is indicated at 95%.

24 Assessing Trauma Why measure trauma? The Adverse Childhood Experiences (ACE) Scale The Post Traumatic Diagnostic Scale (PDS)

25 In more detail: The ACE What does the ACE measure? How was it developed? Constructs Childhood psychological abuse Childhood physical abuse Childhood dysfunctional household Childhood exposure to sexual abuse

26 In more detail: The ACE Sample items: Filitti, M. J., Anda, R. F., Nordenberg, D., et al. (1998). Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults. American Journal of Preventative Medicine, 14(4)

27 What does the ACE tell us? Higher scores indicate greater risk Adverse Childhood Experiences are major risk factors for leading causes of illness and death as well as poor quality of life in the United States

28 Prevalence Rates: The ACE Percentage of participants who experienced negative childhood experiences

29 Prevalence Rates: Female Criminal Justice Populations Number of Adverse Childhood Experiences Women in Study by Messina & Grella (2006)* (sample size of 491) Women at OHI (sample size of 59) % 11.9% % 22.0% % 15.3% % 18.6% % 8.5% 5 or more 21.2% 23.7% *Messina, N. & Grella, C. (2006). Childhood trauma and women s health outcomes in a California prison population. American Journal of Public Health. 96(10):

30 In more detail: The ACE How can we track ACE data? How can we use ACE information? Referral for further evaluation and treatment

31 Additional Trauma Screening Instrument: Post Traumatic Diagnostic Scale (PDS) For use by psychologists, social workers, and counselors Forty-nine item measure, assesses severity of PTSD symptoms related to a single traumatic event Must be purchased Can be used over time to monitor changes in symptoms Use at OHI Foa, E., Cashman, L., Jaycox, L., & Perry, K. (1997). The validation of a self-report measure of PTSD: The Posttraumatic Diagnostic Scale. Psychological Assessment, 9, Foa, E. (1996). Posttraumatic Diagnostic Scale Manual. Minneapolis, MN: National Computer Systems.

32 SIP Short Inventory of Problems Measures the negative consequences of substance abuse across five domains: Physical Interpersonal Intrapersonal Impulse Control Social Responsibility Was originally developed for alcohol use, but has recently been validated on drug using populations (Bender, Griffin, Gallop, and Weiss, 2007).

33 Where does the SIP come from? Allensworth-Davies, Cheng, Smith, Samet, Saitz. (2012). The Short Inventory of Problems-Modified for drug use (SIP-DU): Validity in a primary care sample. The American Journal of Addictions, 21:

34 More about the SIP Self-administered Higher scores indicate increased levels of drugrelated consequences An effective measure of client change and improvement in client functioning when pre and post test scores are compared Provides insight into areas where clients require assistance

35 SIP Sample Items Bender, R. E., Griffin, M. L., Gallop, R. J., & Weiss, R. D. (2007). Assessing negative consequences in patients with substance use and bipolar disorders: psychometric properties of the short inventory of problems (SIP). The American Journal on Addictions American Academy of Psychiatrists in Alcoholism and Addictions, 16(6),

36 SIP Sample Items per Domain Physical Because of using drugs, I have not eaten properly. Interpersonal A friendship or close relationship has been damaged by my drug use. Intrapersonal My drug use has gotten in the way of my growth as a person. Impulse Control When using drugs, I have done impulsive things that I regretted later. Social Responsibility I ve had money problems because of my drug use.

37 SIP How is it utilized at OHI? To measure the severity of substance-related problems among drug court participants at intake, 6 months, and discharge Baseline measures are compared with follow-up measures Provides an indication of client s progress while in the program

38 SIP Tracking Scores SPSS spreadsheet snapshot

39 SIP Sharing Results SIP can not only be used to determine program effectiveness, but also to help inform treatment staff. At OHI, the research department had created individualized reports to share with treatment staff. Treatment staff was to consider this report in program planning for client needs.

40 SIP Individualized Reports Example of an individualized report:

41 SIP Presenting Results Used in determining program effectiveness: Analysis show significant reductions in problems related to drug use: Average SIP Scores for Graduates Over Time (Physical Subscale) SIP Physical Scores Intake 6 Month Follow-Up Discharge Time

42 Some Lessons Learned 1. Must be sure to not only train staff how to administer/score assessments but how to use them 2. Be sure to use a tracking method that is right for your project, staff and resources 3. Use the same tool throughout your organization to measure a construct

43 Discussion 1. What are some assessments that you have used in your own agencies? 2. How have you used the results of the assessments?

44 Questions?