Chapter 4. Symptoms and Functioning Severity Scale (SFSS)

Transcription

1 Chapter 4 Symptoms and Functioning Severity Scale (SFSS) Background Purpose The Symptoms and Functioning Severity Scale (SFSS) serves as the core indicator of treatment progress in the PTPB. It is a general measure of youths emotional and behavior problems intended to measure change over time in closely timed repeated measurements (e.g., two weeks apart). The SFSS is designed to provide systematic feedback on the youth s global level of severity in regard to symptoms and functioning. The SFSS is comparable to other existing clinical outcome measures but has the advantage of being short (it takes only five to seven minutes to complete) allowing for frequent assessment over time. Theory The emphasis of this battery is on the assessment of both treatment process and treatment progress. Treatment progress is to be assessed frequently so that adjustments to the treatment plan can be made while treatment is still ongoing. But, what is understood as treatment progress? In his popular book, Authentic Happiness, Martin Seligman (2002) proposes that if we consider individual well-being to range from -10 to 10 on a scale, then you can either focus to move a person from -2 to 0 or you could try to move a person from 0 to +2. The former could be described as a reduction of problems such as typical mental health symptoms (e.g., difficulty controlling oneself, feelings of depression, experience of anxiety) and the latter as an increase in happiness and hope. In the PTPB we included measures that allow the user to assess progress on both sides of that scale. The SFSS described in this chapter is a measure assessing progress in the reduction of symptom severity and increase of functionality. Thus, it is intended to assess the progress from a high severity level at the beginning of treatment, often referred to as the clinical range, into a comparably normal low level of severity. The SFSS is not too different from existing youth mental health outcome measures. Probably the most popular of these is the Child Behavior Checklist (CBCL; Achenbach, 1991) and the parallel versions of the Youth Self Report (YSR; Achenbach, 1991) and the Teacher Rating Form (TRF; Achenbach, 1991). Among other things, the Achenbach scales are popular because they provides a total problem score plus eight subscale scores (e.g., anxious/depression and attention problems). However, it requires the respondents to answer 118 questions. Completing this questionnaire can easily take the complete clinical session and is, thus, not a feasible option for frequent administration. In contrast, the SFSS provides only two subscale scores (internalizing and externalizing) in addition to the total score, and uses only 33 items, or less if one of the shorter versions is used. 30

2 The 33 items cover behaviors and experienced emotions that are linked to the most typical mental health disorders for youths including ADHD, conduct/oppositional disorder, depression and anxiety. In addition, the SFSS includes items related to peer and family relationship problems. However, the SFSS is not intended as a diagnostic instrument. It provides a global indicator for the overall reduction in symptom severity and increase in functionality. If a more targeted assessment is desired, we recommend administering a specialized scale on a less frequent basis (e.g., every three months) in addition to the SFSS. Since many youths experience co-morbid conditions, a global indicator covering a range of symptoms has important advantages in monitoring treatment progress. We recommend using all three versions (youth, adult caregiver, and clinician) of the SFSS simultaneously. There are significant differences in the perception of the problem by the different stakeholders in the treatment process. In a meta-analysis by Achenbach and his colleagues (Achenbach, McConaughy, & Howell, 1987), for example, they found that while the agreement between similar informants (e.g., two parents) is relatively high (mean r = 0.60), the correlation between different types of informants (e.g., parent and teacher) is low (0.28) and even lower if the scores of the child or youth are compared to those of others (0.22). A more recent review by Meyer and colleagues (2001) generally confirms these findings. Consequently, it seems advantageous if not necessary to triangulate these three perspectives. The brevity and the simplicity of the corresponding SFSS versions facilitate this type of comprehensive assessment. History of Development The SFSS has a long history of development. An earlier form of the Symptoms and Functioning measure was included in the Child and Adolescent Measurement System (CAMS; Doucette & Bickman, 2001) which was used in several child and adolescent mental health research projects. Experience with this earlier form of the scale led to several revisions. One of the major revisions we undertook was to create a better balance among externalizing and internalizing items. Using the American Psychological Association s Diagnostic and Statistical Manual of Mental Disorders Text Revision (DSM-IV-TR; 2000) as a general orientation, we made sure to include items that cover the main disorders described for youths: ADHD, conduct/oppositional disorder, depression and anxiety. In addition, the SFSS includes items related to peer and family relationship problems. The revised form was then reviewed by the clinical leadership of two mental health provider organizations. We also conducted focus groups with several clinicians at a provider organization in Nebraska. In addition we completed three pilot tests at three different provider organizations (the provider in Nebraska, a provider of home-based clinical services in Tennessee and Florida, and a crisis center in New York). Finally, we conducted cognitive interviews with youths and their caregivers. These reviews and tests led to more revisions that resulted in the 33-item form of the SFSS (SFSS-33) tested in the current psychometric study of which the results are presented in this chapter. In addition to the SFSS-33, we also evaluated two parallel shorter forms of the SFSS each containing half of the items. In order to create these shorter forms, we matched items on 31

3 their content area (e.g., ADHD) and their measure score (the item difficulty level). Then, one half of the matched pair was included in the SFSS Short Form A and the other half in the SFSS Short Form B. The idea of these short forms is to have two very short measures assessing the same construct (severity level) at a comparable difficulty level so that the total scores of these two measures can be compared across administrations. The type of information each single item represents will be covered over the course of two measurement instances, while the same total score can be calculated for each measurement instance. Testing and analyses in addition to the ones presented in this chapter are planned in regard to the construct and predictive validity and the sensitivity to change of all forms. Structure The SFSS has three forms of different length for each respondent group: adult caregiver, clinician, and youth. The three forms are a 33 item form (SFSS-33), and two equivalent short forms (Form A and Form B). The SFSS-33 includes a question about self-harm. This self-harm question often requires immediate action by the clinician, which sometimes is difficult to do if the SFSS is not processed immediately. Excluding this item does not change the psychometric properties of the scale in any significant way. Thus, the clinician has the option to simply exclude this item from the scale. The short Form A consists of 17 items while Form B includes 16 items. Each item asks the respondent to rate how frequently the youth experienced certain emotions or showed certain behaviors within the last two weeks (e.g., how often they got into trouble). The frequency is rated using a five-point Likert scale and includes the following answer categories: never, hardly ever, sometimes, often, and very often. The SFSS Total Score is calculated based on a linear transformation of the item average if at least 85 % of the items are completed. Since the average is practically identical for all different lengths of each respondent version this results in the same type of score ranging from 42 to 105 for adult caregivers and clinicians, and 32 to 107 for youths. The total score for a youth can be compared for the same youth over time as a trend and it can be compared to comparison samples, such as the psychometric sample presented in this manual. Based on these comparison samples, one can assess whether a score should be considered high, medium, or low. In addition, we matched the SFSS Total Score to those of the total problem score of the CBCL as explained further below. Clinicans who are familiar with CBCL scores may find this a useful way of interpreting the SFSS Total Scores. In addition to the total score, there are also two subscale scores available: Internalizing and Externalizing. In order to create these subscale scores, we divided the items into internalizing and externalizing symptoms, as it is commonly done in clinical practice. Internalizing disorders, such as depression and anxiety, are expressed within the individual and are focused on a clinically problematic affective and/or emotional state. Externalizing disorders, such as conduct disorder and attention deficit hyperactivity disorder (ADHD) are expressed overtly. It is important to note that these categorizations were done using theoretical criteria prior to the empirical analyses. While there is some evidence in the psychometric data that confirms the division of the items into 32

4 internalizing and externalizing symptoms, the results of the confirmatory factor analyses do not meet stringent scientific criteria in satisfactory ways. Therefore, these subscale scores should serve mainly as an orientation whether certain types of items (internalizing or externalizing) scored higher or lower during a given instance of measurement. For tracking progress on the reduction of the youth s severity level, we recommend using the total score. 33

5 Table 4.1 below shows how the items of the SFSS relate to the short forms, the subscales, and the DSM-IV-TR categories that were used to develop the scale. Table 4.1 SFSS Items in Relation to Short Forms, Subscales, and DSM-IV-TR Categories Item SFSS- DSM-IV-TR Item Subscale No. Short Form Category 1 Throw things when mad B E C 2 Eat more or less A I D 3 Feel unhappy or sad B I D 4 Get into trouble A E C 5 Have little or no energy B I D 6 Disobey adults B E C 7 Interrupted others A E I 8 Lie to get things A E C 9 Hard time controlling temper A E C/I 10 Use drugs B Neither Other 11 Worry about a lot of things A I A 12 Hard time getting along w/ others B E C 13 Threaten or bully others B E C 14 Feel worthless A I D 15 Drink alcohol A Neither Other 16 Have a hard time having fun A I D 17 Feel afraid others laugh B I A 18 Have a hard time waiting turn A E I 19 Sleep more than normally do B I D 20 Hang out with kids in trouble A E C 21 Feel nervous/shy A I A 22 Have a hard time paying attention B E I 23 Get into fights with family/friends B E C 24 Lose things needed A E I 25 Have a hard time sitting still B E I 26 Have a hard time sleeping A I A 27 Feel tense B I A 28 Cry easily B I D 29 Annoy others on purpose A E C 30 Argue with adults B E C 31 Think you don't have friends A Neither Other 32 Feel too scared to ask questions B I A 33* Think about hurting self A I D DSM-IV-TR category codes: C = conduct or opposition; D = depression; A = anxiety; I = hyperactivity or impulse; C/I = conduct or impulse. *This Item requires immediate action and may be excluded at the discretion of the clinician. Items 10, 15, and 31 were uncategorized as they could be associated with any disorder. 34

6 Administration The SFSS may be administered during all phases, intake through follow-up, as indicated in Table 4.2, and is to be completed by the youth. The SFSS-33 may be administered or the short Form A and B may be used alternately instead. Parallel versions are available for adult caregivers and clinicians. The suggested frequency of administration is every other week or at least once a month. Table 4.2 Administration of SFSS by Phase Intake Treatment Discharge Follow-Up Y A C Y A C Y A C Y A C Y = Youth (age 11-18); A = Adult Caregiver; C = Clinician Recommended Frequency: Every two weeks or at least once a month The suggested administration schedule of all the measures in the Peabody Treatment Progress Battery is presented in Appendix A. All PTPB measures with self-scoring forms can be found in Appendix B: Measures and Self-Scoring Forms. Description Basic Descriptives The possible range of total and subscale scores for all different forms of the SFSS is 42 to 105 for adult caregivers and clinicians, and 32 to 107 for youths. Using the well-known criteria established for the CBCL 4 the mean value of approximately 65 for the adult SFSS scales (see Table 4.3) indicates that, on average, the adult caregivers in this sample rated their children to be approximately 1.5 SDs above the mean value of 50 for a nonclinical population. Thus, on average, the youths are considered to be in the clinical range according to their caregivers. The distribution is approximately normal with no significant skewness or kurtosis. As can be seen in Table 4.6, 25% of the adult caregivers rated the severity of their children to be 73 or higher while another quarter rated the severity to be rather low with scores of 58 or less. The distributions of the 4 To make the scores comparable to a well-known clinical measure, we used linear equating to match the mean and standard deviation to that of Achenbach s popular Child Behavior Checklist (CBCL; 1991). This method is based on scores in which the mean and standard deviation of the nonclinical population are (50, 10). A score of 65, 1.5 SDs above average, is often considered as in the clinical range. This method of scaling scores goes back to the Minnesota Multiphasic Personality Inventory (MMPI) in the 1930 s and is familiar to most clinicians. The comparability of the SFSS scores to the CBCL scales was confirmed by fact that the means and standard deviations of the adult caregiver and the youth version of the SFSS are very close to those of the CBCL and YSR for the 116 cases that completed both scales (SFSS and CBCL/YSR). 35

7 internalizing and externalizing subscales are listed in Table 4.3 as well. As expected, the average score for the externalizing scale is higher (by almost seven points) than the internalizing average. The standard deviations of the two subscales are slightly inflated. The correlation of the two subscales scores is relatively high with r = 0.65 but not so high that they are indistinguishable. Table 4.3 Descriptive Statistics for SFSS-Adult Caregiver Summary Scores Scale N Mean Std Dev Skewness Kurtosis Min Max SFSS SFSS Short Form A SFSS Short Form B Internalizing Externalizing The scoring of the clinician versions of the SFSS is parallel to the adult caregiver versions. As can be seen in Table 4.4, the average clinician scores in the psychometric sample were slightly lower compared to the adult caregiver ratings, suggesting that clinicians reported slightly fewer problems than did caregivers. The spread of scores as expressed by the standard deviations is also a slightly lower (about two units) than the variability in the adult caregiver scores. Again, the distribution of scores is approximately normal with no significant skewness or kurtosis. A quarter of the clinicians thought the youths to be at a high score of 69 or higher, while another 25 % at low severity of 57 or lower (see Table 4.6). The distributions of the internalizing and externalizing subscales show that the mean for the externalizing subscale is higher by about seven points. The standard deviations of the subscales are also higher compared to total score of the full scale. The correlation of the two subscales scores are lower compared to the adult caregiver and youth version with r = That is, the two subscales share 25% of the variance. Table 4.4 Descriptive Statistics for SFSS-Clinician Summary Scores Scale N Mean Std Dev Skewness Kurtosis Min Max SFSS SFSS Short Form A SFSS Short Form B Internalizing Externalizing Youth tended to perceive their level of severity lower as compared to their adult caregivers and clinicians. This is not surprising as we mentioned the relatively low correspondence among different types of reporters in the introduction of this chapter. Thus, the lower average ratings of the comparison sample of youth should be taken into 36

8 account when interpreting the SFSS scores. As can be seen in Table 4.5, the average score is between 54 and 55. It important to note that these relatively low scores imply that, on average, the youth in the psychometric sample consider themselves to be comparable to a nonclinical population of children. Only about a quarter of the youth rate themselves to be in the clinical range (see Table 4.6). Also noticeable is the higher variability in scores. Compared to the adult caregivers, the standard deviations in the youth sample are approximately 2 units higher while they are about four units higher than the clinician standard deviations. Thus, on average, youth spread their answers more across the scale range. The distribution of scores is approximately normal with low levels of skewness and kurtosis. The average score on the externalizing subscale is again higher than the internalizing one, but the difference is less pronounced than for the adult caregivers and clinicians (a difference of about four compared to seven points). The correlation of the two subscales is high with r = 0.66 but significantly less than the internal reliability estimates (see Table 4.8). Table 4.5 Descriptive Statistics for SFSS-Youth Summary Scores Scale N Mean Std Dev Skewness Kurtosis Min Max SFSS SFSS Short Form A SFSS Short Form B Internalizing Externalizing Quartiles High scores are in the top quarter, with low scores in the bottom quarter as presented in Table 4.6. For example, for the SFSS-Adult Caregiver Total Score, a score greater than 73 is considered high, while a score less than 58 is considered low. To aid interpretation, the quartiles were used to create low, medium, and high scores and percentile ranks based on comparison to the psychometric sample. This information is presented in the last section of this chapter. Table 4.6 SFSS Quartiles Quartile Adult Caregiver Clinician Youth 100% Max % Q % Median % Q % Min

9 Evidence of Reliability Reliability Coefficients The Cronbach s alpha internal consistency reliability correlations are presented in Table 4.7. These alphas suggest a satisfactory degree of internal consistency for the total score. The short forms have, as expected, a smaller internal consistency estimate compared to the SFSS-33, as can be seen in Table 4.7. However, the difference is small and the alphas for the short forms are still in the satisfactory range, suggesting that the short forms can be used with comparable reliability. Table 4.7 Cronbach s Alphas for the SFSS Scale SFSS-33 Form A Form B SFSS-Youth SFSS-Adult Caregiver SFSS-Clinician The internal consistency reliability of the two subscales is presented in Table 4.8. The reliability estimates for these subscales are also high, suggesting that the subscale scores can be used with satisfactory internal reliability based on this psychometric sample. Table 4.8 Cronbach s Alphas for the SFSS Subscales Scale Internalizing Externalizing SFSS-Adult Caregiver SFSS-Clinician SFSS-Youth Comprehensive Item Psychometrics Tables 4.9 to 4.11 present the comprehensive item psychometrics. Shaded cells indicate that a criterion was out of the range of sought values, as described previously in Table 2.2 in Chapter Two. Only items with two or more shaded cells are considered problematic. Overall results for the adult caregiver, clinician, and youth versions were similar. Two items with near-floor means (alcohol and drugs) had many psychometric warnings for all three respondents. These were likely due to their high kurtosis, low variance and poor fit to the Rasch measurement model. Given the complexity of the construct and the relatively large number of items, it is not unusual to find items that do not meet all of the psychometric criteria in a satisfactory way. While they are not effective items in this 38

10 sample, they were retained because alcohol and drug use have content validity in a mental health index. In the youth sample, the item asking about unusual sleep pattern is also flagged as a potential problematic item. The relatively low item-total correlation (0.40) suggests that this item is not the strongest predictor of severity. However, it was determined that the deviations from the recommended criteria of quality in all three cases were so slight that the item could remain as it is. Besides using drugs and alcohol, the item about hanging out with kids in trouble is flagged as problematic in the adult caregiver sample. The high infit and outfit values as well as the relatively low discrimination value suggest that this item behaves differently from the other items in the scale and is not the most efficient item to use in differentiating youth with different severity levels. However, because of it content validity and the acceptable item-total correlation, we decided to retain the item in the scale. The item about unusual sleeping patterns is flagged again in the counselor sample. The rationale to keep this item is similar to the one for the youth version. In addition, it was considered an important goal to keep all three versions the same to have parallel versions. 39

11 Table 4.9 Comprehensive Item Analysis for the SFSS-Youth Item N Mean St Dev Kurtosis Worry Feel unhappy Temper Disobey Trouble Paying attention Eat more/less Argue with adults Interrupt Family Lose things Sitting still Annoy others Fights with family Cry easily No energy Nervous around others Sleep Lie Tense Kind in trouble Hard time sleeping Waiting your turn Item-Total Std CFA Loadings Measure Infit Outfit Discrimination 40

12 Throw when mad Hard to have fun Afraid of laugh Worthless Scared to ask No friends Bully Hurting self Drugs Alcohol Note: Items listed in ascending order by item difficulty (Measure) Table 4.10 Comprehensive Item Analysis for the SFSS-Adult Caregiver Item N Mean St Dev Kurtosis Item-Total CFA Loadings Measure Infit Outfit Discrimination Disobey adults Hard time c temper Interrupt others Hard time paying attention Argue with adults Feel unhappy or sad Getting along w/ family Get into trouble Worry a lot Lie to get things

13 Annoy other people Hard time sitting still Fights Lose things you need Eat a lot more or less Hard time waiting turn Feel tense Have little or no energy Afraid others would laugh Threaten or bully others Hang with kids in trouble Feel nervous around people Sleep a lot more Throw things when mad Don't have any friends Hard time having fun Cry easily Feel worthless Hard time sleeping Too scared to ask in class Think about hurting yourself Use drugs non-medical Drink alcohol Note: Items listed in ascending order by item difficulty (Measure). 42

14 Table 4.11 Comprehensive Item Analysis for the SFSS-Clinician Item N Mean St Dev Kurtosis Item-Total CFA Loadings Measure Infit Outfit Discrimination Argue with adults Disobey adults Getting along w/ family Feel unhappy or sad Hard time c temper Worry a lot Get into trouble Fights Hard time paying attention Interrupt others Annoy other people Lie to get things Feel tense Hang with kids in trouble Hard time sitting still Hard time waiting turn Have little or no energy Feel nervous around people Afraid others would laugh Hard time having fun Feel worthless Lose things you need Threaten or bully others

15 Hard time sleeping Don't have any friends Sleep a lot more Eat a lot more or less Cry easily Throw things when mad Too scared to ask in class Think about hurting yourself Use drugs non-medical Drink alcohol Note: Items listed in ascending order by item difficulty (Measure). 44

16 Standard Errors of Measurement For the SFSS, the standard errors of measurement (SEMs) are presented in Table For example, on the adult caregiver SFSS-33, we can say that the true score is between approximately ± 2 SEMs, or 2.62 points on a one to five point scale with 95% certainty. It is not surprising that the SEMs for the short forms are higher given their somewhat lower internal consistency reliabilities. The higher standard error for the youth versions also suggest that there is less certainty in the exact level of the youth s rating of severity at any given measurement instance. Having multiple measurement instances from the same youth allows direct comparisons to the youth s prior ratings. Table 4.12 Standard Errors of Measurement by SFSS Form and Informant SFSS Form Adult Caregiver Clinician Youth SFSS SFSS Form A SFSS Form B Reliable Change Index The reliable change threshold for each form and informant on the SFSS is reported in Table For example, the reliable change index (RCI) for the youth self-report score on the SFSS-33 is 5.36 points, giving us 75% confidence that a difference of more than 5.36 points is not due to chance. If the change is in a negative direction (i.e., decrease in score value) it represents an improvement in perceived youth symptoms and functioning, while a change in the positive direction (i.e., increase in score value) indicates that the youth s global severity is worsening. Table 4.13 Reliable Change Thresholds by SFSS Form and Informant SFSS Form Adult Caregiver Clinician Youth 75% 75% 75% SFSS SFSS Form A SFSS Form B Test-Retest Reliability Another test of a scale s reliability is its test-retest correlation. For this purpose data are typically collected from the same group of individuals at two different time points. For scales that measure a more or less constant construct, such as an intelligence test, one would expect high correlations between the first time an individual was assessed and the second time, even if the time between those two measurement instances is substantial (e.g., a year). For a construct like symptom severity, change is expected over time, especially if the youth is in treatment. On average, one would not expect major changes in a short time period (e.g., two weeks). Since the average time interval of seven days 45

17 between administrations in our test-retest sample (SD = 3.8; range 1-14 days), we would expect a relatively high correlation of 0.7 or higher. Table 4.14 provides the correlations for our test-retest samples, in which correlations range between 0.68 and These results suggest some variability over time, as expected since the youth are in treatment, as well as a relatively strong relationship between the two assessments. For example, approximately 75% of the variance in adult caregiver ratings at the second assessment can be explained by using the ratings from the first assessment. In comparison, the testretest reliability is the strongest for adult caregivers, followed by clinicians, and the lowest for youth. It is important to note that the observed correlations with samples of participants are only approximate due to sampling error. Future analyses with larger samples will have to confirm the robustness of these results. Table 4.14 SFSS Test-Retest Correlations Respondent r N Adult Caregiver Clinician Youth Evidence of Validity The objective of this section is to provide evidence for the construct validity of the SFSS. The construct the SFSS is intended to measure is complex. Weiss, Susser & Catron s (1998) model of psychopathology suggests that it has wide-band common features as well as narrow-band specific features (internalizing-externalizing and more specific aspects such as anxiety versus depression). According to the DSM-IV-TR, psychopathology occurs in distinct subtypes, each defined by distinct profiles of symptoms and deficits. The psychometric sample includes youth with a wide variety of problems and the SFSS is a broad spectrum estimate, including anxiety, depression, behavior problems, as well as other problems youth typically experience often at the same time. Thus, we did not expect to find as clear of a factor structure as found in specialized measures, such as Brekke s (2002) five aspects of social functioning in schizophrenics, for example. Interpretation of the factor analyses presented below should take this complexity into account. Scree Plot A combined scree plot of eigenvalues for the three informants of the SFSS, youth, adult caregiver and clinician, appears in Figure 4.1. The plots suggests that the three respondents respond in a similar fashion, with one large and robust principle component, and four additional ones that contribute to less than 4% of the variance. Since we used theoretical considerations prior to the empirical analyses when we determined whether 46

18 each of the items should be considered part of either the internalizing or externalizing subscale, we present the results of the CFA next. Figure 4.1 Combined Scree Plot of Eigenvalues for SFSS Confirmatory Factor Analysis We evaluated three different models using a confirmatory factor analysis (CFA) conducted with SAS CALIS in a mode emulating Bentler & Wu s (1995) EQS. The first model was a one-factor model where all 33 items were loading onto the one single factor. The second model is also based on one factor but only those items that were not ambiguous in regard to the internalizing or externalizing subscales were included (see Table 4.1). The third model included the same items as the second model but is based on two correlated factors (internalizing and externalizing) according to the theory (see Table 4.1). The second and the third model are nested and can be compared directly, while the first model can only indirectly compared to the other two. As can be seen in Tables 4.15, 4.16, and 4.17, none of the models meet the established criteria of fit (CFI, GFI, and RMSEA) in a satisfactory way. It is interesting to note that in the youth sample, the factor structure is the clearest, while in the clinician sample the models fit the data the poorest. Comparatively, however, the third model has a clear superior fit relative to the other two models, which, for the last two models is confirmed 47

19 by the significant chi-square differences tests. We consider this indirect evidence and support for the validity and use of the two subscale scores. Given this and the high internal reliability of the subscales and the difference of their average scores for all three respondent types, we feel comfortable recommending the use of the two subscales as a general orientation in addition to the total score. The standardized factor loadings for the items on the internalizing and externalizing factors in the third model ranged for the adult caregiver version from 0.47 to 0.76 for internalizing, and 0.43 to 0.83 for externalizing. For the clinician version, they ranged from 0.52 to 0.74 and 0.47 to 0.81 respectively. The range for the youth version is 0.44 to 0.73 and 0.52 to These loadings should be considered with caution, especially for the adult caregiver and clinician version, because of the unsatisfactory fit of the overall model to the data. Table 4.15 Evaluation of the SFSS Factor Structure Adult Caregiver Model χ 2 df χ 2 / χ 2 Bentler Joreskog diff df CFI GFI RMSEA One-Factor All 33 Items One-Factor Selected Items Two Correlated Factors For the CFI and GFI, values greater than 0.90 indicate good fit between a model and the data. For the RMSEA, a value of 0.05 indicates close fit, 0.08 fair fit, and 0.10 marginal fit (Browne & Cudeck, 1993). Table 4.16 Evaluation of the SFSS Factor Structure Clinician Model χ 2 df χ 2 / χ 2 Bentler Joreskog diff df CFI GFI RMSEA One-Factor All 33 Items One-Factor Selected Items Two Correlated Factors For the CFI and GFI, values greater than 0.90 indicate good fit between a model and the data. For the RMSEA, a value of 0.05 indicates close fit, 0.08 fair fit, and 0.10 marginal fit (Browne & Cudeck, 1993). 48

20 Table 4.17 Evaluation of the SFSS Factor Structure Youth Model χ 2 df χ 2 / χ 2 Bentler Joreskog diff df CFI GFI RMSEA One-Factor All 33 Items One-Factor Selected Items Two Correlated Factors For the CFI and GFI, values greater than 0.90 indicate good fit between a model and the data. For the RMSEA, a value of 0.05 indicates close fit, 0.08 fair fit, and 0.10 marginal fit (Browne & Cudeck, 1993). Convergent and Discriminant Validity In order to provide additional evidence for the construct validity of the SFSS, we also evaluated criteria for discriminant and convergent validity. Discriminant validity showed that using the scale the construct can be measured in a way that it can be differentiated from related but distinct constructs. In convergent validity, we examined the degree to which the operationalization is similar to (converges on) other operationalizations to which it should be theoretically similar. Evidence for convergent validity was established by showing that the scale scores correlated highly with other established scales that measure the same or a very similar construct. Sometimes, this is also referred to as concurrent validity. Evidence for this convergent and discriminant validity for the SFSS is presented in Chapter 3. Using the SFSS Scoring Use the self-scoring forms (Tables 4.19, 4.20, and 4.21) to calculate the SFSS Total Score. These tables are also available with the measures in Appendix B. Enter the value for the selected responses in fields A-GG for the SFSS-33, or fields A-Q or A-P for Form A or Form B respectively. Calculate the total score by adding the scores and then divide the sum by the number of items, 33 for the SFSS-33, 17 for the Form A, and 16 for Form B. Finally, do a linear transformation by multiplying the average score by the factor U and then add the factor V as shown in Table There are no reverse coded items in the SFSS. The self-scoring forms are also available in Appendix B: Measures and Self- Scoring Forms. Table 4.18 Factors for Linear Transformation of SFSS Scores U V Youth Adult Caregiver / Clinician

21 Use the scoring forms in the case where measures are fully completed (100% response rate). Otherwise, in cases with missing data, scoring can be done by computing the mean of completed items. Determining when too much missing data occurs for computing summary scores is at the discretion of the user. The analyses presented in this chapter required 85% of the items to have valid answers. Table 4.19 Self-Scoring Form SFSS-33 Item Never Hardly Ever Values for Responses Sometimes Often Very Often Enter value for selected responses here and calculate scores as instructed Subscale E Subscale I A B C D E F G H I J K L M N O P Q R S T U V W X Y Z AA BB CC DD EE FF GG 50

22 Subscale E Subscale I Total Score Sum of A, D, F-I, L, M, R, T, V-Y, CC, and DD HH / 16 Perform calculation noted below HH Sum of B, C, E, K, N, P, Q, S, U, Z-BB, FF, and GG For Youth, multiply by and add For Adult Caregiver or Clinician, multiply by and add SFSS Subscale E Score = JJ SFSS Subscale I Score = MM SFSS-33 Total Score = QQ II JJ KK / 14 KK Perform calculation noted below MM Sum of J, O, and EE LL Sum of HH, KK, NN OO / 33 Perform calculation noted below NN OO PP QQ Table 4.20 Self-Scoring Table - SFSS Form A Values for Responses Item Never Hardly Ever Sometimes Often Very Often Enter value for selected responses here and calculate scores as instructed A B C D E F G H I J K L M N O P 51

23 Q Sum of A-Q: R R / 17: For Youth, multiply S by and add For Adult Caregiver or Clinician, multiply S by and add SFSS Form A Total Score = T S T Table 4.21 Self-Scoring Form- SFSS Form B Values for Responses Item Never Hardly Ever Sometimes Often Very Often Enter value for selected responses here and calculate scores as instructed A B C D E F G H I J K L M N O P Sum of A-P: Q Q / 16: For Youth, multiply R by and add For Adult Caregiver or Clinician, multiply R by and add SFSS Form B Total Score = S R S 52

24 Interpretation The SFSS assesses symptoms and functioning during the previous two weeks. Scores on the SFSS can fluctuate significantly from one administration to the next. Thus, it is important to interpret whether these changes represent clinically significant change and if so, what may have caused it. It will also be important to administer the SFSS frequently so that general trends can be reliably assessed. A negative trend indicates that the youth is experiencing fewer problem behaviors and emotions. Low, Medium, High Scores In general, the scores of the SFSS can range from 42 to 105 for adult caregivers and clinicians, and 32 to 107 for youths, where a high score represents high severity while a low score indicates low severity. The low, medium, high scores presented in Table 4.22 apply to all forms of the SFSS, and help to judge whether a score should be considered relatively low, medium, or high, compared to the current psychometric sample. Respondents who rate their (or the youth s) severity as high are experiencing behaviors or emotions that are causing problems in their life. When a youth reports low severity it indicates that the youth is experiencing few problem behaviors or emotions. In addition, the scores can be interpreted relative to the CBCL. The high correlation with the CBCL and linear transformation conducted for the total SFSS scores allow the use of the criteria established for the CBCL and YSR. In general, scores of 65 or higher are considered to be in the clinical range. Table 4.22 SFSS Low, Medium, and High Scores Scale / Subscale Low Medium High SFSS-Youth < > 63 Internalizing < > 64 Externalizing < > 68 SFSS- Adult Caregiver < > 73 Internalizing < > 70 Externalizing < > 80 SFSS-Clinician < > 69 Internalizing < > 68 Externalizing < > 75 Percentile Ranks Percentile ranks for the youth, adult caregiver and clinician psychometric samples appear in Table For example, as shown in the table for youth respondents, a score of 50 is in the 38 th percentile. This means that in the psychometric sample, 38% scored 50 or lower and 62% scored higher. 53

25 Table 4.23 SFSS Percentile Ranks for Total Scores SFSS-Youth SFSS-Adult Caregiver SFSS-Clinician Score Percentile Score Percentile Score Percentile Note: Percentiles on the SFSS short forms are extremely similar. 54

26 References Achenbach, T. M. (1991). Integrative guide for the 1991 CBCL/4-18, YSR, and TRF profiles. Burlington, VT: University of Vermont, Department of Psychiatry. Achenbach, T. M., McConoughy, S. H., & Howell, C. T. (1987). Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101, American Psychological Association (2000). Diagnostic and statistical manual of mental disorders test revision DSM-IV-TR (text revision). Washington, DC: Author. Bentler, P. M. & Wu, E. J. C. (1995). EQS for Windows user s guide. Encino, CA: Multivariate Software, Inc. Brekke, J. S., Long, J. D., & Kay, D. (2002). The structure and invariance of a model of social functioning in schizophrenia. Journal of Nervous and Mental Disease, 190(2), Browne, M. W., & Cudeck, R. (1993). Alternative ways of accessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp ). Newbury Park: Sage. Doucette, A., & Bickman, L. (2001). Child Adolescent Measurement System (CAMS). Nashville, TN: Author. Meyer, G. J., Finn, S. E., Eyde, L., Kay, G. G., Moreland, K. L., Dies, R. R., Eisman, E. J., Kubiszyn, T. W., & Reed, G. M. (2001). Psychological testing and psychological assessment: A review of evidence and issues. American Psychologist, 56, Seligman, M. E. P. (2002). Using the New Positive Psychology to Realize Your Potential for Lasting Fulfillment. New York, NY: Simon & Schuster. Weiss, B., Susser, K. & Catron, T. (1998). Common and specific features of childhood psychopathology. Journal of Abnormal Psychology, 107(1),