CASII. American Academy of Child and Adolescent Psychiatry. AACAP Work Group on Community Systems of Care for Seriously Emotionally Disturbed Children

Transcription

1 CASII Child and Adolescent Service Intensity Instrument Background information and Initial Data Analysis American Academy of Child and Adolescent Psychiatry AACAP Work Group on Community Systems of Care for Seriously Emotionally Disturbed Children Principal Investigator - Theodore Fallon, Jr., MD, MPH CoPrincipal Investigator - Andres Pumariega, MD Work Group members: Mark Chenven, M.D., Emilio Dominguez, M.D., Katherine Grimes, M.D., Graeme Hanson, M.D., William Heffron, M.D., Robert Klaehn, M.D., Larry Marx, M.D., Tom Vaughan, Jr., M.D., Nancy Winters, M.D., and Al Zachik, M.D. AACAP Staff: Kristin Kroeger Ptakowski, Director of Clinical Affairs June 2001

2 INTRODUCTION When a child or adolescent presents for mental health care, either in the private or public sector, whether it be in the medical, mental health, juvenile justice, or child protective system, there has been no standardized way to link the presentation of the child with a level of service that the child/adolescent needs. This has been true despite the fact that managed care, class action suits and consent decrees, such as Willy M. (in North Carolina), have all been predicated on such an established link. In addition, the concepts of wraparound and serious emotional disturbance require a link between presentation and level of service if these concepts are to move forward and be more generally accepted by the public. These facts beg for a standard that can provide a link between the presentation of a child or adolescent and a recommendation for level of service. The American Academy of Child and Adolescent Psychiatry has an opportunity to take the lead in providing this public standard, which up to now has not existed. There have been a number of previous attempts to use clinical assessments as a method of determining the appropriate level of care for children and adolescents. There has, however, never been a clearly defined method available in the public domain for linking the results of a clinical assessment to a level of care best suited for treatment. Traditional psychiatric assessment instruments such at the Diagnostic Interview Schedule for Children (DISC) or the Child Behavior Checklist (CBCL) provide information on clinical status with regard to mood, anxiety, behavior or thought process, but they did not have a connection with treatment needs. These instruments fail to take into account the interaction that the child or adolescent has with his/her environment. Other instruments such as the Child and Adolescent Functional Assessment Scale (CAFAS), like the CBCL, uses a dimensional approach to assess the child or adolescent s interaction with the environment, but fail to take into account the environment itself, which creates stresses for and supports the child or adolescent. There are proprietary instruments such as the Level of Care Assessment Tool (LOCAT) used by USHealthCare that do take into account aspects of the environment, but these have not been normed and have not been available for public reference or use. Another approach to patient treatment and placement has focused on criteria specific to a particular program. For example, a day hospital might have a set of criteria that would describe the type of patient deemed most appropriate for that program. This idea has evolved into the concept of level of care, which attempted to group services of similar intensity together. But again, there is no overall structure that links patient needs with intensity of service. 2

3 INSTRUMENT DEVELOPMENT With these challenges in mind, the AACAP s Work Group on Community Systems of Care, in collaboration with the American Association of Community Psychiatrists, developed the Child and Adolescent Service Intensity Instrument, formerly called the Child and Adolescent Level of Care Utilization System (CALOCUS). The CASII instrument is a tool to determine the appropriate level of care placement for a child or adolescent. It was designed to facilitate communication between personnel involved on both sides of the equation, clinical need as well as resource availability, and to create a standard context in which all participants are working together. The CASII takes into consideration child development and the importance of the parents and the community in supporting the child. It takes into consideration developmental disorders such as mental retardation, autism and delinquency, and to consider the contributions of the child/adolescent as well at the parent and family. This scale also draws from clinical experience, and considers a number of values, theories, and resources. Levels of care are clearly defined within interagency systems of care philosophy using a wraparound process, where services are child-centered, family focused and driven, culturally competent and individualized to the child s multiple needs and context. This instrument draws from clinical experience and a number of values, theories, and resources, including: CASSP /Guiding Principles for Systems of Care (Stroul and Friedman, 1986), which advocates for community-and family-centered treatment in the least restrictive, most normative clinically appropriate environment; Developmental theory, which describes the trajectory of normative physical, emotional, cognitive, and social changes of childhood and adolescents, which must be addressed in both assessment and treatment; Family empowerment, in which the family is respected and regarded as the lead agent in assessment, treatment, and case management activities for the child or adolescent and is viewed as the primary agent for fostering healthy development and growth in the child or adolescent; Cultural competence (Cross, Bazrow, Dennis, and Issacs, 1998; Pumariega and Cross, 1997), which embodies respect for people of all ethnic backgrounds, accommodation of their special needs (e.g. culturally appropriate assessment and treatment, linguistic support) and whenever possible, provision of services by culturally competent professionals and staff members whose ethnic diversity mirrors that of the populations served. Cultural factors often impact the assessment of co-morbidity, level of functioning, environmental support, and treatment and engagement, thus directly biasing the level of care placement. 3

4 Wraparound concepts, which entail the integration of a comprehensive network of professional and support services for the child or adolescent and family using naturally existing community supports; as well as inter-organizational structures capable of providing blended funding streams to provide service (VanDenBerg and Grealish, 1996). This model supports the use of a strength-based, individualized service plan (ISP) for each child and adolescent. Clinical expertise provided by two international organizations comprised of psychiatrists serving children and adolescents, and adults with a variety or psychiatric, substance use, and developmental disorders (American Academy of Child and Adolescent Psychiatry and American Association of community Psychiatrists). The CASII links a clinical assessment with standardized levels of care and has a method for matching the two. The method consists of quantifying the clinical severity and service needs on six dimensions (eight ratings) that are standardized using anchor points. The ratings are quantified in order to convey information easily, but also provide a spectrum along which a child/adolescent may lie on any given dimension. This allows for a broad range of users to employ the CASII. This can be done for any child/adolescent in any setting regardless of diagnosis or the system the child is involved with. The instrument also considers three distinct types of disorder: psychiatric disorders, substance use disorders, or developmental disorders (including autism and mental retardation), and has the ability to integrate these as overlapping clinical issues. Once the dimensional ratings are done, the scores are combined to generate a level of care recommendation. This integration of multiple dimensions is the essence of the CASII instrument. It is this that guides the user to an appropriate CASII level of care assignment. In order to develop an instrument applicable to a wide variety of treatment environments (including rural areas where established mental health services are rare) and child or adolescent needs, it was important to develop a set of definitions for levels of care that described the resource intensities needed at each specific level of care. These definitions needed to be flexible and adaptable, in order to be broadly applicable to the wide variety of treatment environments in which care can be given. This approach was chosen to allow for service providers to give adequate clinical services and quality care in the most economic and realistic fashion. How user friendly the instrument was also important. It was anticipated that ease of use, time, and universal adaptability would be critical factors in establishing broad acceptability of CASII. This instrument is anticipated to lead to the establishment of a single child and adolescent standard for use by insurance and governmental agencies, service providers and consumers. CASII employs multi-disciplinary/multi-informant perspectives on children and adolescents and is designed to be used by a variety of mental health professionals. Although it is primarily used for initial level of care placement decisions, it can be used at all stages of treatment to assess the level of intensity of services needed. An important aspect of CASII is its potential use for feefor-service utilization management. Many instruments in the past have developed separate 4

5 criteria for hospital admissions, continuing care and discharge planning. The CASII instrument makes it unnecessary to use different criteria because of the dynamic nature of the quantifiable dimensional ratings. CASII can also be applied to activities such as treatment planning, outcome monitoring and program development. There are a number of things that CASII will not do. It will not prescribe program design, but rather, it recommends the type and intensity of resources that need to be available in that program. It does not specify treatment intervention, and it does not invalidate the importance of clinical judgement. The CASII also does not limit creativity in developing specific treatment programs that meet the needs of special populations or localities. INSTRUMENT STRUCTURE DIMENSIONAL RATING SYSTEM The CASII dimensional rating system is used to determine the intensity of needed services. It operationalizes the factors clinicians consider in determining the most appropriate services and level of care needed. CASII has six dimensions: RISK OF HARM: This dimension is the measurement of a child or adolescent's risk of harm to self or other as well as an assessment of his/her potential for being a victim of physical or sexual abuse, or neglect. FUNCTIONAL STATUS: This dimension measures the impact of a child or adolescent's primary condition on his/her daily life. It is an assessment of the child's ability to function in all age-appropriate roles: family member, friend, and student. It is also a measure of the effect of the primary problem on such basic daily activities as eating, sleeping and personal hygiene. CO-MORBIDITY: This dimension measures the co-existence of disorders across four domains: Medical, Substance Abuse, Development Disability or Delay and Psychiatric. RECOVERY ENVIRONMENT: This dimension is divided into 2 subscales: 1) Environmental Stress, and 2) Environmental Support. An understanding of the strengths and weaknesses of the child or adolescent's family is essential to choosing an accurate rating in this dimension. It is also a measure of the neighborhood and community's role in either worsening or improving the child or adolescent's condition. RESILIENCY AND TREATMENT HISTORY: Resiliency refers to a child or adolescent's innate or constitutional emotional strength, as well as the capacity for successful adaptation. The concept of resiliency is familiar to clinicians who treat children or adolescents who have the most severe disorders and/or survive the most traumatic life circumstances, yet who either maintain high functioning and developmental progress, or use treatment for a rapid return to that state. This dimension also measures the extent to which the child or adolescent and his/her family has responded to past treatment. 5

6 ACCEPTANCE AND ENGAGEMENT (Scale A--Child/Adolescent, Scale B-- Parents/Primary Caretaker): This dimension is divided into two subscales to allow for measurement of both the child or adolescent's and his/her family's acceptance and engagement. Clearly the child or adolescent's treatment benefits when the family is proactively and positively engaged, and conversely, treatment suffers when the family is disinterested, disruptive or openly hostile toward the process. Only the highest subscale score (the subscale indicating the most significant challenge to treatment) is used in calculating the composite score. Each dimension has a five-point rating scale. For each of the five possible ratings within each dimension, a set of criteria is clearly defined. Only one criterion needs to be met for that rating to be selected. Therefore, for each dimension, the highest rating, in which at least one of the criteria is met, is the rating that should be assigned. LEVELS OF CARE The levels of care in CASII are organized in a unique way. The focus is on the level of resource intensity, which is more flexibly defined in order to meet the child or adolescent s needs. Each level of care is defined by a combination of service variables: physical facilities (care environment), clinical services, support services, crisis stabilization and prevention services. Some levels of care may contain the same resources found at other levels of care. With higher levels of care, a greater number and variety of services are utilized. In addition, the need for active case management of services will increase at the higher levels. In CASII, there are seven levels of care: Level 0: Level 1: Level 2: Level 3: Level 4: Basic Services. This is a basic package of prevention and health maintenance services that are assumed to be available to all people in the community Recovery Maintenance and Health Management. This level of service is usually reserved for those stepping down from higher levels of care who need minimal system involvement to maintain their current level of function or need brief intervention to return to their previous level of functioning. Examples of this level of service include: children or adolescents who only need ongoing medication services for a chronic condition or brief crisis counseling. Outpatient Services. This level of care most closely resembles traditional once/week visits. Intensive Outpatient Services. This level of service can range from a couple visits per week up to a few hours for three days per week, and may include multiple services (e.g. big brother, church services, mental health services) necessitating coordination (case management). Intensive Integrated Service Without 24-Hour Medical Monitoring. This level of 6

7 care best describes the increased intensity of services necessary for the multisystem, multi-problem child or adolescent requiring more extensive collaboration between the increased number of providers and agencies. A more elaborate Wraparound plan is also required, using an increased number of formal supports. Additional supports may include respite, homemaking services or paid mentors. In more traditional systems, this level of service is often provided in a day treatment or partial hospitalization setting. Active case management is essential at this level of care. Level 5: Level 6: Non-Secure, 24-Hour, Medically Monitored Services. Traditionally, this level of care has provided a safe residence and has including group home, foster care or a residential facility, but can also be provided by a tightly knit array of Wraparound services in the community. Secure, 24-Hours, Medically Managed Services. Most commonly, these services are provided in inpatient psychiatric settings or highly-programmed residential facilities. If security needs can be met through the wrap-around process, then this level of intensity of service could also be provided in a community setting. Case management remains essential to make sure that the time each child spends at this level of care is held to the minimum required for optimal care and that the transition to lower levels of care are smooth. FIELD TESTING BACKGROUND Testing of the CASII in a variety of settings has been done to establish both reliability and validity. These studies were funded in part from a grant by the Center for Mental Health Services, a branch of Substance Abuse and Mental Health Services Administration (SAMHSA). The study protocols were reviewed and approved by East Tennessee State University Institution Review Board. The sites for this field study were chosen from among sites volunteered to participate in collecting and submitting data. These sites include: 1. Philadelphia - The outpatient community mental health programs at the Children s Hospital of Philadelphia, the children s hospital of the University of Pennsylvania. 2. Oregon - A consortium of residential, day treatment and inpatient programs centered around Portland, Oregon. 3. The State of Hawaii public mental health system. 4. Selected public mental health sites in the State of North Carolina. At each of these sites, clinicians were trained in the CASII. These clinicians then collected data on new patients, which was collated and submitted for entry and analysis. Only clinicians who were trained and completed at least 2 vignettes were included in this study. 7

8 STUDY 1 - RELIABILITY Purpose: To determine reliability of the CASII among different types of clinicians. Method: Seven clinical vignettes were constructed. These vignettes were given to 16 child psychiatrists and 78 non-psychiatrists (mostly case managers). The 16 child psychiatrists all had assisted in the construction of the CASII and thus were very familiar with the instrument. Each of these psychiatrists rated each of the 7 vignettes for a total of 105 ratings. The 78 non-psychiatrists were trained on the CASII in a 6 hour workshop. These non-psychiatrists were mostly Masters trained social workers with an average of 5 years experience (see Table 1 below) from 4 sites around the country (Philadelphia, Oregon, Hawaii, North Carolina). At the end of their training, these clinicians used the CASII to rate at least 2 of the 7 vignettes chosen at random for a total of 157 ratings. Intraclass correlations coefficients (ICC 2,2) as described by Shrout and Fleiss (1979) were calculated for physicians and non-psychiatrists separately. Table 1 - Non-Psychiatrist Raters Training Number Average Years of experience post training BA Training years Masters training years PhD training years Results: As seen in Table 2 below, intraclass correlation coefficients for the sub-scales for physicians ranged between 0.73 and 0.93 while the composite score is For the non-physicians, the subscale scores ranged from 0.57 to 0.95, while the composite score coefficient is For all of the vignettes, non-psychiatrist rated cases 1.9 points higher than psychiatrists. Table 2 - Intraclass Correlation Coefficients Comparing Raters on CASII Scores (ICC2,2) Child Psychiatrist Ratings Non-Psychiatrist Ratings Risk of Harm Function Co-Morbidity Stress

9 Support Resilience Parent Acceptance Child Acceptance Composite Score Conclusion: These results indicate that the CASII can be used reliably by psychiatrists and non-psychiatrists, even with a relatively brief training. The general trend is that subscale scores for the child psychiatrist were more consistent, but the composite score balances out the inconsistencies for the non-psychiatrists providing an extremely reliable summary score even for clinicians with less extensive training. Another finding was that Psychiatrists tended to rate slightly lower (less severe) than nonpsychiatrists. This is ideal, since it would be preferable to have less experienced clinicians be more cautious, particularly with regard to safety issues. STUDY 2 - VALIDITY Purpose: To determine validity of the CASII by comparing these ratings to the rating of two highly used instruments, the CGAS and the CAFAS. Method: After training on the CASII, 78 non-psychiatrists, (as described above), completed routine clinical evaluations and then rated the patients with the CASII and either the Child Global Assessment Scale (CGAS), as described by Schaffer et al., or the Child and Adolescent Functional Assessment Scale (CAFAS), as described by Hodges. CAFAS scores were computed using the 8 CAFAS sub-scales. Patients ages 6 to 18 years old came from inpatient, outpatient and residential settings. Modalities for outpatient treatment included individual, family, group psychotherapies, case management, and wraparound services. (Pearson correlation coefficients compared the CASII ratings with the CGAS and CAFAS scores). Results: CGAS scores, in this population of patients (n=182), varied from 23 to 81 with a mean of 40. CASII composite scores varied from 8 to 34 with a mean of 20. Correlation of the CGAS with the sub-scale scores of the CASII varied between 0.41 to 0. Those sub-scale correlations related to clinical presentation, which would be expected to correlate with CGAS (function, risk of harm and resilience) were 0.41 to 0.26; while those sub-scales having to do with environment and not related to the child directly (environmental support, parent 9

10 acceptance) were close to 0. Co-morbidity also correlated poorly with the CGAS. Although it would be expected that CGAS might take co-morbidity into account, it appears that it does not. Environmental stress is highly correlated with CGAS, whereas environmental stress is related to the child s clinical state. This correlation might be expected to be lower than resilience. During the training, it became obvious that less well trained clinicians have difficulty sorting out what is environmental stresses and what are disruptions that the child/adolescent themselves had created, (such as getting expelled from school). Table 3 - Correlation of CASII Scores With CGAS Scores (n=182) Correlation with CGAS Risk of Harm -.37 Function -.41 Co-Morbidity -.05 Stress -.28 Support -.05 Resilience -.26 Parent Acceptance -.02 Child Acceptance -.24 Composite Score -.33 All patients who had CGAS ratings also had CAFAS ratings. In addition, there were 432 patients who had only CAFAS ratings (total n =614 for CAFAS/CASII rating combinations). Mean CASII composite score on these 614 patients was 20 with a range of 8-34 while CAFAS composite score mean was 96 with a range of 0 to 200. Table 4 shows the correlations between the CASII scores and the CAFAS composite score. As with the CASII/CGAS correlations, those CASII scales that reflect attributes about the child are moderately correlated with the CAFAS composite score: Risk of Harm, Function and Resilience. With the CAFAS scores, the Co-Morbidity scale is more highly correlated than with the CGAS. Also, just as with the CGAS comparison, the CASII sub-scales having to do with environment and not related to the child directly (environmental support, parent acceptance) are lower ( ). For comparison, for those patients who had both CAFAS and CGAS scores (182), this correlation was computed to be

11 Table 4 - Correlation of CASII Subscale and Composite Scores With CAFAS Composite Score (n=614) Pearson Correlations with CAFAS composite score Risk of Harm.51 Function.52 Co-Morbidity.41 Stress.35 Support.22 Resilience.50 Parent Acceptance.11 Child Acceptance.31 Composite CASII Score.62 CGAS (n=182).50 Conclusion: These results indicate that there is moderate correlation between conventionally used scales (CGAS and CAFAS) and the CASII, although there seems to be a higher correlation between the CASII and the CAFAS. - particularly the composite scores. This study trained clinicians in the use of the CASII but not in the use of the CAFAS or the CGAS. Both of these scales were being used routinely at the test sites. We did not test the clinicians on their proficiency of the CAFAS or the CGAS. It has been shown in the literature that the results of the CGAS, and the comparison of the CAFAS and CGAS suggest that the fidelity of the CGAS is less than optimal in the field. It is also encouraging to see that those CASII sub-scales that measure the child alone correlate more highly with the CAFAS scores. While those sub-scales that measure environment or engagement have much lower correlation - as would be expected. CONCLUSION The CASII is reliable when used by a broad range of clinicians. It is also valid when compared with the CAFAS, not only in the CASII subscales related to the child, which are correlated with the CAFAS composite, but also in the CASII subscales relating to environment and engagement, which are not correlated with the CAFAS scores. The CASII seems to be measuring what we expect it to measure (Face Validity). 11