The Integrated Developmental Model of Supervision: Scale Development and Validation Procedures

Transcription

1 Professional Psychology: Research and Practice 1992, Vol. 23, No. 6, Copyright 1992 by the American Psychological Association, Inc /92/$3.00 The Integrated Developmental odel of Supervision: Scale Development and Validation Procedures Brian W cneill Department of Educational and Counseling Psychology Washington State University John S. C. Romans Department of Applied Behavioral Sciences Oklahoma State University Cal D. Stoltenberg Department of Educational Psychology University of Oklahoma Despite the growing empirical evidence for developmental models of supervision, the need for reliable, valid assessment procedures for identifying a trainee's level of development exists. This study sought to address this need by administering an instrument designed to assess constructs relevant to Stoltenberg and Delworth's (1987) Integrated Developmental odel (ID) to counseling and clinical psychology trainees of varying levels of graduate education, counseling, and supervision experience. Preliminary data provided support for the ID and suggested construct validity for the Supervisee Levels Questionnaire-Revised because scores on the instrument differed for certain trainee groups who varied in amount of previous training and experience. In 1984, Russell, Crimmings, and Lent concluded that developmental conceptualizations of the supervisory process held considerable promise in terms of synthesizing theory from diverse orientations and guiding research on supervision. Since then, empirical evidence has been mounting in support of the theoretical conception of counselor supervision and training as a developmental process (see reviews by Stoltenberg & Delworth, 1987; Worthington, 1987). However, Russell et al. (1984) stressed the need for the construction of reliable and valid assessment procedures for identifying a trainee's level of development. Holloway (1984) has also emphasized the need for psychometrically sound instruments to evaluate trainee perception of clinical supervision outcome or effectiveness. In an attempt to address the concerns of Russell et al. (1984) and Holloway (1984), cneill, Stoltenberg, and Pierce (1985) tested some of the specific constructs posited by Stoltenberg's (1981) model of counselor supervision and development. Trainees' perceptions of their own counseling and supervision behaviors were assessed by an instrument designed to measure the primary constructs of the counselor complexity model (i.e., the BRIAN W CNEILL received his PhD from Texas Tech University in He is currently an associate professor in the Counseling Psychology program at Washington State University. CAL D. STOLTENBERG received his doctorate from the University of Iowa in He is currently the chairperson of the Department of Educational Psychology at the University of Oklahoma. JOHN S. C. ROANS received his doctorate from the University of Kansas in 1990 and is currently an assistant professor in the Department of Applied Behavioral Sciences at Oklahoma State University. This INVESTIGATION WAS SUPPORTED by University of Kansas General Research Allocation CORRESPONDENCE CONCERNING THIS ARTICLE should be addressed to Brian W cneill, Department of Educational and Counseling Psychology, Washington State University, Pullman, Washington Supervisee Levels Questionnaire [SLQ]). Congruent with predictions made by the model, results indicated that trainees possessing increasing amounts of counseling and supervision experience and more graduate education reported greater selfawareness, autonomy, acquisition of counseling skills, and understanding of theory than did trainees at lower experience levels. Thus, the results of cneill et al. (1985) were viewed as providing support for the constructs proposed by Stoltenberg (1981), and the SLQ demonstrated promise as an assessment instrument for researchers and supervisors. ost recently, however, Stoltenberg's (1981) model has been revised and expanded by incorporating aspects of other developmental conceptualizations (e.g., Loganbill, Hardy, & Delworth, 1982), as well as available research findings, resulting in the Integrated Developmental odel (ID; Stoltenberg & Delworth, 1987). In his earlier model, Stoltenberg (1981) neglected to directly address the possibility that trainees may be functioning at different levels of counselor development for various activities associated with counselor and psychotherapist behavior. Gross measures of development were used when a trainee was considered to be largely functioning within Stage 1,2, or 3. This assumption was reflected in instruments designed to assess trainee level (e.g., the original SLQ, cneill et al., 1985, and Wiley & Ray, 1986, among others). Loganbill et al. (1982) more completely addressed this issue by proposing that trainees need to resolve eight critical issues in their development into master therapists. However, their model maintained a developmental path that relied on recycling through three general sequential stages in a manner similar to the "spiraling" notion later proposed by Hess (1986). Although intuitively appealing, recycling or spiraling through developmental stages is inconsistent with most theories of development that assume (except for the possibility of brief regression) that development is permanent and not subject to reenactment within a given area (see Stoltenberg & Delworth, 1987, for a detailed discussion). The ID deals with this issue by proposing three key structures that describe 504

2 DEVELOPENTAL ODEL OF SUPERVISION 505 trainee development through stages across eight separate (although not orthogonal) domains of therapist activity. This reconceptualization of trainee development accounts for trainees exhibiting behaviors consistent with more than one level of development at any given time across various domains. Thus, one does not recycle through stages but progresses through the stages (as marked by changes in the structures) for different domains at different points in time. According to this model, a therapist who is an advanced Level 3 in one area, for example, individual therapy, may be at Level 1 or 2 with respect to family therapy or psychological assessment. In brief, the integrated model proposes changes in three structures that trace the progress of trainees through four different developmental stages or levels. These structures include (1) Self and Other Awareness, (2) otivation, and (3) Dependency-Autonomy. For example, at the first developmental level, trainees are hypothesized to be dependent on the supervisor for advice and direction, overfocused on their own anxieties, and high in motivation based on a strong desire to become a counselor, yet lacking an in-depth understanding of the complexity of the role or process of counseling. At Level 2, trainees experience a dependency-autonomy conflict in regard to dependence on the supervisor for guidance. As he or she gains experience with the process of counseling and concomitant successes and failures, a questioning of one's skills occurs, resulting in fluctuating levels of motivation. At this point, the trainee shifts from a self-focus to a predominant focus on the experiences of the client, which often results in confusion or emotional turmoil. The Level 3 trainee has weathered the previous conflicts and functions on a more autonomous level, with a higher degree of confidence. otivation again becomes high and consistent, based on an understanding of the limitations and complexities of the counseling process. These advanced trainees are also able to be reflective as well as empathic, thus integrating their own emotional and cognitive responses to a client with an awareness of the client's experience. The three overriding structures are applied to various domains of counselor training and activities (e.g, intervention competence, client conceptualization), and development is thought to occur sequentially at different rates across these domains depending on training experiences and individual counselor characteristics. The final stage, Level 3 integrated, is conceptualized by Stoltenberg and Delworth (1987) as a generally integrative stage where Level 3 structures are present in a number of domains and the counselor achieves integration across domains. Supervision environments are carefully designed and matched to the developmental level of the trainee in order to enhance his or her progress through the levels. For example, in the case of the Level 1 dependent trainee, an educational environment high in structure and direction is needed to attend to the basic needs of the beginner. In contrast, the Level 3 advanced trainee benefits from a less structured environment that is designed not only to support but also to challenge the trainee to function and think independently. Although a number of instruments have been used to measure trainee development in a manner congruent with Stoltenberg's (1981) developmental model (e.g., Krause & Allen, 1988; cneill et al. 1985; Reising & Daniels, 1983; Wiley & Ray, 1986), none of these instruments adequately addresses development as conceptualized by the ID. The crucial structures of the ID, which describe development across domains for the four levels, are not addressed in any current instruments. For example, Wiley and Ray's (1986) and Krause and Allen's (1988) instruments rely on descriptions of trainees' behaviors and training needs consistent with constructs from Stoltenberg's (1981) model, which assumes that trainees will be in a single stage of development, with no reference to developmental structures or domains. Similarly, Reising and Daniels (1983) used Likert-type items tapping various trainee and supervisory needs in an examination of Hogan's (1964) model, which was later incorporated into Stoltenberg's (1981) model. Again, developmental structures and domains of activity were not addressed. Thus, with the recent presentation of the ID, our intent was to develop and validate an instrument that reflects the current theoretical constructs of supervision proposed by Stoltenberg and Delworth (1987) that are not addressed by the SLQ or other instruments. Thus, we hoped to construct a brief instrument for application to the general domain of counseling and supervision that might benefit supervisors' applied work with trainees and researchers conducting investigations of the model. Subjects ethod The subjects were 66 female and 39 male volunteers from eight geographically diverse training programs in counseling and clinical psychology, representing eastern, midwestern, western, and southern sections of the United States. Four of the sites were doctoral-level training programs in counseling psychology, two were clinical psychology programs, and two were internship sites that included practicum training. All program sites included master's-level trainees. With the exception of one clinical training program, all sites were accredited by the American Psychological Association. Training directors or their representatives were contact persons, solicited on the basis of informal professional contacts, who volunteered to participate in data collection and follow-up. They were then asked to recruit trainees currently enrolled in a practicum or internship and provide the investigators with the number of trainees who fit these criteria. The total sample was 69.6% counseling psychology students and 30.4% clinical psychology students. All intern-level participants were enrolled in counseling psychology doctoral programs. Of 159 trainees contacted, 105 returned the completed measures, resulting in a return rate of 66%. Instrumentation The Supervisee Levels Questionnaire-Revised (SLQ-R) was designed to tap characteristics on a continuum of development associated with the levels hypothesized by Stoltenberg and Delworth (1987). The instrument was derived by using the previous 24 items from the SLQ as well as additional items generated to tap the overall constructs and overriding structures proposed by Stoltenberg and Delworth (1987). The 47 self-report items were then rationally organized by three expert judges into three subscales reflecting the overriding structures of self- and other-awareness, motivation, and dependencyautonomy, with an emphasis on application to the domains of intervention competence, client conceptualization, and interpersonal assessment. Differences in classification were resolved through discus-

3 506 B. cneill, C. STOLTENBERG, AND J. ROANS sion. 1 The response format was a 7-point Likert scale with never and always as polar anchors. Higher scores reflect higher levels of development as described in the model. Demographic information tapping trainees' gender, graduate education, counseling experience, and amount of supervision received was also collected. Procedure The SLQ-Rs, along with the demographic questionnaire, were mailed to the contact persons, who then distributed them to trainees, who then volunteered to participate. Subjects were sampled approximately 6 weeks into the academic semester. All forms were anonymously completed and returned to the contact persons, who returned the completed materials to the investigators. Analysis Participants were classified into groups we defined as beginning, intermediate, and advanced trainees on the basis of counseling experience (range = 1-26 semesters), supervision experience (range = 1-22 semesters), and graduate education (range = 1-8 years). Past research has relied on classifying trainees according to current practicum status. We believe that this method ignores other pertinent experiences that trainees bring to supervision. Thus, similar to cneill et al. (1985), we attempted to account for multiple criteria in classifying trainees developmentally by simply assigning each trainee a value for each index of experience (e.g., 1, 2, 3, etc.) and summing those three values to create a score for a trainee experience variable (which accounted for semesters of previous counseling and supervision experience, as well as years of graduate education). We then divided our sample into three groups who differed in terms of trainee experience and labeled them as beginning, intermediate, and advanced trainees as generally specified by the ID. This procedure resulted in what we conceptualized as 22 beginning trainees with one semester of supervision and counseling experience and 1 to 2 years of graduate education (Trainee Experience < 4), 48 intermediate trainees with two to four semesters of counseling and supervision and 3 years of graduate education (Trainee Experience 5-7), and 35 advanced trainees with five or more semesters of supervision and counseling and 4 or more years of graduate education (Trainee Experience > 7). Reliability Results Of the original 47 items administered to the subjects, those items that did not correlate significantly with both the total score and appropriate subscale score at the.0001 level were discarded, and some items were reclassified into different subscales on the basis of high item-subscale correlations. This procedure resulted in 12 items composing the Self and Other Awareness subscale (score range = 12-84; e.g., I feel genuinely relaxed and comfortable in my sessions, I am able to adequately assess my interpersonal impact on clients and use that knowledge therapeutically), 8 items composing the otivation subscale (score range = 8-56; e.g., Sometimes the client's situation seems so hopeless, I just don't know what to do, Sometimes I question how suited I am to be a counselor/therapist), and 10 items composing the Dependency-Autonomy subscale (score range = 10-70; At times I wish my supervisor could be in the counseling/therapy session to lend a hand, It is important that my supervisor allow me to make my own mistakes). The complete SLQ-R consisted of a total of 30 items (score range = ). Cronbach alpha reliability coefficients were calculated for the three subscales, resulting in reliability estimates of.83,.74,.64, and.88 for the Self and Other Awareness, otivation, and Dependency-Autonomy subscales, and total scores, respectively. Validity To assess the construct validity of the SLQ-R, differences were examined in subscale and total scores between the beginning, intermediate, and advanced groups. Pearson correlation coefficients calculated on SLQ-R subscale scores indicated that the three subscales were significantly related for Self and Other Awareness and Dependency-Autonomy, r =.53 p <.001; for Self and Other Awareness and otivation, r =.58 p <.001; and otivation and Dependency-Autonomy, r =.43 p <.001. Although these intercorrelations are statistically significant, in our view they are not so high as to suggest that the three subscales are measuring the same attribute. Nonetheless, to initially explore for differences in SLQ-R scores between the groups, we used a multivariate analysis of variance (ANOXA) using trainee experience as the independent variable and the SLQ-R subscales as dependent variables. Hotelling's test of significance indicated that the three trainee groups differed on a linear combination of SLQ-R subscale scores, F(6,198) = 2.45 p <.026. An analysis of variance (ANOVA), again using trainee experience as the independent variable, indicated that the total SLQ-R scores of the groups differed significantly, F(2,102) = 7.37 p <.001. In comparison, an ANOVA examining the total SLQ scores (cneill et al., 1985) of the current trainee groups only approached significance, F(2,102) = 2.77, p <.067, thus demonstrating the comparative superiority of the SLQ-R in tapping differences among trainees. Because we expected that SLQ-R subscale and total scores would increase as a result of trainee experience, as recommended by Rosenthal and Rosnow (1985), we used a series of focused, one-way planned contrasts to test our hypotheses and further specify the nature of our obtained effects. Results of the series of focused one-tailed t tests are summarized in Table 1. Using a.05 alpha level, we consistently found significant differences in mean subscale and total SLQ-R scores between the beginning and advanced trainee groups and the intermediate and advanced trainee groups. Although the group means differed slightly in the expected directions, no significant differences were obtained between the beginning and intermediate groups. We also used the product-moment correlation as a measure of effect size (Rosnow & Rosenthal, 1988). Obtained correlations for significant effects consistently fell in the "medium" range as defined by Cohen (1977), who labeled effect sizes on a continuum from small to large, and were reflective of magnitudes previously found in counseling psychology research (Haase, Waechter, & Solomon, 1982). Table 2 summarizes the mean scores for each of the groups on the SLQ-R subscales and total scores, along with standard deviations, ranges, and a 95% confidence interval. 1 The judges were the three authors who had detailed knowledge of the model. Any differences in subscale classification were discussed and unanimously agreed on by the judges. The SLQ-R, information on items, and other analyses are available from Brian W cneill.

4 DEVELOPENTAL ODEL OF SUPERVISION 507 Table 1 Preplanned Contrasts of Supervisee Levels Questionnaire-Revised for Trainee Experience Level Score Self and Other Awareness Dependency-Autonomy otivation Total score df Beginning vs. intermediate Intermediate vs. advanced Beginning vs. advanced T r r T T r value probability r df value probability r df value probability r Discussion Overall, our preliminary data from this investigation indicate that the SLQ-R measures the constructs associated with Stoltenberg and Delworth's (1987) integrated model of counselor development with some degree of reliability and validity. In terms of reliability, the Cronbach alpha scores for the subscales and total scores represent a substantial improvement over the previous version of the SLQ, with the exception of the Dependency-Autonomy subscale. Results of the planned contrasts demonstrated no significant differences between what we had conceptualized as groups of beginning and intermediate trainees. Confidence intervals indicated substantial overlap in the SLQ-R scores of these groups. On the other hand, the scores of both beginners and intermediates were consistently lower than those of our advanced group, with little or no overlap as evidenced in the confidence intervals. Thus it is possible that our hypothesized distinctions between what we considered beginning and intermediate trainees were overly finite. That is, the subjects in this study may be more reflective of only beginning (Level 1) and intermediate (Level 2) supervisees as defined in the ID. Indeed, the lack of ceiling effects on the SLQ-R scales suggests a higher possible range for scores of counselors who possess higher levels of experience. Similar effects were reported by Wiley and Ray (1986), as differences in supervised experience were found only between supervisor-rated Level 1 and Level 4 and Level 2 and Level 4 graduate program trainees, respectively. Consequently, future studies will need to include more intern-level trainees as well as postdoctoral professionals in order to assess the full range of scores on the SLQ-R. Examination of our obtained results suggests that supervisees with varying amounts of counseling, supervision, and graduate education demonstrate self-reported characteristics consistent with the constructs of the ID as measured by the SLQ- R. It appears that the Self and Other Awareness subscale and total SLQ-R scores were most sensitive to the differences reported by the trainee groups, followed by the otivation and Dependency-Autonomy subscales, respectively. Because the scores on the otivation subscale progressed in a hierarchical fashion, it appears that this subscale taps the type of motivation posited to occur in the Level 3 individual in the ID. The comparatively lower performance of the Dependency-Autonomy subscale may be due to the difficulty in measuring this construct because the ID notes that Level 2 trainees will vacillate in level of independent functioning. This finding suggests a need to refine the Dependency-Autonomy subscale. Analyses performed on both the total scores of the SLQ-R and the SLQ (cneill et al., 1985) indicated that the SLQ-R appears to better discriminate between the trainee groups than did the previous version of the instrument. Thus, although we viewed our results as providing construct validity for the SLQ-R, we also interpreted the present findings as preliminary support for Stoltenberg and Delworth's ID. These results are also consistent with previous studies specifically documenting trainees' self-reported differences in counseling and supervision behaviors across experience levels (e.g., Heppner & Roehlke, 1984; cneill et al., 1985; Reising & Dan- Table 2 Descriptive Statistics for Supervisee Levels Questionnaire-Revised Subscale and Total Scores Supervisee level Beginning Intermediate Advanced Score Self and Other Awareness Dependency- Autonomy otivation Total score to to to to to to to to to to to to Note. CI = confidence interval.

5 508 B. cneill, C. STOLTENBERG, AND J. ROANS iels, 1983) as well as the growing body of evidence supporting developmental conceptualizations of the training process in general (see Worthington, 1987). However, it is important to acknowledge that other investigators have outlined the methodological limitations of developmental supervision literature and provided alternative explanations to developmental processes per se in psychology trainees (see Holloway, 1987; Stoltenberg & Delworth, 1988). Similar to the original SLQ, the present version focuses solely on supervisee perceptions of their own development and needs but does not address preferences for specific supervisory environments. Although this is important to the ID, such a focus goes beyond the intent of the present instrument. Future revisions of existing instruments (e.g., the Supervisee Needs Questionnaire; Stoltenberg, Pierce, & cneill, 1987) will need to be done to address changes in assessing supervision environments as described by the ID. It is also important to note that our obtained differences reflect the perceptions of trainees in regard to their counseling and supervision behaviors and that experience level within the training process only approximates the developmental constructs hypothesized to occur at a particular state of training. Although we believe that the viewpoints of our trainees concerning the supervisory process should not be minimized, as Borders (1989) noted, future research is needed to compare the responses of trainees on the SLQ-R with perceptions of supervisors and actual assessment of trainee behaviors posited to occur according to the ID (e.g., ability to conceptualize clients). Future research is also necessary to compare our rationally-empirically derived subscale structure with empirically derived factor-analytic confirmatory methods and models that require a larger sample size. Thus, the present investigation was limited in this respect. It is our hope that dissemination of these findings will generate future investigations of this type. We are currently in the process of developing a larger subject pool in order to apply such methodologies. In this way, we hope to continue to refine the SLQ-R as an instrument useful to both researchers and supervisors working from the ID for both purposes of assessment and evaluation. References Borders, L. D. (1989). A pragmatic agenda for developmental supervision research. Counselor Education and Supervision, 79, Cohen, J. (1977). Statistical power analysis for the behavioral sciences (2nd ed.). New \brk: Academic Press. Haase, R. E, Waechter, D.., & Solomon, G. S. (1982). Average effect size of research in counseling psychology. Journal of Counseling Psychology, 29, Heppner, P. P., & Roehlke, H. J. (1984). Differences among supervisees at different levels of training: Implications for a developmental model of supervision. Journal of Counseling Psychology, 30, Hess, A. K. (1986). Growth in supervision: Stages of supervisee and supervision development. The Clinical Supervision, 4, Hogan, R. A. (1964). Issues and approaches in supervision. Psychotherapy: Theory, Research, and Practice, 1, Holloway, E.L. (1984). Outcome evaluation in supervision research. The Counseling Psychologist, 12, Holloway, E.L. (1987). Developmental models of supervision: It is development? Professional Psychology: Research and Practice, 19, Krause, A. A., & Allen, G. J. (1988). Perceptions of counselor supervision: An examination of Stoltenberg's model from the perspective of supervisor and supervisee. Journal of Counseling Psychology, 35, Loganbill, C, Hardy, E., & Delworth, Y(1982). Supervision: A conceptual model. The Counseling Psychologist, 10, cneill, B. W, Stoltenberg, C. D, & Pierce, P. A. (1985). Supervisees' perceptions of their development: A test of the counselor complexity model. Journal of Counseling Psychology, 32, Reising, G. N., & Daniels,. H. (1983). A study of Hogan's model of counselor development and supervision. Journal of Counseling Psychology, 30, Rosenthal, R., & Rosnow, R. L. (1985). Contrast analysis: Focused comparisons in the analysis of variance. Cambridge, England: Cambridge University Press. Rosnow, R. L., & Rosenthal, R. (1988). Focused tests of significance and effect size estimations in counseling psychology. Journal of Counseling Psychology, 35, Russell, R. K., Crimmings, R.., & Lent, R. W (1984). Counseling training and supervision: Theory and research. In S. D. Brown & R. W Lent (Eds.), Handbook of counseling psychology (pp ). New York: Wiley. Stoltenberg, C. D. (1981). Approachingsupervision from a developmental perspective: The counselor complexity model. Journal of Counseling Psychology, 28, Stoltenberg, C. D, & Delworth, U. (1987). Developmental supervision: A training model for counselors and psychotherapists. San Francisco: Jossey-Bass. Stoltenberg, C. D, & Delworth, V. (1988). Developmental models of supervision: It is development-response to Holloway. Professional Psychology: Research and Practice, 19, Stoltenberg, C. D, Pierce, R. A., & cneill, B. W (1987). Effects of experience on counselor trainees' needs. The Clinical Supervisor, 5, Wiley,. D, & Ray, P. B. (1986). Counseling supervision by developmental level. Journal of Counseling Psychology, 33, Worthington, E. L. (1987). Changes in supervision as counselors and supervisors gain experience: A review. Professional Psychology: Research and Practice, 19, Received September 10,1991 Revision received ay 6,1992 Accepted ay 6,1992