Securing treatment integrity is necessary in experimental

Transcription

1 Regular Articles Measurement of Transference Interpretations Kjell-Petter Bøgwald, M.D. Per Høglend, M.D., Ph.D. Øystein Sørbye, M.D. The authors present a cost-efficient process rating scale for detailed measurement of how much transference interpretations and related therapist interventions are used in brief dynamic psychotherapy. Theoretical and methodological considerations on how to operationalize and quantify such therapeutic interventions are discussed. The scale had highly satisfactory interrater reliability for three raters, who rated 60 whole sessions from an ongoing randomized study of two manualized forms of brief dynamic psychotherapy. In one treatment group, moderate emphasis on transference analysis was intended. In the other, minor or no use of the studied component was intended. The two treatment groups differed significantly in the use of transference interpretations and related interventions. There was no significant difference in therapists general therapeutic skill or use of supportive interventions. The treatment differentiation was consistent with the manuals. (The Journal of Psychotherapy Practice and Research 999; 8: ) Securing treatment integrity is necessary in experimental psychotherapy research. The basic concepts embodied in treatment integrity are competence or skill, adherence to the treatment modality, and differentiation of treatments that should be different. Treatment integrity is most often measured by using independent evaluators who rate audiotaped or videotaped sessions with one or more process rating scales. In recent years, a substantial number of different process rating scales have been developed, 8 but so far none have been accepted as standard instruments. Many of the published process rating scales contain items intended to measure different aspects of the analysis of the patient-therapist relationship. However, we did not find any of them suitable for detailed description of different aspects of such therapeutic work. Furthermore, several adherence scales have so many items that they are cumbersome to use, and the interrater reliability of the individual items is often unacceptably low.,3 Global concepts like transference interpretations or working with the patient/therapist relationship or focus on the here and now are too vague and general to be used as items for measuring different technical aspects of the analysis of the patient/therapist relationship. Interpretations of transference in a strict sense Received April 6, 999; accepted June 3, 999. From the Department of Psychiatry, University of Oslo, Norway. Address correspondence to Dr. Bøgwald, Department of Psychiatry, University of Oslo, P.O. Box 85 Vinderen, 039 Oslo, Norway; Copyright 999 American Psychiatric Association 264 J Psychother Pract Res, 8:4, Fall 999

2 Bøgwald et al. (e.g., genetic interpretations) are used sparsely in mainstream short-term dynamic psychotherapy. 9,0 Questions about how the patient might feel and fantasize about the therapist are often used to prime the patient s own understanding of transference phenomena or neurotic repetitional patterns. In our study, we needed to include a wider range of different interventions that might be used to explore possible transference reactions than is covered in most existing adherence scales. We therefore developed a new rating scale, Specific Therapeutic Technique (STT; Høglend, 995, unpublished manuscript), for measuring therapist interventions that address the patient/therapist interaction and transference phenomena. The new scale was based on elements from existing adherence scales, especially from the Vanderbilt Therapeutic Strategies Scale. CONSIDERATIONS IN CONSTRUCTING THE SCALE When therapists actively use the patient/therapist relationship to increase patient insight, they commonly employ diverse verbal interventions such as questions, confrontations, clarifications, and interpretations that intend to make the patient aware of possible transference phenomena. The use of different but mutually dependent interventions tends to make molecular frequency ratings of therapist interventions (e.g., separate talking turns) potentially misleading for assessment of the work with transference phenomena. Questions, clarifications, or confrontations used early in a session may become meaningful and pregnant parts of a transference interpretation made by the therapist near the end of the session. That the result (communicated transference interpretation) might be more than the sum of the parts (the separate therapist utterances) is an important argument for using long sampling units (a molar method, using, e.g., full sessions) rather than small session segments (a molecular method, using, e.g., speaking turns). Heaton et al. 2 found little correspondence between molar and molecular methods, whereas Connolly et al. 9 and Ogrodniczuk 3 found a relatively high correlation between therapist actions measured with molar and molecular methods. It could be argued that with global (molar) methods, raters may tend to overrate interventions if these interventions are particularly striking or nearer in time to the actual rating process and therefore more readily available in memory. This phenomenon is called availability heuristic. 4 On the other hand, we argue that a more aggregated rating of specific therapist techniques may be more clinically valid than a measurement based on molecular rated frequency. Ascertaining the extent to which high ratings on the transference subscale really correspond to what expert clinicians consider frequent use of transference interpretations was a priority for us. Identification and assessment of transference-related interventions require a certain degree of inference. In our opinion, such ratings are probably best done by highly qualified clinicians with extensive training in both short-term and long-term psychoanalytically oriented therapies. DESCRIPTION OF THE SCALE The STT is a 5-item process scale. The STT can be divided into three parts: a transference subscale (STTtransference) consisting of 5 items, an extratransference subscale with 5 items (STT-extratransference), and 5 additional items covering work on termination of therapy and focal adherence. In this article, we focus on the STT-transference subscale. We decided to include five different elements that we consider important when measuring how much the therapist has focused on the patient/therapist relationship and interpreted possible transference phenomena in a given psychotherapy session. It is important to emphasize that our purpose is to capture the therapist s active utilization of possible transference phenomena that goes beyond mere use of transference interpretations in a strict sense. For instance, the therapist addressing the therapeutic relationship by asking the patient how he or she feels about the therapy is no interpretation. But addressing the patient s feelings or thoughts about the therapeutic relationship is often a precursor to transference interpretations. A short description of the five different elements that constitute the transference subscale appears in Table. We see that item can be simply allusions to the therapeutic setting, and items 2 and 4 are often questions that may facilitate the patient s own understanding of transference phenomena. Items 3 and 5 involve interpretations in a more traditional sense. The items in the STT-extratransference subscale are very similar, but they describe interventions where the therapist focuses on interpersonal relations other than the patient/therapist relationship. RATING INSTRUCTIONS The frequency, either as the raw score or as a ratio of all ratings in the session, is certainly an important factor J Psychother Pract Res, 8:4, Fall

3 Measurement of Transference Interpretations when assessing how much transference interventions have been used in a given session. However, with a global rating method the exact number of different interventions is not available. Neither does the frequency of a certain intervention necessarily give a valid measure of how important this type of intervention was in a given session. How explicit or clear an intervention was should be given weight in the rating process. Transference interpretations might be highly explicit if, for instance, the therapist talks clearly into the relationship and uses words like me and you. On the other hand, the therapist may talk primarily about the patient s relations to others and merely allude to the patient/therapist relationship, or the therapist may talk in a less personal manner and use theoretical or abstract formulations. Such unclear allusions to the therapeutic relationship may not be heard by the patient, and although the skilled clinical rater recognizes the therapist s attempt to focus on transference phenomena, such unclear interventions should be given relatively less weight in the rating process. The emphasis or the underlining the therapist gives the intervention is also important. This includes the voice volume used when giving the interpretation and whether the statement sounds like ending with a question mark (tentatively), a period (in a neutral manner), or an exclamation mark (with conviction). The therapist might ask the patient questions to ensure that the interpretation has been understood or otherwise emphasize that he or she considers the transference-related statement to be important. An example of a low degree of emphasis would be presenting the transference interpretation in a low-voiced, hesitating, or noncommittal way that might make it easy for the patient to ignore the interpretation. The raters using STT were instructed to consider both the relative frequency (relative to all interventions) TABLE. Transference intervention items of the Specific Therapeutic Technique scale (STT). Focus on patient/therapist relationship (preparation for interpretation) 2. Questions about patient s feelings toward therapist/therapy 3. Interpretation involving patient/therapist relationship and one or several dynamic elements (impulse, anxiety, defense) 4. Questions about patient s thoughts about what therapist might feel toward patient 5. Linking of repetitional patterns (including genetic interpretations) Note: The complete text of the STT is available from the first author. of the transference-related interventions and how explicitly and with what degree of emphasis the interventions were delivered. On the basis of a synthesis of these considerations, the expert clinician rates each item on a Likert five-step scale ranging from 0 ( not at all ) to 4 ( very much ). Concepts like the correspondence between the transference interpretations and psychodynamic formulations, 5 focal adherence, 6 the degree of accuracy, 7,8 or plan compatibility 9 of transference interpretations might all be important factors in process ratings, but they should not be part of the transference subscale measuring to what extent such interventions have been used. We aim at the purest possible estimate of transference-related interventions given by the therapist unaffected by the timing, accuracy, degree of friendliness/hostility, or correctness of these interventions. The timing, quality, and meaningfulness of the interventions, as well as the patient s immediate responses to the transference focusing, should be measured independently of the transference subscale. If the patient s response were to be given weight in rating the STT-transference, then relationships between these interventions and outcome would be confounded by patient characteristics. METHODS Patients The patients described in this article are the first 30 completers in a randomized experimental study. The majority of the patients in this sample were professional, middle-class individuals with an average formal education of about 5 years. Their age ranged from 22 to 57 years (median 38). DSM-IV 20 Axis I diagnoses were mostly adjustment disorders, anxiety and affective disorders, and problems not due to mental disorder. Twelve patients (40%) had one or more Axis II disorders, primarily dependent, avoidant, and depressive personality disorders. The mean Global Assessment of Functioning (GAF) score at the initial dynamic evaluation was 60.8 (SD 7.0, range 46 79), and the mean Global Severity Index (GSI) score from the Symptom Checklist-90 revised (SCL-90-R) 2 was 0.92 (SD 0.45, range ). The patient sample is typical of a group of outpatients generally offered explorative dynamic psychotherapy. 266 J Psychother Pract Res, 8:4, Fall 999

4 Bøgwald et al. Therapies, Therapists, and Raters In an ongoing large-scale randomized study (Oslo Transference Study) with dismantling strategy, two forms of brief dynamic psychotherapy are compared: both groups are treated with dynamic psychotherapy, but in one group, therapist interventions addressing patient/therapist interaction (transference) are meant to be used minimally or avoided. In the other group, analysis of the therapist/patient relationship and transference interpretations are to be employed in the way that is usual in mainstream dynamic therapy. Simple randomization allocated 7 patients to the treatment group where transference-related interventions should be used actively (program group) and 3 patients to the group where such interventions should be avoided (comparison group). The therapies in both groups consist of weekly sessions for up to year or a maximum 40 sessions. We intended to have two treatment groups that both were given therapy by a skillful following of basic psychodynamic principles but that could be differentiated on one component, namely, the degree of transference focusing. It has been shown that high emphasis on transference analysis in brief dynamic psychotherapy may be causally related to less favorable outcome 22,23 or deterioration of therapeutic alliance. 24 Overzealous use of transference analysis should therefore be avoided, and the therapists were trained to use moderate amounts of transference interpretations even to the patients in the program group. The therapists in the study, six psychiatrists and one clinical psychologist, are all experienced therapists (mean clinical experience 20 years), and all are well educated in both short-term and long-term dynamic therapy, several with psychoanalytic education. They underwent 3 years of pilot training in order to be able to implement the techniques of both treatments with ease and competence. The three raters were psychiatrists. All were highly experienced with psychodynamic treatment (mean clinical experience 20 years). Measures Together with the new 5-item process rating (STT) scale presented above, a more general process rating scale, the Interpretive and Supportive Technique Scale (ISTS), 3 with 4 items, and an eight-item General Interpersonal Skill instrument (Høglend, 995, unpublished manuscript) were used. The last instrument is a competence rating scale that measures the therapist s general skill in a given session on a five-level scale from 0 ( not at all ) to 4 ( very much ). The items intend to measure quality factors that adherents of most modalities of psychotherapy would agree on, such as Therapist responds to the patient in an accepting and understanding manner and Therapist interventions seem meaningful and well timed. The ISTS is a relatively new instrument for measuring different supportive and expressive interventions in psychotherapy. This scale has shown good psychometric properties in two large-scale studies. 3 The ISTS and the STT have similar rating instructions and scale formats (e.g., full-session sample segments, Likert scales of 0 to 4). Both instruments have relatively few items. The ISTS consists of an interpretive subscale of seven items and a supportive subscale of seven items. We grouped the seven interpretive items into two new subscales: the 4 items not containing transference interpretations but containing other explorative interventions (named ISTS-explorative), and the three items with transference-related interpretations (ISTS-transference). Prior to randomization, the patients were evaluated with a comprehensive battery of self-report and clinician-rated instruments. In this article, pretreatment variables reporting the patient s own symptom description, the Symptom Checklist-90 revised, 2 and the independent evaluator-rated Global Assessment of Functioning Scale from DSM-IV 20 will be analyzed. Procedure The raters used the three process rating scales independently after listening to the audiotaped 45-minute therapy sessions. The raters were generally blind to the patient s treatment group (except as noted below), but they were given a written pretreatment psychodynamic formulation in order to assess how well the therapist s interventions adhered to this pretreatment formulation. All sessions in the 30 treatments were audiotaped, and from the pool of sessions, one from the early stage (session 7) and one from the middle phase (session 6) were selected for process ratings. On the basis of the interrater reliability estimates achieved by the three raters at session 7, we decided to use only two of the three raters when rating session 6. For practical reasons, and in order to test if knowledge of the actual treatment group J Psychother Pract Res, 8:4, Fall

5 Measurement of Transference Interpretations would bias the ratings, we decided to include all cases where the therapists rated their own patients. Thus, 8% of the total number of ratings were done by raters not blind to the treatment group. Data Analysis The interrater reliability estimates were calculated as intraclass correlations 25 (ICC[2,k]), where rater is considered a random factor and k is the number of raters. Differences in use of transference (STT-transference), extratransference (STT-extratransference), and supportive versus interpretive interventions and of general skill in the two treatment groups were tested with independent-sample t-tests. The classification of the individual cases was done by discriminant analysis (SPSS version ), using treatment group as grouping variable and STT-transference score as independent variable. To test the possibility of biased ratings by the raters who were also therapist to the patient rated, we compared the therapist-rater with the two other raters in two separate ways. First we compared the STT-transference scores to see if knowledge of actual treatment group yields more extreme ratings: for example, did raters who knew that the therapist should focus on transference give higher ratings on STT-transference than raters blind to treatment group? Second, we used the Moses test of extreme reactions 26 to test the hypothesis that therapist-raters had more extreme ratings when rating their own patients. As part of the investigation of convergent and divergent validity, we correlated STT-transference ratings with scores from other process scales, using Pearson s correlation coefficients. Furthermore, exploratory analysis to see if we could find any associations between pretreatment patient variables and the therapists interventions was done by using multivariate analysis of variance. RESULTS The interrater reliability estimates for STT-transference measured in sessions 7 and 6 are shown in Table 2. The estimates are highly satisfactory for both sessions across all the patients. The lower bound estimates (95% confidence intervals) indicate that the STT-transference subscale can be rated reliably with only one rater. The intraclass correlations are also reported for the separate treatment groups. As expected, and reported in other studies,,3 the reliability estimates are not as high when measured within modality. The main reason for this difference is the intended lower variance on this measure within each treatment group. This makes it more difficult to obtain good reliability by using ICC as the method for estimation of agreement. The within-group reliability estimates at session 6 are satisfactory, probably because of higher variance in the use of transference interpretations at this stage in the treatment. Even in session 7, the within-group reliabilities are good if based on mean scores of the three raters (model ICC[2,3]), with estimates of 0.90 and 0.82 for the two groups. Four of the five individual items of the STT-transference subscale were measured with acceptable to excellent reliability, even based on a single rater model: ICC(2,) values of 0.77 (item 3), 0.78 (item 2), 0.79 (item 5), and 0.90 (item ). Item 4 had ICC(2,) of 0.56 but improved to an acceptable 0.79 when based on the average score of three raters (model (ICC[2,3]). The lower reliability estimate on this item is probably due to the low base rate and limited variability of this intervention. The individual item ratings showed that, as expected, the transference-related interventions most used in the group assigned for transference focusing were those that contain simple statements about the patient/therapist relationship (mean score 3.0, considerable ) and questions about the patient s feelings toward therapy or the therapist (mean 2.8, considerable ). These interventions are often preparations to interpretations of dynamic elements involving the patient/therapist relationship (mean 2.3, moderately ) and linking of repetitional patterns (mean 2.4, moderately ). Direct questions about the patient s thoughts about what the therapist might feel toward the patient may be potent interventions to illuminate and make transference phenomena conscious, but they may be anxiety-provoking when used early in therapy. This intervention was used relatively seldom (mean.4, very little ) in sessions 7 and 6. In the treatment group where transference focusing should be avoided or minimally used, this abstention was accomplished in almost all the cases, with mean ratings on all of the five items in the range 0.03 to 0.40, corresponding to a use of these interventions between not at all and very little. An acceptable coefficient of internal consistency (Cronbach s alpha) was found for STT-transference: The alpha for the five-item STT-extratransference subscale 268 J Psychother Pract Res, 8:4, Fall 999

6 Bøgwald et al. was The alpha was only 0.50 if we combined all 0 items from both subscales, indicating that our subscales measure different constructs. When therapists rated their own patients from the high transference group, the mean STT-transference score was 2.6 (SD 0.8) compared with 2.7 (SD 0.8) for the two raters blind to treatment group. Patients in the low transference group were on average rated 0.08 (SD 0.2) on STT-transference by their therapists, compared with an average of 0.02 (SD 0.04) for the two other raters. These results indicate that knowledge of treatment group did not bias the ratings. Blind raters tended to discriminate just as well or better between the groups. Even if the average scores are very similar, one might expect more extreme ratings on individual patients when rated by their own therapist. However, the Moses test of extreme reactions was nonsignificant for all possible comparisons. To test if STT can be used to distinguish the two treatment groups, we compared the groups by using independent-sample t-tests. As shown in Table 3, the STT-transference subscore was significantly different in the two groups. The STT-extratransference score was significantly higher in the treatment group where focus on the patient/therapist relationship was avoided, indicating that analysis of other interpersonal relationships was given more space and emphasis. Transference-related interventions measured with ISTS were also different in the two groups. Other explorative interventions and supportive interventions were not significantly different in the two groups. Our measure of competence (general skill) was very similar in the two groups. We found satisfactory treatment integrity in the project regarding general competence, use of explorative and supportive techniques, adherence to specific techniques, and differentiation between treatment groups on the component that should be different. Discriminant analysis using treatment group as grouping variable and STT-transference score as independent variable gave the classification results shown in Table 4. Using the most robust classification (crossvalidation), where each case is classified by the functions derived from all cases other than that case, we find that only one case was misclassified; 97% of the cases were correctly classified. The use of transference focusing (STT-transference) for the individual cases is illustrated in Figure. In this illustration, one column represents one case. We can see the mean ratings of sessions 7 and 6 for STT-transference. All of the 7 patients randomized to the program group had actually been given such interventions. All patients except one were given no or a minimal amount of transference focusing in the comparison group. As part of the validity testing of the STT, correlations between STT-transference and other process ratings from the same sessions and raters were computed. All variables are based on average ratings across sessions and raters, using three raters at session 7 and two raters at session 6. The results show high positive correlation with transference ratings from a different process scale (ISTS-transference), but low and nonsignificant correlations with several other scales that measure different aspects of the therapist s activity (Table 5). These findings are consistent with good convergent and discriminant validity of the STT-transference subscale. To explore possible influence of patients pretreatment condition on therapists use of transference interpretations, supportive interventions, or their general therapeutic skill, an analysis of variance using multivariate general linear modeling (GLM) 26 was done (Table 6). We did not find that a measure of the patient s selfreported symptoms using GSI from SCL-90-R or pretreatment GAF, rated by independent clinical experts, had any significant influence on the therapist s compe- TABLE 2. Interrater reliability for STT-transference on 30 patients: estimates for use of single rater (model ICC[2,]) with 95% confidence interval Statistic All Patients (N 30) Session 7 Session 6 Group 0 Group All Patients Group 0 (n 3) (n 7) (N 30) (n 3) Group (n 7) ICC(2,) %CI ( ) ( ) ( ) ( ) ( ) ( ) Note: STT Specific Therapeutic Technique scale; ICC intraclass correlation coefficient; CI confidence interval; Group 0 treatment group where transference interpretations should be avoided; Group treatment group with active use of transference interpretations. J Psychother Pract Res, 8:4, Fall

7 Measurement of Transference Interpretations TABLE 3. Independent-sample t-tests Subscale Group 0 (n 3) Scores (mean SD) Group (n 7) t Effect Size (pooled SD) 95% CI (mean difference) STT-transference (5 items) * 4.00 ( ) ISTS-transference (3 items) * 3.40 ( ) STT-extratransference (5 items) *.93 ( ) ISTS-explorative (4 items) ( ) ISTS-supportive (7 items) ( ) Competence (8 items) ( ) Note: Group 0 treatment group where transference interpretations should be avoided; Group treatment group with active use of transference interpretations; CI confidence interval; STT Specific Therapeutic Technique scale; ISTS Interpretive and Supportive Technique Scale; Competence General Interpersonal Skill scale. *P TABLE 4. Discriminant analysis: classification results a Predicted Group Membership Group trans trans Total Count trans trans 6 7 Percentage trans trans Note: Cross-validated. trans signifies no or minimal focus on therapist/patient relationship; trans signifies active focus on therapist/patient relationship. a 96.7% of cross-validated grouped cases correctly classified. FIGURE. 4 Transference ratings on 30 individual cases. The black column indicates the case where transference interpretation was used to a degree that violated the randomization. tence, use of transference interpretations, or use of supportive interventions. The more severely disturbed patients did not get more supportive interventions than the better functioning patients. Furthermore, regardless of pretreatment distress and global functioning, all the patients were treated with high degrees of explorative dynamic techniques. The different therapists had given the same amounts of transference interpretations. Only one of the seven therapists differed significantly in skill level and use of support. Treatment group influenced variance in the use of transference interpretations, as intended, but it did not significantly influence the therapist s use of support or general skill in the studied sessions. No significant therapist-by-group interactions regarding use of transference, support, or skill were found, indicating that the relatively long pilot training of the therapists had made them able to implement the techniques of both groups with high competence and adherence. Mean STT Transference 3 2 TABLE 5. Correlation between STT-transference and other process ratings (N 30) Subscale Pearson Correlation Significance (two-tailed) STT-transference.000 ISTS-transference (3 items) 0.959* ISTS-support (7 items) ISTS-extratransference (4 items) Competence (8 items) No Transference Focus Transference Focus (n 3) (n 7) Treatment Group Note: STT Specific Therapeutic Technique scale; ISTS Interpretive and Supportive Technique Scale; Competence General Interpersonal Skill scale. *Significant at P 0.0 (two-tailed) 270 J Psychother Pract Res, 8:4, Fall 999

8 Bøgwald et al. TABLE 6. Influence of pretreatment distress (GSI/GAF), therapist, and treatment group on process ratings: test of between-subjects effects using multivariate general linear modeling (N 30) Source Dependent Variable df F P GSI GAF Therapist Group Therapist by Group DISCUSSION The reliability estimates for the transference subscale, and also for the individual items of the scale, were satisfactory. Earlier studies 22,23 have shown that similar transference interventions can be measured reliably (R ) when the sampling unit is short segments like separate statements. We have shown that it is possible to achieve the same high interrater reliability with global scores and the full session as the sampling unit. Such a procedure is more cost-efficient and, in our view, gives a more meaningful description of the clinical valence of these interventions. Even within each modality (the two different treatment groups), the reliabilities were acceptable and were better than those reported in some other studies.,3 We decided to include ratings (8% of the total number of ratings) where the therapist rated his or her own cases and therefore was aware of the actual treatment group. This did not bias the ratings: knowledge of treatment group did not give more extreme ratings in any direction. Using therapists as raters of their own patients is often avoided because of the presumed biases. Our findings indicate that therapists can do process ratings such as STT reliably. STT-transference ISTS-support Competence STT-transference ISTS-support Competence STT-transference ISTS-support Competence STT-transference ISTS-support Competence STT-transference ISTS-support Competence a 0.03 a 0.00 a Note: GSI Global Severity Index; GAF Global Assessment of Functioning; STT Specific Therapeutic Technique scale; ISTS Interpretive and Supportive Technique Scale; Competence General Interpersonal Skill scale. a Significant findings. The STT-transference subscale was able to differentiate the two experimental groups both on a group level (t-tests) and on a case level (discriminant analysis). We found a high correlation coefficient between our transference subscale and a three-item subscale describing transference-related interventions from the ISTS. 3 On the other hand, the correlation between STT-transference and the remaining four interpretive items or the seven-item support subscale from ISTS was low. These results indicate elements of convergent and discriminant validity for the STT. Further validation of the construct measured is warranted. As part of this effort, we are in the process of testing the association between the STT-transference scores from audiotaped sessions and the posttreatment assessment of the therapeutic ingredients both from the patient s and the therapist s perspective. One important aspect of the present study was to test if experimental manipulation of one component of the therapy, namely focus on the patient/therapist relationship, would influence other techniques used by the therapists or their general therapeutic skill. When the therapist did not use the patient/therapist relationship actively as a vehicle to increase the patient s insight, more time was available for focusing and interpretation of other (here labeled as extratransference ) interpersonal relationships. In the treatment group where transference focusing was avoided, we measured significantly higher scores on the five-item extratransference subscale of STT. When we used the Interpretive and Supportive Technique Scale (ISTS), 3 our estimated interrater reliability for the interpretive subscale was comparable with the findings of the developers of ISTS (ICC[2,] 0.82). However, our estimate on the supportive subscale was much lower (ICC[2,] 0.48). The low reliability coefficient for the supportive items seems to have one main explanation: the average use of supportive interventions was low and had little variance. When interrater reliability is measured with the ICC, low variance necessarily makes it difficult to achieve high coefficients. The two treatment groups did not differ regarding use of supportive interventions. Pretreatment patient distress and global functioning as measured with SCL-90-R and GAF, did not influence the use of specific techniques, degree of support, or general skill of the therapists. For most of the cases, our raters did not know the patients pretreatment symptom levels and could not compensate for pretreatment cli- J Psychother Pract Res, 8:4, Fall

9 Measurement of Transference Interpretations ent difficulty when doing competence ratings, as some authors 27 have advocated. The fact that pretreatment dysfunction did not influence use of supportive versus explorative interventions might be explained by a relatively homogeneous patient sample, well selected by experienced clinicians for dynamic psychotherapy and generally in little need of supportive interventions. Alternatively, the controlled experimental design might have been carried through by the therapists in an overcontrolled manner, with rigid adherence to the instructions regardless of the patients conditions and needs. The general high ratings on the competence scale do not support the latter hypothesis. Although the use of transference interventions was convincingly different in the two groups, the use of transference interpretations was moderate even in the cases randomized to focus on possible transference phenomena. That the patients pretreatment symptoms and dysfunctions did not predict the therapists competence or how much they would use specific techniques is consistent with findings reported by Barber et al. 28 in a study of depressed patients treated with Supportive-Expressive Dynamic Psychotherapy. The current study has some limitations that should be considered. The high interrater reliability estimates may have resulted partly from the raters being highly familiar with the scales. Compared with many other studies using different process rating scales, 29 the raters in this study were far more experienced as clinicians. As stated, the rating of STT requires a certain degree of inference, and it is questionable whether undergraduates, for instance, would achieve the same high interrater reliabilities. Interrater reliability should be further tested using other groups of raters and also different patient populations. Only 2 sessions, out of a mean number of 35 sessions, have been rated from each of the 30 treatments so far. Is the mean of these 2 sessions representative for all the sessions? Based on the relatively high correlation between STT-transference ratings on the two occasions (r 0.74, P 0.00), and intersession reliability as high as 0.85 (using ICC[2,k] model according to Shrout and Fleiss, 25 where k [number of different sessions] 2), we have found high stability in the use of transference interpretations across the two sessions. It can be argued that the substantial difference in use of transference-related interventions between the two groups (effect size 4.0) may be due in part to rater biases. For example, an early interpretation of the patient/therapist relationship might have influenced the raters by creating an anchoring heuristic 4 : after forming an early estimate and a guess of treatment group based on a single intervention, the raters might fail to adjust their estimates accurately on the basis of subsequent data. Our acceptable interrater agreement, even within treatment modality, is inconsistent with a strong influence of such biases. CONCLUSIONS We have developed an instrument, the Specific Therapeutic Technique scale, that is designed to measure specific use of transference focusing and interpretations. The STT appears to be a useful tool for measuring and monitoring these psychodynamic interventions. We achieved good interrater reliability, even at item level. When we employed the instrument in an ongoing project with two different treatment groups, we were able to distinguish the two groups, consistent with some aspect of discriminant validity of the scale. Using different amounts of transference focusing did not influence the measured use of supportive interventions, explorative interventions other than transference interpretations, or the general therapeutic skill in this study. The treatment integrity of the ongoing Oslo Transference Study seems promising regarding competence, adherence, and differentiation. This research was supported by the National Council for Science and Humanities, Norway; Norwegian Council for Mental Health; and Diakonhjemmet Hospital, Oslo, Norway. An earlier version of this work was presented at the 29th annual meeting, Society for Psychotherapy Research, Snowbird, UT, June 24 28, 998. REFERENCES. Hill CE, O Grady KE, Elkin I: Applying the Collaborative Study Psychotherapy Rating Scale to rate therapist adherence in cognitive-behavior therapy, interpersonal therapy, and clinical management. J Consult Clin Psychol 992; 60: Piper WE, Debbane EG, de Carufel FL, et al: A system for differentiating therapist interpretations from other interventions. Bull Menninger Clin 987; 5: Barber JP, Crits-Christoph P: Development of a therapist adherence/competence rating scale for supportive-expressive dynamic psychotherapy: a preliminary report. Psychotherapy Research 996; 6: Svartberg M: Manualization and competence monitoring of short-term anxiety-provoking psychotherapy. Psychotherapy 989; 26: J Psychother Pract Res, 8:4, Fall 999

10 Bøgwald et al. 5. Windholz MJ, Silberschatz G: Vanderbilt Psychotherapy Process Scale: a replication with adult outpatients. J Consult Clin Psychol 988; 56: Hoyt MF, Marmar CR, Horowitz MJ, et al: The Therapist Action Scale: instruments for the assessment of activities during dynamic psychotherapy. Psychotherapy Theory, Research and Practice 98; 8: Suh CS, O Malley SS, Strupp HH, et al: The Vanderbilt Psychotherapy Process Scale (VPPS). Journal of Cognitive Psychotherapy (special issue: Client-therapist Relationship and Cognitive Psychotherapy) 989; 3(2): Bond M, Banon E, Grenier M: Differential effects of interventions on the therapeutic alliance with patients with personality disorders. J Psychother Pract Res 998; 7: Connolly MB, Crits-Christoph P, Shappell S, et al: Therapist interventions in early sessions of brief supportive-expressive psychotherapy for depression. J Psychother Pract Res 998; 7: Piper WE, Debbane EG, Bienvenu JP, et al: Relationships between the object focus of therapist interpretations and outcome in short-term individual psychotherapy. Br J Med Psychol 986; 59(pt ):. Butler SF, Henry WP, Strupp HH: Measuring adherence in time-limited dynamic psychotherapy. Psychotherapy 995; 32: Heaton KJ, Hill CE, Edwards LA: Comparing molecular and molar methods of judging therapist techniques. Psychotherapy Research 995; 5: Ogrodniczuk JS, Piper WE: Measuring therapist technique in psychodynamic psychotherapies: development and use of a new scale. J Psychother Pract Res 999; 8: Tversky A, Kahneman D: Judgment under uncertainty: heuristics and biases. Science 974; 82: Piper WE, Joyce AS, McCallum M, et al: Concentration and correspondence of transference interpretations in short-term psychotherapy. J Consult Clin Psychol 993; 6: Høglend P, Piper WE: Focal adherence in brief dynamic psychotherapy: a comparison of findings from two independent studies. Psychotherapy 995; 32: Crits-Christoph P, Cooper A, Luborsky L: The accuracy of therapists interpretations and the outcome of dynamic psychotherapy. J Consult Clin Psychol 988; 56: Crits-Christoph P, Barber JP, Kurcias JS: The accuracy of therapists interpretations and the development of the therapeutic alliance. Psychotherapy Research 993; 3: Silberschatz G, Fretter PB, Curtis JT: How do interpretations influence the process of psychotherapy? J Consult Clin Psychol 986; 54: American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders, 4th edition. Washington, DC, American Psychiatric Association, Derogatis LR, Lipman RS, Covi L: SCL-90: an outpatient psychiatric rating scale preliminary report. Psychopharmacol Bull 973; 9: Piper WE, Azim HF, Joyce AS, et al: Transference interpretations, therapeutic alliance, and outcome in short-term individual psychotherapy. Arch Gen Psychiatry 99; 48: Høglend P: Transference interpretations and long-term change after dynamic psychotherapy of brief to moderate length. Am J Psychother 993; 47: Henry WP, Strupp HH, Butler TE, et al: Effects of training in time-limited dynamic psychotherapy: changes in therapist behavior. J Consult Clin Psychol 993; 6: Shrout PE, Fleiss JL: Intraclass correlations: uses in assessing rater reliability. Psychol Bull 979; 86: SPSS Inc: Statistical Package for the Social Sciences (SPSS). Version 8.0. Chicago, Waltz J, Addis ME, Koerner K, et al: Testing the integrity of a psychotherapy protocol: assessment of adherence and competence. J Consult Clin Psychol 993; 6: Barber JP, Crits-Christoph P, Luborsky L: Effects of therapist adherence and competence on patient outcome in brief dynamic therapy. J Consult Clin Psychol 996; 64: Moras K, Hill CE: Rater selection for psychotherapy process research: an evaluation of the state of the art. Psychotherapy Research 99; :3 23 J Psychother Pract Res, 8:4, Fall