Computers in Human Behavior

Transcription

1 Computers in Human Behavior 26 (2010) Contents lists available at ScienceDirect Computers in Human Behavior journal homepage: Internet administration of self-report measures commonly used in research on social anxiety disorder: A psychometric evaluation Erik Hedman a, *, Brjánn Ljótsson a, Christian Rück a, Tomas Furmark b, Per Carlbring c, Nils Lindefors a, Gerhard Andersson a,c,d a Department of Clinical Neuroscience, Psychiatry Section, Karolinska Institutet, Stockholm, Sweden b Department of Psychology, Uppsala University, Uppsala, Sweden c Department of Behavioural Sciences and Learning, Linköping University, Linköping, Sweden d Swedish Institute for Disability Research, Linköping University, Linköping, Sweden article info abstract Article history: Available online 11 February 2010 Keywords: Self-report questionnaire Internet Psychometrics Social anxiety disorder The Internet has become increasingly popular as a way to administer self-report questionnaires, especially in the field of Internet delivered psychological treatments. Collecting questionnaire data over the Internet has advantages, such as ease of administration, and automated scoring. However, psychometric properties cannot be assumed to be identical to the paper-and-pencil versions. The aim of this study was to test the equivalence of paper-and-pencil and Internet administered versions of self-report questionnaires used in social phobia research. We analyzed data from two trials in which samples were recruited in a similar manner. One sample (N = 64) completed the paper-and-pencil version of questionnaires and the second sample (N = 57) completed the same measures online. We included the Liebowitz Social Anxiety Scale-self-assessment (LSAS-SR), the Social Interaction and Anxiety Scale (SIAS), and the Social Phobia Scale (SPS) as measures of social anxiety. Also included were the Montgomery Åsberg Depression Rating Scale-self-assessment (MADRS-S), the Beck Anxiety Inventory (BAI), and the Quality of Life Inventory (QOLI). Results showed equivalent psychometric properties across administration formats. Cronbach s a ranged between 0.77 and There was an indication of a somewhat higher construct validity when participants filled out questionnaires using paper-and-pencil. We conclude that the LSAS-SR, SIAS, and SPS can be administered via the Internet with maintained psychometric properties. Ó 2010 Elsevier Ltd. All rights reserved. 1. Introduction With the advent of the Internet and the development of Internet-administered psychological treatments (Andersson, Bergström, Carlbring, & Lindefors, 2005; Spek et al., 2007), researchers and clinicians are increasingly using the Internet to administer self-report questionnaires. While the number of controlled trials showing promising effects of guided Internet-delivered Cognitive Behavior Therapy (CBT) is increasing rapidly (Andersson et al., 2008), fewer studies in the field of clinical psychology have focused on the use of the Internet for information gathering purposes. Internet administration of questionnaires can have some advantages. For example, they can be completed in the patients own home with instant data storing, which may reduce risks that are associated with paper questionnaires, e.g., lost in the mail. In addition, the internal drop-out rate in terms of uncompleted items can be reduced by * Corresponding author. Address: Department of Clinical Neuroscience M 57, Karolinska University Hospital Huddinge, SE Stockholm, Sweden. Tel.: addresses: erik.hedman.2@ki.se, erik.hedman@sll.se (E. Hedman). making it impossible to skip items. Furthermore, scripts can be used to calculate scores, which can be easily exported to statistics software. However, when self-report questionnaires are transferred to the Internet, psychometric properties cannot be assumed to be identical to their paper version equivalents (Andersson, Ritterband, & Carlbring, 2008; Buchanan, 2003). Several studies have investigated the effect of administration mode of questionnaires in various contexts. Internet-administered questionnaires are strongly correlated with scores obtained using paper-and-pencil versions and internal consistencies tend to be equal (Andersson, Kaldo-Sandström, Ström, & Strömgren, 2003; Austin, Carlbring, Richards, & Andersson, 2006; Carlbring, Brunt, et al., 2007; Richter et al., 2008). Still, it has been suggested that each Internet-administered questionnaire should be evaluated separately (Buchanan, 2003), in particular when establishing norms. One large clinical population for which Internet administration of questionnaires could seem attractive are people suffering from social phobia, or social anxiety disorder (Erwin, Turk, Heimberg, Fresco, & Hantula, 2004; Shepherd & Edelmann, 2005). However, psychometric research on this topic is scarce and separate norms might be needed. One reason for doing so is to investigate whether respondents /$ - see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi: /j.chb

2 E. Hedman et al. / Computers in Human Behavior 26 (2010) consistently give higher severity ratings when giving their answers via the Internet, which was one of the first reported systematic differences between Internet-administered questionnaires and the original versions (Joinson, 1998). To our knowledge, very little research has been conducted to test the psychometric properties of Internet-administered questionnaires commonly used in the assessment of social anxiety disorder. Given the increasing use of Internet-administered social anxiety disorder questionnaires, due to the development of Internet-delivered CBT and the high prevalence of the disorder, we find it very important to evaluate their psychometric properties. The aim of this study was to test the equivalence of paper-andpencil and Internet administered versions of self-report questionnaires that measure social anxiety disorder, general anxiety, depression, and quality of life. More specifically, we compared mean scores, internal consistencies and construct validity across administration formats. To estimate construct validity we used the criteria for intercorrelations proposed by Campbell and Fiske (1959). 2. Method 2.1. Participants Participants were 121 individuals who had registered and fulfilled the inclusion criteria for two studies evaluating the effect of Internet-delivered treatment for social anxiety disorder (Andersson et al., 2006; Carlbring, Gunnarsdottir, et al., 2007). The participants were allocated to two separate, but nearly identical studies. In the first study, in which paper-and-pencil were used to fill out self-report questionnaires, the average age was years (SD = 10.22) and the proportion of women was 51.56%. In study 2, in which the participants completed all self-report questionnaires via the Internet, the average age was years (SD = 9.21) and the proportion of women was 64.91%. For marital status and educational level of the participants, see Table Recruitment Participants were recruited by means of newspaper articles in national and regional papers, notices in health magazines, and by an Internet link from the home page of the Swedish National Anxiety Association. A web-page was created for the study, which included general information about Cognitive Behavior Therapy (CBT) and an application form for participation in the study. Participants were selected by means of a computerised screening interview consisting of the Social Phobia Screening Questionnaire (Furmark et al., 1999), the self-rated version of the Montgomery Åsberg Depression Rating Scale, MADRS, (Svanborg & Åsberg, 1994), and 10 additional questions regarding current and past treatments. Following the screening eligible participants were interviewed either in a live interview or by telephone. All included participants fulfilled the DSM IV (American Psychiatric Association, 2000) criteria for social phobia according to the Structured Clinical Interview for DSM IV (First, Gibbon, Spitzer, & Williams, 1998). For more specific details regarding the inclusion and exclusion criteria see the primary trials (Andersson et al., 2006; Carlbring, Gunnarsdottir, et al., 2007). A total of 121 participants were included, with 64 in the first trial and 57 in the second trial. Descriptive statistics of the samples presented in Table Materials In total six questionnaires were completed and in the present report. First, we included the Liebowitz Social Anxiety Scale-self report version (LSAS-SR; Baker, Heinrichs, Kim, & Hofmann, 2002; Liebowitz, 1987). The LSAS measures fear in and avoidance of 24 social situations (13 performance and 11 interaction situations), that are assumed to be difficult for people suffering from social anxiety disorder. Fear and avoidance in each situation is rated on a four-point scale ranging from 0 (no fear/never avoid) to 3 (severe fear/usually avoid). The LSAS was originally developed as a clinician administered scale for outpatients with social anxiety disorder, but its straightforward format makes it suitable for use as a self report measure. The two versions have similar reliabilityand validity properties (Fresco et al., 2001). Moreover, the two versions are strongly correlated (r = 0.85) (Fresco et al., 2001). LSAS-SR has a high internal consistency (Cronbach s a = 0.95) (Baker et al., 2002), as well as a high test retest reliability over 12 weeks (r = 0.83; Fresco et al., 2001). The convergent and discriminant validity of LSAS-SR has been shown to be strong, and the scale is sensitive to change and is therefore often used in treatment research (Baker et al., 2002). The second and third social anxiety questionnaires used were the Social Phobia Scale (SPS) and Social Interaction Anxiety Scale (SIAS) (Mattick & Clarke, 1998). The SPS measures anxiety in 20 performance situations, while the SIAS is constructed to measure affective, behavioral, and cognitive reactions in 20 social interaction situations. Each situation is rated on a five-point scale, ranging from 0 to 5. The scales were developed to assess social anxiety in research studies as well as in clinical settings. In the validation of SIAS and SPS three clinical groups that consisted of individuals with social anxiety disorder panic disorder and simple phobia were employed. These groups were also compared to a sample of healthy controls. Both scales have high test retest reliability (r = for SPS and Table 1 Demographic description of the participants. Variable Paper-and-pencil Internet n % n % Gender a Women Men Age b Mean age (SD) Min max Marital status c Married/living together Widow Divorced Never married a v 2 = 2.2, df =1,p <.10. b t (119) = 2.6, p =.015. c v 2 = 0.3, df =3,p <.95.

3 738 E. Hedman et al. / Computers in Human Behavior 26 (2010) r = for SIAS) (Mattick & Clarke, 1998). The internal consistency has also been found to be high (Cronbach s a = 0.89 for SPS and 0.83 for SIAS). Lastly, SPS and SIAS have been shown to have good discriminant validity (Mattick & Clarke, 1998). In addition, three secondary measures were administered as symptoms of depression, general anxiety often are included in social anxiety research. We used the Montgomery Åsberg Depression Rating Scale-self-assessment, MADRS-S (Svanborg & Åsberg, 1994) to measure symptoms of depression. MADRS-S consists of nine items measuring nine different symptoms and each symptom is rated on a seven-point scale with four predefined anchor labels and three non-defined anchor labels in between. MADRS-S was validated in a clinical setting and has shown to be able to discriminate between DSM-IV Axis I disorders in a group of psychiatric inand outpatients. The test retest reliability of MADRS-S is high with rs ranging from 0.80 to In a comparative study, Svanborg and Åsberg (2001) showed that MADRS correlated highly (r = 0.87) with the Beck Depression Inventory, BDI, indicating acceptable convergent validity (Beck, Ward, Mendelson, Mock, & Erbaugh, 1961). As a measure of general anxiety symptoms we used the Beck Anxiety Inventory, BAI, (Beck, Epstein, Brown, & Steer, 1988). The BAI consists of 21 anxiety symptoms, and the respondent rates on a four-point scale (0 3) to which extent he or she has experienced them during the last week. The scale was validated in a group of more than 1000 psychiatric outpatients with affective and anxiety disorders. The BAI demonstrates good discriminant and convergent validity, high internal consistency (Cronbach s a = 0.92) and good test retest reliability over 1 week (r = 0.75). Finally, we administered the Quality of Life Inventory (QOLI) (Frisch, Cornell, Villanueva, & Retzlaff, 1992). The QOLI assesses subjective quality of life by tapping the degree of satisfaction in 16 different life domains (e.g., work). For each domain, the respondent is asked to rate importance on a three-point scale (0 2) and the degree of satisfaction on a six-point scale ( 3 to +3). By multiplying importance by satisfaction, each domain yields a value from 6 to +6. The total score is the average score of domains rated as important or very important. The scale was validated on three different samples: psychiatric inpatients, outpatients from a university counselling center and a nonclinical sample consisting of undergraduate students and criminal offenders. The QOLI demonstrates high internal consistency (Cronbach s a = ; Carlbring, Brunt, et al., 2007), good one month test retest reliability (r = 0.92) and good discriminant, convergent, and nomological validity (Frisch et al., 1992) Procedure The data were collected during the pre-treatment phase of both trials. In the first study the paper-and-pencil version was used (N = 64) and in the second all instruments were administered via the Internet (N = 57). Participants in the paper-and-pencil group (PP) were sent a package of the measures in the order LSAS-SR, SIAS, SPS, BAI, MADRS-S, QOLI and asked to fill them out and return them in a postage-paid envelope as soon as possible. Participants in the Internet (IT) group were instructed to go to a specific webpage, requiring password-access, and fill out the questionnaires. Unique codes were used to identify respondents. The questionnaires were administered in the same order as in the PP-group and it was possible to go back and review all answers before submitting them. No experimenters were present during either of the administration formats. Both studies were approved by the local ethics committee. All measures were in the Swedish language and have been translated and validated previously (Furmark et al., 1999). To estimate construct validity of the questionnaires measuring social anxiety disorder, we used the following criteria derived from Campbell and Fiske (1959): (a) The intercorrelations of LSAS, SIAS, and SPS should be large and significantly differ from zero (convergent validity). (b) The correlations between these measures should be larger than their respective correlations to the other questionnaires (MADRS-S, BAI, and QOLI) because of the latters intention to measure other constructs (discriminant validity). (c) The reliability coefficient, in this report Cronbach s a, should be larger than all other correlations Statistical analyses Statistical analyses were conducted using SPSS version 15.0, Chicago: SPSS, Inc. Cronbach s a was used to calculate internal consistency. Although measurement data were ordinal we employed parametric tests since scale levels exceeded 20 and other parametric requirements were met. Under these circumstances it has been suggested that use of parametric tests is legitimate (Clark-Carter, 1997). Multivariate analysis of variance (MANOVA) and Welch s ANOVA were used to analyze differences between the two administration formats. We used Levene s test to investigate homogeneity of variance and Box s M to test homogeneity of covariance matrices. Differences in demographic nominal variables were tested using v 2 -tests. We used Pearson s product moment correlation to investigate intercorrelations between the six questionnaires. 3. Results 3.1. Internal consistency Cronbach s a ranged between 0.77 and The a-values for each questionnaire and administration format are presented in Table 2. The differences in internal consistency across administration format were small, the largest difference being found for SIAS (a = 0.93 vs. 0.86). Within administration formats and across questionnaires the a-values ranged from 0.77 to 0.94 and 0.81 to 0.94 for paper-and-pencil and Internet, respectively Effects of administration format Mean values and standard deviations for the selected questionnaires across administration formats are presented in Table 3. Overall, differences were small and of no clinical importance. However, the MANOVA identified a close to significant trend F (6,113) = 2.12, p =.06, using Hotelling s Trace multivariate test. Hence, we continued and tested for univariate effects (see Table 3). The subsequent univariate between-subjects tests for each dependent variable revealed that only the QOLI showed a significant difference between the Internet group and paper-and-pencil group. A Table 2 Internal consistencies (Cronbach s a) for the two administration formats for each questionnaire. Measure Paper-and-pencil Internet LSAS 0.94 (n = 64) 0.94 (n = 57) SIAS 0.93 (n = 60) 0.86 (n = 57) SPS 0.93 (n = 59) 0.89 (n = 57) MADRS-S 0.82 (n = 64) 0.86 (n = 57) BAI 0.84 (n = 64) 0.89 (n = 57) QOLI 0.77 (n = 64) 0.81 (n = 57) LSAS, Liebowitz Social Anxiety Scale; SIAS, Social Interaction Scale; SPS, Social Phobia Scale; MADRS-S, Montgomery Åsberg Depression Rating Scale-Self report; BAI, Beck Anxiety Inventory; QOLI, Quality of Life Inventory.

4 E. Hedman et al. / Computers in Human Behavior 26 (2010) Table 3 Means (SD), between group differences (paper-and-pencil vs. Internet) for each self-report questionnaire. Source Measure Administration format Mean (SD) F (1,118) p-value Administration format LSAS Paper-and-pencil (n = 64) (21.53) Internet (n = 57) (22.48) Administration format SIAS Paper-and-pencil (n = 64) (14.42) Internet (n = 57) (12.57) Administration format SPS Paper-and-pencil (n = 63) (14.98) Internet (n = 57) (15.69) Administration format MADRS-S Paper-and-pencil (n = 64) (6.59) Internet (n = 57) (7.26) Administration format BAI Paper-and-pencil (n = 64) (6.90) Internet (n = 57) (8.41) Administration format QOLI Paper-and-pencil (n = 64) 1.29 (1.29) Internet (n = 57) 0.58 (1.84) Levene s test of error variance indicated that the assumption of equality of variance was not met regarding QOLI, F (1,118) = 4.20, p =.043. However, the difference between the IT-group and PPgroup remained significant when analyzed with the more robust Welch s test (F 0 = 5.82, p =.018). Overall, assumptions of MANOVA were met to an acceptable standard. Scatter plots showed that all dependent variables were normally distributed, test of homogeneity of covariance matrices were non-significant (Box s M =.08) and homogeneity of variance was indicated by non-significant Levene s tests for LSAS, SIAS, SPS, BAI, and MADRS-S, F (1,118) = , p = The between group effect sizes were small (g p 2 = ) for all questionnaires except the QOLI, which showed a moderate effect size (g p 2 =.048). Hence, in spite of being from different studies and completing the questionnaires in different formats the differences between the two samples recruited in a similar manner were small and of no major clinical importance Intercorrelations and construct validity Intercorrelations between measures are presented in Table 4. In the IT-group, all measures were significantly correlated in the expected direction. In the PP-group, 11 of the 15 possible intercorrelations were statistically significant. The non-significant correlations were between MADRS-S and SPS, BAI and SIAS, QOLI and SPS, QOLI and BAI, respectively. As seen in Table 4 the differences between correlations were not just caused by the somewhat smaller sample size in the second sample. 4. Discussion The aim of this study was to investigate the validity of Internetadministered social anxiety questionnaires as compared to their Table 4 Intercorrelations of questionnaires measuring social anxiety, depression, general anxiety and quality of life. Upper panel shows Internet sample, lower panel paperand-pencil sample. Measure LSAS.67 **.77 **.60 **.58 **.46 ** 2. SIAS.67 **.47 **.40 **.36 **.35 ** 3. SPS.73 **.60 **.54 **.69 **.33 * 4. MADRS-S.25 *.32 ** **.57 ** 5. BAI.27 * **.34 **.29 * 6. QOLI.36 **.43 ** **.00 LSAS, Liebowitz Social Anxiety Scale; SIAS, Social Interaction Scale; SPS, Social Phobia Scale; MADRS-S, Montgomery Åsberg Depression Rating Scale-self-assessment; BAI, Beck Anxiety Inventory; QOLI, Quality of Life Inventory. * p <.05. ** p <.01. standard paper-and-pencil format. Results suggest that the psychometric properties are adequate for the three Internet-administered social anxiety questionnaires (LSAS-SR, SIAS, SPS) in our study. We also found good psychometric properties for the instruments assessing general anxiety (BAI) and depression (MADRS-S). Multivariate analysis showed a trend towards difference between the Internet group and the paper-and-pencil group, which was mostly explained by differences in responses to the QOLI. However, since the study involved data from two separate studies, albeit recruited and assessed in a similar manner for identical purposes, it can be expected that the two groups would differ on at least one variable. The overall findings showed that the two groups were very similar, and if a systematic bias in favor of one administration format had been present it is likely that the differences would have been larger. The internal consistencies proved to be high for the questionnaires in their Internet administration format, with Cronbach s a ranging from 0.81 to This is well above the level of 0.70, which can be regarded as an acceptable a-value (Clark-Carter, 1997). The study lends no support to the idea that respondents consistently give higher severity ratings when giving their answers via the Internet, which was suggested early in the history of psychological Internet research when disinhibition on the Internet was observed (Joinson, 1998). When it comes to construct validity, both administration formats showed high convergent validity and fully met the criterion of correlations being strong among social anxiety disorder questionnaires. Regarding discriminant validity the social anxiety disorder measures were uncorrelated or correlated to a lesser degree with QOLI than with each other, indicating discriminant validity. This finding was similar in the two groups. For the two questionnaires measuring depression and general anxiety, there appeared to be a difference between administration formats in terms of correlations with the social anxiety disorder measures, favoring the paper-and-pencil group. That is, when using paperand-pencil, the ability to discriminate between different emotional states could be somewhat better. Of course, this could reflect true differences, but another possible explanation is that the paperand-pencil format in a better way signals a new set of questions with different meaning. In contrast, when responding to more than 300 individual items on the Internet with no clear separation between the different sets of questions, it is probably easier to get stuck in the same response style. Our suggestion is that it may be important to reproduce the original design of paper-and-pencil questionnaires in as great detail as possible, for example by having similar typeface and colors. Although the present results strongly indicate that the Internet administration format can be reliable and valid, the present study holds some limitations. First, the design makes it impossible to completely isolate the effect of administration format. The fact that

5 740 E. Hedman et al. / Computers in Human Behavior 26 (2010) respondents were recruited from two separate studies means that the effects of administration format can be confounded with differences in symptom severity, i.e., true group differences could mask potential effects of administration mode. In other words, it could be that Internet administration actually results in higher severity ratings, but that this effect cannot be detected due to a true milder degree of symptom severity in the Internet group. However, true differences between the groups are unlikely for several reasons. First, all participants were recruited in a similar manner. Second, the recruitment went on during the same time period and the inclusion criteria were similar with the exception that participants in the first study were required to attend a live structured interview. Third, the treatment response in the two studies was largely similar. There were also small differences between the groups in terms of demographic characteristics. Nevertheless, the ideal design to control for these issues would be to let each participant fill out both types of questionnaires, and randomize order of administration (Carlbring, Brunt, et al., 2007). Unfortunately, this was not done in this study. A second limitation is the relatively small sample size. This study included 121 participants, which could be compared to the 494 participants included in the study conducted by Carlbring, Brunt, et al. (2007), where the objectives were similar to the ones in this study. However, power calculations in this study yield that there was a probability slightly lower than 78% of detecting a moderate effect size of d > 0.5 given an alpha level of.05, which we view as acceptable (Clark-Carter, 1997). Finally, one issue of concern is the generalizability of the results to real life clinical settings. The participants were all self-referred through a web application, and no referrals were solicited from primary care or psychiatric clinics. Although all participants were diagnosed using the Structured clinical interview for DSM-IV Axis I Disorders (SCID-I), it could be that social anxiety disorder patients in psychiatric settings are less comfortable using the Internet. On the other hand, this potential problem is highly likely to diminish with time since the use of Internet is already widespread in Sweden, and keeps expanding to larger parts of the population (Internetworldstats, 2008). It would be of value to replicate this study using a balanced, randomized design with administration format as the independent variable (Carlbring, Brunt, et al., 2007). Nonetheless, taken together with previous studies in the area of panic disorder we conclude that the Internet administration format does not compromise psychometric properties of measures used in research on social anxiety disorder. Acknowledgements This study was sponsored in part by grants from the Swedish Council for Working and Life Research, the Swedish Research Council, and the Söderström Königska Foundation to the last author (PI). We thank all coworkers in the Sofie-project for their assistance in the data collection. References American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ex ed.). Washington, D. C: American Psychiatric Press. Andersson, G., Bergström, J., Buhrman, M., Carlbring, P., Holländare, F., Kaldo, V., et al. (2008). Development of a new approach to guided self-help via the Internet. The Swedish experience. Journal of Technology in Human Services, 26, Andersson, G., Bergström, J., Carlbring, P., & Lindefors, N. (2005). The use of the Internet in the treatment of anxiety disorders. Current Opinion in Psychiatry, 18, Andersson, G., Carlbring, P., Holmstrom, A., Sparthan, E., Furmark, T., Nilsson-Ihrfelt, E., et al. (2006). Internet-based self-help with therapist feedback and in vivo group exposure for social phobia: A randomized controlled trial. Journal of Consulting and Clinical Psychology, 74, Andersson, G., Kaldo-Sandström, V., Ström, L., & Strömgren, T. (2003). Internet administration of the Hospital Anxiety and Depression Scale in a sample of tinnitus patients. Journal of Psychosomatic Research, 55, Andersson, G., Ritterband, L. M., & Carlbring, P. (2008). A primer for the assessment, diagnosis and delivery of Internet interventions for (mainly) panic disorder. Lessons learned from our research groups. Clinical Psychologist, 12, 1 8. Austin, D. W., Carlbring, P., Richards, J. C., & Andersson, G. (2006). Internet administration of three commonly used questionnaires in panic research: Equivalence to paper administration in Australian and Swedish samples of people with panic disorder. International Journal of Testing, 6, Baker, S. L., Heinrichs, N., Kim, H. J., & Hofmann, S. G. (2002). The Liebowitz social anxiety scale as a self-report instrument: A preliminary psychometric analysis. Behaviour Research and Therapy, 40, Beck, A. T., Epstein, N., Brown, G., & Steer, R. A. (1988). An inventory for measuring clinical anxiety: Psychometric properties. Journal of Consulting and Clinical Psychology, 56, Beck, A. T., Ward, C. H., Mendelson, M., Mock, J., & Erbaugh, J. (1961). An inventory for measuring depression. Archives of General Psychiatry, 4, Buchanan, T. (2003). Internet-based questionnaire assessment: Appropriate use in clinical contexts. Cognitive Behaviour Therapy, 32, Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait multimethod matrix. Psychological Bulletin, 56, Carlbring, P., Brunt, S., Bohman, S., Austin, D., Richards, J., Öst, L.-G., et al. (2007). Internet vs. paper and pencil administration of questionnaires commonly used in panic/agoraphobia research. Computers in Human Behavior, 23(3), Carlbring, P., Gunnarsdottir, M., Hedensjö, L., Andersson, G., Ekselius, L., & Furmark, T. (2007). Treatment of social phobia: Randomised trial of internet-delivered cognitive-behavioural therapy with telephone support. British Journal of Psychiatry, 190, Clark-Carter, D. (1997). Doing quantitative psychological research: From design to report. East Sussex, UK: Psychology Press Ltd. Erwin, B. A., Turk, C. L., Heimberg, R. G., Fresco, D. M., & Hantula, D. A. (2004). The Internet: Home to a severe population of individuals with social anxiety disorder? Journal of Anxiety Disorders, 18, First, M. B., Gibbon, M., Spitzer, R. L., & Williams, J. B. W. (1998). SCID-I: Structured Clinical Interview for DSM-IV Axis I Disorders (J. Herlofson, övers.). Danderyd: Pilgrim Press. (Original publicerat 1997). Fresco, D. M., Coles, M. E., Heimberg, R. G., Liebowitz, M. R., Hami, S., Stein, M. B., et al. (2001). The Liebowitz Social Anxiety Scale: A comparison of the psychometric properties of self-report and clinician-administered formats. Psychological Medicine, 31, Frisch, M. B., Cornell, J., Villanueva, M., & Retzlaff, P. J. (1992). Clinical validation of the Quality of Life Inventory. A measure of life satisfaction for use in treatment planning and outcome assessment. Psychological Assessment, 4(1), Furmark, T., Tillfors, M., Everz, P., Marteinsdottir, I., Gefvert, O., & Fredrikson, M. (1999). Social phobia in the general population: Prevalence and sociodemographic profile. Social Psychiatry and Psychiatric Epidemiology, 34, Internetworldstats. (2008). Internet usage in Sweden. Available from < Accessed Joinson, A. (1998). Causes and implications of disinhibited behavior on the Internet. In J. Gackenbach (Ed.), Psychology and the Internet. Intrapersonal, interpersonal and transpersonal implications (pp ). San Diego: Academic Press. Liebowitz, M. R. (1987). Social phobia. Modern Problems of Pharmacopsychiatry, 22, Mattick, R. P., & Clarke, J. C. (1998). Development and validation of measures of social phobia scrutiny fear and social interaction anxiety. Behaviour Research and Therapy, 36, Richter, J. G., Becker, A., Koch, T., Nixdorf, M., Willers, R., Monser, R., et al. (2008). Self-assessments of patients via Tablet PC in routine patient care: Comparison with standardised paper questionnaires. Annals of the Rheumatic Diseases, 67, Shepherd, R.-M., & Edelmann, R. J. (2005). Reasons for internet use and social anxiety. Personality and Individual Differences, 39, Spek, V., Nyklicek, I., Smits, N., Cuijpers, P., Riper, H., Keyzer, J., et al. (2007). Internet-based cognitive behavioural therapy for subthreshold depression in people over 50 years old: A randomized controlled clinical trial. Psychological Medicine, 37, Svanborg, P., & Åsberg, M. (1994). A new self-rating scale for depression and anxiety states based on the Comprehensive Psychopathological Rating Scale. Acta Psychiatrica Scandinavica, 89, Svanborg, P., & Åsberg, M. (2001). A comparison between the Beck Depression Inventory (BDI) and the self-rating version of the Montgomery Åsberg Depression Rating Scale (MADRS). Journal of Affective Disorders, 64,