Development and psychometric properties of the Problematic Internet Use Questionnaire

Transcription

1 Development and psychometric properties of the Problematic Internet Use Questionnaire Andrew Thatcher Psychology Department, University of the Witwatersrand, Johannesburg, Private Bag X3, WITS, 2050, South Africa Shamira Goolam Psychology Department, University of the Witwatersrand, Johannesburg, South Africa Since the inception of the Internet, research has suggested that a small group of people have a tendency to abuse the Internet and thereby cause damage to their health, their personal life and/or their professional life. Numerous terms, including Internet Addiction Disorder and pathological Internet use, have been used to describe this behaviour and its consequences. More recently, the term, problematic Internet use, has been favoured to describe people who cannot resist engaging with the Internet. This article describes the development and psychometric properties of the Problematic Internet Use Questionnaire (PIUQ). The development of the PIUQ was a two-stage process involving a pilot study on a smaller sample (N = 279) and a validation study on a much larger sample (N = 1795). The results of the validation study provided good evidence for the reliability and construct validity of the PIUQ. The exploratory factor analysis suggests that the PIUQ may consist of three sub-scales (i.e., Online preoccupation, Adverse effects and Social interactions), each with good internal reliability. The discussion provides recommendations for future studies using and validating the PIUQ. Keywords: concurrent validity; factor validity; internal reliability; Internet Addiction Disorder; Problematic Internet Use Questionnaire; psychometric properties The Internet has greatly improved the degree of communication and access to information for many millions of people. The advantages of the Internet are clearly evident in the marked increase in Internet use in recent years. Nua Internet Surveys (2002) estimates that there are now in excess of 600 million people globally with regular Internet access. Unfortunately, along with the benefits there is also increasing evidence of the abuse of the Internet (Griffith, 2003). The primary concern of this article is the Psychological Society of South Africa. All rights reserved. South African Journal of Psychology, 35(4) 2005, pp ISSN

2 Andrew Thatcher and Shamira Goolam psychological problems associated with excessive Internet use. Studies have shown that some people neglect their work, family, social and academic responsibilities, suffer social isolation, face financial difficulties, and even report physical problems as a result of their overuse of the Internet (Griffiths, 2000; Young, 1996b; 1999). Given problems experienced due to the excessive use of the Internet, it has been suggested that some people might become Internet addicts (Brenner, 1997). The issue of whether people do indeed become addicted to the Internet is still a matter of much debate. The DSM-IV classifications recognise dependence on a substance or an activity that is characterised by intemperance, increased tolerance levels, withdrawal symptoms, and loss of control (American Psychiatric Association, 2000). However, behavioural impulse-control disorders such as gambling (Lesieur & Blume, 1987), eating disorders (Lacey, 1993), sexual addictions (Goodman, 1993) and technological addictions (Griffiths, 1995) are also widely recognised. It has been argued that the excessive use of the Internet should be placed into the category of behavioural impulse-control disorders. There have been numerous phrases used to describe problems associated with excessive Internet use. Young initially preferred the term Internet Addiction Disorder (Young, 1996b) and later pathological Internet use (Young, 1997). More recently, the term problematic Internet use (Shapira, Goldsmith, Keck, Khosla & McElroy, 2000) has gained in popularity, although many researchers still refer to Internet addicts. There have been numerous studies that have attempted to assess addictive Internet behaviours. In the earlier studies each investigation devised its own measure attempting to define and conceptualise the key concepts. In one of the earliest studies into Internet addiction, Egger and Rauterberg (1996) devised a 46-item measure that looked at usage patterns, feelings and experiences of Internet use. The questionnaire was posted on a Website and elicited 450 responses mainly from Switzerland. A total of 10% of the respondents identified themselves as Internet addicts on the basis of a single item. The Internet addict group spent significantly more time online. Egger and Rauterberg (1996) also found that the Internet addict group felt more anxious if their Internet use was restricted and would be more likely to feel guilty or depressed if they spent a long time online. Brenner (1997) devised a 32-item true false questionnaire of Internet experiences (Internet-related Addictive Behaviour Inventory) based on the behavioural criteria for substance abuse found in the DSM-IV. Brenner (1997) reported a good internal consistency (α = 0.87) for the scale. In his sample of 563 responses, Brenner (1997) found that younger users experienced more Internetbased problems despite spending as much time online as older users. Unfortunately, the article reports very few results and no information on the scale (other than the internal reliability and 14 sample items). In another online study, Petrie and Gunn (1998) devised their own 16-item Internet Use and Attitudes Scale that looked at Internet use, beliefs and attitudes. A factor analysis of these items suggested that a single factor explains 39% of the variance. Internet addicts were identified on the basis 794

3 Development and psychometric properties... of a single item that asked respondents to identify themselves as Internet addicts. In Petrie and Gunn s (1998) sample of 445 respondents, 46% of respondents identified themselves as Internet addicts. The respondents who scored high on their scale were more likely to spend more time online and to hold more positive attitudes towards the Internet. Using Beck s Depression Inventory and Eysenck s Introversion/Extraversion Scale, they also found a significant relationship between high Internet use and both depression and introversion. Young (1996a) was the first person to link excessive Internet use to a DSM-IV diagnosis. According to Young (1996a), excessive Internet use was most closely associated with behavioural impulse-control disorders as defined in the DSM-IV. In her earlier work, she devised an Internet addiction test that consisted of 8 yes/no items, corresponding with the eight criteria that she defined for Internet addiction (these criteria approximate the criteria for pathological gambling). In this initial Internet addiction test, Young (1996a) considered any respondent who answered yes to 5 or more of the items (consistent with the cut-off for pathological gambling) as Internet dependants. In a sample of 396 respondents, Young (1996a) found that 60% were classified as Internet dependent. The Internet dependants were more likely to report negative consequences of Internet use, were more likely to use the interactive functions of the Internet and were more likely to have recently started using the Internet. In later assessments of Internet addiction, Young (1999) devised the Internet Addiction Test (IAT). The IAT included the 8 original items based on DSM-IV criteria as well as 12 new items. Of course, there are numerous problems with using a DSM-IV classification. The main concern with this classification is that comparisons of excessive Internet use are made with pathological gambling. A more detailed critical analysis of these DSM-IV criteria is made in Thatcher and Goolam (in this issue). Morahan-Martin and Schumacher (2000) assessed pathological Internet use (PIU) using a self-developed scale with 13 yes/no items. They considered a respondent to be a pathological Internet user if she or he answered positively to at least four of the items. On a sample of 277 University students, Morahan-Martin and Schumacher (2000) reported an internal reliability of 0.88, although they provided no information on the development of the PIU or the rationale for determining the pathological Internet use cut-off. In their study, Morahan-Martin and Schumacher (2000) also provide information on a self-developed, 25-item Internet Behaviour and Attitudes Scale (IBAS) using a Likert-type scale. no internal reliability was given for the IBAS, although an exploratory factor analysis revealed six factors (social confidence, socially liberating, competence, ease of communication, disadvantages of use, and lurking) accounting for 59% of the variance. More recently, attempts at developing a measure of excessive Internet use have focused on factor-analytic approaches. Pratarelli and Browne (2002) used a 74-item questionnaire on computer and Internet-related experiences. A factor analysis of these items on a sample of 524 student respondents suggested a three-factor model, an 795

4 Andrew Thatcher and Shamira Goolam addiction factor (α = 0.89), a sex factor (α = 0.72) and a user factor (α = 0.57). Structural equation modelling of the factors suggested that an addictive performance profile (i.e., the addiction factor) was predictive of excessive behaviours towards the Internet for sexual purposes (i.e., the sex factor) and for functional use (i.e., the user factor). However, Pratarelli and Browne (2002) acknowledged that their model was not robust and concluded that more research in this regard was necessary. In another factor-analytic conceptualisation, Davis, Flett and Besser (2002) developed and validated the Online Cognition Scale (OCS) to measure pathological Internet use. The OCS was based on Davis s (2001) cognitive-behavioural model which distinguishes between specific conditions (i.e., users that use the Internet for a specific purpose, such as online gambling or online sex) and generalised conditions, both of which result from some underlying psychopathology (e.g., depression, anxiety and substance dependence). Items in the OCS were drawn from symptoms described in the literature on problematic Internet use, as well as items adapted from measures of procrastination, depression, impulsivity and pathological gambling. Their confirmatory factor analysis of the 36-item OCS on a sample of 211 student respondents confirmed a four-factor model (i.e., loneliness/depression factor, α = 0.77; diminished impulse control factor, α = 0.84; distraction factor α = 0.81; and social comfort factor, α = 0.87). The factors of the OCS were positively correlated with the length of time spent online and the IBAS. Furthermore, the interactive applications of the Internet were most related to scores on the OCS. There have been numerous suggested predictors of problematic Internet use. The obvious predictor of Internet addiction is Internet usage and the most commonly reported predictor in this regard is length of time spent online (e.g., Davis et al., 2002; Morahan-Martin & Schumacher, 2000; Petrie & Gunn, 1998; Widyanto & McMurran, 2004; Young, 1996a) the longer the time spent online, the greater the likelihood of experiencing problems with the Internet. Additionally, Widyanto and McMurran (2004) found that using the Internet for personal use was a better predictor than using the Internet for work purposes. Some studies have found that younger users (Brenner, 1997; Widyanto & McMurran, 2004) and those who have recently started to use the Internet (Widyanto & McMurran, 2004) were also most likely to experience symptoms of problematic Internet use, although it is more likely that younger users would also have been most likely to have recently adopted the Internet. Furthermore, the interactive functions of the Internet (e.g., newsgroups, chat rooms and online games) have been found to be more addictive and to cause more psychological problems (Morahan-Martin & Schumacher, 2000; Young, 1996a). Numerous studies suggest a significant relationship between depression and problematic Internet use (e.g., Davis et al., 2002; LaRose et al., 2003; Petrie & Gunn, 1998), and Morahan-Martin and Schumacher (2000) found that loneliness and social liberation were significantly related to problematic Internet use. 796

5 In general, then, studies on problematic Internet use have tended to use self-devised multi-item measures with few reported psychometric properties. The psychometric properties that are reported are usually from the same authors who developed the instrument. The single exception to this trend was Widyanto and McMurran (2004) who reported on the psychometric properties of Young s (1999) 20-item IAT. Widyanto and McMurran s (2004) factor-analysis (with a small sample of 86 respondents) suggested that the IAT had six factors (two of the factors with only 2 items each: salience, excessive use, neglect work, anticipation, lack of control, neglect social life). The factor analytic approaches (Davis et al., 2002; Pratarelli and Browne, 2002) are impressive with large numbers of items and complex statistical analyses, but the sample sizes are moderate in relation to the number of items and their results (even by their own admission) suggest that further investigation is necessary. This article reports on the development of the Problematic Internet Use Questionnaire (PIUQ) and the psychometric properties of the PIUQ in a follow-up investigation. In terms of concurrent validation, we expect that high scores on the PIUQ would be related to a higher amount of time spent online, that the strength of this relationship would increase with the amount spent online for personal use, that highest PIUQ scores would be found with younger Internet users and that people who use the interactive functions of the Internet would score higher on the PIUQ. METHODOLOGY Development and psychometric properties... Initial development and pilot study on the PIUQ Young s (1999) Internet Addiction Test (IAT) was available at the time of data collection but the psychometric properties of this test had not yet been reported (these were only recently reported by Widyanto and McMurran, 2004). Davis et al. s (2002) 36-item OCS, Morahan-Martin and Schumacher s (2000) 13-item PIU and Pratarelli and Browne s (2002) 74-item scale had each not yet been published at the time of conceptualising the PIUQ in The questions for the PIUQ pilot study were formulated by examining a frequently cited pathological gambling questionnaire (Lesieur and Blume s, 1987, South Oaks Gambling Screen (SOGS)), Young s (1996a) criteria for Internet addiction, as well as items drawn from the published literature on symptoms of Internet addiction. The draft PIUQ consisted of 16 items where respondents were asked to indicate in a five-point Likert-type format (from never to always ), the frequency of various symptoms or behaviours of problematic Internet use. The sample for the pilot study consisted of self-selected respondents to the draft PIUQ. The PIUQ was converted into a Webpage and posted on the Website of an Internet development company. A large Internet service provider, a prominent national newspaper and a Johannesburg university all posted a message and a link from the homepage of their Websites. Finally, an interview was undertaken on a popular, regional South African talk radio station where potential participants were given the 797

6 Andrew Thatcher and Shamira Goolam Website address and invited to complete the online questionnaire. In a five-week period in late 2000, a total of 279 people responded to the questionnaire. The majority of the sample was male (71%) below the age of 30 years (13 18 years old: 20%; years old: 37%; years old: 22%), with a high level of education (26% of the sample had a postgraduate qualification and a further 38% of the sample had a post-matriculation qualification). A large proportion of the sample consisted of students (43%) and unmarried (70%) respondents. A large proportion of the sample had been using the Internet for longer than one year (65%). Respondents were most likely to use the Internet for five days a week (32%), although a fair proportion of respondents used the Internet every day of the week (28%). Most respondents spent fewer than 15 hours a week online (64%) and a small proportion spent more than 40 hours a week on the Internet (9%). The internal reliability of the pilot PIUQ was high with a Cronbach alpha coefficient of 0.90 and with item total correlations between 0.45 and An exploratory principal component factor analysis was conducted to determine whether there were any sub-scales. The unrotated factor analysis was indicative of a one-factor solution with the first factor explaining 41% of the variance. Three factors met Kaiser s criterion of unity (i.e., eigen values greater than one). The subsequent factors explained minimal variance (i.e., 8% for Factor 2 and 6% for Factor 3). Gorsuch (1974) argues that if a factor does not add much to the variance already explained, then it is not worth interpreting. A promax rotation produced a solution that was clearly interpretable with the three factors labelled as Adverse effects (eight items), Time online (five items) and Social interaction (three items). In establishing the concurrent validity of the pilot PIUQ, it was found that scores on the PIUQ were significantly related to respondents time spent online (F(1, 277) = 11.28, p < 0.01) time spent online was a categorical variable in the pilot study. Item refinement on the PIUQ Despite the excellent internal reliability of the pilot PIUQ in the pilot study, it was felt that additional refinements to the items were necessary. In particular, one item had vague phrasing: Have you ever suffered any serious adverse consequences because of your use of the Internet? This item did not specify the type of adverse consequences. The item was therefore expanded into three items, each specifying a particular category of adverse consequence: Have you ever suffered any adverse physical or health-related consequences because of your use of the Internet? Have you ever suffered any adverse psychological consequences because of your use of the Internet? Have you ever suffered any adverse financial consequences because of your use of the Internet? It was also felt that two of the eight Internet addiction criteria were not covered adequately in the pilot PIUQ. For the criterion of needing to spend more and more time online to achieve satisfaction, the following item was added: Do you find that you need to spend more and more time on the Internet to feel 798

7 Development and psychometric properties... satisfied? For the preoccupation with the Internet criterion, the following item was added: Do you find yourself thinking about the next time you will be able to get onto the Internet?. Two items were not in the five-point Likert-type format from never to always but instead required respondents to answer in a dichotomous yes/no format. It was decided to keep these two items in this format to allow for variability in the response formats. The revised PIUQ therefore consisted of 20 items, with 18 items requiring respondents to indicate whether they agreed or disagreed with statements on a five-point Likert-type scale and 2 items were dichotomous in a yes/no response format. The 18 five-point items were scored on a scale of 1 to 5, where 1 indicated low problematic Internet use and 5 indicated high problematic Internet use. Similarly, the two dichotomous items were scored as 1 or 5. The revised PIUQ therefore was aimed at assessing problematic Internet use on a scale of 20 to 100 (where 100 is the highest indicator of problematic Internet use). Validation sample The PIUQ was converted into a Webpage and a link was placed on the homepage of a prominent South African online magazine. The link took interested viewers to a brief explanation of the research and an invitation for volunteers to complete the questionnaire. If a respondent volunteered to participate they were able to read the instructions and complete the questionnaire online. Once respondents had completed the questionnaire they were directed to a Webpage that provided information about IAD and problematic Internet use. Contact details of a free counselling service were also provided. The Website address for the questionnaire was also submitted to two popular South African search engines and a media release, which included the Website address, was made to prominent South African newspapers and radio stations. Questionnaire responses were collected over a two-week period in September A total of 1795 usable questionnaires were obtained. Given the content area of the PIUQ and the collection of data using an online questionnaire, it is possible that the sample was biased towards respondents who favoured using the online environment for the communications and interactions. The majority of respondents were male (75%) and white (83%). Most of the respondents were young, between the ages of 19 and 35 years (19 24 years old: 24%; years old: 29%; and years old: 16%). Respondents were most likely to be single (48%), although a fair proportion were married (38%), cohabiting with a partner (6%) or were divorced (5%). Most respondents had a post-matriculation qualification (post-matriculation diploma: 30%; university degree, 28%; or postgraduate qualification, 18%). The majority of respondents were employed in a professional capacity (62%) or a semi-professional capacity (13%) and a large proportion were working in the information technology (IT) sector (50%). This is probably because the popular online magazine used to gather the sample specifically targeted IT professionals. 799

8 Andrew Thatcher and Shamira Goolam Respondents were most likely to access the Internet from work (83%) and/or home (72%). Most respondents had access to the Internet from one (36%) or two (52%) locations, while 2% accessed the Internet from five or more locations. The majority of respondents had been using the Internet for between two and eight years (77%), with a fair proportion of the respondents having used the Internet for longer than eight years (15%) and a smaller proportion of respondents that had used the Internet for fewer than two years (7%). Respondents were most likely to use the Internet for between four and eight hours a day (55%), although a large proportion used the Internet for fewer than four hours a day (24%) and a small proportion used the Internet for more than 12 hours a day (6%). The most common use of the Internet was for the World- Wide Web (99%) and (98%); while online messaging (39%), online chatting (34%), transferring files (51%), and online interactive games (15%) were also used by a substantial proportion of respondents. Instruments The survey posted on the Website of a popular online IT magazine consisted of five sections. The first section contained eight biographical questions. The second section contained ten questions related to respondents use of the Internet (e.g., type of Internet access, frequency of use, type of applications and whether Internet use was related to work or recreation. The third section contained a list of 20 words and asked respondents to indicate which of the words they would associate most with their time spent online. The fourth section contained eight questions corresponding to Young s (1996a) criteria for Internet Addiction. The fifth section contained the twentyrevised items of the PIUQ. RESULTS Internal reliability The Cronbach alpha coefficient was 0.90 with item total correlations ranging from 0.38 to The item total correlations suggest that while there is good internal reliability there is also sufficient variability between the individual items of the scale. Eight items would alter the Cronbach alpha coefficient if removed, but these eight items would only decrease the coefficient from 0.90 to 0.89 and so there was no statistical justification to remove these items from the scale. There were only two items where all respondents did not answer the item. These were the two items that required dichotomous responses. In the one item, 8 respondents (0.4% of respondents) failed to respond to the question and in the other, 18 respondents (1.0% of respondents) failed to respond. These results suggest that there is evidence that the use of different response formats should be revisited. 800

9 Development and psychometric properties... Table 1. Factor loadings for rotated factor analysis (varimax solution) Question no. Question Do you find that you need to spend more and more time on the Internet to feel satisfied? 1 Do you ever find that you stay on the Internet much longer than intended? 8 Do you find yourself looking forward to spending time on the Internet and feeling as if you can t wait to be online? Do you spend as long as possible online? Do you find yourself thinking about the next time you will be able to get onto the Internet? 20 Do you find yourself relying on the Internet to brighten up your life? 6 Do you find that you keep secrets from others regarding your time spent on the Internet? 17 Do you go online when you know there are more important things you should do? 18 Do you feel misunderstood by people who don t see the attraction of the Internet? 3 Do you feel distressed when you cannot connect to the Internet? 12 Have you ever suffered any serious adverse psychological consequences because of your use of the Internet? 11 Have you ever suffered any serious adverse physical/healthrelated consequences because of your use of the Internet? 13 Have you ever suffered any serious adverse financial consequences because of your use of the Internet? 19 Does your use of the Internet cause problems in your daily life? 10 Has your use of the Internet resulted in the loss of a significant relationship, job or career opportunity? Have you ever tried unsuccessfully to stop using the Internet? Have you experienced a situation where you tried to escape problems by going onto the Internet? 16 Do you prefer online socialising to other forms of socialising? Do you find it easier to interact with others online as opposed to face to face? 7 Do you tend to seek out certain individuals on the Internet? 0.64 Eigen value % variance explained Cronbach alpha Notes: Factor 1: Online preoccupation. Factor 2: Adverse effects. Factor 3: Social interactions

10 Andrew Thatcher and Shamira Goolam Exploratory factor analysis An exploratory principal components factor analysis was performed to determine whether the same factor structure was attained as in the initial pilot study. Owing to the two dichotomous items, the factor analysis was conducted on the correlation matrix (as opposed to the covariance matrix). While the initial unrotated factor analysis was indicative of a single-factor solution with the first factor explaining 37% of the variance, the scree-plot and Kaiser s criterion of unity (i.e., eigen values >1) suggested three factors from the principal components factor analysis. The subsequent factors explained 7% and 6% of the variance, respectively. In total, the first three factors explained 50% of the variance. In examining the factor loadings across the three factors, it was evident that all but one of the items ( Have you ever suffered an serious adverse psychological consequences because of your use of the Internet? ) loaded highest on the first factor. In the orthogonal iterations, the varimax rotation produced the simplest and easiest interpretable solution. Factor loadings of 0.40 and above were chosen, as these loadings were statistically significant (Hair, Anderson, Tatham & Black, 1998). The results of the varimax rotation factor loadings are given in Table 1. Factor 1 consists of ten items with a Cronbach alpha of Each of the items in this factor refers to the respondent thinking about being online or wanting to spend more time online. This factor has therefore been labelled Online preoccupation. Factor 2 consists of seven items with a Cronbach alpha of The items in this factor each refer to negative outcomes that respondents might experience as a result of their online activities. This factor has therefore been labelled Adverse effects. Finally, Factor 3 consists of three items, with a Cronbach alpha of Each of the items in this factor refers to the respondent using the Internet for social interaction activities and the factor has been labelled Social interactions. Table 2. Correlations between the three PIUQ dimensions and with Young s eight criteria PIUQ Total Online preoccupation Adverse effects Social interactions Adverse effects 0.63* Social interactions 0.60* 0.47* Young s eight criteria 0.72* 0.68* 0.63* 0.44* Note: * p < 0.01 Correlations of PIUQ factors The three PIUQ factors all correlated significantly with one another, with correlations ranging from r = 0.63 to r = 0.47 (see Table 2). The strongest correlation was found between Online preoccupation and Adverse effects and the weakest correlation between Social interactions and Adverse effects. 802

11 Development and psychometric properties... Concurrent validity Concurrent validity was established by correlating the PIUQ score with Young s (1996a) eight criteria for Internet Addiction. Young s (1996a) criteria consist of eight yes/no items corresponding with the diagnostic definition of Internet addiction. Thus, scores range from 0 to 8. The PIUQ was significantly correlated with Young s criteria (r = 0.72, p < 0.01). The correlations with the PIUQ factors ranged from 0.68 (Online preoccupation) to 0.44 (Social interactions) as shown in Table 2. Correlations of PIUQ/factors with Internet use behaviours Correlations between the total PIUQ score, the three factors and the Internet use behaviours are presented in Table 3. The only Internet use behaviour that was not significantly correlated with the PIUQ was the length of time since starting to use the Internet (r = 0.01, p>0.05). The frequency of Internet use (days per week) was significantly related to the PIUQ (r = 0.21, p<0.01) and to each of the factors. There was also a significant relationship between the length of time of each Internet session and the PIUQ score (r = 0.37, p<0.01) and with each of the PIUQ factors. The length of time per week spent online was also significantly correlated with the PIUQ (r = 0.46, p<0.01) and each of the factors. Comparing the amount of time spent online at different periods of the day, the correlations with the PIUQ were strongest for the amount of time spent online between 23:00 and 08:00 (r = 0.41, p<0.01 to r = 0.32, p<0.01), followed by 18:00 to 23:00 (r = 0.39, p<0.01 to r = 0.27, p<0.01) and then 08:00 to 18:00 (r = 0.15, p<0.01 to r = 0.12, p<0.01). Comparing the factors of the PIUQ, the correlations were generally strongest with the Online preoccupation factor, followed by the Social interactions factor and weakest for the Adverse effects factor. Table 3. Correlations between the PIUQ and Internet use behaviours PIUQ Total Online preoccupation Adverse effects Length of use Number of locations 0.20* 0.17* 0.19* 0.15* Days per week 0.21* 0.20* 0.15* 0.19* Time per session 0.37* 0.33* 0.30* 0.34* Online time per week 0.46* 0.42* 0.35* 0.42* Time online: 08:00 18: * 0.13* 0.12* 0.14* Time online: 18:00 23: * 0.36* 0.27* 0.37* Time online: 23:00 08: * 0.38* 0.32* 0.36* Note: * p<0.01 Social interactions 803

12 Andrew Thatcher and Shamira Goolam Functional usage of the Internet and psychological associations Comparisons were made between those respondents who participated in online gaming, online chat and online messaging services and those who did not. Owing to the inequality of variances, t-tests using the Satterthwaite approximation for degrees of freedom were used. Respondents who participated in online gaming (t318 = , p<0.01), online chat (t918 = , p<0.01), and online messaging services (t1271 = -8.95, p<0.01) had significantly higher scores on the PIUQ than those respondents who did not participate in these activities. These results are presented in Table 4. The t-tests were also computed, comparing those respondents who associated their time online with depression, isolation or loneliness and those respondents who did not. Statistically significant differences were found for depression (t46.7 = -6.32, p<0.01), isolation (t45.7 = -3.66, p<0.01) and loneliness (t70.5 = -6.77, p<0.01). Table 4. Functional usage and psychological associations PIUQ mean Absence (N) SD PIUQ mean Presence (N) SD df t-value Online games (1524) (271) * Internet Chat (1189) (606) * Online messaging (1099) (696) * Depression (1749) (49) * Isolation (1750) (45) * Loneliness (1728) (67) * Notes: N is given in parentheses below the mean. Presence refers to respondents participating in an online activity, or expressing an association with depression, isolation or loneliness. * p<

13 In each case, those respondents who indicated that they associated their online time with depression, isolation or loneliness had significantly higher PIUQ scores. These results are also presented in Table 4. DISCUSSION Development and psychometric properties... Internal reliability The results of the internal reliability analyses suggest that the PIUQ has good internal reliability, with an overall α = The Cronbach alpha coefficient demonstrates suitable internal reliability for scales in the social sciences (Rosenthal & Rosnow, 1991). However, based on a systematic item analysis, it was evident that the policy of having two dichotomously scored items should be revisited. These two items were the only items where a small proportion of respondents did not answer the item. This might suggest that some respondents would feel more comfortable with a wider range of response options for these questions. In addition, a closer inspection of Items 8 ( Do you find yourself looking forward to spending time on the Internet and feeling as if you can t wait to be online? ), 10 ( Has your use of the Internet resulted in the loss of a significant relationship, job or career opportunity? ) and 11 ( Have you ever suffered any serious adverse physical/health-related consequences because of your use of the Internet? ) suggests that the double-barrelled nature of these items might require their separation into more than one item, as they may confound respondents answers. Construct validity Construct validity was partially established, using exploratory factor analysis. The results of the factor analysis presented in this article confirm a similar factor structure as that found in the pilot study. In the pilot study, the factor analysis suggested a three-factor solution: adverse effects, time online and social interaction. The three factors for the pilot study approximate the three factors from the validation study (i.e., Adverse effects, Online preoccupation and Social interactions) although the number of items in each factor has obviously changed due to items being added to the PIUQ. These three factors (Factor 1: Online preoccupation ten items; Factor 2: Adverse effects seven items; Factor 3: Social interactions three items) explain a significant portion of the total variance (50%). The internal reliability of the three sub-scales of α = 0.88 (Online preoccupation), α = 0.77 (Adverse effects), and α = 0.74 (Social interactions) showed good internal consistency. The three sub-scales were also found to be positively correlated with one another. Further evidence of construct validity was that the PIUQ and its sub-scales correlated significantly with other external measures that other researchers have found to be significantly related to problematic Internet use. Time spent online was significantly correlated with the PIUQ and its factors with the strongest correlation for time spent online between 23:00 and 08:00 and between 18:00 and 23:00 (i.e., for time spent 805

14 Andrew Thatcher and Shamira Goolam online outside of normal working hours). These significant relationships are supported by evidence from numerous studies that have demonstrated the significant relationship between time spent online and problematic Internet use (e.g., Davis et al., 2002; Morahan-Martin & Schumacher, 2000; Petrie & Gunn, 1998; Widyanto & McMurran, 2004; Young, 1996a). Consistent with the findings of the Widyanto and McMurran (2004) study, this study also found that using the Internet outside of normal working hours was more strongly related to problematic Internet use. Furthermore, the results of this study have shown statistically significant positive relationships between the PIUQ and the following Internet usage activities: time spent online for each Internet session, the number of days per week that a respondent uses the Internet and the number of different locations from where a respondent has access to the Internet. However, Widyanto and McMurran (2004) found a negative relationship between the time since starting to use the Internet and problematic Internet use which was not supported by the results of this study. The results of this study would suggest that there is no relationship between how long it has been since the individual started to use the Internet and problematic Internet use. Consistent with the results of the Young (1996a) and Morahan-Martin and Schumacher (2000) studies, this study found significant relationships between problematic Internet use and the interactive functions of the Internet (online chatting, online messaging and online gaming). However, Widyanto and McMurran found no significant relationships. The non-concordance with Widyanto and McMurran s results may be a function of their small sample size (86 valid responses). This study also supported the results from the studies of Petrie and Gunn (1998), Davis et al. (2002) and LaRose et al. (2003), who found significant relationships between depression and problematic Internet use. In addition, this study found a significant relationship between problematic Internet use and loneliness and isolation. Morahan-Martin and Schumacher (2000) also found a significant relationship between problematic Internet use and loneliness. The highly significant correlation (r = 0.72) with Young s eight criteria provides strong support for the construct validity of the PIUQ. The high correlations should not be surprising, since the PIUQ was partially based on these eight criteria. While the correlation is very high, it must be pointed out that a correlation of r = 0.72 demonstrates 50% shared variance between the two measures, indicating that there is still 50% unexplained variance. Future research is needed to establish the PIUQ s convergent/divergent validity with other measures of problematic Internet use (e.g., Young s, 1999, IAT or Davis et al s, 2002, OCS). The correlations with the sub-scales of the PIUQ were rather interesting, with the lowest correlation (although still significant at r = 0.44) between Young s eight criteria and social interactions. Further research is necessary to determine whether social interactions are a positive or negative consequence of Internet use. Owing to the significant positive correlation with Adverse effects (r = 0.47), the results of this study suggest that social interactions, as measured by the PIUQ, are likely to assess negative consequences. 806

15 Development and psychometric properties... Limitations and directions for future research The first obvious limitation is the generalisability of the sample. The validation sample consisted of respondents recruited by means of a South African-based online IT magazine. It is difficult to establish how generalisable these results are even to a South African population, let alone a wider international sample. Further research is required with different samples to broaden the population validity of the PIUQ. The sample size for the validation study in particular was impressive, compared to other similar studies on problematic Internet use. However, Azar (2000) has highlighted some of the problems with collecting valid data from volunteer Internet-based samples. In an Internet-based questionnaire it is more difficult to establish the reasons why respondents volunteer to complete psychological questionnaires. The nature of the volunteer bias is therefore difficult to establish and may also be exaggerated with large sample sizes. In particular, we are concerned that people who suspect that they may be susceptible to problematic Internet use may be more inclined to respond to this survey. The possible inclusion of items to assess for social desirability might be explored in future versions of the PIUQ. While the results of this study are supportive of the psychometric properties of the PIUQ, we have not yet established the diagnostic potential of the instrument. Further studies are needed to determine the diagnostic potential of the PIUQ. Studies are needed where the PIUQ is correlated with clinical diagnoses and case study evidence. Through studies of this nature, reliable and valid cut-off scores on the PIUQ for determining problematic Internet use might be established. As mentioned previously, there is also a need to correlate the PIUQ with other measures of problematic Internet use (e.g., IAT or OCS). In the literature review we criticised previous attempts to develop measures of problematic Internet use as being based on factor analytic studies on scales developed by the same authors. In this respect, while the psychometric properties of the PIUQ are impressive, the criticism is no different. We would therefore encourage independent validation (or invalidation) studies of the PIUQ. CONCLUSIONS This article has described the development and psychometric properties of the PIUQ, an instrument designed to assess the possible negative psychological aspects of using the Internet. Items for the PIUQ were developed from the theoretical definitions and criteria of problematic Internet use as defined in Young (1996a) and from items in a commonly used pathological gambling questionnaire. The PIUQ was pilot-tested, refined and then tested on a validation sample. The PIUQ was shown to have good internal reliability (α = 0.90) and a factor analysis suggested a three-factor solution explaining 50% of the variance: Online preoccupation, Adverse effects, and Social interactions (each with good internal reliability). The PIUQ was shown to have strong correlations with other predictors of problematic Internet use, including total time 807

16 Andrew Thatcher and Shamira Goolam spent online (r = 0.46), feelings of depression, isolation and loneliness, the types of online activities (online gaming, Internet chatting and Online messaging), and a strong significant correlation with the Young s eight criteria of Internet addiction. The predictive validity of the PIUQ for the clinical diagnosis of problematic Internet was not established in this study. The PIUQ is therefore not a clinical diagnostic instrument, although it has the potential to be utilised as an initial screening device for follow-up clinical diagnoses. REFERENCES American Psychiatric Association (1995). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: American Psychiatric Association. Azar, B. (2000). Online experiments: ethically fair or foul? Monitor in Psychology, 31(4). (accessed on 4 April 2005). Brenner, V. (1997), Psychology of computer use. XLVII. Parameters of Internet use, abuse and addiction: The first 90 days of the Internet Usage Survey. Psychological Reports, 80, Davis, R. A. (2001). A cognitive-behavioural model of pathological Internet use. Computers in Human Behaviour, 17, Davis, R. A., Flett, G. L. and Besser, A. (2002). Validation of a new scale for measuring problematic Internet use: Implications for pre-employment screening. CyberPsychology & Behavior, 5, Egger, O. and Rauterberg, M. (1996). Internet behaviour and addiction. id.tue.nl/ g.w.m.rauterberg/ibq/report.pdf (accessed on 20 September 2004). Goodman, A. (1993). Diagnosis and treatment of sexual addiction. Journal of Sex and Marital Therapy, 19, Gorsuch, R. L. (1974). Factor Analysis. Philadelphia: W.B. Saunders. Griffiths, M. (1995). Technological addictions. Clinical Psychology Forum, 71, (2000). Does Internet and computer addiction exist? Some case study evidence. CyberPsychology & Behavior, 3, Griffiths, M. (2003). Internet abuse in the workplace: Issues and concerns for employers and employment counsellors. Journal of Employment Counselling, 40, Hair, J. F., Anderson, R. E., Tatham, R. L. and Black, W. C. (1998). Multivariate data analysis (5th ed.). London: Prentice-Hall. Lacey, H. J. (1993). Self-damaging and addictive behaviour in bulimia nervosa: A catchment area study. British Journal of Psychiatry, 163, LaRose, R., Lin, C. A. and Eastin, M. S. (2003). Unregulated Internet usage: addiction, habit or deficient self-regulation. Media Psychology, 5, Lesieur, J. D. and Blume, S. B. (1987). The South Oaks Gambling Screen (SOGS): A new instrument for the identification of pathological gamblers. American Journal of Psychiatry, 144, Morahan-Martin, J. and Schumacher, P. (2000). Incidence and correlates of pathological Internet use among college students. Computers in Human Behaviour, 16, Nua Internet Surveys (September, 2002). How many online? how_many_online/index.html (accessed on 9 September 2004). 808

17 Development and psychometric properties... Petrie, H. and Gunn, D. (1998). Internet addiction? The effects of sex, age, depression and introversion. Paper presented at the British Psychology Society Conference. herts.ac.uk/sdru/helen/inter.html (accessed on 13 December 2004). Pratarelli, M. E. and Browne, B. L. (2002). Confirmatory factor analysis of Internet use and addiction. CyberPsychology & Behavior, 5, Rosenthal, R. and Rosnow, R. L. (1991). Essentials of Behavioral Research. Methods and data analysis. New York: McGraw-Hill. Shapira, N. A., Goldsmith, T. D., Keck, P. E., Khosla, U. M.,and McElroy, S. L. (2000). Psychiatric features of individuals with problematic Internet use. Journal of Affective Disorders, 57, Widyanto, L. and McMurran, M. (2004). The psychometric properties of the Internet Addiction Test. CyberPsychology & Behavior, 7, Young, K.S. (1996a, August). Internet addiction: The emergence of a new clinical disorder. The 104th Annual Meeting of the American Psychological Association, Toronto.. (1996b). Addictive use of the Internet: a case that breaks the stereotype. Psychological Reports, 79, (1997, August). What makes the Internet addictive: Potential explanations for pathological Internet use. The 105th Annual Meeting of the American Psychological Association, Chicago.. (1999). Internet addiction: symptoms, evaluation and treatment. In L. Van de Creek and T. Jackson (Eds), Innovations in clinical practice: A source book (pp ). Sarasota, Florida: Professional Resource Press. 809