Child: care, health and development

Transcription

1 bs_bs_banner Child: care, health and development Original Article doi: /cch Confirmatory factor analysis of the Strengths and Difficulties Questionnaire in Singaporean kindergartners R. Bull,* K. Lee,* I. H. C Koh and K. K. L Poon* *National Institute of Education, Nanyang Technological University, Singapore, and Utrecht University, Utrecht, Netherlands Accepted for publication 19 August 2015 Keywords confirmatory factor analysis, kindergarten, SDQ, Singapore Correspondence: Rebecca Bull, National Institute of Education, Nanyang Technological University, 1 Nanyang Walk, , Singapore rebecca.bull@nie.edu.sg Summary Background The Strengths and Difficulties Questionnaire (SDQ) assesses behavioural adjustment in children aged 3 to 16 years. To ascertain the appropriateness of the scale for a specific population, it is important to examine whether the distinctiveness of the scale dimensions can be verified empirically. Aims Confirmatory factor analysis was used to test explicitly which of three models better explain our data, and whether model fit was improved by the addition of method factors. Methods Parents of 411 Singaporean kindergartners completed the SDQ. Results A four-factor multi-trait multi-method model (Prosocial, Conduct, Hyperactivity, Internalizing and two method factors) provided the best fit to the data. There was strong evidence for convergent and discriminant validity. However, differences in configural loading pattern indicated gender-related differences in the mapping of the SDQ items. Discussion Differences in factor structure across countries and gender may reflect differing conceptions of the underlying dimensions, as well as differences in normative expectations. However, our findings may allow its use as a screening tool to identify Singaporean children at risk of emotional and behavioural difficulties. In Singapore, approximately 5% of children aged 6 12 years exhibit evidence of externalizing problems such as aggression, hostility and antisocial behaviour, whilst 12% exhibit internalizing problems, as assessed by the Child Behaviour Check List (CBCL; Woo et al., 2007). However, the provision of psychological services for Singaporean children is faced with many challenges, including the lack of qualified mental health professionals, a high ratio of students to school psychologists, social stigma associated with seeking professional help (Ooi et al., 2014) and the lack of local norms for many psychological scales. The Strengths and Difficulties Questionnaire (SDQ; Goodman, 1997) assesses behavioural adjustment in children and adolescents aged 3 to 16 years. It has been used with both clinical and community samples in countries throughout Europe, the USA and Scandinavia (e.g. Dickey & Blumberg, 2004; Shojaei et al., 2009; McCrory & Layte, 2012), the Middle East and Asia (e.g. Du et al., 2008; Matsuishi et al., 2008; Lai et al., 2010; Liu et al., 2013) to identify children at risk of developing social, emotional, or behavioural difficulties and for evaluating the effectiveness of intervention. Here, we examine the factor structure of the parent SDQ in a community sample of Singaporean kindergartners. Preventative actions are more effective for young children than for older children (Loeber & Farrington, 2000), making it imperative to identify children with behavioural problems at a young age. The SDQ subscales Emotion, Conduct, Hyperactivity, Peer, and Prosocial behaviour, each includes five items, and on 2015 John Wiley & Sons Ltd 109

2 110 R. Bull et al. each item a respondent indicates the degree to which he/she agrees with a particular behaviour using a 3-point scale (0 = not true, 1 = somewhat true and 2 = certainly true). The Emotion, Conduct, Hyperactivity, and Peer subscales are aggregated to form a Total Difficulties Score; a higher score indicates poorer behavioural adjustment. On the Prosocial scale a higher score represents more positive rating of child prosocial behaviours. The SDQ is reported to have good concurrent, predictive and discriminant validity (see Stone et al. (2010) for a review) and has been shown to have good convergent validity: Hyperactivity and Conduct subscales of the SDQ correlate with externalizing subscales in the CBCL, and items from the Emotion and Peer SDQ scales correlate with the internalising scale of the CBCL (e.g. Liu et al., 2013). However, whether the scale functions identically across cultures has been questioned. In determining the cultural appropriateness of this measure, it is important to examine whether the number and distinctiveness of these dimensions can be verified. Several studies found support for Goodman s (1997) five-factor structure for the parent report SDQ (e.g. Smedje et al., 1999; Sanne et al., 2009; Niclasen et al., 2012). Other studies have been unable to replicate a five-factor structure. Dickey and Blumberg s (2004) exploratory analysis suggested that a three-factor solution better fit the data: (a) an externalising factor consisting of Hyperactivity and Conduct items, (b) an internalizing factor (Emotion and Peer items) and (c) a method factor comprising Prosocial and positively worded items from the Conduct and Peer scales. Liu et al. (2013) proposed a four-factor model consisting of Prosocial, Internalising, Conduct and Hyperactivity. The Conduct and Hyperactivity scales reflected Goodman s (1997) original factor structure, although the Prosocial scale resembled the method factor described by Dickey and Blumberg (2004). To address the methodological concern of positively and negatively worded items impacting on the factor structure, a number of studies have adopted a multi-trait multi-method approach (MTMM) in modelling their data. Van Roy et al. (2008) and McCrory and Layte (2012) found support for the five-factor model but found an improvement in model fit when a positive construal factor was added to the model. Data from East Asia (e.g. Hong Kong, China) generally indicate greater difficulty and less prosociality than European countries (Table 1) and differences in factor structure, e.g. Liu et al. s (2013) four-factor model for Chinese children. Gender differences have also been found. Boys tend to score higher than girls on the Conduct, Hyperactivity and Peer subscales, and in the total difficulties score (e.g. Du et al., 2008; Matsuishi et al., 2008; Lai et al., 2010; Mieloo et al., 2012; Niclasen et al., 2012; Liu et al., 2013). Girls sometimes score higher than boys on the Prosocial and Emotion scales. However, interpretation of these results is complicated by findings of gender-related differences in factor structure (Hill & Hughes, 2007). Many studies used exploratory factor analysis to determine factor structure of the SDQ. Although exploratory factor analysis is useful for initial investigations of the underlying structure of new scales, the availability of a wealth of prior findings allow us to use a more theory guided approach. Here we used confirmatory factor analysis to test explicitly which of three previously established models better explain our data. We also tested whether models were improved by the addition of method factors. Previous findings of gender-related differences motivated us to examine the same issue with our data. Finally, we make recommendations for cut-off scores to define children in clinical and borderline risk categories. Method Participants and procedure The sample consisted of 411 (192 females) children attending preschools in Singapore, aged from 52 to 87 months (M = 68.64, SD = 8.37). The sample was similar to the wider Singapore population in ethnic distribution; Chinese (60.2%), Malay (20.9%), Indian (13.4%) and others (5.4%). Approximately 76% of the sample attended preschools run by the largest notfor-profit providers in the country, which serve the majority of average-to-lower-income families in the population. Of these approximately 14% of children were recruited from centres serving disadvantaged children and children deemed to be at risk for a learning difficulty. The data were obtained from three separate studies that examined a range of child outcomes (conceptual understanding, early number and reading skills and social-emotional development). For one study we are not able to estimate response rate as recruitment was handled directly from the preschools. For the remaining two studies response rate was approximately 69%. Parents were provided with information about the study via an information sheet using consent procedures approved by the university Institutional Review Board. All parents completed the SDQ independently and returned it to their child s preschool. Results Initial analyses considered mean scores on each subscale and total difficulties and compared these with means from samples in other countries. Also considered were the cut-off scores that

3 SDQ in Singaporean kindergartners 111 Table 1. Comparison of mean subscale scores across studies and recommended cut-off scores for clinical risk (highest 10% of scores) and borderline risk (next 10% of scores) Present study Hong Kong China Netherlands Age range 5 6 years 6 12 years 3 10 years 5 6 years Prosocial (8.6) Male 6.41 (1.92) Female 6.81 (1.88) Borderline (5) 5 5 Clinical (0 4) Hyperactivity (3.6) Male 4.47 (2.17) Female 3.39 (2.12) Borderline (6) 6(M) 5(F) 7 Clinical (7 10) 7 10(M) 6 10 (F) 8 10 Emotional (1.9) Male 1.68 (1.63) Female 1.84 (1.94) Borderline (4) 3 4 Clinical (5 10) Conduct (1.6) Male 2.46 (1.78) Female 1.93 (1.62) Borderline (3) 4(M) 3(F) 3 Clinical (4 10) 5 10(M) 4 10(F) 4 10 Peer (1.4) Male 2.11 (1.42) Female 1.97 (1.40) Borderline (3) 3 5 Clinical (4 10) Total Diffs (8.6) Male (4.81) Female 8.91 (5.22) Borderline (14 16) 14 16(M) 13 16(F) Clinical (17 40) For each subscale, means in italics indicate a significant difference between males and females. For recommended clinical cut-offs, where these differ for males and females, they are reported separately. British norms for 5 10 years old as reported in Lai et al. (2010). Du et al. (2008). Mieloo et al. (2012). would be recommended in the local sample applying the criteria of the highest 10% of the sample being at clinical risk, and the next 10% of the sample being of borderline risk (Table 1). The mean scores generally show less prosocial behaviour and more difficulties on all subscales (particularly for boys), with the exception of Emotion, than in European samples. This is in line with findings from other Asian samples, e.g. Hong Kong and China. However, the normal score range (encompassing 80% of the sample) is very similar to the score range for a UK sample; the only noticeable differences in recommended risk cut-off scores are for Conduct (higher score needed to be classified as borderline or clinical risk) and Emotion (lower score needed to be classified in the risk categories). A series of confirmatory factor models were analysed to identify which of previously established models would best fit our data (Fig. 1). The models were (a) Goodman s (1997) five-factor model, (b) a four-factor model (Liu et al., 2013) and (c) a threefactor model (Dickey & Blumberg, 2004). We also tested each model using a MTMM approach. In the MTMM models, each item loaded on two factors: a trait factor (e.g. Hyperactivity) and a method factor (Positive or Negative). The data were estimated using Mplus (version 7.1; Muthén & Muthén, 2013), with missing data (less than 5% of each item) modelled using the full information maximum likelihood approach. Both the three and five-factor models fitted the data poorly (see Table 2 for fit indices). The

4 112 R. Bull et al. Figure 1. Five-factor, four-factor, and three-factor models considered in the analysis. The UK preschool version of the SDQ was used [two conduct items regarding antisocial behaviour ( often lies or cheats and steals from home, school or elsewhere ) are replaced by oppositionality items ( often argumentative with adults and can be spiteful to others )]. Table 2. Tested models and their goodness of fit Model number Model description (d.f.) χ 2 RMSEA CFI WRMR (d.f.) Δχfs 2 CFI 1 five factors (265) four factors (269) three factors (272) five-factor MTMM (239) four-factor MTMM (243) three-factor MTMM (246) four-factor MTMM with two items re-designated (243) No trait factors (testing for construct validity) (274) (31) No Prosocial factor (testing for construct validity) (255) (12) As a rule of thumb, a model that provides a good fit to the data has the following values: root mean square error of approximation (RMSEA) < 0.06, comparative fit index (CFI) 0.95, weighted root mean residual (WRMR) < 0.90 (Schreiber et al., 2006). The preferred model is shown in bold four-factor model provided the best fit to the data but was still poor by conventional standards. Furthermore, the Hyperactivity and Conduct factors were correlated strongly (r = 0.829). Using a MTMM model, the-five factor MTMM model converged with an error that is likely caused by a very high correlation between the Prosocial and Conduct factors (r = 0.916); this correlation suggests that though the model provided a good fit, there is no substantive separation between Prosocial and Conduct. Between the three and four-factor MTMM models, the latter is preferable: the three-factor model showed that the Prosocial items loaded poorly, which rendered their corresponding factor unsustainable. Inspection of the factor loadings from the four-factor model showed that two items failed to load strongly. The first item, Able to get on better with adults than with other children, was described by Goodman (1997) as a neutral item, and Liu et al. (2013) discarded this item to improve internal validity. Our data suggested that allowing this item to load on Prosocial would produce a better fit. This is reasonable as parents may view being able to get on with adults as prosocial rather than as a peer-related problem. The item constantly fidgeting or

5 SDQ in Singaporean kindergartners 113 squirming was classified as Hyperactivity by Goodman but was an indicator for Conduct in Liu et al. (2013). Our data showed that it failed to load on the Conduct subscale. For this reason, we re-estimated the model with this item reclassified as an indicator for Hyperactivity. With these changes, the model produced a reasonably good fit (Fig. 2 and Model 7 in Table 2). With the exception of the gets on better with adults item, all construct measures loaded significantly onto their respective factors. Prosocial was correlated negatively with Hyperactivity, Internalising and Conduct. Internalizing (a mix of items from the Emotional and Peer scales) was correlated positively with Hyperactivity and Conduct. We fitted two additional models to test for convergent validity. Given the improvement in model fit when the method factors were added, we tested the possibility that variance in the data reflected only variation in whether items were positively or negatively phrased. This is a particular concern for the Prosocial factor because it shared all but one of its indicators with the Positive method factor. We tested a strong version of this hypothesis by comparing the four-factor MTMM model to a model that contained only the method factors (no trait factors). This model produced a markedly poorer fit to the data. We also tested a weaker version of the hypothesis by removing only Prosocial and retaining the rest of the original four-factor MTMM. This too produced a poorer fitting model. At the individual parameter level, two-thirds of the indicators loaded more strongly on the traits than on the method factors. These findings provide support for convergent validity. A bootstrap analysis (using draws) was conducted to examine the extent to which parameter estimates are affected by our modest sample size. Of particular interest is that none of the positively worded items were now loaded strongly onto their method factor. The negatively worded factor remains viable. Inter-correlations between the factors also altered, with only Prosocial correlating marginally with Conduct (P =0.051) and significantly with Hyperactivity (P < 0.001). Low correlations between the factors provide further support for the discriminant validity of the substantive constructs. One of our previous concerns was that most of the Prosocial items cross-loaded onto the positively worded factor; the findings that Prosocial remains viable despite the absence of substantive cross-loadings onto the positively worded factor provide additional evidence for the discriminant validity of the Prosocial factor. To examine whether there are gender-related differences in how individual items mapped onto their underlying constructs, the four-factor MTMM model was fitted to data from boys and girls separately. The model converged without errors for girls. Findings on Prosocial and Positive were similar to those found in the bootstrapped analysis. Most of the indicators loaded strongly onto Prosocial, but cross-loaded weakly on Positive. Notably, half of the indicators from the three difficulties factors failed to load onto their respective difficulties factors. In contrast, findings from the boys showed that most of the prosocial indicators loaded weakly; instead, they cross-loaded strongly on the Positive factor. The findings also differed from the girls in that all but one of the difficulty indicators loaded strongly on their respective difficulties factors. One caveat on the boys findings is that the model yielded an inadmissible correlation between the two method based factors (r > 1), suggesting that the model did not provide a good fit. Nevertheless, these differences in model fit and loading patterns are indicative of gender-related differences in the functioning of the SDQ items. Discussion The present study investigated the optimal factor structure of the SDQ in a South East Asian community sample and considered whether the structure was invariant across gender. The best-fitting model was a MTMM model consisting of four trait factors (similar to Liu et al. (2013) for a Chinese sample) and two method factors. The trait factors were Prosocial (which included three positively worded Conduct and Peer items) Conduct, Hyperactivity and Internalizing (which included Peer and Emotion items). Whilst previous studies using confirmatory factor analysis have found Goodman s five factors to provide a satisfactory model fit (Van Roy et al., 2008; Mieloo et al., 2012), these studies did not consider whether an alternative model would provide a better fit to the data. A unique strength of our study is that we took several models that had previously been found to provide the best description for the SDQ and tested them explicitly. A number of previous studies have questioned the discriminatory validity of the SDQ, because of substantial correlations between the subscales. Van Roy et al. (2008), for example, reported high correlations between the Hyperactivity and Conduct scales, and between the Peer and Emotion scales, suggesting conceptual overlap between the subscales. In the current analysis, the four-factor MTMM model was compared against a model in which the traits were not differentiated. Although the Prosocial factor was correlated strongly with Conduct and with Hyperactivity, the deterioration in model fit for the undifferentiated model, together with modest correlations between the Internalizing, Hyperactivity and Conduct subscales, provides support for discriminant validity of the four SDQ factors. If replicated, our findings suggest that interpretation

6 114 R. Bull et al. Figure 2. Four factor MTMM model. Pros = Prosocial, cond = Conduct, inter = Internalizing, hype = Hyperactivity, pos = positively phrased items, neg = negatively phrased items. Cronbach alpha for each scale was Prosocial = 0.767, Conduct = 0.591, Internalizing = and Hyperactivity =

7 SDQ in Singaporean kindergartners 115 of the SDQ should focus on the four substantive constructs identified in the present study, rather than the original five factors. Our results also suggest that there are gender-related differences in the mapping of the SDQ items; the difficulty indicators were more likely to load on their respective difficulty factor for boys than for girls. However, the reverse was true in the case of the prosocial items. Previous studies have reported higher item-factor loadings for boys than for girls (Hawes & Dadds, 2004). Niclasen et al. (2012) found that individual items generally loaded on the same factors across genders, but that reliability estimates of the subscales were higher for boys compared to girls. These findings suggest that parents of boys and girls are not interpreting or responding to items in the same way. For these reasons, when making decisions on social and emotional risk, comparisons of scores within each gender, rather than across boy and girls, are recommended, as direct comparisons of scores for boys and girls may not be meaningful. With regard to cultural differences, data from other East Asian contexts (e.g. Hong Kong, China) generally indicate greater difficulty and less prosociality than European countries. This is also seen in our Singaporean sample; mean scores generally show less prosocial behaviour and more difficulties on all subscales (particularly for boys), with the exception of Emotion. However, the normal score range (encompassing 80% of the sample) is very similar to the score range for the UK sample. The only noticeable differences are for Conduct (where parents generally report more difficulties, and hence, a higher score is required to be classified in a risk category) and Emotion (where parents tend to report fewer difficulties). It is not clear whether these variations reflect real cultural differences or whether they are because of differences in the way the various items map onto underlying constructs. Putting aside the factor analytical differences, cross-national differences in scores do not necessarily reflect comparable differences in the prevalence of difficulties. Mieloo et al. (2013) indicated that there are differences in the validity and reliability of the SDQ subscale scores across different ethnic groups, which may be because of some behaviours being valued differently across cultures Outward display of aggression is discouraged in Asian countries, while self-control, emotional restraint and social inhibition are encouraged (Woo et al., 2007). This may help explain why in Asian samples the behaviours encompassed under Conduct and Hyperactivity remain distinct, rather than being subsumed under one externalising factor; Conduct items represent outward aggression and bad behaviours, whilst Hyperactivity items do not have the same aggressive element. Such cultural factors may greatly influence children s manifestations of emotional and behavioural problems, and parent thresholds of what they consider to be problematic behaviours (Weisz et al., 1988). This suggests that if the SDQ is to be used for screening purposes, the cut-off scores need to be culturally specific to take account of typical expectations regarding children s social and emotional behaviours. As there are very few measures standardized for local use, the data presented here will provide a reference point against which local practitioners can gauge a child s strengths and difficulties. Data from future studies will be used to confirm the factor structure and the proposed risk boundaries, and will be expanded to consideration of both teacher and parent reports. Conclusion Emotional and behavioural difficulties developed in childhood show stability across time and can progress into adult psychiatric disorders. This makes it particularly important that there are instruments available to assess for behavioural and emotional problems early in development and which are relevant to that culture. The present findings provide support for a modified factor structure of the SDQ in line with findings from other Asian populations. The SDQ may be a viable scale for screening and highlighting for further assessment those children at risk of emotional and behavioural difficulties. Our findings also suggest that comparisons of scores across gender and countries must be performed with caution. Differences in factor structure across populations may reflect differing conceptions of the underlying dimensions or differences in setting and normative expectations. Key messages Confirmatory factor analysis was used to test the fit of three previously identified factor structures of the SDQ. The best-fitting model was a multi-trait multi-method model consisting of four trait factors and two method factors. The structure was different across gender. The findings are similar to those found in other Asian community samples but indicate that cross-country and gender comparisons should be made with caution because of different parental expectations of appropriate child behaviour. The SDQ may provide an efficient tool for communitybased screening and identification of at-risk children in Singapore.

8 116 R. Bull et al. Acknowledgements This research was partially funded by grants from the Office of Educational Research (OER16/12RB and OER15/08SW). Views expressed in this article do not necessarily reflect those of the National Institute of Education. References Dickey, W. C. & Blumberg, S. J. (2004) Revisiting the factor structure of the Strengths and Difficulties Questionnaire: United States. Journal of the American Academy of Child and Adolescent Psychiatry, 43, Du, Y., Kou, J. & Coghill, D. (2008) The validity, reliability and normative scores of the parent, teacher and self-report versions of the Strengths and Difficulties Questionnaire in China. Child and Adolescent Psychiatry and Mental Health, 2. Available at: capmh.com/content/2/1/8 (last accessed 9 April 2013). Goodman, R. (1997) The Strengths and Difficulties Questionnaire: a research note. Journal of Child Psychology and Psychiatry, 38, Hawes, D. J. & Dadds, M. R. (2004) Australian data and psychometric properties of the Strengths and Difficulties Questionnaire. Australian and New Zealand Journal of Psychiatry, 38, Hill, C. R. & Hughes, J. N. (2007) An examination of the convergent and discriminant validity of the Strengths and Difficulties Questionnaire. School Psychology Quarterly, 22, Lai, K. Y. C., Luk, E. S. L., Leung, P. W. L., Wong, A. S. Y., Law, L. & Ho, K. (2010) Validation of the Chinese version of the strengths and difficulties questionnaire in Hong Kong. Social Psychiatry and Psychiatric Epidemiology, 45, Liu,S.K.,Chien,Y.L.,Shang,C.Y.,Lin,C.H.,Liu,Y.C.&Gau,S.S.F. (2013) Psychometric properties of the Chinese version of Strength and Difficulties Questionnaire. Comprehensive Psychiatry, 54, Loeber, R. & Farrington, D. (2000) Young children who commit crime: Epidemiology, developmental origins, risk factors, early interventions, and policy implications. Development and Psychopathology, 12, Matsuishi, T., Nagano, M., Araki, Y., Tanaka, Y., Iwasaki, M., Yamashita, Y. & Kakuma, T. (2008) Scale properties of the Japanese version of the Strengths and Difficulties Questionnaire (SDQ): a study of infant and school children in community samples. Brain & Development, 30, McCrory, C. & Layte, R. (2012) Testing competing models of the Strengths and Difficulties Questionnaire s (SDQ) factor structure for the parent-informant instrument. Personality and Individual Differences, 52, Mieloo, C., Raat, H., van Oort, F., Bevaart, F., Vogel, I., Donker, M. & Jansen, W. (2012) Validity and reliability of the Strengths and Difficulties Questionnaire in 5 6 year olds: differences by gender or by parental education? PLoS ONE, 7, e Mieloo, C., Bevaart, F., Donker, M., van Oort, F., Raat, H. & Jansen, W. (2013) Validation of the SDQ in a multi-ethnic population of young children. European Journal of Public Health, 24, Muthén, B. O. & Muthén, L. K. (2013) Mplus (Version 7.1). Muthén & Muthén, Los Angeles, CA, USA. Niclasen, J., Teasdale, T. W., Andersen, A. N., Skovgaard, A. M., Elberling, H. & Obel, C. (2012) Psychometric properties of the Danish Strengths and Difficulties Questionnaire: the SDQ assessed for more than 70,000 raters in four different cohorts. PLoS ONE, 7, e Ooi, Y. P., Ang, R. P., Ibrahim, N. H., Koh, D., Lee, P. Y., Ong, L. P., Wong, G. & Fung, D. S. S. (2014) The continued development and practice of school psychology in Singapore: using REACH as an illustration. School Psychology International, 35, Sanne, B., Torsheim, T., Heiervang, E. & Stormark, K. M. (2009) The Strengths and Difficulties Questionnaire in the Bergen child study: a conceptually and methodologically motivated structural analysis. Psychological Assessment, 21, Schreiber, J. B., Nora, A., Stage, F. K., Barlow, E. A. & King, J. (2006) Reporting structural equation modelling and confirmatory factor analysis results: a review. Journal of Educational Research, 99, Smedje, H., Broman, J. -E., Hetta, J. & von Knorring, A. -L. (1999) Psychometric properties of the Swedish version of the Strengths and Difficulties Questionnaire. European Child & Adolescent Psychiatry, 8, Shojaei, T., Wazana, A., Pitrou, I. & Kovess, V. (2009) The Strengths and Difficulties Questionnaire: validation study in French schoolaged children and cross-cultural comparisons. Social Psychiatry and Psychiatric Epidemiology, 44, Stone, L. L., Otten, R., Engels, R. C., Vermulst, A. A. & Janssens, J. M. (2010) Psychometric properties of the parent and teacher versions of the Strengths and Difficulties Questionnaire for 4- to 12-Year- Olds: a review. Clinical Child and Family Psychology Review, 13, Van Roy, B., Veenstra, M. & Clench-Aas, J. (2008) Construct validity of the five-factor Strengths and Difficulties Questionnaire (SDQ) in pre-, early, and late adolescence. Journal of Child Psychology and Psychiatry, 49, Weisz, J. R., Weiss, B., Walter, B. R., Suwanlert, S., Chaiyasit, W. & Anderson, W. W. (1988) Thai and American perspectives on overcontrolled and undercontrolled child behaviour problems exploring the threshold-model among parents, teachers, and psychologists. Journal of Consulting and Clinical Psychology, 56, Woo, B. S. C., Ng, T. P., Fung, D. S. S., Chan, Y. H., Lee, Y. P., Koh, J. B. K. & Cai, Y. (2007) Emotional and behavioural problems in Singaporean children based on parent, teacher, and child reports. Singapore Medical Journal, 48,