CHAPTER 7 VALIDITY 7.1 INTRODUCTION THE CONCEPT OF VALIDITY THE METHOD OF DETERMINING VARIDITY TYPE OF VALIDITY...

Transcription

1 235 CHAPTER 7 VALIDITY CONTENTS PAGE 7.1 INTRODUCTION THE CONCEPT OF VALIDITY THE METHOD OF DETERMINING VARIDITY TYPE OF VALIDITY CONTENT VALIDITY CONSTRUCT VALIDITY CONCURRENT VALIDITY SUMMARY OF THE VALIDITY OF THE SCALE REFERENCE CHAPTER 7 254

2 236 CHAPTER 7 VALIDITY 7.1 INTRODUCTION : A test is expected to prove its worth. If the test does not fulfil its worth, it is not worth of anything. Ross and Stanley (1963)* have rightly pointed out: : Although high reliability is no guarantee that the test is good, low reliability does not indicate that it is poor. Validity is always the first to be sought in a test, and granted that reliability is a valuable auxiliary." In the process of scale construction and standardization, proof of justification of purpose is known as the scale validation. Consequently, validation of a test score is the most important and significant step in the process of standardization of any scales. Most of the users before selecting the scale for the use, look carefully into the values of validity, therefore, the constructor of the scale should make clear the concept of the validity. 7.2 THE CONCEPT OF VALIDITY : The validity is an important characteristic of the scale. It depends upon the efficiency with which it measures that it attempts to measure. In other words it is defined as the accuracy with which the scale measures what it claims to measure. The term validity and purpose are very closely associated with each other. A scale which fulfils the purpose for which it is designed is called a valid scale. This led to say that work culture scale should measure the work culture and nothing else. Therefore, in the course of valid work culture scale, the primary school students who have strongly agreement

3 should get more scores than those who are slightly agreement. Garrett (1965)2 has rightly put it as: 237 "The validity of the test or of any measuring instrument, depends upon the validity with which it measures what it purports to measure." Freeman (I960)2 has defined validity index as: "An index of validity shows the degree to which a test measures what it purports to measure, when compared with accepted criteria." Freeman suggested that for validating the test, it must be compared with some accepted standards or other criteria which are regarde by experts as the evidence of the traits or ability to be measured by the scale. Consequently, the selection of validation criteria is of the prime importance in the process of a scale validation. 7.3 THE METHODS OF DETERMINING VALIDITY : The scale constructors often speak of operational validity and functional validity. Some constructors speak of empirical and rational validity. All these types of validities are discussed in the present chapter. Procedures for determining validity of the scales are primarily concerned with the relationship between performance on the scale and other independently observable facts about the behaviour characteristics under consideration. There are number of techniques that are described as various techniques. Anastasi (1970)* has presented them as follows: "The validity technique recommendations has classified these procedures under four categories, designated as content, predictive concurrent and construct validity. Out of these four categories of validity the two, namely content and construct or concept validity are described under the headings of rational validity, by many authors.

4 238 Similarly concurrent, predictive and congruent validity are described under the heading of empirical or statistical validity. In these methods the validity is estimated by means of statistical technique". 7.4 TYPE OF VALIDITY Fundamentally all procedures for determining the test validity are concerned with the relationship between performance on the test and other independently observable facts about the behaviour characteristic under consideration. The techinque that are employed for investigating relationship are many and have been described by various names. Anastasi (1970) have rightly put: "The APA Technical recommendations has classified these procedures under four categories, designed as content, predictive, concurrent and construct validity". Out of these four categories of validity the two, namely content and construct or concept validity are described under the headings of rational validity, by many authors. Similarly concurrent, predictive and congruent validity are discribed under the heading of empirical or statistical validity. In these methods the validity is estimated by means of statistical techniques. The high reliability is the first esential quality of a valid test. Several procedure to find out the index of validity are available. They are different types of method for establishing the validity of the scale. 1) CONTENT VALIDITY 2) PREDICTIVE VALIDITY 3) CONSTRUCT VALIDITY 4) CONCURRENT VALIDITY

5 239 5) CONGRUENT VALIDITY 1) CONTENT VALIDITY Content validity is also known as logical or curricular validity, which is based on judgement. The test maker may ask a number of experts to examines the items systematically and indicate whether or not they represent sufficiently well there theoretical universe form which they were drawn. All the major aspects of the content area must be adequately covered by the test items, and they must be in the correct propertions. All the major objectives of instruction must be included, as well as the subject matter. 2) PREDICTIVE VALIDITY Predictive validity refers to the extent to which a test can predict the future performance of individuals. One say a test has predictive validity when it can effectively indicate how a person will do on a latter task. This type of validity.is especially important for tests that are used for selection and classification purposes, such as tests that have been constructed for the selection of applicant who wish to enter colleges and universities. 3) CONSTRUCT VALIDITY Construct validity refers to the extent to which a test reflects constructs presumed to underline the test performance and also the extent to which it is based on theories regarding these constructs. Instead of asking how well this test samples a content area, or how well it predicts future performance, we ask how well the relationship between the test items and the corresponding behaviour could be explained. Thus this kind of validity requires the use of hypotheses drawn from theories regarding the nature of qualities that are to be

6 240 measured and from observations and experiences with these traits and qualities. In fact, construct validity, developed out of researcher s needs for a way of validating the instruments they were using in theory development. The problem of validity arises because psychological measurement is indirect. Under such circumstances, it is never possible to be completely certain that a test measures the precise characteristics for which it is designed. Thus, it is always necessary to gather some sort of evidence which provides confidence that a test score really represent what in pears to represent. The determination of the extent to which a text is affected by the constant errors involves a varidity of measurment procedure, representing the validity of the test. So attempts are made to establish the validity of the scale. 4) CONCURRENT VALIDITY Anastasi Anne (1970) have rightly put : "Evidence of validity may be obtained from the relationship with other currently obtainable information about an individual the relationship between test-scores Mid indices of criterion status obtained at approximately the some time is known as concurrent validity. Generally the criterion status obtained by asking teachers who are in close contact with the pupils to give their estimate about the pupils ability under the investigation. Rating s have been emp^ytp in the validation of almost every type of test". Their estimates of rating will serve as the criterion. The correlation between teacher s estimate and scores on the test is then computed. The obtained correlation is known as the co-efficient of validity. 5) CONGRUENT VALIDITY This type of validity is essentially estimated by the means of a

7 241 statistical technique. For this, the set of scores on the present scale is correlated with the set of criteria of a similar measure. It means, it is correlated with some available well known powerful scale of the similar nature. The correlation of the new scale with the existing scale would show to what extent the two scales measure the same behaviour characteristics. If the correlation coefficient is very high between these two sets, it is inferred that the new scale is valid. Since it measure what the criterion scale is supposed to measure. The type of evidence just proposed is some what circular for this condition is that the criterion test must be fully valid otherwise the correlation between two scales would not be much dependable and this type of validity would be misleading. The researcher could not find out any such work culture scale for the primary school students in Thai language. So the researcher could not establish the congruent validity. For the present scales, the content validity, construct validity and concurrent validity were employed to estimate the validity indix by the statistical techniques. The scale constructor has established the following types of validity for the present scale. 7.5 CONTENT VALIDITY The method to examine the content validity based on the judgement of twenty experts. For establishing this type of validity, the researcher took the help of experts in the field of sociology, psychology, education, and lawyer, who know right type of evaluation process for analysing the statements of the scale. After the careful analysis of the scale statements describing the behaviour characteristics of work culture with the mentioned experts. The basic characteristics of work culture

8 242 presented in the form of statements were approved by the experts. 7.6 CONSTRUCT VALIDITY: In the case of determining the construct validity the first task is to define the measure. In the present scale the work culture should be defined. Thorndike and Hagen (1957) have rightly explained what the phrase 'Construct* really mean: "Again we are thrown back on rational analysis, but this time we are trying to analyse concept and see what is implied by it, rather than to make a catalogue of content." For establishing the construct validity of the present scale, the concept of work culture was translated into components. According to the term work culture, it is analysed in the behaviour components. It is against this analysis the researcher should check her own scale to see the construct validity. Thorndike and Hagen (1957) have rightly put: "The analysis and the evaluation are now concerned not with content or subject matter acted upon but with the function of process that are applied to some content." For estimating the construct validity, the Item-Test Correlation was applied to the data obtained from 100 primary school students. The Pearson Product Moment was calculated for the *rxy value from the total scores and the score of each item. Table 7.1 presents the 'rxy* value calculated by the item-test correlation method.

9 243 TABLE 7.1 THE CONSTRUCT VALIDITY OF THE WORK CULTURE SCALE CALCULATED BY PEARSON PRODUCT MOMENT FOR THE ITEM-TEST CORRELATION ITEM NO. rxy ITEM NO. t xy ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** 0.05 a = 0.25 * s SIGNIFICANT AT 0.01

10 244 TABLE 7.1 (CONTINUED) ITEM NO. rxy ITEM NO. r xy ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** = 0.25 * = SIGNIFICANT AT 0.01 From table 7.1, it can be seen that the score of each item obtained from 100 primary school students indicates the *rxy value between which were significant at 0.01 level.

11 CONCURRENT VALIDITY: The evidence of the validity be obtained from the relationship with other currently obtainable information about an individual. Anastasi (1970)^ has defined the concurrent validity as follows: "The relation between test scores and indices of criterion status obtained at approximately the same time is known as concurrent validity." The criterion status is obtained by asking the professors of the respondents who were in close contact with the respondents to give their estimation about the work culture of primary school students under investigation. This estimation about the work culture of the primary school students were considered to be the criteria with which the score obtained on present scale was compared with the opinion of teachers. The obtained coefficient of correlation between two sets of score indicates the measures of concurrent, validity. The obtained correlation is known as the coefficient of validity. For establishing concurrent validity of the present scale two techniques were used: Technique A and Technique B. In technique A, the rating done by primary school teachers who were in close contact of primary school students. In technique B, peer rating technique was used. It was decided to administer the work culture scale to 100 primary school students in Bangkok who situated in different areas, AND sex. The representive sample for the calculation the validity is presented as follow :

12 246 TABLE 7.2 THE REPRESENTATIVE SAMPLE FOR THE CALCULATION OF THE CONCURRENT VALIDITY OF THE WORK CULTURE SCALE SEX AREAS OF SAMPLE TOTAL BOYS GIRL Pranakhon District Nongkhem District Dusit District Bangkoknoi District Bangkokyai District Thonburi District Chatujak District Prakhanong District Bangplat District Talingchan District Total

13 247 TECHNIQUE : A : PRIMARY SCHOOL TEACHERS* RATING: For getting the rated work culture of the primary school students by the respective teachers. The teachers were asked to read each statement and rate the students keeping his/her work culture characteristics before him. Each teacher was fully explained how to rate the student about their work culture. Over the evaluative proforma duly filled in for all individual student, were then collected from their respective teachers. The response categories were converted into scores by assigning weight of 3 to A, 2 to B and 1 to C for positively worded evaluative statements and weight of 1 to evaluative statements. A, 2 to B and 3 to C for negatively worded The total score of each student was found out by summing up the scores earned by them on individual statements. The score would be between 63 and 189. In all for 100 students two sets of scores were obtained. The Pearson Product Moment (rxy) of the obtained score is presented in Table: No. 7.3.

14 TABLE THE SCATTER DIAGRAM OF TWO SETS OF SCORES OBTAINED BY THE PRIMARY SCHOOL TEACHERS RATING AND SELF RATING ON THE BASIS OF WORK CULTURE SCORES OF PRIMARY SCHOOL TEACHERS RATING CLASS Total INTERVAL SCORES OF SELF RATING ON THE BASIS OF WORK CULTURE r = 0.75** SE = X 1.96 = 0.08 for the level of X 2.58 = 0.10 for the level of 0.01 ** significant at 0.01 level r = range r = 0.85 to 0.65

15 249 It is observed from table 7.3 that the correlation coefficient between the two sets of scores is So correlation coefficient could not larger than 0.83 and could not smaller than 0.67 at 0.05 level of significant. In the same way the correlation coefficient could not larger than 0.85 and could not smaller than 0.65 at 0.01 level of significant. The obtained correlation between the students self eating and their teachers rating is highly significant at 0.01 level. Hence, it could be concluded that the scale has good concurrent validity.

16 250 TECHNIQUE : B : PEERS RATING : As discussed in the technique A, in establishing concurrent validity a correlation coefficient is established between the set of scores of the present scale and set of scores obtained on the evaluative proforma. Particularly for this the opinions of teachers or other individuals who know the subjects fully well, generally all the subjects are rated by a single individual who knows the subjects or a group of trained judges knowing the subject well. In the present situation, there was no similar well known standard scale available. Hence, the researcher converted the present self rating form of work culture scale into the peer rating form. Grammatically the subject parts of each of the statements were changed. The first person singular I* was substituted by the third person singular He/she. The rest of the part of the statements were kept as they were without substantial change in the content. Some illustrations are supplied here under: (i) Self Ratins form: I do not like to work hard. Peer-Rating form // \ // ). He/she ^gjpt like to work hard. (ii) Self Rating form I dislike to change the practice of my work. Peer-Rating form He/she dislike to change the practice of his/her work. (iii) Self Rating form I listen to other friend s idea. Peer-Rating form He/she listen to other friend s idea.

17 251 In the introduction form, the instructions and illustrations were little bit modified to suit the peer rating form. The transformed form of the present self rating form into peer rating form of work culture scale was then got of printed. After this a group of students was selected. They were explained the purpose, instructions and the illustration of the peer rating attempts. Each student was asked to fill up at least two booklets about their different classmates with whom they live, sit within the class and share views on work culture. After collection of the peer rating booklets, they were sorted out and filled booklets on the same students who were pinned together. The scores were subjected to the arithmetic mean. This means scores of each student was regarded as the criterion score. Thus, two sets of scores of each student were listed down in the form of pair. From the pair of scores of two sets, the correlation coefficient of validity was computed. The Pearson Product Moment (rxy) is presented in Table: 7.4

18 TABLE THE SCATTER DIAGRAM OF TWO SETS OF SCORES OBTAINED BY THE PRIMARY SCHOOL STUDENTS RATING AND PEERS* PRIMARY SCHOOL STUDENTS RATING ON THE BASIS OF WORK CULTURE SCORES OF PRIMARY SCHOOL STUDENTS CLASS Total INTERVAL SCORES OF PEERS PRIMARY SCHOOL STUDENTS r = 0.82** SE a X 1.96 = 0.06 for the level of X 2.58 = 0.08 for the level of 0.01 ** significant at 0.01 level r a range r = 0.90 to 0.74

19 253 It is observed from table 7.4 that the correlation coefficient between the two sets of scores is So correlation coefficient could not larger than 0.88 and could not smaller than 0.76 at 0.05 level of significant. In the same way the correlation coefficient could not larger than 0.90 and could not smaller than 0.74 at 0.01 level of significant. The obtained correlation between the students self rating and peers rating is highly significant at 0.01 level. Hence, it could be concluded that the scale has good concurrent validity. 7.8 SUMMARY OF THE VALIDITY OF THE SCALE The obtained validity of the scale determined by the construct validity and the concurrent validity (technique A and technique B) has shown in summary in table 7.5. TABLE 7.5 COMPARATIVE VIEW OF VALIDITY OBTAINED BY DIFFERENT MEfflODS No. METHODS OF VALIDITY VALIDITY VALUE 1. CONSTRUCT VALIDITY SIGNIFICANT AT 0.01 IN EVERY ITEMS 2. CONCURRENT VALIDITY - TECHNIQUE A r TECHNIQUE B r» From table 7.5, it can be seen that the validity value obtained by using different methods are mostly indentical. It can concluded that the validity of the work culture scale are quite highly satisfactory.

20 254 REFERENCE CHAPTER 7 1 Ross, C.C. and Standley. Meassurement in Today s School. (New York : Englewood, Cliffs, Prentice Hall, Inc.), Garrett, H.E. Statistics in Psychology and Education. (Bombay : Vakils, Feffer and Simons Private Ltd., 1965), p Freeman, George M. Theory and Practice of Psychological Testing. (New York : Holt, Rinehart and Winston, Inc., 1960), p Anastasi, Anne. Psychological Testing. (London : The McMillan Co.Collier McMillan), Thorndike, R.L. and Hagen, E. Measurement and Evaluation in Psychology and Education. (New York : John Wiley and Sons, Inc., 1957), p Thorndike, R.L. and Hagen, E. Measurement and Evaluation in Psychology and Education. (New York : John Wiley and Sons, Inc., 1957), p Anastasi, Anne. Psychological Testing. (London : The McMillan Co.Collier McMillan), Allen L. Edwards. Experimental Design in Psychological Research. (New York : McGraw-Hill Book Company, 1971) p Best, John W. Research in Education. (New Jersey : Prentice-Hall, 1977), p Cochran, William G. Saapling Technique. (New York : John Wiley & Sons. Inc., 1966) p Cronbach, L.J. Essentials of Psychological Testing. (New York : Harper and Row, 1966), p.46.

21 Ebel, R.L. Measuring Educational Achievement. New Delhi : Prentice Hall of India Pvt.Ltd., Flanagan, J.C. A Proposed Procedure for Increasing the Efficiency of Objective Tests. Journal of Educational Psychology, Ferguson George A. Statistical Analysis in Psychology and Education. (New York : McGraw-Hill Book Company), Green, H.A. and Other. Measurement and Evaluation in Secondary School. (New York : The Prentice Hall Company, 1957), p Guilford, J.P. Psychometric Method. (New York : McGraw Hill Book Company, Inc., 1954), p Kerlinger, Fred N. Foundations of Behavioral Research. (Englewood, Clifts N.J.Prentice Hall Inc., 1968). 18 Lado, Robert. Language Testung-The Construction and use of Foreign Tests ; A Teachers Book. London : Longman Green and Co.Ltd., 1962, P Lindquist : Measurement in Education. (Washington D.C. : American Council of Education, 1959), p Manroe, W.S. Encyclopedia of Educational Research. (New York : The Prentice Hall Company, 1959), p Martin Fishbein. reading in Attitude Theory and Measurement. (New York : Jonh Wiley and Sons Inc., 1967), P Pungrat Thaverat. Research in Behavioural Social Science. (Bangkok : Book Centre Chulalongkorn University, 1997), p

22 23 Remmers, M.H. and Gage, N.L. A Practical Introduction to Measurement and Evaluation. (Delhi : Delhi University Book Stall, 1967), p Travers, Robert M.W. An Introduction to Educational research. (New York : McMillan), Van Dalen, Deobold B. Understanding Educational Research. (New York : McGraw-Hill Book Company), Walker, H.M. and Lev, J. Elementary Statistical Method. (New York : Oxford and IBH Publishing Co., 1971), p Yamane, Taro. Statistics : An Introductory Analysis. (New York Harper and Row), 1967.