FACTOR ANALYSIS OF THE ABILITY TO COMPREHEND TIME-COMPRESSED SPEECH 1

Transcription

1 40 Journal of Reading Behavior Vol. 4, No. 1, Winter FACTOR ANALYSIS OF THE ABILITY TO COMPREHEND TIME-COMPRESSED SPEECH 1 Ronald P. Carver, Raymond L. Johnson, and Herbert L. Friedman* Abstract Sources of individual differences in the listening comprehension of speech presented at different rates were investigated by factor analyzing 11 variables, including measures of comprehension and field-independence. Twenty-two different tests, two for each variable, were administered to 50 college students. It was concluded that: (a) the cloze type of test includes a large component of variance unrelated to comprehension and quite specific to the technique itself, and (b) the comprehension of highly speeded speech probably involves a perceptual ability to be field-independent in addition to a separate ability involved in comprehending speech at normal rates. The correlates of the ability to comprehend time-compressed speech have been studied (see Friedman and Johnson, 1968, and Johnson and Friedman, 1970), but data relevant to the extent to which this ability can be distinguished from the ability to comprehend normal speech are sparse. The ability to comprehend time-compressed speech may depend upon the ability of the individual to disregard the unique stimulus characteristics of speeded speech and attend more to meaning or content, i.e., be more field-independent (Witkin, Dyk, Faterson, Goodenough and Kamp, 1962). Field-dependency is a theoretical construct which has been introduced to explain the difficulties of some people in recognizing familiar stimuli when embedded in unfamiliar contexts, in "keeping things apart" in the perceptual field, in imposing order and avoiding disruptive stress responses in unfamiliar or novel situations. Field-dependent people tend to be stimulus-bound, unable to differentiate the variant from the invariant. The ability to understand compressed speech would seem to require the listener to remain undistracted by the superfluous novelty of speeded speech, and to attend more to the content of the message than to the rate of presentation. It was hypothesized that field-independent listeners would be better able to cope with time-compressed speech than field-dependent listeners. The two primary purposes of the following research were: (a) to investigate the sources of individual difference in the comprehension of speech presented at different rates, and (b) to explore the relationship between field-independency and the ability to comprehend * Dr. Carver, Dr. Johnson, and Dr. Friedman are on the staff of The American Institutes for Research, Washington Office. 1 1 This research was supported by a General Research Support Grant, National Institutes of Health, and by the American Institutes for Research.

2 speeded speech. 2 This investigation was accomplished by factor analyzing the correlations among eleven variables selected to include a comprehension factor and a field-independency factor. Method Subjects Fifty college student volunteers were paid to participate in the study. All were native speakers of English and were without hearing defects. Procedure The Ss were tested in three groups on three consecutive Saturday sessions. Upon arrival, the Ss were given a schedule for the testing session and a form which contained the dollar amount of bonus for each correct answer on each test. The 5s bonuses were in addition to their $4.00 minimum payment. The Ss were asked to do their best on each test and were also informed that their tests would be graded during the testing session so that their dollar amount of bonuses earned could be given to them at the completion of the testing. To provide control over fatigue, refreshments were provided during three breaks in the four and one-half hours of testing. To insure uniform procedures, all instructions for all tests were tape recorded, with opportunity given for Ss to ask questions. Tests Passages and test items from Form A and Form B of the Carver- Darby Chunked Reading Test (CDCRT) 3 were used to make five listening test variables. The CDCRT was designed to measure information storage during reading and has been suggested as a good measure of comprehension (Carver, 1970a; Carver 1970b; Carver & Darby, 1970a; Carver & Darby, 1970b). The words contained in each reading passage have been grouped, i.e., "chunked," with one to five words in each chunk. In a consecutive set of five chunks, one chunk has been replaced on the test with a different chunk. The task for S is to recognize the incorrect chunk on the test passage. The inserted chunks were written to change the meaning of the listening passage, and the incorrect chunks have been empirically developed to discriminate between readers and nonreaders of the original passage. Each of the ten passages on the two forms of the CDCRT contains exactly 100 chunks but the passages vary from 310 to 356 in total words. The passages 2 The general idea for exploring the relationship between compressed speech and field-independence was contributed by Johnson and Friedman while Carver was largely responsible for the other primary purpose as well as the design, analysis, and reporting of the research. 3 The Carver-Darby Chunked Reading Test is a publication of the American Institutes for Research, Washington Office,

3 42 were tape recorded by a professional announcer and then the passages were time-compressed to different durations. Table 1 contains the durations and word rates for the subtests that comprise the five chunked listening comprehension variables Slowed Chunked, Normal Chunked, Speeded Chunked 1, Speeded Chunked 2, and Highly Speeded Chunked. Table 1 Descriptive Characteristics of the Five Chunked Listening Comprehension Variables Variables Duration (minutes) Form A Word Rate (words/min.) CDCRT Passage No. Duration (minutes) FormB Word Rate (words/min.) CDCRT Passage No. Slowed Rate Normal Rate Speeded Rate 1 Speeded Rate 2 Highly Speeded The following four test variables all involved taking a cloze test on a passage. The passages were taken from the Miller and Coleman (1967) set of passages which have been scaled for readability. The cloze tests on the passages for this study were constructed by deleting every fifth lexical work (i.e., noun, verb, adverb, adjective) and an eight minute time limit was allowed for each test. In the succeeding descriptions of tests the particular passage used has been identified by its order position as it has been presented by Aquino (1969). In parentheses following the number identification, the readability of each passage has been indicated in percent (i.e., the percent correct responses on the "every fifth word deleted" measure of readability). The Fill-in Cloze variable consisted of filling in the blanks on Passage 19 (55.7%) and Passage 25 (47.0%) without having the benefit of listening to the passages prior to completing the cloze items. Tests on both passages contained twenty items each. The Normal Cloze variable consisted of Passage 23 (53.2%) and Passage 22 (56.2%) tape recorded and played at the normal rate, 152 and 160 wpm, respectively. Tests on both passages contained twenty items each. The Highly Speeded Cloze variable consisted of Passage 20 (55.7%) and Passage 21 (58.7%) tape recorded and played at the highly speeded rates of 428 and 391 wpm, respectively. Passage 20 had a twenty-item corresponding cleze test and Passage 21 had a nineteen item test. The Competing Messages Cloze variable consisted of Passage 15

4 (63.0%) and Passage 17 (63.0%) played at a normal rate with two twenty-item cloze tests on the content. Passage 35 (31.7%) was played at the same time on the same speaker as Passage 25, and Passage 32 (35.8%) was played at the same time on the same speaker as Passage 17. Immediately prior to the presentation of the listening materials, Ss were told the general topic of both the test passage and the distracting passage and 5s were also instructe'd to try to disregard the distracting passages and pay attention only to the content of the test passage. Flowers and Crandall (1967) advanced the notion that this competing messages task taps a central auditory ability. The competing messages type of test was constructed for the present study to produce an alternative measure of field-independence using the auditory mode of connected discourse as test content. That is, it was advanced as an a priori hypothesis that the ability to listen to the test passage without being distracted by the superimposed passages would involve an ability to separate the stimulus from its field, i.e., be field-independent. The following two tests were from the battery of Reference Tests for Cognitive Factors (French, Ekstrom and Price, 1963) and are conventional traditional paper and pencil tests of field-independency. The Hidden Figures Test variable is a thirty-two-item test administered in two separate and equal parts. Each part contains sixteen embedded figures which must be detected and marked using a five-choice multiple-choice item. Each part of the test is of ten minutes duration and is timed separately. The Hidden Patterns Test variable present geometric line shapes which requires S to respond with a check mark if a master shape is contained in the test shape. This test is also administered in two parts with a two-minute time limit for each part which contains 200 patterns. Table 2 Testing Schedule 43 8:30 8:35 9:35 9:40 10:20 10:25 10:50 11:15 11:35 11:50 12:15 12:20 12:55 Introductory Remarks Chunked Listening Test (Form A) Break Chunked Listening Test (Form B) Break Two Fill-in Cloze Tests Two Normal Cloze Tests Two Highly Speeded Cloze Tests Break Hidden Figures Test Hidden Patterns Test* Two Double Talk Cloze Tests Completion of Testing (Award of bonuses) * Subjects were given 1 per correct answer on all tests except the Hidden Patterns Test which had a bonus payment rate of 10 per 10 correct answers.

5 44 Design Table 2 contains the schedule for testing. This schedule presents the order for the administration of each test as well as approximate durations for each test. Results and Discussion Table 3 presents the intercorrelations, means, and standard deviations for the eleven test variables. The score for each variable was the raw score sum of the two subtests comprising each variable. Table 4 contains the results of three analyses. A principal component factor analysis was performed on the data and these results are presented as Analysis 1 in Table 4. Both the scree test (Cattell, 1966) and the number of eigen values greater than one (Kaiser, 1960) resulted in the decision to extract and rotate (varimax) three factors, and these factors accounted for 72.3% of the variance. TO aid in the interpretation of the factors, the rotated oblique primary factor loadings were also calculated and these are presented as Analysis 2 in Table 4. Notice in Analysis 2 that the five chunked listening comprehension variables define Factor 1 as a comprehension Factor. Factor II is clearly a Field-Dependency factor since it loads mainly on the Hidden Pattern Test and Hidden Figures Test variables. Factor III is a factor that involves an ability to fill in the missing parts of a passage and this factor has been labeled a Cloze factor. It should be noted that every variable that involved a cloze test loaded on Factor III. Table 3 Intercorrelations :, Means, and Standard Deviations for the Eleven Variables (N = 50) Variables Slowed Chunked Normal Chunked Speeded Chunked 1 Speeded Chunked 2 Highly Speeded Chunked Fill-in Cloze Normal Cloze Highly Speeded Cloze Hidden Figures Test Hidden Patterns Test Competing Messages Cloze l Mean S.D It was somewhat surprising that the cloze comprehension tests and the chunked comprehension tests loaded mainly on separate factors. However, Weaver and Kingston (1963) found a similar separation between traditional comprehension tests and clpze tests using both standardized reading and listening tests. Although their reading and listening cloze tests were not comprehension tests (since they did not

6 Table 4 Three Analyses of the Relationships among the Eleven Test Variables (N = 50) Analysis 3** Analysis 1 Analysis 2** Factor Load- Varitnax Factor Load- Rotated ings from ings Orthogonally Oblique Analysis 1 Extracted by Principal Primary Factor Corrected for Component Method Loadings Attenuation Variables r xx * I II III h= I II III I II III Slowed Chunked Normal Chunked Speeded Chunked Speeded Chunked Highly Speeded Chunked Fill-in Cloze Normal Cloze Highly Speeded Cloze Hidden Figures Test Hidden Patterns Test Competing Messages Test * Reliability of test as estimated by the Spearman-Brown Formula. ** Only loadings above.30 have been included. allow Ss to read or listen to the test passages in an undeleted form), their results might be expected to be similar to the present results due to the usually high correlation found between cloze tests given with and without an opportunity to comprehend the passage (see Rankin, 1965); The chunked type of test has been shown to measure changes in reading comprehension better than a traditional standardized test (Carver and Darby, 1970a), yet individual differences on this test still correlate highly with other traditional comprehension tests (Carver and Darby, 1970b). Thus, the present results seem to be compatible to those of Weaver and Kingston. Bormuth (1969) factor analyzed traditional multiple-choice comprehension variables and cloze variables and found that one factor fit adequately. Bormuth compared this result to that of Weaver and Kingston and explored certain differences between the two studies that might account for the discrepancy. The present results do little to explain why results of past studies are inconsistent, yet these data do tend to suggest that the ability to complete the cloze items is somewhat different from the ability to do well on other measures of comprehension. It would seem reasonable that the ability to fill-in the missing one-fifth of a passage, given four-fifths, would involve skills highly related to but separate from the ability to comprehend while reading or listening. In the present study, the Cloze factor was empirically separated from the Comprehension factor, yet these two factors correlated.47 in the oblique analysis presented in Analysis 2 of Table 4. This correlation,.47, is comparable to the corresponding.61 corre- 45

7 46 lation between these two factors for the Weaver and Kingston (1963) data, as determined by a re-analysis of their reported correlation matrix. Hopefully, future research and rationale will adjudicate the precise status of the cloze technique when measuring comprehension. At present, it appears that the cloze technique when used as a measure of comprehension, includes a large component of variance which is specific to the cloze technique itself. The relationship between the ability to do well on cloze tests and the ability to comprehend can be analyzed in more detail via the orthogonal factors in Analysis 1 which force a zero correlation between factors. That is, the orthogonal loadings permit each test variable to be analyzed into its uncorrelated component parts. However, instead of directly interpreting the orthogonal factor loadings in Analysis 1 of Table 4, another analysis was completed which allows the unreliability of the tests to be taken into account. Table 4 also includes the reliability estimates from each variable. These were calculated by substituting the correlation between the two subtests, which comprise each of the eleven variables, into the Spearman-Brown formula. Notice that the reliabilities estimates ranged from.32 to.90. With this range of reliabilities, the direct interpreta^ tion of the sizes of factor loading in Analysis 1 seemed undesirable since factor loadings depend directly upon the size of the correlations among the test variables, and these correlations are in turn directly affected by the reliability of each variable. Therefore, Analysis 3 was completed which corrects the loadings in Analysis 1 for attenuation due to the unreliability of the test variables. This correction is possible since the loadings in Analysis 1 are the correlations between the test variable scores and the factor scores. (In those cases where the test variable correlated with another variable higher than the Spearman- Brown reliability estimate, the highest correlation was used as the reliability estimate.) Reliability, being an imperfect estimate, produced the situation wherein some of the loadings in Analysis 3 are greater than unity. Yet, these estimated loadings permit a more reasonable interpretation of the results, and at the same time serve to accent the relative unreliability of the loadings themselves. Each of the eleven test variables were again analyzed in terms of its loadings in Analysis 3. The first four chunked listening test variables load only on Factor I, the Comprehension factor. The size of these loadings (.99,.93,.98, and.98) suggest that these variables are pure measures of the Comprehension factor. The Highly Speeded Chunked variable loaded highly on the Comprehension factor (.88), but it also loaded substantially on the Cloze factor (.52). It seems reasonable that the Highly Speeded Chunked variable would load on the Cloze factor since at low levels of comprehension the chunked test must involve skills similar to those involved

8 in guessing the correct cloze fill-ins. That is, given little or no information regarding the nature of the original passage, the ability to recognize which one-fifth is incorrect on the chunked test would likely involve skills similar to those required for being able to fill in one-fifth of the words on the cloze test. The cloze test variable which involved no opportunity to comprehend, i.e., the Fill-in Cloze variable, loaded only on the Cloze- factor (1.03) and this result helps make the definition of the Cloze factor quite precise. The Normal Cloze variable loaded highest on the Cloze factor (1.10), yet it also loaded substantially on the Comprehension factor (.42). This latter result lends support for the definition of Factor I as a Comprehension factor. The Highly Speeded Cloze variable was the only variable which loaded on all three factors. Its primary loading was on the Field- #Independency variable (.72). It also loaded quite high on the Cloze factor (.63), which is unsurprising since the test was a cloze type task. The loading of.45 on the Comprehension factor suggests that the ability to do well on this test also involved a substantial amount of comprehension ability. The Hidden Pattern Test variable and the Hidden Figure Test variable define Factor II by their high loadings,.85 and.95, and they load on no other factor. The Competing Messages Cloze variable loads mainly on the Cloze factor which might have been expected since the tests on the competing messages listening passages were cloze tests. As was also expected, the ability to do well on the competing messages test involved the ability to be field-independent. Although the above relationships might have been expected, the sizes of the loadings were not. The competing messages test was designed to provide an auditory measure of fieldindependency. Yet, its loading on this factor was small,.43. It is impossible to evaluate how well it achieved its designed objectives since the Cloze factor accounted for the predominant portion of its variance (1.03 loading). Also, not expected was the lack of a loading on the Comprehension factor, which suggests that the ability to understand the competing messages type of presentation does not require any of the normal comprehension ability. In general, the above loadings for each variable serve to substantiate the interpretations given to each factor and are in agreement with what might be expected from an a priori consideration of each variable. The detailed analysis of the loadings also supports the previous interpretation that the cloze type of comprehension test mainly involves an ability or skill which is related to the ability to comprehend but is separate from it. The fact that the cloze test given without the opportunity to comprehend did not load on the Comprehension factor, whereas the cloze test given with the opportunity to compre- 47

9 48 hend did load on the Cofhprehension factor, provides support for this interpretation. With regard to the relationship between speeded speech comprehension and field-independency, the evidence is not clear-cut, but is highly suggestive. The Highly Speeded Cloze variable loaded very highly on the Field-Independency factor (.72), whereas the Highly Speeded Chunked variable did not load at all on this factor. It is possible that the reason for this difference is due to the higher rates of the two cloze passages (428 and 391 wpm) as compared to the two chunked passages (345 and 351 wpm). However, an explanation that seems more plausible for the above discrepancy involves practice or experience in listening to speeded speech. Although the first four rates on the chunked listening passages may be perceived as simply incremental rate changes, the highly speeded version may be qualitatively different in that this high rate of speech is a distinctively novel experience for most individuals. It could be that the more field-independent, individuals are better able to comprehend the meaning embedded in this novel stimulus, but they are not able to do this upon the first exposure to the task. That is, the reason that the chunked test did not load on the field-independency factor may have been because this test happened to have been presented first, during the adaptation phase for these individuals. This effect of different factors being involved in different trials or stages of a task was first reported by Fleishman (1955). Certain empirical evidence in this study seems to support the above interpretation. Table 5 presents the correlations among the two field-independency variables and the four sub-variables involved in the two speeded speech test variables. Each correlation in Table 5 has been corrected for attenuation due to the unreliability of the two variables involved in the correlation. Notice that the secondly administered variable of each subset involved a substantially higher correlation than the first, whether it be for the Hidden Pattern Test variable or the Hidden Figure Test variable. Notice also that for the Hidden Table 5 Correlations Among the Two Field-Independency Variables and the Four Highly Speeded Subtests of Comprehension (N = 50) Variable Highly Speeded Chunked (Form A) Highly Speeded Chunked (Form B) Highly Speeded Cloze (Passage 20) Highly Speeded Cloze (Passage 21) Presentation Order First Second Third Fourth Field-Independency Variable Hidden Patterns Test.22* Hidden Figures Test Each correlation has been corrected for the unreliability of the two variables.

10 Figures Test variable, the four correlations increase incrementally from the first presentation,.22, to the fourth presentation,.95, and for the Hidden Patterns Test variable the correlations almost perfectly parallel those of the Hidden Figures Test variable. On the basis of preceding research and the results of this study, it was concluded that: (a) the cloze test used as a measure of comprehension includes a large component of variance unrelated to comprehension and quite specific to the cloze technique itself, and (b) the ability to comprehend highly speeded speech probably involves an ability to be field-independent in addition to the separate ability involved in comprehending speech at normal rates. References AQUINO, M. R. The validity of the Miller-Coleman readability scale. Reading Research Quarterly, 1969, 4, BORMUTH, J. R. Factor validity of cloze tests as measures of reading comprehension. Reading Research Quarterly, 1969, 4, 34, CARVER, R. P. What is reading comprehension and how should it be measured? In G. B. Schick (Ed.) Nineteenth Yearbook of the National Reading Conference, Milwaukee: National Reading Conference, CARVER, R. P. Analysis of "chunked" test items as measures of reading and listening comprehension. Journal of Educational Measurement, 1970, 7, CARVER, R. P. and DARBY, C. A., Jr. Development and evaluation of a test of information storage during reading. Journal of Educational Measurement, 1970, in press. (a) CARVER, R. P., and DARBY, C. A., Jr. Analysis of the chunked reading test, 1970, Unpublished manuscript. (b) CATTELL, R. B. The scree test for the number of factors. Mullivariate Behavioral Research, 1966, 1, FLEISHMAN, E. A., and HEMPEL, W. E. The relation between abilities and improvement with practice in a visual discrimination reaction test. Journal of Experimental Psychology, 1955, 49, FLOWERS, A. and CRANDALL, E. W. Relations among central auditory abilities, socio-economic factors, speech delay, phonic abilities and reading achievement: A longitudinal study. Report. U.S. Office of Education Grant No Research in Education, No. ED , FRIEDMAN, H. L. and JOHNSON, R. L. Compressed speech: Correlates of listening ability. The Journal of Communication, 1968, 18, FRENCH, J. W., EKSTROM, R. B., and PRICE, L. A. Manual for kit of reference tests for cognitive factors. Princeton, N. J.: Educational Testing Service, June JOHNSON, R. L. and FRIEDMAN, H. L. Effects of temporal spacing on listening comprehension: a source of individual differences. Proceedings of Second Louisville Conference on Rate and/or Frequency Controlled Speech. Louisville, KAISER, H. F. The application of electronic computers to factor analysis. Educational and Psychological Measurement, 1960, 10, MILLER, G. C. and COLEMAN, E. B. A set of thirty-six passages calibrated for complexity. Journal of Verbal Learyiing and Verbal Behavior, 1967, 6, RANKIN, E. F., Jr. The cloze procedure a survey of research. In E. L. Thurston and L. E. Hafner, eds., Fourteenth Yearbook of the National Reading Conference. Milwaukee: National Reading Conference, 1965, WEAVER, W. W. and KINGSTON, A. J. A factor analysis of the cloze procedure and other measures of reading and language ability. The Journal of Communication, 1963, 13, WITKIN, H. A., DYK, L. B., FATERSON, H. F., GOODENOUGH, D. R., and KARP, S. Psychological differentiation. New York: Wiley,