Working memory and intelligence, looking at its relationship through Brunswik s lens. Werner W. Wittmann. University of Mannheim, Germany

Transcription

1 Working memory and intelligence, looking at its relationship through Brunswik s lens Werner W. Wittmann University of Mannheim, Germany Symposium Working memory and intelligence: Controversy or consensus organized by Phillip L. Ackerman, Georgia Institute of Technology Presented at APS - 15 th Annual Convention Atlanta, GA - May 29 June 1, 2003 The results presented here are based on extensive research by my Mannheim group on working memory, intelligence and complex computer based business games performances.the whole project was funded by the German Research Foundation (DFG) via grants Wi 1390/1-4. The most important coworkers are: Dr. Heinz-Martin Süß, now professor of psychology at University of Magdeburg, Germany Dr. Klaus Oberauer, now research associate at University of Potsdam, Germany Dr. Oliver Wilhelm, now junior professor at Humboldt University, Berlin Dr. Ralf Schulze, now assistent professor at University of Münster, Germany Dr. Florian Schmiedek, now research associate at the Max-Planck-Institute for Human Development, Berlin Nicolas Sander, Diplom Psychologe at University of Mannheim 1

2 Clinical prediction paradigm schematized by Brunswik s lens model (After Hammond, Hursch, and Todd, 1964) Validity (achievement), r y c y j X 1 r x 1 y c r x1 y j r x 2 y c X 2 r x2 y j Criterion score y c y j Clinician s prediction r x 3 y c r x 3 y j r x 4 y c X 3 r x4 y j Empirical validity of cues, r x i y c y = b x + b x + b x + b x c c 1 1 c 2 2 c 3 3 c 4 4 X 4 Input data (cues) Cue utilization by clinician, r x i y j y = b x + b x + b x + j j 1 1 j 2 2 j 3 3 b x j 4 4 The lens model has all virtues of principles of symmetry,which are the key concepts for all successful sciences! Validity is perfect only under perfect symmetry. The predictor and the criterion model must use the same indicators,cues or manifest variables and weight them to the same degree. How principles of symmetry lead to success in science can be seen in the work of Michael Faraday, Richard Feynmanor Murray Gell-Mann. 2

3 Egon Brunswik (Photo courtesy of Department of Psychology, University of California, Berkeley) Thank you Egon and congratulations to your 100 th anniversary Egon Brunswik was one of the most misunderstood psychologists of his time. He introduced the lens model and the representative design among many other things. His contemporaries scolded him about using regression and correlation analysis as the method of the devil in analyzing his lens model, which will never lead to any good causal evidence compared to the randomized experiments, being so successful for advancing science. Yet they forgot that principles of symmetry are of the same importance for scientific theories and new discoveries. There had been a small group of psychological scientist who consistently capitalized and improved on Brunswik s legacy. The most prominent one is Kenneth R. Hammond who impressively improved on Brunswik s work, especially in judgment and decision making with considerably impacts for data analysis, research design and policy making. He founded the International Brunswik Society and without any doubt is the researcher who contributed most in keeping Brunswik s legacy alive. 3

4 THE HIERARCHICAL BRUNSWIK LENS MODEL PREDICTOR AREA CRITERION AREA CR1 CR1 PR A CR2 CR2 CR A CR3 CR3 PR g CR4 CR4 CR g PR B CR5 CR5 CR B CR6 CR6 The original lens model can easily be extended to a hierarchical one. It immediately tells us, that we should be careful in looking for corresponding levels of generality in relating predictors or treatments and criteria. Otherwise asymmetry results and disappointing low relationships and misleading conclusions. 4

5 The Berlin Model of Intelligence (BIS) CONTENT g F figural/ spatial V verbal N numerical operative parcels O P E R A T I O N S simple mental speed M short term memory C creativity R reasoning tasks, total number correct BD OE ZS OG FM WE LO ZF OJ ZK AN CH BG FA Aw TG KW UW ST WM PS EF MA IT AM WA TM SV WA Sl XG SI RZ ZP ZZ ZW DR TN ZG ZR ZN SC TL RD BR fi Sparcel 1 fi Sparcel 2 fi Sparcel 3 fi Mparcel 1 fi Mparcel 2 fi Mparcel 3 fi Cparcel 1 fi Cparcel 2 fi Cparcel 3 fi Cparcel 4 fi Rparcel 1 fi Rparcel 2 fi Rparcel 3 fi Rparcel 4 fi Rparcel 5 Each parcel is build as an aggregate of three z-scored tasks, e.g.: Sparcel1 = (Z BD + Z TG + Z XG ) The Berlin model of intelligence structure distinguishes between two modes of intelligence. The content mode refers to figural/spatial, verbal and numerical tasks. The operation mode distinguishes between mental speed(s) assessed via relative simple tasks, where only speed counts, between tasks which assess short term memory(m), between tasks where the number of different ideas count and between complex reasoning tasks( R ), where one has to think more deeply and longer to find the correct solution. The two modes are crossed and each single test is put in one cell of that 3*4 matrix. In theory derived parceling each single test is standardized to z-scores( mean zero and sd of one). Then a parcel is built by averaging over one row of the matrix. Any differences in performance differences of the content mode are thus diminished, giving the individual differences in operation a better chance to show up. If this theory is correct, one should find mainly four operative factors by factoring the 15 parcels. This is exactly what happens and it has continously been replicated over many samples. The same strategy is applied in building parcels over the operation facets. Factoring these parcels consistently leads to three content factors. 5

6 Intelligence and many unanswered questions Berlin Model of Intelligence (BIS-4 test) What predicts and explains intelligence SINGLE TASK LEVEL (BIS-4) CELL LEVEL LEVEL OF OPERATIVE LEVEL OF GROUP FACTORS AND CONTENT GENERAL OF INTELLIGENCE MODE INTELLIGENCE WORKING MEMORY MF MN MV SF SN SV CF CN CV RF RN RV F N V M S C R Content Factor g c Operative Factor g f g The Berlin model of intelligence structure (BIS) can be organized into the hierarchical model shown at the right. The highest level of generality is Spearman s g-factor at the apex. It is mostly assessed as the aggregate over all tasks building a positive manifold or as Art Jensen recommends as the first unrotated principle components of all tasks factored. The left hand question marks ask what predicts and explains intelligence. The Gestalt principles of the hierarchical lens model force one immediately to search for a hierarchic corresponding symmetrical model once we think about working memory as an explanatory construct. 6

7 The set of working memory tasks in the WMC95-study Working Memory Capacity Switching verbal Category gen. Alpha Span Verbal Coord Switching figurative Random gen. Reading span Verbal span Tracking Spatial Coord Spatial Integration MU spatial STM Spatial STM verbal Spatial WM Pattern Transform. MU Spatial STM MU numerical Spatialfigurative Comp. Span Backward digit span MU numerical Math span Contents numerical Storage & Processing Gauß Switching numerical Star counting Supervision Coordination Functions We started your research in the early nineties and looked for all types of working memory tasks used at that time mainly by experimental cognitive researchers. We adapted these tasks for assessing individual differences and classified them into the 3*3 lattice shown in the slide according to content and functions. Factoring these tasks resulted into three factors shown next. Interestingly we could not fractionate those with verbal and numerical content, they loaded on one common factor. But the spatial tasks could be fractionated from them and also as a third factor a switching factor popped up which maps mainly information processing speed. 7

8 WORKING MEMORY MODEL (WMC_95) BERLIN MODEL OF INTELLIGENCE LEVEL OF GENERAL WORKING MEMORY FACTOR LEVEL OF ORTHOGONALIZED WORKING-MEMORY GROUP-FACTOR SINGLE WORKING MEMORY TASK SINGLE TASK LEVEL (BIS-4) CELL LEVEL LEVEL OF OPERATIVE LEVEL OF GROUP FACTORS AND CONTENT GENERAL OF INTELLIGENCE MODE INTELLIGENCE WMC-g WMC- WMC- SPAT WMC- NV Switching Short Term Memory (F) Memory updating (F) Spatial (F) coordination Reading Span (V) Computation Span (N) Memory updating (N) Switching (N) Switching (V) Switching (F) MF MN MV SF SN SV CF CN CV RF RN RV F N V M S C R Content Factor g c Operative Factor g f g We organized the results from the working memory tasks in the hierarchical model shown on the left and started asking how the different levels of generality are related to each other. 8

9 Relating the three WMC-group factors to the operative factors (WMC95_study) WORKING MEMORY GROUP FACTORS BIS-OPERATIVE GROUP FACTORS WMC-NV WMC-SPEED.33*.33*.48*.24*.25* REAS CREA.74 E 4 R 2 = E 5 R 2 = * -.48*.26* MEM.93 E 6 R 2 = * WMC-SPAT EQS Summary Statistics Method: = ML Chi-Square: = df = 9 pvalue = BBNFI = * -.17* BBNNFI: = CFI: = Set-R 2 = N = * SPEED.83 E 7 R 2 =0.31 This slide shows how the working memory group factors are related to the operative group factors of the BIS. The most appropriate data-analytic tool for this purpose is Jack Cohen s set correlation system which is a multivariate generalization of the product-moment correlation and it can be seen that both sets are highly related to one another. The set of the working memory factors predicts 83.4% of the generalized variance of the operative intelligence factors and vice versa. But the structural equation model also shows sources of variance not accounted for in the BIS once the WMC group factors are held constant. This can be seen in the negatively correlated error terms between reasoning and the three other non-reasoning factors, which are all highly significant and have to be introduced to get good model fit. These correlations map profile differences in the operative factors. E.g. there are subjects which have relatively higher scores on reasoning (REAS) compared to their short-term memory (MEM) scores and vice versa. There are subjects which can store a lot of information in short term memory but are obviously relatively poorer than others in processing and digesting them in the sense of deeper thinking and making sense out of it, i.e. what information processing as reasoning means. In data-analysis not shown here this contrast factor contributed substantial incremental variance additionally to the working memory g-factor for a whole bunch of independent criteria measuring learning success as knowledge and complex problem solving performance. We are reminded with this contrast factor to what memory researchers like the late Donald Broadbent meant with his Maltese cross metaphor. The Maltese cross visualizes bottleneck problems with information processing in the sense of reasoning and deeper thinking. 9

10 Relating the BIS-operative group factors to the three WMC-group factors (WMC95_study) BIS-OPERATIVE GROUP FACTORS WORKING MEMORY GROUP FACTORS REAS.48*.28*.37* WMC-NV.80 E 1 R 2 =0.35 CREA.36*.47* MEM.28*.23*.17* WMC-SPEED.81 E 2 R 2 = * SPEED.19*.49*.16* WMC-SPAT.81 E 3 R 2 =0.34 Here we simply have changed the former working memory predictor set to the criterion set and the operative intelligence factors to the predictor set. Again holding operative intelligence constant it was necessary to model correlated errors. They again map profile differences in working memory factors. Some subjects have relatively higher scores on spatial working memory than on the numerical/verbal factor. We are reminded here to Alan Baddeley s two slave systems, namely the phonological loop (WMC-NV residual) and the visual sketch pad (WMC-SPAT residual). We are tempted with these results to conclude that there are subjects which are better visualizers than verbalizers and vice versa. Numerical content often can and has to verbalized so the failure to fractionate the verbal from the numerical content may be no surprise. Interestingly the working memory speed factor residual is positively related to the WMC-NV residual but not to the spatial WMC factor. The reason for this might be the relatively much more overlearning of verbal/numerical content compared to figural/spatial content. 10

11 Facet Taxonomy for Working Memory Tasks WMC_97 study FUNCTION VERBAL NUMERICAL SPATIAL 1 SPATIAL 2 Storage Word span Digit span Dot span Pattern span Processing a) CRT categories b) CRT syllables a) CRT odd even b) CRT high low a) CRT arrows up down b) CRT arrows above below a) CRT patterns parts b) CRT patterns symmetry Storage + Processing Word span (dual) Digit span (dual) Dot span (dual) Pattern span (dual) Coordination Coordination + Storage Monitoring verbal (no-memory) Monitoring verbal (memory) Monitoring numerical (no-memory) Monitoring numerical (memory) Flight control (no-memory) Flight control (memory) Finding squares (no-memory) Finding squares (memory) Processing + Supervision Switching a) + b) Switching a) + b) Switching a) + b) Switching a) + b) Notes: The first column represents the hypothesized components involved in the tasks. CRT = choice reaction time task. In our 97 study we extended the set of working memory task but kept the best markers of the WMC95 factors. At the functional level at one hand we used tasks which supposedly assessed purely storage, coordination and simple speed related processing, on the other we used tasks which combine two types of functions. 11

12 Relating the three WMC-group factors to the operative factors (WMC97_study) WORKING MEMORY GROUP FACTORS BIS-OPERATIVE GROUP FACTORS WMC 97-SP.40* MEM.92 E 1 R 2 =0.16 WMC 97-CO WMC 97-PROC EQS Summary Statistics Method: = ML Chi-Square: = 5.81 df = 9 pvalue = BBNFI = *.30*.24* -.22*.18* -.23*.21* BBNNFI: = CFI: = Set-R 2 = N = 131 REAS SPEED CREA.77 E 3 R 2 = E 4 R 2 = E 2 R 2 = * -.26* -.18* -.21* Factoring the WMC97 tasks again resulted into three factors, but which are now related to functions only, namely a storage and processing (WMC97- SP) and coordination (WMC97-CO) and a processing (WMC-PROC) factor. Surprisingly the set-correlation squared is pretty much lower in the 97 study compared to 95 study. It dropped down from 83.4% to 68.2%. We have included new types of working memory tasks discussed in the literature but this did obviously not result in a closer relationship between the function related working memory and the operative intelligence group factors. 12

13 Schmid-Leiman models for working memory and intelligence (WMC_97).61 E 1 E 2 E 3 CO1V CO1N COF Co_res D 4 R2= ROP1 ROP2 ROP REAS_res E 5.30 E 6 SPV SPN wmcg 1.00 g MOP1 MOP2 MEM_res.40 E 7 SPF2 MOP3 E 8 E 9 E 11 PROCV PROCN PROCF Pro_res.68 SOP1 SOP2 SOP Speed_res.17 Chi sq.=184,71 P=0.00 CFI=0.95 RMSEA=0.06 Given the results above we tried a new and more differentiated model. In this model we first aggregated all working memory tasks to composites representing one function and one content, i.e. coordination, storage and processing and simple speed related processing for verbal, numerical and spatial content only. The Schmid-Leimanstrategy takes out a general factor first and we did this by mapping a working memory and an intelligence g- factors correspondingly. Doing this we also constrained the pathcoefficients (loadings) for these g-factors to be equal. Not all path coefficients are shown to avoid cluttering of the slide. Once these g-factors are modeled, the Schmid-Leiman strategy maps the residuals into orthogonal group factors. Thus group factors orthogonal to the g factors result. The surprise is now that using such a strategy this type of a WMC-g and an intelligence g are perfectly related. Yet the coordination residual (Co-res) and the reasoning residual (REAS-res) factors orthogonal to their respective g-factors are also highly and significantly related (i.e..61!). In the parlance of classical canonical data analysis two independent canonical factors show up. These results demonstrate that what we get is a function of how we aggregate our measures. 13

14 Disentangling (Cor)relationships in the Brunswik-Symmetry framework WMC95-study WMC- g WMC-NV WMC-SPAT MF MN MV SF SN SV CF Factors NON -R used as THEORY-DERIVED SUPPRESSORS M S C Operative Factor gf g WMC-SPEED CN CV R RF RN RV This slide demonstrates how Brunswik symmetry can be used for better construct validation. Is reasoning ability little more than working memory capacity The BIS- model tells us what reasoning ( R )is not, namely M,S and C, so could it be that the working memory model also contains these M,S and C related sources of variance To test the construct validity of working memory g or that of the WMC group factors we can use the M,S and C factors as theory derived suppressors, i.e. counterfactuals. 14

15 Experimenting with theory derived suppressors in the Brunswik-symmetry framework Reasoning and WMC -g R =.648 R 2 =.419 adj. R 2 =.415 N = 135 Reasoning and WMC -g with non-reas (*) factors as suppressors R =.712 R 2 =.507 adj. R 2 =.492 Increment dr²=.088 Reasoning and WMC group factors R =.669 R 2 =.447 adj. R 2 =.435 Reasoning and WMC group factors with non- Reas (*) factors as suppressors R =.724 R 2 =.524 adj. R 2 =.502 Increment dr 2 =.077 PREDICTOR STD COEF PREDICTOR STD COEF PREDICTOR STD COEF PREDICTOR STD COEF WMC -g.648 * WMC -g BIS-S BIS-C BIS-M WMC -NV.495 WMC -SPAT.422 WMC -SPEED.155 * WMC -NV WMC -SPAT WMC -SPEED BIS-S BIS-C BIS-M All beta-weights p<.01 The results demonstrate higher symmetry at a lower level of generality. This tool is very helpful for differential-correlational research in explaining and understanding relationships between constructs. All suppressors are orthogonal to the criterion, i.e. their zero-order correlations are all zero, based on the theoretical model used! But introducing them into the regression equation suppresses variance in the WMC-predictors not related to reasoning. Thus we learn about reliable but unwanted variance in the predictors, which after being removed (partialled out) enhances the relationship. We too often look in the wrong direction when a relationship is not perfect. It may be the case that we have already all the information, but our predictors contain more reliable information than we need for prediction and explanation. The conclusion here is, that the working memory factors either the general one or the group factors contain variance not related to reasoning and thus working memory cannot be identical with reasoning! These theory derived suppressor factors can be regarded as counterfactuals to the construct we try to predict, explain and understand. Using variables or constructs as counterfactuals is basically nothing else than applying convergent and discriminant validation strategies in the sense of Campbell and Fiske. The differential researcher who cannot use randomized subjects as control groups has thus a powerful tool for construct validation and disentangling causal relationships. 15

16 Testing working memory and intelligence at real-life criteria General(g) and not(g) factors as intelligence as process factors from the WMC95_study Model of performances in complex computer based business games opi_g: General problem solving capacity Chi sq.=18.68 P=0.95 CFI=1.00 RMSEA=0.00 The relationship between working memory and intelligence so far has lead to some puzzling and interesting results. But to still better understand these constructs we should learn about their predictive validities. Do they differentially predict criteria of practical importance and can we thus learn more about their construct validity as well For this purpose we used surrogates for real life performances, namely a set of computer based business games which represent individual differences in complex problem solving. TAILOR (shop) models a company producing and selling shirts, PPLANT is a power plant where the production of energy has to be matched as good as possible to the changing demands of the market(high scores in that games represents poorer performance,hence the negative loading), LEARN is a system modeling a high-tech IT market, where one has to compete against three other companies simulated by the computer. This game was developed by Peter Milling a colleague of mine from the Mannheim business school and top-expert in business and system dynamics and we are thus confident that it actually mirrors real-life demands on a manager. All games were either replicated or gave two performances scores. Thus we a-priori modeled correlated errors controlling for game specific variance. As regards working memory and intelligence g we see that they differentially predict and explain the real-life general complex problem solving capacity latent. Bis-g has the higher path coefficient compared to WMC-g. Two Schmid-Leiman group factors contribute significant incremental variance. M_R_notg is the orthogonal group factor discussed above contrasting short-term memory with reasoning and WMC_S_notg, representing mostly speed, is orthogonal to WMC_g. The clear conclusion we have now is that working memory and intelligence as assessed in our WMC95-study cannot be identical, they differ in their predictive validity! 16

17 Testing Ackerman s PPIK-theory WMC95_study Chi sq.=28.51 P=0.87 CFI=1.00 RMSEA=0.00 In the slide before we saw that Bis_g had the strongest direct causal impact on general real-life problem solving capacity. Is such a theory true Ackerman claims that intelligence as knowledge is always the best predictor of performance, so we used his PPIK-theoretical framework and distinguished between intelligence as process and intelligence as knowledge which represents more the fluid and crystallized parts of intelligence in his extension of Cattell s investment theory. For all our business games we independently assessed the knowledge subjects had developed after a few practice trials and aggregated these knowledge tests to KNOW_g representing intelligence as knowledge (K in PPIK). The results impressively bolster Ackerman s theory. Intelligence as knowledge has to strongest causal impact and intelligence as process now as mainly effective indirectly via KNOW_g! 17

18 Reasoning is a little bit more than working memory capacity! Wmc97_study Chi sq.=52,76 P=0.12 CFI=0.98 RMSEA=0.04 The results above about the predictive validity of working memory and intelligence are related to the WMC95 study, for the WMC97 study we had no data about real-life performance. But to deal with the question whether reasoning is little more than working memory we can use an additional variant in choosing the reasoning factor in that study as the only criterion to be predicted and explained from the working memory model. It can be seen that WMC-g fairly well predicts reasoning, but the coordination residual factor adds incremental variance as well. So again in the 97 study, reasoning IS a little bit more than working memory!! 18

19 Summary and conclusions The relationship between working memory and intelligence is close but not identical. It depends pretty much on what is defined as a working memory task and as an intelligence task. Experimental cognitive research too often uses a single task only, thus the problem of level of generality is rarely visible and dealt with, odds for mismatch and asymmetry are high and the generalization of results is heavily endangered. Differential cognitive researchers favorite g-factor at one level of generality easily can be a more circumscribed group factor at a more general level. Principles of Brunswik-symmetry can help in improving prediction and explanation while searching for better understanding of construct validity. Symmetry is a key concept in all successful sciences.thank you Egon Brunswik! 19