The neural basis of cross-modal numerosity perception

Size: px

Start display at page:

Download "The neural basis of cross-modal numerosity perception"

Clementine Tate
5 years ago
Views:

1 The neural basis of cross-modal numerosity perception Stimuli generation 1 and EEG pilot Sushrut Thorat Advisors: Marco Buiatti and Manuela Piazza Independent Studies Project Report June 26, 2016 CIMeC - Center for Mind/Brain Sciences University of Trento, Italy 1 The stimuli generation codes can be found at:

2 Contents 1 Introduction 2 2 Generation of visual stimuli: The procedure and rationale Relevant factors Stimuli generation Circular envelope constraint Stimulus generation algorithm Paradigm: Cross-modal numerosity perception EEG pilot: My participation 8 4 Concluding Remarks 10 1

3 Chapter 1 Introduction Izard et.al. [5] showed that newborn infants have the ability to distinguish between abstract numbers, across sensory modalities (visual and auditory, in this case). This suggested that abstract numerical concepts are innate to humans, and might be language and culture independent. They did so by subjecting infants to visual stimuli composed of simple shapes such as circles, while playing sequences of sounds, with numerosity as the variable of interest. The measure of the perception of congruency of numerosity across the two sensory modalities, was looking time. Looking time does not directly provide information about which parts of the brain are involved in information processing. Using EEG with ERPs would be a way to understand how the various regions of the brain behave during this task and would help us locate the neural correlate of cross-modal abstract number perception. Unfortunately, ERP requires a large amount of trials with the same stimulus, something which isn t feasible with newborn infants with their short attention spans. So, we resort to measuring frequency-tagged neuro-electric responses [6]. We do this by sinusoidally modulating the intensity of the stimuli, hoping that the neural responses will follow the modulation. Information about the level of attention to the stimuli can be assessed via the power of the tagged frequency in the visual cortex. Phase differences in the tagged responses could provide information about the relative time differences between different processing steps leading to number perception. As we are tracking the sinusoidal response at a particular frequency, we can avail of a stimulus-specific response within a few oscillations i.e. in a matter of tens of seconds. We expect to learn about the responses of regions involved in number perception, such as the intra-parietal sulcus, as a function of numerosity congruency or incongruency across modalities in the stimuli. Before moving on to infants, we wanted to check if frequency tagging actually worked with our paradigm and about the information we could expect to gain from it, by running a pilot study on adults. I was involved in designing the stimuli and in carrying out the EEG experiments on adults. I detail my work in the next two chapters, first about the stimuli generation, and then about the EEG experiment. 2

4 Chapter 2 Generation of visual stimuli: The procedure and rationale Our experiments assess numerosity perception using visual stimuli (the other stimulimodality is auditory) which depict identical objects with the desired numerosity. Non-numerical factors such as item size, item spacing [3], item distribution [4, 7], and item nesting [2] also influence numerosity perception. Item distribution is factored out by generating multiple stimuli, wherein items are distributed randomly on the screen (and close to the center, for foveal stimulation), subject to item size and spacing constraints. Our stimuli do not contain nested items. To make individual items discriminable, a minimum inter-item distance is set. The factors which are non-trivial to be controlled for, are the item size and spacing. We will first define these factors and discuss our method of controlling for them across stimuli. We then will discuss the paradigm used in our experiments, and generation of stimuli sequences for the same. 2.1 Relevant factors The item size is characterised by the Item Surface Area (ISA), and item spacing by Sparsity (St), in [3]. Sparsity is the Field Area (FA), characterised by the convex hull of the stimulus, divided by the number of items in the stimulus. Sparsity isn t independent of ISA. We define a new term, Spacing (Sp), as Sp = St ISA, which is independent of the ISA, and Total Space (TSp) as TS = n Sp. The Total Surface Area (TSA) and Field Area (FA) also contribute to numerosity perception. ISA, St, and Sp are called intensive variables, and TSA, FA, and TSp, extensive variables. As TSA = n ISA, FA = n St, and TSp = n Sp, where n is the number of items, we cannot control both the intensive and extensive variables across stimuli with different number of items. So, across the stimuli in an experimental block (with multiple values of numerosities), we control for either the intensive or extensive variables. 3

5 Figure 2.1: A depiction of the relations in the 4D matrix, for n = 3, 9. All values are in pixels. 2.2 Stimuli generation Given the ranges of ISA and Sp, the numerosity (n), and the number of stimuli to be produced (T), we will generate stimuli with uniformly random ISA and Sp, within the ranges, such that the mean of ISA and Sp across stimuli matches the mean of the extremes of the given ranges. The same procedure is followed when controlling for the extensive variables (TSA and TSp). We now detail a method to generate a stimulus with a given ISA and Sp Circular envelope constraint We do not know of any generic analytical algorithm to generate a stimulus, given its ISA, Sp, and n. We could randomly generate stimuli with the given ISA, and choose the one with the one with the required Sp, such that they fit on the screen. The solution space for stimuli, given this constraint, is huge. A notable observation is that the circle enveloping each stimulus need not span the screen, and the radius of such a circle is related to the Sp, ISA, and n of the stimulus. The matlab function intensive_radii_relation computes the distribution of Spacing, given the radius of the enveloping circle (R), the ISA, and the number of items in the stimulus, and assigns the Spacing with the highest frequency (median) from the distribution, to the corresponding R, ISA, and n. We can use this function to estimate the radius of the envelope that would maximise the chances of the random stimulus generator at converging onto the required Spacing. These 4

relations are stored in a 4D matrix, which are visualised in Fig. 2.

6 relations are stored in a 4D matrix, which are visualised in Fig Stimulus generation algorithm Given the numerosity (n), the Item Surface Area (ISA), and the Spacing (Sp), and the 4D relation matrix, we compute the radius of the envelope that will maximise the chances of creating a stimulus with our constraints (the median Sp as in the 4D relation matrix). We then randomly generate stimuli with the given n and ISA, until the Spacing of some stimulus is within a threshold (that we set) of the required Spacing. We use circles as items in our stimuli to avoid the possible effect of numerosity information in the number of sides of polygons. Figure 2.2: Summary of the stimulus generation algorithm The matlab function stimgen_int generates the polar coordinates of the centres of the n circles (items) to be displayed in the stimulus, maintaining the given Spacing and ISA. We constrain the diameter of the enveloping circle to be maximally equal to the lower dimension of the screen resolution, so that the stimuli do not spill outside the screen. The matlab function imggen_int can generate, display and save an image of the stimulus. Sample stimuli are shown in Fig Now we will describe the stimuli requirements and generation for our experimental paradigm. 2.3 Paradigm: Cross-modal numerosity perception A single run in this paradigm contains an audio stimulus (which has the associated numerosity of either 3 or 9), and two blocks of 30 visual stimuli of each of the two numerosities. The stimuli intensities are varied sinusoidally to tag the resulting neural responses for detection through EEG. These visual stimuli across numerosities are matched either in their intensive or their extensive variables. According to our stimulus generation scheme, if the ranges of the intensive/extensive variables are fixed, the means of those variables across blocks stay constant. So, we need to fix 8 parameters - the maxima and minima of ISA, Sp, TSA, and TSp. Item Surface Area (ISA): The visual angle subtended by each object is constrained to lie between 2.7 and 4.6, as in [5]. This translated to a minimum ISA of 8k pixels and a maximum of 5

7 n = 3, ISA = 10k, Sp = 35k n = 9, ISA = 10k, Sp = 35k n = 3, TSA = 45k, TSp = 90k n = 9, TSA = 45k, TSp = 90k Figure 2.3: Sample stimuli. The top row shows stimuli for 3 and 9 which are matched in their intensive variables, and the bottom row shows stimuli matched in their extensive variables. All the values are in pixels, and the screen resolution is pixels. 24k pixels for our setup. The mean ISA, therefore, is 16k pixels. Spacing (Sp): The optimum packing for non-overlapping circles in 2D is the Hexagonal Packing [1], which will set the lower-bound on Spacing. So, the lower-bound is, ( ) Sp lb = ISA ISA π which, in our case, corresponds to 2, 403 pixels. The upper-bound is set by the constraint that the stimuli need to fit on the screen. The maximum Field Area dictates the upper-bound. So, the upper-bound is, Sp ub = πd2 4n ISA where D is the minimum dimension of the screen, and n is the numerosity. The upper-bound, in our case, corresponds to 77, 788 pixels. But as we are generating stimuli randomly and then selecting the required ones, the time required for the algorithm to converge on a stimulus with extreme Spacing 6

8 is high. We bounded the Spacing of our stimuli to 30k to 45k pixels, to minimise the time taken to generate the stimuli. Total Surface Area (TSA): In keeping the TSA average constant, there is no way to constrain the ISAs for n = 3 and n = 9 between 8k and 24k. We decided to constrain TSA between 24k and 72k pixels (average: 48k pixels), such that the ISA range remains the same for n = 3 across both intensive and extensive controls. Total Spacing (TSp): Following the same line of reasoning as in the case of the TSA, we constrain the TSp between 90k and 135k pixels (average: 112.5k pixels). Given the values of the 8 parameters, the condition (intensive or extensive), the numerosity, and the number of stimuli, the matlab function blockgen_stim generates random stimuli with the required constraints. A generated batch of stimuli (for both the intensive and extensive controls) is visualised in parameter space in Fig Figure 2.4: Stimuli in parameter space. The stimuli in red correspond to intensive variable matching, and those in blue correspond to extensive variable matching (labelled as Int and Ext in the plots). 7

1: 64 electrode layout on a common eeg cap.

9 Chapter 3 EEG pilot: My participation I was involved in the pre- and post-testing tasks of the EEG pilot with adults. We used an EEG cap with 64 electrodes, which is depicted in Fig Figure 3.1: 64 electrode layout on a common eeg cap. The experiment was conducted in an electrically shielded and sound-attenuated cabin. EEG was recorded with a Brain-Amp amplifier (Brain Products, Munich) 8

10 using 64 Ag/AgCl-sintered ring electrodes mounted in an elastic cap (Easycap, Munich) and placed equidistantly according to the 10%-system, with an earlobe reference and ground electrode in AFz. Electrode impedances were kept below 15 kω. Data were sampled at 500 Hz and analog-filtered between and 250 Hz during recording. The pre-test tasks consisted of: Cleaning the forehead. Head size measurement to decide what cap to use. Marking the positions of the nasion and inion. Mounting the cap and aligning the Cz and Fpz electrodes. Applying an electrode at the ear-lobe (Reference). Ensuring that the impedance between the Ground (AFz) and Reference electrodes is under 5 kω, by cleaning the regions enclosed by those electrodes and by using electrolyte gel. Clearing out hair enclosed by the electrodes to expose scalp, and cleaning those regions with alcohol. Filling the electrodes with conducting gel until all electrode impedances w.r.t. reference are under 15 kω. Then the EEG tests commenced. The post-test tasks consisted of: Carefully removing the EEG cap. Asking the subject to wash his/her hair. Washing the cap with pressurised water and soap. Putting the cap in a disinfectant for 30 minutes, and then let it dry - ready for further use. 9

11 Chapter 4 Concluding Remarks I was involved in the development of stimuli towards, and in conducting, the EEG experiment on adults to assess cross-modal numerosity perception. This project provided me with a close look at the methodologies regarding the design of experiments related to frequency-tagging EEG and numerosity perception. Assisting in the EEG experiment made me aware of the preparations essential towards ensuring a high signal-to-noise ratio in EEG (use of gels, etc.). They aren t mentioned in research articles, but constitute the foundations of EEG experiments. I would like to thank Profs. Marco Buiatti and Manuela Piazza for guiding me through the project through stimulating discussions. I would also like to thank Sophia Snipes for introducing me to the pre- and post-experimental to-dos in EEG experimentation. 10

12 Bibliography [1] H. C. Chang and L. C. Wang. A Simple Proof of Thue s Theorem on Circle Packing. ArXiv e-prints, [2] Dana L. Chesney and Rochel Gelman. Visual nesting impacts approximate number system estimation. Attention, Perception, & Psychophysics, 74(6): , [3] Nicholas K. DeWind, Geoffrey K. Adams, Michael L. Platt, and Elizabeth M. Brannon. Modeling the approximate number system to quantify the contribution of visual stimulus features. Cognition, 142: , [4] Norman Ginsburg. Effect of item arrangement on perceived numeros1ty: Randomness vs regularity. Perceptual and motor skills, 43(2): , [5] Véronique Izard, Coralie Sann, Elizabeth S Spelke, and Arlette Streri. Newborn infants perceive abstract numbers. Proceedings of the National Academy of Sciences, 106(25): , [6] Anthony M Norcia, L Gregory Appelbaum, Justin M Ales, Benoit R Cottereau, and Bruno Rossion. The steady-state visual evoked potential in vision research: a review. Journal of vision, 15(6):4 4, [7] Matteo Valsecchi, Matteo Toscani, and Karl R. Gegenfurtner. Perceived numerosity is reduced in peripheral vision. Journal of vision, 13(13):7 7,

Supporting Information

Supporting Information ten Oever and Sack 10.1073/pnas.1517519112 SI Materials and Methods Experiment 1. Participants. A total of 20 participants (9 male; age range 18 32 y; mean age 25 y) participated