TITLE:&The"organisation"of"working"memory"networks"is"shaped"by"early"sensory"experience"

Transcription

1 TITLE:&Theorganisationofworkingmemorynetworksisshapedbyearlysensoryexperience VeliaCardin 1,2,3,MaryRudner 1,RitaFDeOliveira 4,JosefineAndin 1,MerinaSu 5,LilliBeese 2,Bencie Woll 2 andjerkerrönnberg 1 1.! LinnaeusCentreHEAD,SwedishInstituteforDisabilityResearch,DepartmentofBehavioural SciencesandLearning,LinköpingUniversity,Sweden. 2.! DeafnessCognitionandLanguageResearchCentre,DepartmentofExperimentalPsychology, UniversityCollegeLondon,49GordonSquare,LondonWC1H0PD,UnitedKingdom. 3.! SchoolofPsychology,UniversityofEastAnglia,NorwichResearchPark,NorwichNR47TJ,United Kingdom. 4.! SchoolofAppliedScience,LondonSouthBankUniversity,103BoroughRoad,LondonSE10AA, UnitedKingdom. 5.! DevelopmentalNeurosciencesProgramme,UCLGOSInstituteofChildHealth,30GuilfordStreet, LondonWC1N1EH,UnitedKingdom. & Corresponding&author:&Dr.VeliaCardin,velia.cardin@gmail.com,&SchoolofPsychology,Universityof EastAnglia,NorwichResearchPark,NorwichNR47TJ,UnitedKingdom.+44(0) Running&title:Workingmemoryprocessinganddeafness Numberoffigures:5 Numberoftables:7 1

2 ABSTRACT&& Earlydeafnessresultsincrossmodalreorganisationofthesuperiortemporalcortex(STC).Herewe investigatedtheeffectofdeafnessoncognitiveprocessing.specifically,westudiedthereorganisation, due to deafness and sign language (SL) knowledge, of linguistic and nonalinguistic visual working memory(wm).weconductedanfmriexperimentingroupsthatdifferedintheirhearingstatusand SLknowledge:deafnativesigners,hearingnativesigners,hearingnonasigners.Participantsperformed a 2aback WM task and a control task. Stimuli were signs from British Sign Language or moving nonsenseobjectsintheformofpointalightdisplays. We found characteristic WM activations in frontoaparietal regions in all groups. However, deaf participantsalsorecruitedbilateralposteriorstcduringthewmtask,independentlyofthelinguistic contentofthestimuli,andshowedlessactivationinfrontoaparietalregions.restingstateconnectivity analysisshowedincreasedconnectivitybetweenfrontalregionsandstcindeafcomparedtohearing individuals.wmforsignsdidnotelicitdifferentialactivations,suggestingthatslwmdoesnotrelyon modalityaspecificlinguisticprocessing. These findings suggest that WM networks are reorganised due to early deafness, and that the organisationofcognitive networks is shaped by thenatureof the sensory inputs availableduring development. 2

3 INTRODUCTION&& Earlysensorylossresultsincrossmodalreorganisationofsensorycortices,whereregionsthatusually processinputsfromthemissingmodalityarerecruitedbytheremainingsensorymodalities(merabet &PascualaLeone,2010).Understandingthisreorganisationisfundamentalforgaininginsightsintothe principlesofneuralplasticityandthefunctionalcapabilitiesofthehumanbrain,includingitspotential forrehabilitationandenhancement. Early deafness results in crossmodal reorganisation of regions which typically serve an auditory function,suchasthesuperiortemporalcortex(stc)(finneyetal.2001;karnsetal.2012;cardinet al.2013).itislikelythatsensorylosswillalsoimpacttheorganisationofcognitivenetworks,giventhe stronginterconnectivitybetweenregionsofthebrainthatsupportsensoryandcognitiveprocessing. However,itisnotknowntowhatextentreorganisationofSTChaswiderconsequencesforcognitive networks.thechallengeinstudyingthisissueindeafhumansisthatcorticalreorganisationisnotonly aconsequenceoflackofaudition,butalsooflanguagebeingacquiredthroughvision,withacquisition oftendelayed(cardinetal.2013).thislateandinsecurelanguagedevelopmenthasconsequencesin adultlanguageproficiencyinsignedandspokenlanguage(cormieretal.2012),withcorresponding effectsinneuralfunction(macsweeneyetal.2008b;mayberryetal.2011;pénicaudetal.2013).thus, differencesincorticalreorganisationandcognitiveperformancebetweendeafandhearingindividuals canbedrivenbythemodalityoflanguageusedbyeachgroup,byadelayinlanguageacquisition,or byauditorydeprivation(lynessetal.2013). VisualWMrepresentsaninterestingcaseforthestudyofcognitionanddeafness,asitisoneofthe domainsinwhichbehaviouraladvantageshavebeenobservedfordeafindividuals(rudneretal.2016; seekeehnerandatkinson2006,forareview).itisexpectedthatthisbehaviouraladvantagewillmap intoreorganisedcorticalpathwaysthatsupportenhancedperformanceinthedeaf.however,the neuroimagingevidenceisinconsistent(buchsbaumetal.2005;bavelieretal.2008;dingetal.2015). In2015,Dingetal.(2015)showedrecruitmentofSTCduringthedelayperiodofavisualshortaterm 3

4 memorytaskindeafindividuals.thiscontradictedapreviousstudybybavelieretal.(2008),which showedrecruitmentoftypicalfrontoaparietalregions,butnoactivationofstcduringmaintenance. Buchsbaumetal.(2005)alsoshowedSTCrecruitmentforvisualshortatermmemoryinthedeaf,but constrainedtoaposteriorregionthatisalsorecruitedforspokenlanguageandsignlanguagewmin hearingsigners,suggestingthatthisregionisgenerallyinvolvedinlinguisticwm,andnotreorganised asaconsequenceofdeafness. Severalfactorsmayhavecontributedtothediscrepancies,butlanguageacquisitionislikelytobean important one, given its relevance for cognitive development (Gathercole and Baddeley 1993). Buchsbaumetal.(2005)andBavelieretal.(2008)testeddeafnativesigners,apopulationinwhich signlanguageisacquiredfrombirth,andwherelanguagedevelopmentachievesthesamelandmarks intimeasthoseofhearingchildrenacquiringaspokenlanguage.instead,thegrouptestedbydinget al.(2015)washeterogeneous,withlanguageacquisitionlikelytohavebeenlateandpoor.itisunlikely thatlanguageproficiencywilldirectlyimpactperformanceonapurelyvisuoaspatialtask,suchasthe oneusedbydingetal.(2015).however,giventhatlanguagehasbeenshowntomediateexecutive functionindeafchildren,includingnonaverbalwm(marshalletal.2015;bottingetal.2016),itislikely thatdelayedandinsecurelanguageacquisitionduringchildhoodcouldhaveresultedinadifferent corticalreorganisationofcognitivenetworksduringdevelopment(macsweeneyandcardin2015).in a scenario in which superior temporal regions are not stimulated by auditory information, and in additiondonotreceivethenecessaryenvironmentallanguageinformationtofullydeveloparolein languageprocessing,itispossiblethatstcregionscouldenduptakingonothercognitivefunctions suchasworkingmemory.incontrast,inthecaseofnativesigners,theenvironmentallanguageinput willbestrong,andthereforethestcispotentiallylesslikelytoberecruitedforworkingmemory. Therefore, differences in language development and proficiency in the groups of deaf individuals testedinpreviousstudiesarelikelytoimpactcrossmodalreorganisationandthereorganisationof cognitive networks, potentially explaining the contradictory results of previous working memory studies. 4

5 Giventheeffectoflanguageproficiencyoncognitivedevelopmentindeafindividuals(Marshalletal. 2015;Bottingetal.2016),itisimportanttodeterminewhetherdeafnessaffectslinguisticandnona linguisticwmdifferently.previously,linguisticwmprocessingindeafindividualshasbeenstudied usingsignlanguagestimuli(buchsbaumetal.2005;bavelieretal.2008).resultsofthesestudiesshow thatsignlanguagewmindeafindividualsrecruitsparietalandoccipitalregions(buchsbaumetal. 2005;Bavelieretal.2008).Giventhatthispatternofresultshasalsobeenfoundwhenstudyingsign languageworkingmemoryinhearingsigners,ithasbeensuggestedthatthiswasaneffectofmore spatiallyorientedlinguisticprocessinguniquelypertainingtosignlanguages(rönnbergetal.2004; Buchsbaumetal.2005;Rudneretal.2007;Bavelieretal.2008;Paetal.2009).However,previous studieshavefocusedonisolatinglinguisticprocessingbycomparingsignlanguagewmtospoken languagewm.therationalebehindthisdesignisthattheyarebothlinguisticwm,butrelyingon different sensoryandmotorprocesses: while sign languages are visualamanuallanguages,spoken languages are auditoryaoral languages. However, neuroimaging studies have not compared sign languagewmtopurelyvisualwm.therefore,itisnotknownwhetherdifferencesinrecruitmentof parietal and occipital regions are related to modalityaspecific linguistic processing or to sensory processing. In other words, it is not clear whether these differences are due to more spatially orientatedlinguisticprocessinginsignlanguagethaninspokenlanguage,oriftheyaredrivenby differentsensoryprocessingofthestimuli.furthermore,noneofthosestudieshasshownadditional recruitmentofmiddleandanteriorstcregionsindeafindividuals. Thus, several questions about the organisation of working memory networks in the context of auditorydeprivationremainunanswered: 1.! DoesdeafnessresultinrecruitmentofSTCduringworkingmemory,suggestingreorganisation ofcognitivenetworksasaconsequenceofauditorydeprivation,oristhisaneffectofdelayed languageacquisition? 2.! IsSTCrecruitedforWMindeafindividualsonlywhenthestimuliarepurelyvisuoaspatial? 5

6 3.! AretherelinguisticWMmechanismsspecificforsignlanguage,anddotheydifferbetween deafandhearingindividuals? TheaimofthisstudyistoaddressthesequestionsbyunderstandingwhethertheSTCandcognitive networks involved in WM are reorganised as a consequence of early deafness, independently of delayedlanguageacquisition.inaddition,weaimtoaddresswhetherdifferencesinsignedandspoken languagewmareduetosensoryorlinguisticprocessing. WeconductedanfMRIexperimentwhileparticipantsperformedlinguisticandnonalinguisticWMand controltasks.ourmaingroupofinterestwereindividualswhowerecongenitallydeafnativesigners ofbritishsignlanguage,andwhothushavenormaldevelopmentofafirstlanguage.hearingnative signers and nonasigners were also tested to determine whether effects were driven by auditory deprivation,orbysignlanguageknowledge. Tosegregateeffectsthataredrivenbygeneralvisuoaspatialprocessingofsignlanguagestimulifrom thosethataredrivenbylinguisticprocessingofsigns,weusedsignsofbritishsignlanguage(bsl)and nonsenseobjects.aproblemwhencomparingsignsandothervisuoaspatialstimuli,suchasnonsense objects,isthattherearedifferencesbetweenstimulinotonlyinlinguisticcontent,butalsoinbasic visualfeatures,suchastexture,colour,contrastandshape.toavoidtheselowalevelconfounds,we designedallourstimuliusingpointalightdisplays.therefore,inallourexperimentalconditions,our stimulihad,onaverage,comparablelowalevelvisualfeatures. Wehypothesisedthatifdeafnessimpactsthereorganisationofcognitivenetworks,differencesin cortical recruitment for WM should be observed between deaf and hearing individuals. These differences should be independent of sign language knowledge, of the linguistic content of the informationtoberemembered,andbeaccompaniedbyanetworkawisereorganisationforcognitive processing. Furthermore, if parietal and occipital areas are specifically recruited for sign language workingmemoryasaconsequenceoflinguisticprocessing,weshouldobservestrongeractivationsin 6

7 these regions when deaf and hearing signers remember BSL stimuli, than when they remember nonsenseobjects. METHODS& Participants& Therewerethreegroupsofparticipants(Table1): A)! Deafsigners(N=12):congenitallyseverelyatoaprofoundlydeafindividuals,whohaveat leastonedeafparent,andarenativesignersofbritishsignlanguage(bsl). B)! Hearingnativesigners(N=16):hearingindividualswhowereborntoatleastonedeaf parentwhocommunicatedwiththemfrominfancyusingbsl. C)! HearingNonaSigners(N=16):hearingindividualswhoarenativespeakersofEnglishand ofanotherspokenlanguage(nativebilinguals),andwhohadnopreviousknowledgeof BSL. Thedecisiontoincludenativebilingualsoftwospokenlanguageswasmadebecausenativesignersof BSL,bothdeafandhearing,arealsonativeorveryproficientbilinguals,usingspokenandwritten EnglishaswellasBSL.DeafsignersinBritainandinothercountries,areusuallybilingualtosome extent,havingdifferentdegreesofknowledgeofthemostusedspokenlanguageofthecountrywhere theyreside.however,intheliterature,theyareoftencomparedtogroupsofmonolingualspeakers. Here,wewantedtoensurethatallthreegroupsarebilingualinordertocontrolforcognitiveeffects potentially related to using more than one language in everyday communication (Bialystok et al. 2012). 7

8 ParticipantswererecruitedfromtheUCLPsychologySubjectPool,theSubjectDatabaseoftheUCL InstituteofCognitiveNeuroscience,andtheParticipantsDatabaseoftheUCLDeafness,Cognitionand Language Research Centre. Participants were all rightahanded (selfareported), had normal or correctedatoanormalvisionandnohistoryofneurologicalproblems(selfareported).allparticipants gavetheirwritteninformedconsenttoparticipateinthestudy,andwerecompensatedfortheirtime, travelandaccommodationexpenses.allproceduresfollowedthestandardssetbythedeclarationof Helsinki,andwereapprovedbytheUCLEthicscommittee. Werecruiteddeafandhearingnativesignersbasedonthecriteriadescribedabove.Hearingnona signerswerepreascreened,andparticipantswereselectedtomatchtheothertwogroupsonageand nonaverbalintelligence,asassessedwiththeblockdesignsubtestofthewechslerabbreviatedscale of Intelligence (Wechsler, 1999; Table 1). To assess participants WM skills, we also conducted a computerizedversionofthecorsiblockatappingtask(corsi,1972)asimplementedinpeblsoftware ( and an adapted computerized version of the operation span task (TurnerandEngle1989),asreportedinAndinetal.(2013).Theoperationspantaskrequiredsolving arithmetical operations and simultaneously remembering specific items. It consisted of twelve sequencesofequations,eachcontaining2,3,4or5equations.sequencesstartedwith2equations, andprogressivelyincreasedinsizeafterthreesequencesofeachlength.intotal,therewerefortyatwo equations,eachcontainingeitheramultiplicationordivision,andasubtractionoraddition(e.g.3x2 +1=7).Aftereachoperationwaspresented,participantshadtoreportbypressingakeywhetherthe resultoftheoperationwascorrect,whileatthesametimerememberingthedigitpresentedasthe resultoftheequation(ignoringifcorrectorincorrect).aftereachsequence,participantshadtoreport theremembereddigitsinthesameorderaspresented.themaximumspanlengthfortheoperation spanwas5.tocorrectforceilingeffectsandoccasionalinattention,theoperationspanwasratedas the sum of the span of each correctly remembered sequence, times the proportion of times a sequenceofsuchspanwasretrievedcorrectly.twoasampletatestswereconductedtodetermine whetherperformancewassignificantlydifferentbetweengroups.hearingsignershadasignificantly 8

9 shorterspaninthecorsitaskthandeafsigners(t(26)=3.4;p=.002)andhearingnonasigners(t(30)= 2.2;p=.040).Hearingnonasignersscoredhigherthandeafsignersintheoperationspantask(t(26)= 2.1;p=.049).NoneoftheotherbetweenagroupscomparisonsofperformanceforCorsiandoperation spantasksweresignificantlydifferent(p>.05).duetothesesignificantdifferences,performancein thecorsiandoperationspantaskswereenteredascovariatesinallthebehaviouralandneuroimaging analyses. Gender was also entered as a factor in the analyses described below, because groups differedinnumbersofmalesandfemales. Stimuli& Our aim was to have stimuli that differed in terms of linguistic content, but were otherwise comparable.inparticular,wewantedtominimisedifferencesinvisualfeatures(e.g.colour,texture, contrast,illumination)thatcoulddriveresponsesinlowalevelvisualareas,butalsopotentiallyengage differentwmstoragemechanismsduetotheirvisualpropertiesratherthantheirlinguisticstatus.to this purpose, and based on theories of dynamic visual event perception (Runeson and Frykholm, 1983),allourstimuliwerecreatedaspointalightdisplays.Inthisway,theuniquecomponentsand globalmotionofthearraywerethesame,butdifferencesinthespatiallocationofthedotswillresult intheperceptionornotofbiologicalmotionofthehands. Stimuli consisted of video clips of 21 signs of British Sign Language (BSL) and 21 nonsense 2a dimensionalmovingobjects,presentedasmultiacolouredpointalightdisplays(johansson,1973)ona black background (see Figure 1; Sup. Fig. 1). The BSL signs were created using a Qualysis Motion CaptureSystem(QualysisAB,Sweden),withmarkersoneachofthejointsoftherighthandofthe model.3dcoordinatesfromeachjointwereextracted,processedanddisplayedusingmatlab2010 (Mathworks,MA)andCogent( spatiallocationandglobalmovementofthesignstimuli,butchangingthearrangementofthedots fromspecificjointstononsenseshapes. 9

10 SignswereoneahandedBSLlexicalsigns:ANGER,DANCE,DELICIOUS,DENMARK,DISAPPOINTED,DISLIKE,FEEL, KETTLE,LAUGH,LIKE,LIVE,PAINFUL,PETROL,PHONE,POOR,PREGNANT,RICE,SCHOOL,SHOCK,SMOKEANDWANT.Onea handed signs were chosen in which the palm was parallel to the body, to avoid occlusion of the articulatorswhichwouldmakethepointalightdisplaydifficulttocaptureanddiscriminate.repetition ofhandashapeacrosssignswasmatchedwithrepetitionofshapeacrossobjects.averagenumberof pointalightsanddurationofthedisplaywerealsomatchedbetweensignsandobjects. Eachsigndisplaycontainedthreereferencepoints:onecorrespondingtothenoseofthemodel,and onecorrespondingtoeachoftheshoulders.toavoidsmalljitteringbetweenthelocationofthese pointsduetosmallsmovementsofthemodel,theaveragepositionofthenoseandshoulderswas calculatedfromthesignstimulianddisplayedineveryvideotoprovidestaticbodyreferencesforthe viewer.thesereferencepointswerealsodisplayedfortheobjectstimuli,toavoidgrossdifferences in visual features between signs and objects. Participants were asked to fixate their gaze on the referencepointcorrespondingtothenose,whichwaspresentthroughouttheexperiment,including periodsinwhichstimuliwerenotdisplayed.infutureinstancesinthispaperwewillrefertothenose referencepointas fixationdot. ToensurethatpointalightsignswererecognisedaslexicalBSLsigns,andthatnonsenseobjectswere notmisperceivedaslexicalsigns,a largersetof 49 stimuliwasincludedinalexicaldecisionpilot experiment.fourfluentbslsigners(threenative,1earlylearner)tookpartinthistest.participants wereshowneachstimulusonce,andtheywereaskedtodecidewhethereachdisplaywasarealbsl signornot.followingthebehaviouralpreatest,stimuliwererevisedtoexcludeambiguousitems.only itemsthatwerecorrectlyjudgedbyatleast3ofthe4participantswereincludedinthefinalsetof stimuli. 10

11 Tasks& Participantsperformedtwotypesoftask:a2abackworkingmemorytask(Braveretal1997)anda control colour task (Fig. 1; Sup. Fig. 1). In the 2aback WM task block, pointalight displays were presentedsequentially,andparticipantshadtoindicatewhethertheoverallshapeandmovementof thedisplaywasidenticaltoordifferentfromtheoneseentwostepsback.inthewmtask,thecolour ofthedotswasalwaysvariedtoavoidparticipantsperformingthetaskbyrememberingthecolourof specificdots.inthecontrolcolourtaskparticipantshadtoindicateiftherewereanyyellowdotsin thedisplay.inshort,foreachdisplay,theyhadtoanswera Yes / No question: Isthisdisplayidentical totheone2astepsback? or Arethereanyyellowdotsinthedisplay?.Participantsheldabuttonbox withboththeirhands.theywereinstructedtopressonebuttonwiththeirthumbtoindicate YES, andusetheotherthumbtoindicate NO.Rightandleftthumbswerealternatedfor YES and NO responsesacrossparticipants. Beforetheexperiment,thetaskswereexplainedtoparticipantsintheirpreferredlanguage(BSLor English),andwritteninstructionswerealsoprovidedinEnglish.Ashortpracticesessionensuredthat participantswerecomfortablewithalltasks.pointalightdisplaysforthepracticesessionweredifferent fromthoseusedinthemainexperiment.theexperimentersmonitoredparticipants performance during practice, answering questions and offering additional clarification as necessary in BSL or English. Experimental&Design& Our experiment had two types of stimuli and two types of task (2 x 2 design), resulting in four experimentalconditions:wmsigns,wmobjects,coloursigns,andcolourobjects.eachscanning sessionhad4experimentalruns;eachrunhad12taskblocks(3ofeachcondition),andeachblock had12trials.blockslasted~28seach,andtheywereseparatedbyeightshortfixationperiods(2a3 11

12 s), and four long fixation periods (12 a 16 s). These were semiarandomly intercalated, avoiding occurrenceoftwolongperiodsafterconsecutiveblocks.overall,eachrunlasted~8.5min. At the beginning of each block, a cue was presented for 1.5 a 2.2 s indicating the type of task participantswouldhavetoperform(either memory or colour ).Thiswasfollowedbyagreenfixation dotfor1s,toindicatethetaskwasgoingtostart.afterthepresentationofthelaststimulusofthe block,thefixationdotchangedcolourtoredfor1stoindicatetheendoftheblock. Eachblockhadeitherfive2abackmatches,orfour2abackmatchesand1lure(rangingfrom1a7back steps).thisarrangementwasusedforboththewmtaskandthecolourtask,buttheidentityofthe display was not relevant for the colour task, and the colour task could not be performed by rememberingtheidentityofthedisplay.ofthe21signsand21objectsinthestimulusset,7unique items were chosen on each block of a particular condition. To specify the identity of the stimuli displayed,48blocksequencesweregenerated(4runsx12blocks).toavoidanypotentialactiveor passive prediction, sequences were not repeated across conditions, and each participant was presented with each of the 48 sequences once. Sequences were randomly allocated to different conditionsforeachparticipant,meaningthatallsequenceshadthesameprobabilitytobeusedfor thewmtaskorthecolourtask.inthecolourtask,yellowdotsappearedin4 5trialsofeachblock, so that the ideal distribution of yes and no answers across tasks was the same. The set of sequencesthatdeterminedtheappearanceofyellowdotswasdifferentfromthatwhichdetermined theidentityofthedisplay.asexplainedabove,colourcouldnotbeusedtosolvethewmtask,and viceversa. Image&Acquisition&& ImageswereacquiredattheBirkbeckaUCLCentreforNeuroimaging(BUCNI)inLondon,UK,usinga 1.5TSiemensAvantoscanneranda32achannelheadcoil. 12

13 Thereweretwovideocamerasinthemagnet sbore.onewasusedtomonitortheparticipant sface andensuretheywererelaxedandawakethroughoutscanning;theothermonitoredtheparticipant s left hand which was used by deaf participants for manual communication with the researchers betweenscans.athirdvideocamerainthecontrolroomwasusedtorelaysignedinstructionstothe participantviathescreen.anintercomwasusedforcommunicationwithhearingparticipants.all volunteersweregivenearaprotection. StimuliwerepresentedusingMATLABandCogent.Allstimuliwereprojectedontoascreenhungin frontofthemagnet sbore;participantswatcheditthroughamirrormountedontheheadcoil. There were five functional scans: 1 resting state and 4 task fmri. Functional imaging data were acquiredusingagradientaechoepisequence(36slices,tr=3060ms,te=50ms,fov=192mm,2mm thickness,distancefactor=50%)givinganotionalresolutionof3x3x3mm.thefirstsevenvolumes ofeachrunwerediscardedtoallowfort1equilibrationeffects.eachexperimentalscanlasted~8.5 min(167volumes).therestingstatescanlasted~10min(196volumes),anditwasconductedatthe beginningofthesession.duringthisscan,participantswereinstructedtoliequietlywiththeireyes open,lettheirmindwanderandnotfallasleep.ahigharesolutionstructuralscanwasacquiredusing magnetizationapreparedrapidacquisitionwithgradientecho(mprage,tr=2730ms,te=3.57ms, 1mm 3 resolution,176slices). Task&fMRI&Data&Analysis TaskafMRI datawereanalysedusingmatlab andstatisticalparametricmappingsoftware(spm8; Wellcome Trust Centre for Neuroimaging, London, UK). Images were realigned, coregistered, normalisedandsmoothed(8 mmfwhmgaussiankernel)followingspm8standardpreaprocessing procedures.firstalevelanalysiswasconductedbyfittingagenerallinearmodel(glm)with4main regressor:wmsigns,wmobjects,coloursigns,andcolourobjects.foreachmainregressor,rtsfor eachtrialweremodelledasafirstaorderparametricmodulator.otherregressorsincluded:righthand 13

14 response,lefthandresponse,cueperiodindicatingawmblock,andcueindicatingacolourblock.for everyregressor,eventsweremodelledasaboxcaroftherelevantduration,andconvolvedwithspm s canonicalhaemodynamicresponsefunction.motionparameterswerederivedfromtherealignment oftheimagesandincludedinthemodelasregressorsofnointerest.regressorswereenteredintoa multipleregressionanalysistogenerateparameterestimatesforeachregressorateveryvoxel. Foreachparticipantseparately,contrastsforeachindividualconditionweretakentoasecondlevel analysis,wheretheeffectsofdeafness,taskandstimulustypeweretestedasindicatedintheresults. GenderandperformanceintheCorsiandoperationspantaskswereincludedascovariates(analysis withoutcorsiandoperationspanascovariatesrevealedthesamepatternofresultsforallcontrasts). Effectsandinteractionsofinterestweretestedusingspecifiedtacontrasts.Voxelsarereportedasx, y,zcoordinatesinaccordancewithstandardbrainsfromthemontrealneurologicalinstitute(mni). Differencestatisticalthresholdsareusedfordisplayspurposes,butactivationsareonlydiscussedif theyreachedasignificancethresholdofp<.05(fwe,corrected)atpeakorclusterlevel. RestingIstate&functional&connectivity&analysis& Seedatoaseed restingastate functional connectivity analysis was carried out in the CONN toolbox implemented in MATLAB (WhitfieldaGabrieli and NietoaCastanon 2012). Clusters that were differentiallyactivatedbetweenhearinganddeafindividualsduringtheworkingmemorytaskwere used as Regions of Interest (ROI) in the resting state connectivity analysis. The images were preprocessedfollowingthesamestepsasinthetaskfmriexperiment.thesignalfluctuationsover time in the resting state scans were averaged over all the voxels in each ROI and extracted for subsequent correlation analyses. In addition, the Artifact Detection Tools (ART) toolbox ( thedata,whichwereaddedintosubsequentanalysesasadditionalregressorstocorrectformotion artefacts(compcormethod)(behzadietal.2007;chaietal.2012).lastly,abandpassfilterof0.008 a0.09hzwasappliedtodiscardcardiovascularandrespiratorynoise(chaietal.2012).theaveraged 14

15 signalfromeachroi(source)wasthencorrelatedwiththesignalofeveryotherroi(target),and normalisedusingfisher sratoaztransforms. RESULTS& Behavioural&Results&& Duringscanning,participantsperformedaWMtaskandacontrolcolourtask(seeMethods).Reaction timesandd foreachconditionareshownintable2.toevaluatewhetherparticipantsperformedthe taskatequallevels,arepeatedmeasuresanovawasconductedusingreactiontimeasadependent variable.withinasubjectfactorsweretask(wm,colour)andstimulustype(signs,objects);betweena subject factors were Deafness (Yes, No), Sign Language Knowledge (Yes, No), and Gender (male, female).performanceinthecorsiandoperationspantaskswereenteredascovariates.therewere nosignificantmaineffects,buttherewasasignificantinteractionbetweentaskanddeafness(f(1,36) =7.03,p=.012).Investigationofthisinteractionshowedthatdeafindividualsweresignificantlyfaster thanhearingindividualsforthewmtask(table2;f(1,36)=8.4,p=.006),butnotforthecolourtask (F(1,36)=1.1,p=.31). AsimilarrepeatedmeasuresANOVAwasconductedwithd asadependentvariable.therewasamain effectoftask(f(1,36)=4.34,p=.045),whereparticipants performancewassignificantlybetterfor thecolourtask(d =3.3±0.11s.e.m.)thantheWMtask(d =2.2±0.12s.e.m.).Therewasalsoa maineffectofsignlanguageknowledge(f(1,36)=4.84,p=.034),withsigners(deafandhearing)(d =2.9±0.12s.e.m.)performingsignificantlybetterthannonasigners(d =2.5±0.16s.e.m.)acrossall tasksandstimuli.noothermaineffectorinteractionwassignificant(f<2.8andp>.1). Given the significant interaction between deafness and task, RTs for each trial were included as modulators at the first level in the neuroimaging analysis. d was not included in the main neuroimaging analysis because the statistically significant effects were not related to deafness. 15

16 However,amodelinwhichthed foreachrunwasincludedasacovariateinthesecondlevelanalysis producedthesamepatternofresults. fmri&results Stimulus(Effects.Asetofstimulimadeofpointalightdisplayswascreatedforthepurposeofthisstudy. Behaviourally,wecheckedthevalidityofthestimuliinapilotlexicaldecisiontask(seeMethods). Validationofthestimuliwasalsothefirststepintheneuroimaginganalysis,ensuringthatthepointa lightdisplaysresultedintypicalactivationsobservedintheposteriortemporalaoccipitalcortexfor biologicalmotionofhands(pelphreyetal.2005;capeketal.2008).thiswascorroboratedusingthe contrast [Signs > Objects] separately for each group (Fig. 2; Sup. Fig 2; Table 3). In each group, significant activations were observed in posterior temporalaoccipital cortex, a region activated by biological motion of hands (as described above). Group comparisons showed no significant differencesbetweenthegroupsforthecontrast[signs>objects].however,inbothgroupsofsigners, activations associated with linguistic processing of the stimuli were also observed. Specifically, in hearing signers, there was additional recruitment of typical leftalateralised perisylvian regions for language processing. In deaf signers, instead, and presumably as a consequence of crossmodal plasticity, activations were more prominent along the STC. These results are in agreement with literatureonsignlanguageprocessinginhearinganddeafsigners(nevilleetal.1998;macsweeneyet al. 2002, 2008a; Cardin et al., 2013; Emmorey et al. 2014), confirming that our sign stimuli were perceivedasmovingbodyparts,fromwhichlinguisticinformationcouldbeextracted. Working(memory(effects.(NeuralactivationselicitedbyWMwereevaluatedwiththecontrast[WM> Colour]acrossallgroupsofparticipants.Thisresultedintypicalfrontoaparietalactivationsforworking memory(fig.3a;table4),includingbilateraldorsoalateralprefrontalcortex(dlpfc),frontaleyefields, preasupplementary motor area (preasma), insula, intraparietal sulcus (IPS), precuneus, posterior middletemporalgyrus,andthalamus. 16

17 Tounderstandtheeffectsofcongenitaldeafnessoncorticalcognitiveprocessing,weaskedwhether additionalordifferentregionswererecruitedforwmprocessingindeafindividuals.tothisend,we firstevaluatedthedeafnessxtaskinteraction[deaf(wm>colour)>hearing(wm>colour)].this contrastshowedactivationsalongthewholelengthofthestc(fig.3b).toensurethatthisfinding wasduetostrongeractivationsforwminthegroupofdeafindividualsandnotbecauseofweaker activationsinhearingindividuals,welookedatthecontrast[wm>colour]forthedeafgroup,using theinteractioncontrastasamask(table5).figure3bshowsinredthattheposteriorregionsofthe STC,bilaterally,wererecruitedforWMindeafindividuals,butnotinhearingparticipants(signersand nonasigners; for separate results from each of these groups please see Sup. Fig. 3). Parameter estimatesfurtherrevealedthatthedifferentialactivationsinanteriorportionsofthestcobtained withtheinteractioncontrastwereduetodeactivationsduringthewmtaskinhearingindividuals,and nottorecruitmentoftheseregionsforwmindeafindividuals(fig.3c).deactivationsinanteriorstc were much reduced for WM in the group of deaf individuals, hence the significant result in the interaction. ToevaluatewhetherpartoralloftheWMeffectfoundinposteriorSTCinthedeafgroupwasa consequenceoflinguisticprocessing,weevaluatedtheinteractionbetweentaskandstimuliwiththe contrast[(wmsigns>wmobjects)>(coloursigns>colourobjects)].thiscomparisonalsoallowed ustoidentifymechanismsthatwerespecifictosignlanguagewm,andnotgeneralvisualwmortaska independentlinguisticprocessing.wetestedthisinteractionseparatelyforeachofthethreegroups, andnosignificantactivations(p<.05fwe)werefound.asobservedintheparameterplotsinfig.3c, strongeractivationswerepresentforwminbothstimulusconditions:signsandobjects.furthermore, nospecificactivationsforsignlanguagewmwerefoundeitherinsignersorinnonasigners.these resultsshowthatstcisrecruitedforwmwhenthereislinguisticcontentinthestimuli,butalsowhen itdoesnothaveanyexplicitlinguisticinformation. 17

18 The results presented above suggest that deaf individuals have the capacity to assign additional corticalresourcestowm.itispossiblethattheseextraresourcesreducewmprocessingdemandsin frontoaparietalregionstypicallyinvolvedinthisfunction.toevaluatewhetheranycorticalregions wererecruitedtoalesserextentindeafindividualsthaninhearingindividualsduringthewmtask, weevaluatedtheinteractioncontrast[hearing(wm>colour)>deaf(wm>colour)].forsimplicity, fromthispointonwards,wewillrefertothiscontrastasthereverseinteraction.resultsofthereverse interactioninfig4andtable6showseveralfrontoaparietalregionsthatwerelessactiveindeafthan inhearingindividualsduringthewmtask(fig.4,blueclusters),includingtheleftpreasma,dlpfcand inferiorparietallobule(ipl),andbilateralintraparietalsulcus(ips).ascanbeseeninthetoppanelof Fig.4(blueclusters),theresultsobtainedwiththereverseinteractionoverlapwiththefrontoaparietal networkrecruitedforwminhearingindividuals(fig.4,toppanel,yellowclusters),suggestingthat theadditionalrecruitmentofstcforwminthedeafgroupisaccompaniedbynonarecruitment or weaker recruitment of some frontoaparietal regions. To confirm this, we plotted the reverse interactiontogetherwiththecontrast[wm>colour]forthedeafgroup(fig.4,bottompanel,red clusters).theslicesandparameterestimatesinthebottompaneloffig.4showthatregionssuchas preasmaandtheleftiplareindeedrecruitedforwmindeafparticipants,buttoalesserstrength thaninthehearinggroups.instead,partsoftheleftdlpfcandbilateralipsareclearlyrecruitedfor WMinhearingparticipants,butnotindeafparticipants.Theseresultsshowthat,whencomparedto hearingindividuals,deafindividualsrecruitstcforwmwhilealsorecruitingtoalesserextentfrontoa parietalregionstypicallyinvolvedinwm.thisisinlinewithourpreviousstudy,inwhichweshowed additionalrecruitmentofstcforgeneralvisuoaspatialprocessingindeafindividualstogetherwith weakeractivityinoccipitoaparietalregionstypicallyinvolvedinthesefunctions(cardinetal.2016).in thepresentstudy,weobservedasimilartrendinfig.2,whererecruitmentofstcforsignlanguage processingindeafindividualsisalsoassociatedwithweakerrecruitmentofleftalateralisedperisylvian regions(fig.2;sup.fig.2). 18

19 Functional( Connectivity( Analysis. Results from our taskabased fmri study suggest that, in deaf individuals,thestcistakingoversomeofthefunctionsthatinhearingindividualsareperformedby thefrontoaparietalnetworkforwm.ifthisisthecase,thestcshouldsomehowbeincorporatedinto thisfunctionalnetworkoffrontoaparietalregions.suchincorporationcanbemeasuredintermsof functional connectivity during restingastate, when the lowafrequency fluctuations of spontaneous activityoffunctionallyrelatedareasishighlycorrelated(biswaletal.1997).therefore,ifstcispart ofthefrontoaparietalnetworkindeafindividuals,acorrelationbetweenactivityinstcandinfrontoa parietalregionsshouldbefoundinthedeafparticipants,butnotinthegroupsofhearingindividuals. To test this hypothesis, resting state functional connectivity was performed in a completely independentfunctionalscan,whichprecededthetaskabasedfmriandinwhichparticipantswerenot performingatask;therefore,connectivityresultswerenotbiasedbyourfindingsinthetaskabased fmri.wehypothesisedthatifthestcistakingoversomeofthefunctionsoffrontoaparietalregions, achangeinfunctionalconnectivitybetweenthestcandtheregionsthatareweaklyrecruitedforwm shouldbeobservedasaconsequenceofdeafness.clusterssignificantlyactiveforbothinteraction contrastswereusedasroisintheconnectivityanalysis(tables5and6).theseincluded:rightandleft STC, bilateral preasma, left DLPFC, left IPL, and right and left IPS. For each group separately, the averagesignalfromeachroi(source)wascorrelatedwiththesignalofeveryotherroi(target).figure 5(Sup.Fig.4)andTable7showtheresultsofthisanalysis.Inbothgroupsofhearingindividuals, signers and nonasigners, significant positive correlations were found between frontal and parietal regionsathisresultisexpectedfromroiswhicharepartofthesamefunctionalnetwork.rightand leftstcwerenotpositivelycorrelatedtoanyoftheseregions.incontrast,inthedeafgroup,stcis positively correlated to frontal regions. Specifically, in this group, activity in left STC is positively correlatedtopreasma,andactivityinrightstciscorrelatedwithactivityinpreasmaandleftdlpfc. These correlations are not found in either hearing group. Twoasample tatests were conducted to comparethestrengthofconnectivitybetweenstcandallotherseedsindeafandhearingindividuals. Significantdifferenceswerefoundbetweendeafandhearingindividualsinconnectivitybetweenprea 19

20 SMAandbothright(p=.03)andleft(p=.01)STC,andbetweenrightSTCandleftIPL(p=.03;Fig.5; Sup.Fig.4;Table7).AscanbeobservedinTable7,differencesinconnectivitybetweenSTC(bilaterally) andpreasmaareduetoalargerpositivecorrelationinactivitybetweentheseareasindeafsigners, than in the groups of hearing individuals. Between right STC and left IPL, the connectivity is significantlymorenegativeforthehearinggroupsthanforthegroupofdeafindividuals.however, functionalconnectivitybetweenrightstcandleftiplisonlysignificantinthegroupofhearingnona signers;therefore,aneffectofsignlanguageknowledgecannotbeexcludedatthispoint. These results show a difference in functional connectivity between STC and frontoaparietal regions as a consequenceofdeafness,andsuggestthatthestcmaybeincorporatedintoafunctionalnetworkfor cognitivecontrolindeafindividuals. DISCUSSION&& Early sensory experience shapes cortical organisation (e.g. Hensch, 2004). In cases of auditory deprivation in early life, crossmodal visual and somatosensory reorganisation has been found in corticesthatusuallyhaveauditoryfunctions(hickoketal.1997;finneyetal.2001;karnsetal.2012; Cardinetal.2013).Here,weshowthatauditorydeprivationalsoresultsincorticalreorganisationfor cognitiveprocessing.threefindingssupportthis:recruitmentofthestcforvisualwmindeaf,but notinhearingindividuals;weakerrecruitmentoffrontoaparietalregionsforwmindeafthanhearing individuals; and differences in functional connectivity between STC and frontoaparietal regions involvedinwmasaconsequenceofauditorydeprivation.together,theseresultssuggestanetworka widereorganisationforcognitiveprocessingasaresultofauditorydeprivationitself,andnotlateand insecurelanguageacquisition,northeknowledgeanduseofavisuoaspatiallanguage. 20

21 Inaddition,wefoundnoevidenceoflinguisticWMmechanismsexclusivetosignlanguageineither deaforhearingsigners,suggestingthatsignlanguagewmreliesongeneralvisuoaspatialandlanguage networks. Crossmodal&Plasticity:&Preservation&of&function&vs&Functional&Shift& Crossmodallyreorganisedsensorycorticeshavebeenshowntopreservetheiroriginalfunctionbut adapt to respond to a different sensory input (Lomber et al. 2010). This has been causally demonstratedbylomberetal.(2010)intheauditorycortexofcats.coolingspecificcorticalauditory areas,theyshowedthatregionsthatinhearingcatsareinvolvedinprocessingsoundlocalisationand sound movement, in deaf cats are respectively involved in visual localisation and visual motion. Importantly,thisreorganisationconferredondeafcatsbehaviouraladvantagesoverhearinganimals. Thispreservationoffunctioncanalsobeobservedindeafhumans,bothforsensoryandcognitive processes(cardinetal.2013).inthatpaper,itwasshownthattheleftstc,whichisusuallyinvolved inspokenlanguageprocessing,isrecruitedforsignlanguageprocessing,butnotforgeneralvisuoa spatialprocessing. Inthisstudy,wereportadifferenttypeofplasticityasaconsequenceofauditorydeprivation:the recruitment of STC for WM, suggesting a functional shift from the role this cortex has in hearing individuals. This kind of functional shift has also been observed in visual cortical areas of blind individuals,whereresponsestolanguagehavebeenfoundinregionswhichareusuallyinvolvedin lowalevelvisualprocessing(röderetal.2002;amedietal.2004;bednyetal.2015;laneetal.2015). AtypicalWMtaskinvolvesseveralstagesofrepresentation expectation,encoding,maintenanceand retrieval allinfluencedbyselectiveattention(gazzaleyandnobre,2012).thespecificroleofstc duringwmindeafindividualsisnotyetclear.thedifferentialrecruitmentbetweendeafandhearing individualsofstcandfrontoaparietalnetworksforwm,accompaniedbydifferencesinrestingstate connectivitybetweentheseregions,suggeststhatthereorganisedstcmightbeincorporatedintoa network for cognitive control. However, we cannot unequivocally assert this until the type of 21

22 computationthestcisperformingduringwmisidentified.itispossiblethatthestcissimplystoring visual information, in the same way that, in hearing individuals, this region retains auditory informationduringworkingmemorytasks(linkeandcusack2015).thepositivecorrelationbetween restingstateactivityinstcandfrontalregionsindeafindividuals,whenparticipantsdonothaveto doaworkingmemorytask,speaksagainstthisargument,buttoconfidentlyrejectthisinterpretation weneedfurtherevidence,fromneuroimagingandelectrophysiologicalstudies,showingthetypeof informationbeingrepresentedinthiscortex. AnotherpossibilityisthattheactivityobservedinSTCduringtheWMtaskisaneffectoflanguage processing.asexplainedabove,stchasbeenshowntobeinvolvedinlanguageprocessingindeaf individuals(nevilleetal.1998;macsweeneyetal.2002;emmoreyetal.2011;leonardetal.2012; Cardinetal.2013).Replicatingthispreviousresearch,ourresultswiththecontrast[signs>objects]in thedeafgroupshowbilateralactivationofstc.thus,thequestionariseswhetherthiseffectofwm isreallyalanguageeffect.althoughwecannotcompletelyruleoutthisinterpretationatthispoint, severalpiecesofevidencesuggestthatthisisnotthecase:thewmeffectispresentwithbothsigns andnonasigns,sotherecruitmentofstcforworkingmemoryisindependentoftheexplicitlinguistic contentofthestimuli.itcouldstillbearguedthatparticipantsverbalisetheiranswers,andthuscause STCactivations.However,itisnotclearwhyonlydeafparticipantswouldbeusingthisstrategy,asit would be expected that hearing participants would use a similar strategy, also recruiting STC. Furthermore, in their WM study, Ding et al. (2015) used a lowalevel visuoaspatial task in which participantshadtorememberthelocationofagrating.herestcisalsorecruitedforwmindeaf individuals,eventhoughverbalisationisnotlikelytobeausefulstrategygiventhelargenumberof targetlocations. STC&is&recruited&for&working&memory&in&deaf&individuals& PreviousstudiesondeafindividualswerecontradictoryinrelationtotherecruitmentoftheSTCfor WM.Dingetal.(2015)showedrecruitmentofSTCforvisualshortatermmemoryindeafindividuals 22

23 butnotinhearingcontrols.however,itwasnotclearwhetherthiseffectwasmediatedbydeafness per(seorbysecondaryeffectsoflanguageoncorticalreorganisation(seebelow).buchsbaumetal. (2005)alsoshowedSTCrecruitmentintheirWMstudy,butconstrainedtoleftposteriorregions.This resultissimilartothatobtainedinastudyofwmforspeechandsigninhearingsigners(paetal. 2009),suggestingthatthisregionisgenerallyinvolvedinlinguisticWM,andisnotreorganisedasa consequence of deafness. Furthermore, Bavelier et al. (2008) showed recruitment of STC in deaf individuals during visual shortaterm memory encoding, but not during the maintenance period, suggestingasensoryroleforthiscortex.severaldifferencesbetweentheexperimentalgroupsand taskstestedinthesepreviousstudiescouldexplainthecontradictoryresults.buchsbaumetal.(2005) andbavelieretal.(2008)testeddeafnativesigners,apopulationwithnormallanguagedevelopment. Instead,itislikelythatonaveragethepopulationofDingetal.(2015)hadlateandinsecurelanguage development(spokenandsigned),givenearlyonsetofdeafness,andlateonsetofhearingaiduse (averageage10.9years)andlateageofsignlanguageexposure(average6.8years).thisraisesthe possibilitythatsomeoftheeffectsfoundbydingetal.(2015)arenottheresultofdeafnessper(se, butasecondaryeffectoflateandinsecurelanguageacquisition.inthissituation,therecruitmentof STCforworkingmemory,andperhapsothercognitivefunctions,couldbeacompensatorymechanism developed during infancy either because language cannot be used effectively to aid cognition or becausetheamountoflanguageinputisnotenoughtofullydevelopthefunctionofstcinlanguage processing.ineithercase,therecruitmentofstcforwmwillbetheresultoflanguagedelay,or languagedelaycombinedwithauditorydeprivation,butnotaneffectofsensorydeprivationper(se. Thus,iftherecruitmentofSTCforWMisdrivenbyinsecureordelayedlanguagedevelopment,we will not see STC recruitment for WM in native sign language users, given that in this population languagedevelopmentisnormal. Alternatively, contradictory results in previous studies could be due to reorganisation of cortical networksinresponsetolifealongexperiencewithasignlanguage,whichprovidesanadditionalsource 23

24 of variability when there is linguistic content in the tested stimuli. Buchsbaum et al. (2005) and Bavelieretal.(2008)usedWMtaskswithlinguisticstimuli,whereasDingetal.(2015)usedbasicvisuoa spatialstimuli.giventhatthestcisinvolvedinsignlanguageprocessingindeafindividuals(e.g.cardin etal.2013),itispossiblethatpreservingthislanguagefunctioninnativesignerscouldhaveprevented itsinvolvementinwm. Inthisstudy,wespecificallydesignedanexperimenttoaddressthesecontradictions: 1)! Wetesteddeafnativesigners,whodonothavelanguagedelay 2)! Wedesignedourstimulitohavelinguisticandnonalinguisticcontent,butused pointalight displays,sothattheycouldbedirectlycomparedwithoutresultsbeingdrivenbydifferences insensoryfeatures 3)! Wetestedhearingnonasignersandhearingnativesigners,aswellasdeafnativesigners,to exclude the possibility that effects were driven by sign language knowledge, and not by deafnessper(se. Usingthisdesign,wefoundthatposteriorportionsoftheSTCarerecruitedforvisualWMindeaf individuals.thisstcrecruitmentisaccompaniedbyweakerrecruitmentoffrontoaparietalregions typically involved in WM, and by functional reorganisation of networks involved in cognitive processing.ourdesignallowsustoassertthattheseeffectsare: 1)! duetodeafnessper(se,andnotlateandinsecurelanguageacquisition,astheywerefoundin adeafpopulationwhodonothavelanguagedelay 2)! independentoflinguisticprocessingofthestimuli,asstcwasrecruitedforwmwhenstimuli weresignsfrombslandwhentheywerenonsenseobjects. 3)! independentofthenativeuseofavisuoaspatiallanguage,astheseeffectswerefoundindeaf nativesigners,butnotinhearingnativesigners. 24

25 RecruitmentoftheSTCforWMindeafindividualsislargelyconstrainedtoitsposteriorportion,but thespatialextentoftherecruitmentweobservedisbilateralandextendsanteriorlybeyondtheleft STCregionreportedbyBuchsbaumetal.(2005). Bavelieretal.(2010)foundthatregionsintheinferiorparietallobulewereactivatedtoalesserextent indeafsignersthanhearingspeakersduringthemaintenanceperiodofwm.onthecontrary,intheir study,thesameparietalregionsweremoreactiveindeafsignersthanhearingspeakersduringthe recall phase. These results suggest that the differences in activation of parietal regions that we observebetweendeafandhearingindividualsinthisstudy,andperhapsdifferencesalsoinother frontoaparietalregions,mightbeassociatedwiththemaintenancecomponentofworkingmemory, more than encoding or recall. However, this will remain an open question until further studies specificallytestthishypothesis. Sign&language&WM&vs&spoken&language&WM& Signed and spoken languages differ in their underlying sensory and motor processes: while sign languages are visualamanual languages, spoken languages are auditoryaoral languages. From a cognitiveperspective,signlanguagescanbeusedastoolsforinvestigatingtowhatdegreeneural processesarebasedon,orareindependentof,underlyingsensoryandmotormechanisms.previous studieshaveshowngreatcommonalityinthecorticalmechanismssupportingwmforsignedand spokenlanguages(rönnbergetal.2004;rudneretal.2007,2009;paetal.2009),suggestinglargely modalityaindependentmechanismsforlinguisticwm.however,corticalactivationselicitedbysigned andspokenlanguagewmalsodifferedinsomeimportantaspects.specifically,thereareactivations forsignlanguagewminsuperiorparietalandlateraloccipitoatemporalregions.previously,itwasnot clear whether these differences were driven by signaspecific activations reflecting more spatially orientatedlinguisticprocessinginsignlanguagethaninspeech,oriftheywereduetodifferencesin thesensorypropertiesofthestimuli. 25

26 Bycombiningstimulispecificallydesignedtominimisedifferencesinbasicvisualfeatures,andgroups ofparticipantswithandwithoutknowledgeofasignlanguage,inthisstudywewereabletosegregate effectsthataredrivenbygeneralvisuoaspatialprocessingofthestimulifromthosethataredrivenby linguisticprocessing.wefoundnoeffectsthatwerespecifictosignlanguagewm.therefore,we suggestthatlanguagemodalitydifferencesbetweenspeechandsignwmfoundinpreviousstudies of linguistic WM, are at least partially due to sensory differences in visuoaspatial and auditory processing,andnotexclusivetolinguisticwmmechanismsforsignlanguage. Faster&visual&working&memory&reaction&times&for&deaf&signers&& Behavioural results from this study show that deaf individuals were significantly faster than both groupsofhearingindividualsinthewmtasks.thisadvantageispresentforbothtypesofstimuli,signs andobjects,andwasnotobservedinthecolourtask.thisisinagreementwithpreviousliterature showingthatdeafnativesignersoutperformnonasignersonwmtasksinvolvingvisualdynamicspatial information(seekeehnerandatkinson2006,forareview): a! deafsigningchildrenhaveenhancedmemoryforspatiallocations,orientationandmovement onthecorsispatialspantask(wilsonetal.1997). a! deaf signing children and adults are better at analysing and recalling dynamic point light displays, such as Japanese Kanji figures or Chinese pseudocharacters (Poizner et al. 1989; Klimaetal.1999). a! deafchildrenoutperformhearingchildrenonshortatermmemoryforcomplexanddynamic visual figures, provided the task does not rely on serial recall (Todman and Cowdy, 1993; TodmanandSeedhouse,1994;Klimaetal.1999). a! deafadultsshowfasterreactiontimesinavisuoaspatialshortatermmemorytask(dingetal. 2015). 26

27 Ascanbeobservedinthisstudy,theadvantageobservedinreactiontimesintheWMtaskdoesnot generalisetoaccuracyorspan,eitherwithinthesamefmriwmtaskorinthepreascreeningtests(see alsoemmoreyetal.,2017,andandinetal.,2013,forotherexamplesandanextensivediscussion). This supports the view that better performance in deaf individuals occurs at the level of visual processingspeed,oratthelevelofresponsegenerationandselection,assuggestedbyseveralstudies showingfasterreactiontimesfordeafindividualsinvisualdetectiontasks(bottarietal.2010;pavani andbottari,2012). Anothersignificantbehaviouralresultfromourstudyisthatsigners,bothhearinganddeaf,performed betterthannonasignersacrossbothtasks,wmandcolour.itispossiblethatthisresultreflectsareal perceptualadvantagedrivenbytheexperienceofknowingasignlanguage,giventhatithasbeen previously found that deaf signers were better at encoding and reproducing pseudocharacters presentedaspointlightdisplays(klimaetal.,1999).furtherstudieswithpointlightdisplaysinsigners andnonasignerswillbeneededinordertoassertthisconfidently. CONCLUSION&& Here we have shown that deaf individuals recruit bilateral STC during WM, independently of the linguisticcontentofthestimuli.thisisaccompaniedbyareductionintherecruitmentofparietaland frontalregionstypicallyassociatedwithwminhearingindividuals,suggestingthatthestcmightbe takingovercognitivefunctionsusuallyperformedbythesefrontoaparietalregions.usingrestingstate connectivity analysis, we also found a difference in the pattern of connectivity between frontal, parietalandstcregionsbetweenthedeafsignersandhearingindividuals,whetherornottheywere signers. This suggests a functional shift towards cognitive processing in STC as a consequence of crossmodalreorganisation. 27

28 Acknowledgments: This work was supported by the Swedish Council for Working Life and Social Research(2008a0846),theSwedishResearchCouncil(349a2007a8654,LinnaeusCentreHEAD),andby grantsfromtheeconomicandsocialresearchcouncilofgreatbritain(resa ;RESa620a 28a0002)totheDeafnessCognitionandLanguageResearchCentre.WewouldliketothankMischa CookeandIndieBeediefortheirhelpwiththerecruitmentofparticipants,andAnnaaLenaStrohand RebeccaMurphyfortheirhelpwithinitialdatacollection.Wekindlythankallthedeafandhearing participantswhotookpartinourstudy.theauthorsdeclarenocompetingfinancialinterests. REFERENCES& Amedi A, Floel A, Knecht S, Zohary E, Cohen LG Transcranial magnetic stimulation of the occipitalpoleinterfereswithverbalprocessinginblindsubjects.natneurosci.7: AndinJ,OrfanidouE,CardinV,HolmerE,CapekCM,WollB,RönnbergJ,RudnerM.2013.Similardigita basedworkingmemoryindeafsignersandhearingnonasignersdespitedigitspandifferences. FrontPsychol.4:942. Bavelier D, Newman AJ, Mukherjee M, Hauser P, Kemeny S, Braun A, Boutla M Encoding, rehearsal,andrecallinsignersandspeakers:sharednetworkbutdifferentialengagement.cereb Cortex.18: BednyM,RichardsonH,SaxeR Visual cortexrespondstospokenlanguageinblindchildren. JNeurosci.35: BehzadiY,RestomK,LiauJ,LiuTT.2007.Acomponentbasednoisecorrectionmethod(CompCor)for BOLDandperfusionbasedfMRI.NeuroImage.37: BiswalBB,VanKylenJ,HydeJS.1997.SimultaneousassessmentofflowandBOLDsignalsinrestinga statefunctionalconnectivitymaps.nmrbiomed.10:

29 BottariD,NavaE,LeyP,PavaniF.2010.Enhancedreactivitytovisualstimuliindeafindividuals.Restor NeurolNeurosci.28: Botting NF, Jones A, Marshall C, Denmark T, Atkinson J, Morgan G Nonaverbal executive function is mediated by language: A study of deaf and hearing children. Child Dev. doi: /cdev BuchsbaumB,PickellB,LoveT,HatrakM,BellugiU,HickokG.2005.Neuralsubstratesforverbal workingmemoryindeafsigners:fmristudyandlesioncasereport.brainlang.95: BialystokE,CraikFIM,LukG.2012.Bilingualism:consequencesformindandbrain.TrendsCognSci. 16: CapekCM,WatersD,WollB,MacSweeneyM,BrammerMJ,McGuirePK,DavidAS,CampbellR Hand and mouth: cortical correlates of lexical processing in British Sign Language and speechreadingenglish.jcognneurosci20: CardinV,OrfanidouE,KästnerL,RönnbergJ,WollB,CapekCM,RudnerM.2016.Monitoringdifferent phonological parameters of sign language engages the same cortical language network but distinctiveperceptualones.jcognneurosci.28: CardinV,OrfanidouE,RönnbergJ,CapekCM,RudnerM,WollB.2013.Dissociatingcognitiveand sensoryneuralplasticityinhumansuperiortemporalcortex.natcommun.4:1473. ChaiXJ,CastañónAN,ÖngürD,WhitfieldaGabrieliS.2012.Anticorrelationsinrestingstatenetworks withoutglobalsignalregression.neuroimage.59: CormierK,SchembriA,VinsonD,OrfanidouE.2012.Firstlanguageacquisitiondiffersfromsecond language acquisition in prelingually deaf signers: evidence from sensitivity to grammaticality judgementinbritishsignlanguage.cognition.124: CorsiPM.1972.Humanmemoryandthemedialtemporalregionofthebrain(DoctoralDissertation). 29 Daneman M, Carpenter P. University Microfilms No. AA105a Abstracts International.

30 34:819B.McGillUniversity,Montreal. DingH,QinW,LiangM,MingD,WanB,LiQ,YuC.2015.Crossamodalactivationofauditoryregions duringvisuoaspatialworkingmemoryinearlydeafness.brain:138: Emmorey K, Giezen MR, Petrich JAF, Spurgeon E, O'Grady Farnady L The relation between workingmemoryandlanguagecomprehensioninsignersandspeakers.actapsychol.177:69a77. EmmoreyK,McCulloughS,MehtaS,GrabowskiTJ.2014.Howsensoryamotorsystemsimpactthe neuralorganizationforlanguage:directcontrastsbetweenspokenandsignedlanguage.front Psychol.5:484. EmmoreyK,XuJ,BraunA.2011.Neuralresponsestomeaninglesspseudosigns:evidenceforsigna basedphoneticprocessinginsuperiortemporalcortex.brainlang.117: FinneyEM,FineI,DobkinsKR.2001.Visualstimuliactivateauditorycortexinthedeaf.NatNeurosci. 4: GathercoleSE,BaddeleyAD.1993.Workingmemoryandlanguage.HoveandNewYork:Psychology Press. GazzaleyA,NobreAC.2012.Topadownmodulation:bridgingselectiveattentionandworkingmemory. TrendsCognSci.16:129a135. HenschTK.2004.Criticalperiodregulation.AnnuRevNeurosci.27: Hickok G, Poeppel D, Clark K, Buxton RB, Rowley HA, Roberts TP Sensory mapping in a congenitallydeafsubject:megandfrmistudiesofcrossamodalnonaplasticity.humbrainmapp. 5: Johansson G Visual perception of biological motion and a model for its analysis. Percept Psychophys.14: KarnsCM,DowMW,NevilleHJ.2012.Alteredcrossamodalprocessingintheprimaryauditorycortex 30

31 of congenitallydeafadults:avisualasomatosensory fmri study with a doubleaflash illusion. J Neurosci.32: KeehnerM,AtkinsonJ.2006.Workingmemoryanddeafness:Implicationsforcognitivedevelopment and functioning. In SJ Pickering (Ed), Working memory and education. San Diego: Academic Press. Klima ES, Tzeng OJ, Fok YYA, Bellugi U, Corina D, Bettger JG From sign to script: Effects of linguisticexperienceonperceptualcategorization.journalofchineselinguisticsmonographseries. 13:96a129. LaneC,KanjliaS,OmakiA,BednyM Visual cortexofcongenitallyblindadultsrespondsto syntacticmovement.jneurosci.35: LeonardMK,FerjanaRamirezN,TorresC,TravisKE,HatrakM,MayberryR,HalgrenE.2012.Signed wordsinthecongenitallydeafevoketypicallatelexicosemanticresponseswithnoearlyvisual responsesinleftsuperiortemporalcortex.jneurosci.32: LinkeAC,CusackR.2015.FlexibleInformationCodinginHumanAuditoryCortexduringPerception, Imagery,andSTMofComplexSounds.JCognNeurosci.27: LomberSG,MeredithMA,KralA.2010.Crossamodalplasticityinspecificauditorycorticesunderlies visualcompensationsinthedeaf.natneurosci.13: LynessCR,WollB,CampbellR,CardinV.2013.Howdoesvisuallanguageaffectcrossmodalplasticity andcochlearimplantsuccess?neuroscibiobehavrev.37: MacSweeneyM,CapekCM,CampbellR,WollB.2008a.Thesigningbrain:theneurobiologyofsign language.trendscogsci.12: MacSweeneyM,WatersD,BrammerMJ,WollB,GoswamiU.2008b.Phonologicalprocessingindeaf signersandtheimpactofageoffirstlanguageacquisition.neuroimage.40:

32 MacSweeneyM,WollB,CampbellR,McGuirePK,DavidAS,WilliamsSCR,SucklingJ,CalvertGA, BrammerMJ.2002.NeuralsystemsunderlyingBritishSignLanguageandaudioavisualEnglish processinginnativeusers.brain.125: MarshallC,JonesA,DenmarkT,MasonK,AtkinsonJ,BottingN,MorganG.2015.Deafchildren snona verbalworkingmemoryisimpactedbytheirlanguageexperience.frontpsychol.6:527. Mayberry RI, Chen JaK, Witcher P, Klein D Age of acquisition effects on the functional organizationoflanguageintheadultbrain.brainlang.119: MerabetLB,PascualaLeoneA.2010.Neuralreorganizationfollowingsensoryloss:theopportunityof change.natrevneurosci.11: NevilleHJ,BavelierD,CorinaD,RauscheckerJ,KarniA,LalwaniA,BraunA,ClarkV,JezzardP,Turner R.1998.Cerebralorganizationforlanguageindeafandhearingsubjects:biologicalconstraints andeffectsofexperience.procnatlacadsciusa.95: Pa J, Wilson SM, Pickell H, Bellugi U, Hickok G Neural organization of linguistic shortaterm memory is sensory modalityadependent: Evidence from signed and spoken language, J Cogn Neurosci.20: PavaniF,BottariD.2012.Visualabilitiesinindividualswithprofounddeafness:acriticalreview.In: MurrayMM,WallaceM(eds)Frontiersintheneuralbasesofmultisensoryprocesses.CRCPress, BocaRaton, PelphreyKA,MorrisJP,MichelichCR,AllisonT,McCarthyG.2005.Functionalanatomyofbiological motion perception in posterior temporal cortex: An fmri study of eye, mouth and hand movements.cerebcortex.15: PénicaudS,KleinD,ZatorreRJ,ChenJ,WitcherP,HydeK,MayberryR.2013.Structuralbrainchanges linkedtodelayedfirstlanguageacquisitionincongenitallydeafindividuals.neuroimage.66:

33 PoiznerH,FokA,BellugiU.1989.Theinterplaybetweenperceptionoflanguageandperceptionof motion.lang(sciences,(11:267a287. Röder B, Stock O, Bien S, Neville H, Rösler F Speech processing activates visual cortex in congenitallyblindhumans.eurjneurosci.16: RönnbergJ,RudnerM,IngvarM.2004.Neuralcorrelatesofworkingmemoryforsignlanguage.Brain ResCognBrainRes.20: RudnerM,AndinJ,RönnbergJ.2009.Workingmemory,deafnessandsignlanguage.ScandJPsychol. 50: RudnerM,FranssonP,IngvarM,NybergL,RönnbergJ.2007.Neuralrepresentationofbindinglexical signsandwordsintheepisodicbufferofworkingmemory.neuropsychologia.45: RudnerM,KeidserG,HyggeS&RönnbergJ.2016.Bettervisuoaspatialworkingmemoryinadultswho reportprofounddeafnesscomparedtothosewithnormalorpoorhearing:datafromtheuk Biobankresource.EarHear.37: Runeson S, Frykholm G Kinematic specification of dynamics as an informational basis for personaandaactionperception:expectation,genderrecognition,anddeceptiveintention.jexp PsycholGen.112: TodmanJ,CowdyN.1993.Processingofvisualaactioncodesbydeafandhearingchildren:Coding orientationormacapacity?intelligence.17:237a250. TodmanJ,SeedhouseE.1994.Visualaactioncodeprocessingbydeafandhearingchildren.LangCog Processes.9:129a141. TurnerML,EngleRW.1989.Isworkingmemorycapacitytaskdependent?JMemLang.28: WechslerD.1999.WechslerAbbreviatedScaleofIntelligence(WASI).ThePsychologicalCorporation. WhitfieldaGabrieliS,NietoaCastanonA.2012.Conn:Afunctionalconnectivitytoolboxforcorrelated 33

34 andanticorrelatedbrainnetworks.brainconnect.2: Wilson M, Bettger JG, Niculae I, Klima ES Modality of language shapes working memory: EvidencefromdigitspanandspatialspaninASLsigners.JDeafStudDeafEduc.2:150a

35 Table&1.&Groups&demographics&and&pre6screening&tests.&& Age Gender WASI Corsi OperationSpan mean(range) s.e.m mean s.e.m mean s.e.m mean s.e.m HearingNon<Signers(N=16) 28.3(19<52) 2.1 7m/9f * ** 0.28 HearingSigners(N=16) 29.9(21<48) 2.1 5m/11f * DeafSigners(N=12) 25.7(19<33) 1.4 6m/6f * ** 0.57 AllHearing(N=32) 29.1(19 52) m/20f AllSigners(N=28) 28.1(19 48) m/17f Gender:m=male;f=female;*Significantdifferencesbetweenhearingsignersanddeafsigners(t(26)=3.4;p=.002),andbetweenhearingsignersandhearingnon<signers(t(30)=2.2;p=.040).** Significantdifferencesbetweendeafsignersandhearingnon<signers(t(26)=2.1;p=.049).s.e.m.=standarderrormean. 35

36 Table&2.&Performance&in&the&working&memory&(WM)&and&colour&tasks.&& d ReactionTimes(ms) Signs Objects Signs Objects WM colour WM colour WM colour WM colour mean s.e.m. mean s.e.m. mean s.e.m. mean s.e.m. mean s.e.m. mean s.e.m. mean s.e.m. mean s.e.m. HearingNon<Signers HearingSigners DeafSigners Allhearing Allsigners

37 ! Table!3.!Peak!activations!for!the!contrast![Signs!>!Objects]! Peak!Voxel! Name!!! p!(corr)! Z!scores!! x! y! x!! Hearing!NonFSigners! Posteriortemporal/Lateraloccipital R < :73 1 L :45 :79 4 Hearing!Signers! Posteriortemporal/Lateraloccipital R < :70 1 L < :45 :67 7 InferiorTemporalGyrus L :42 :43 :14 InferiorFrontalGyrus L.029* 3.6 : Deaf!Signers! Posteriortemporal/Lateraloccipital R :70 1 SuperiorTemporalCortex L.005* 4.0 :66 :37 7 R.019* :13 :2 Thetableshowstheactivationpeaksforthecontrast[Signs>Objects]foreachgroupseparately.L:left.R:right. Corr:allvaluesFWEcorrectedatp<.05,withtheexceptionof*,whichareFDR:correctedatclusterlevel. 37

38 Table!4.!Peakactivationsforthecontrast[WM>Colour]forallgroupsofparticipants.! Peak!Voxel! Name!!!! Z!scores!! x! y! x!! DLFPC R > L >8 : FrontalEyeFields R > L >8 :27 :1 58 IntraparietalSulcus L >8 :36 :46 43 R >8 42 :43 49 Precuneus L >8 :12 :64 52 R >8 9 :64 52 pre:sma R,L > Insula L >8 :30 23 :2 R > :2 PosteriorMiddleTemporalGyrus L >8 :51 :64 :5 R :52 :5 Thalamus R :10 4 L 5.8 :12 :16 10 Thetableshowsthepeakofactivationsforthecontrast[WM>Colour]forallgroupsofparticipants.L: left.r:right.dlpfc:dorso:lateralprefrontalcortex.pre:sma:pre:supplementarymotorarea.allpeaks significantatp<.0001(fwecorrected). 38

39 Table!5.!Regions!more!active!in!deaf!individuals!for!the!working!memory!task.!! Peak!voxel Name!! p!(corr)! Z!score!! x! y! z! PosteriorSuperior TemporalCortex L < :54 :43 16 R :31 4 Thetableshowsthepeakofactivationsforthe[WM>Colour]inthedeafgroup.Resultswere maskedwithaninclusivemaskforthecontrast[deaf(wm>colour)>hearing(wm>colour)],to includeonlyvoxelsthatweremoreactiveforthewmtaskonlyindeafindividuals(p<.005;inclusive masking).l:left.r:right.corr:pvaluefwecorrected. 39

40 ! Table!6.!Peaks!of!regions!more!active!in!hearing!individuals!for!the!working!memory!task.! Peak!Voxel! Name!!! p!(corr)! Z!scores!! x! y! x!! pre:sma L < : IntraparietalSulcus L :21 :67 43 R.04* :52 43 InferiorParietalLobule L :48 :46 46 DLPFC L : Thetableshowsthepeakofactivationsforthecontrast[Hearing(WM>Colour)>Deaf(WM>Colour)].L:left. R:right.Corr:allvaluesFWEcorrectedatp<.05,withtheexceptionof*,whichiscorrectedatclusterlevel. DLPFC:dorso:lateralprefrontalcortex.SMA:supplementarymotorarea 40

41 Table!7.!RestingFstate!functional!connectivity!results.!! DeafSigners HearingSigners HearingNon:Signers LSTC RSTC 0.637!! 0.631! LSTC DLPFC LSTC pre:sma 0.144*! :0.001 LSTC LIPL :0.024! F0.104! F0.121 LSTC LIPS :0.123! F0.144! F0.095 LSTC RIPS F0.192!! F0.087! F0.110 RSTC DLPFC 0.116! :0.004 RSTC pre:sma 0.163*! : RSTC LIPL :0.001*! :0.060 F0.208 RSTC LIPS :0.161! F0.175! F0.165 RSTC RIPS F0.163!! F0.124! F0.138 DLPFC pre:sma 0.263!! 0.418! DLPFC LIPL 0.328!! 0.569! DLPFC LIPS 0.145!! 0.259! DLPFC RIPS ! pre:sma LIPL ! pre:sma LIPS : :0.049 pre:sma RIPS :0.117 :0.008 :0.086 LIPL LIPS 0.368!! 0.514! LIPL RIPS 0.178!! 0.319! LIPS RIPS 0.617!! 0.627! ThetableshowcorrelationcoefficientsoffMRIactivityinrestingstate.Statisticallysignificantcoefficientsareshowninbold(p <.05).*indicatesconnectivityofSTCissignificantly(p<.05)differentbetweendeafandhearingindividuals.L:left.R:right. DLFPC:Dorso:lateralprefrontalcortex.pre:SMA:pre:supplementarymotorarea.IPS:intraparietalsulcus.IPL:inferiorparietal lobule.therighthemisphereisshownontheright.ds:deafsigners.hs:hearingsigners.hns:hearingnon:signers. 41

42

43

44 A WORKING MEMORY > COLOUR ALL GROUPS x = -38 z = 41 B x = 45 x = 50 x = 55 x = 60 x = 64 right x = -47 x = -51 x = -57 x = -60 x = -64 left C WM Signs Colour Signs WM Objects Colour Objects -0.4 Deaf Signers Hearing Signers Hearing Non-Signers -0.6 Deaf Signers Hearing Signers Hearing Non-Signers Parameter Estimates Deaf Signers Hearing Signers Hearing Non-Signers -1 Deaf Signers Hearing Signers Hearing Non-Signers

45 A x = -3 z = 43 B 0.6 L DLPFC ( ) 2 presma ( ) x = DS HS HNS 0 DS HS HNS 1.2 L IPL ( ) 0.6 R IPS ( ) DS HS HNS z = DS HS HNS 0.7 L IPS ( ) DS HS HNS

46

47 Figure!Legends! Fig.!1.!Experimental!design!and!stimuli.!A:!Diagrammaticrepresentationofascanningrun.ITI:inter: trial interval B:! Structure of an experimental block. Using the same stimulus set, participants performedeitheraworkingmemory(wm)taskoracontrolcolourtask.intheexperiment,point: lights were displayed in colour. For a colour version of the stimuli, please see Sup. Fig. 1. C: Representativeexamplesofthestimuli.Thedashed:arrowrepresentsthepatternofmovementofthe display. Fig.!2.!Sign!stimuli!activate!biological!motion!cortical!areas!in!all!groups,!and!languageFprocessing! regions!in!signers.thefigureshowstheresultsforthecontrast[signs>objects]separatelyforeach groupofparticipants.contrastsaredisplayedatp<.005(uncorrected),butonlycorrectedresultsare discussedandreportedintable3.acolourversionofthisfigurecanbefoundinsup.fig.2.! Fig.! 3.!! Superior! temporal! cortex! (STC)! is! recruited! for! visual! working! memory! (WM)! in! deaf! individuals.a:!resultsofthecontrasts[workingmemory>colour]averagedacrossallgroupsand stimuli.therighthemisphereisshownontheright.contrastdisplayedatp<.05,fwe.b:resultsfor thedeafnessxtaskinteraction[deaf(wm>colour)>hearing(wm>colour)]areshowningreen. Only posterior STC regions are more active for working memory in deaf individuals (red clusters: overlap between the contrast [WM > Colour] in the deaf group only, and the deafness x task interaction).bothcontrastsdisplayedatp<.005forvisualisationpurposes,butallpeakssignificant atp<.05,fwe(seetable5).c:barplotsshowparameterestimatesfromthepeaksoftheredclusters (topplots)andthegreenclusters(bottomplots).peakvoxelsareindicatedbythecrosshairsinthe slices,andtheirmnicoordinatesaredisplayedatthetopofeachgraph.barrepresentmeans±s.e.m. DS:DeafSigners.HS:HearingSigners.HNS:HearingNon:Signers. 42

48 Fig.!4.!Weaker!recruitment!of!frontoFparietal!regions!for!working!memory!(WM)!in!deaf!individuals.! Top!panel:!Thefigureshowsinbluetheresultsofthedeafnessxtaskreverseinteraction[Hearing (WM > Colour) > Deaf (WM > Colour)]. This interaction contrast is shown at p <.005 for display purposes,butresultsarediscussedonlyiftheyachievedsignificanceatcorrected(p<.05,fwe)level (Table6).Thecontrast[Hearing(WM>Colour)]isshowninyellow(p<.05FWE).!Bottom!panel:! Resultsofthedeafnessxtaskreverseinteractionareshowninblue,overlappedwithresultsfromthe contrast[deaf(wm>colour)]inred(p<.001).!thebarplotsshowparameterestimates(means± S.E.M)forthepeaksofthedeafnessxtaskreverseinteraction(blueclusters).LDLPFC,RIPSandLIPS: parameterestimatesfrompeakvoxelsofthereverseinteractionexcludingthosevoxelsthatoverlap withthecontrast[deaf(wm>colour)].pre:smaandlipl:parameterestimatesfrompeaksofthe overlapbetweenthereverseinteractionandthe[deaf(wm>colour)]contrast.l:left.r:right.dlfpc: Dorso:lateralprefrontalcortex.Pre:SMA:pre:supplementarymotorarea.IPS:intraparietalsulcus.IPL: inferiorparietallobule.therighthemisphereisshownontheright.ds:deafsigners.hs:hearing Signers.HNS:HearingNon:Signers. Fig.!5.!Differences!in!functional!connectivity!between!STC!and!frontal!areas!in!deaf!and!hearing! individuals.!thefigureisagraphicalrepresentationoftheresting:statefunctionalconnectivityresults shownintable7.significantresting:statecorrelationcoefficientsareindicatedbyalinejoiningtwo givenrois.blacklinesindicatepositivecorrelations;greylinesindicatenegativecorrelations.stc: Superior Temporal Cortex. DLFPC: Dorso:lateral prefrontal cortex. Pre:SMA: pre:supplementary motorarea.ips:intraparietalsulcus.ipl:inferiorparietallobule.l:left.r:right.sup.fig.4isacolour versionofthisfigure. 43

49 A B Supplementary Figure 1 An example of a scanning run Block Structure ~ 28 s 2-3 s ~ 28 s 2-3 s ~ 28 s s ~ 28 s WM signs ITI Colour signs ITI Colour objects ITI WM objects WM task: Same or different to 2 steps back? Trial... n C signs Stimuli objects Trial 3 Trial 2 Colour task: Are there any yellow dots? Trial 1

50 Supplementary Figure 2 Signs > Objects Hearing Non-Signers Hearing Signers p <.005 Deaf Signers

51 Supplementary Figure 3 WORKING MEMORY > COLOUR (DS > HS) p <.005 WORKING MEMORY > COLOUR (DS > HNS) p <.005