A Computational Model of Dynamic Perceptual Attention for Virtual Humans

Transcription

1 14th Conference on Behavor Representaton n Modelng and Smulaton (BRIMS), Unversal Cty, CA, May 2005 A Computatonal Model of Dynamc Perceptual Attenton for Vrtual Humans Youngjun Km Randall W. Hll, Jr. Davd R. Traum Insttute for Creatve Technologes Fj Way, Sute 600 Marna del Rey, CA yjkm@ct.usc.edu, hll@ct.usc.edu, traum@ct.usc.edu Keywords: Percepton, Attenton, Vrtual human, on-verbal communcaton ABSTRACT: An mportant characterstc of a vrtual human s the ablty to drect ts perceptual attenton to objects and locatons n a vrtual envronment n a manner that looks belevable and serves a functonal purpose. We have developed a computatonal model of perceptual attenton that medates top-down and bottom-up attenton processes of vrtual humans n vrtual envronments. In ths paper, we propose a perceptual attenton model that wll ntegrate perceptual attenton toward objects and locatons n the envronment wth the need to look at other partes n a socal context. 1. Introducton Modelng nteractve vrtual humans has been one of the prmary goals of mmersve vrtual envronments for tranng. An mportant characterstc of a vrtual human s the ablty to drect ts perceptual attenton to objects and locatons n a vrtual envronment n a manner that looks belevable and serves a functonal purpose. ot only must the vrtual human pay attenton to objects related to the tasks t s performng, but t must also be able to cope wth sudden events that demand attenton. It s often the case that the amount of nformaton n the vrtual envronment far exceeds the processng abltes of the vrtual human. In fact, only a small fracton of sensory nformaton can be fully processed and assmlated nto the cogntve model. A successful model of perceptual attenton provdes a way of both prunng the ncomng sensory nput and choosng the most salent nformaton to focus on durng the next step of a decson cycle. The computatonal models of perceptual attenton that we surveyed fell nto one of two camps: top-down and bottom-up. Bologcally nspred computatonal models (Itt, 2001; Courty et al., 2003) typcally focus on the bottom-up aspects of attenton, whle most vrtual humans (Traum and Rckel, 2002; Chopra and Badler 2001; Marco and el, 2002; Conde and Thalmann 2004) mplement a top-down form of attenton. Bottom-up attenton models only consder the mage nformaton (e.g, color, ntensty, orentaton, and moton) wthout takng nto consderaton salency based on tasks or goals. As a result, the outcome of a purely bottom-up model wll not consstently match the beahvor of real humans n certan stuatons. Models lke Itt s (2001) can predct the bottom-up salence of features n an mage at any pont n tme, but such a model s not suffcent to predct where to actually look. Humans are generally task-orented, and t s safe to say that a great deal of one s tme s spent lookng at objects related to the current task. Modelng perceptual attenton as a purely top-down process, however, s also not suffcent for mplementng a vrtual human. A purely top-down model does not take nto account the fact that vrtual humans need to react to perceptual stmul vyng for attenton. For nstance, t s reasonable to expect that a loud nose, lke gunfre or an exploson, wll catch the attenton of vrtual human and nvoke some knd of mmedate reacton. Top-down systems typcally handle ths n an ad hoc manner by encodng specal rules to catch certan condtons n the envronment. The problem wth ths approach s that t does not provde a prncpled way of ntegratng the everpresent bottom-up perceptual stmul wth top-down control of attenton. One of the dstnctons between the work descrbed n ths paper and some of the other work on models of perceptual attenton s the purpose of the model n the context of a vrtual human. In many of the systems we revewed, the purpose of the percepton model was to make the vrtual human behave as though t was attendng to the surroundngs and tasks n a natural way. In contrast, our goal s also to develop vrtual humans that can perform tasks, react to contngences, nteract wth other agents, both vrtual and human, plan, and make decsons about what to do next or at some future tme (Hll, 2000). To accomplsh ths, perceptual attenton s a crtcally mportant mechansm for restrctng the sensory nformaton beng processed by the percepton module and controllng vrtual humans to exhbt goal-drected and reactve behavors. In the frst stage of perceptual attenton, there are mechansms that flter the nformaton that comes through the sensory system. Subsequent processes selectvely strengthen or weaken the prorty of

2 the nformaton. Drectng perceptual attenton toward the nterests of a partcular regon n space can be acheved by two dstngushable shfts; covert and overt shfts of perceptual attenton. It s well known that covert and overt attenton shfts affects gaze drecton to locatons n space (Wolfe, 1994). The sequences of gaze fxatons descrbe the way n whch overt attenton deployed, whereas drectng attenton to locaton n space wthout movng gaze descrbes the way n whch covert attenton s deployed. measured by observng ndvdual vsual attrbutes (e.g., sze, shape, and color). Low Medum 190 o 90 o 30 o Hgh Medum Low In ths paper, we present a computatonal model of perceptual attenton for vrtual humans. Ths model extends a pror model of perceptual resoluton (Hll, 2000) based on psychologcal theores of human percepton. Ths models allows vrtual humans to dynamcally nteract wth objects and other ndvduals, balancng the demands of goal-drected behavor wth those of attendng to novel stmul. Ths model has been mplemented and tested wth the MRE Project (Swartout et al, 2001). Top-down Vew Medum Hgh Medum 30 o 90 o 2. Modelng Percepton n Vrtual Humans Our vrtual humans are mplemented n Soar, a general archtecture for buldng ntellgent agents (ewell, 1990) and buld on the STEVE Archtecture (Rckel and Johnson, 1999) n the mmersve envronment called the Msson Rehearsal Exercse (MRE) (Swartout et al, 2001). The vrtual humans behavor s not scrpted; rather, t s drven by a set of general, doman-ndependent capabltes. The vrtual humans perceve events n the scenaro, by nteractng wth the smulator, reason about the tasks they are performng, and they control the bodes and faces of the PeopleShop anmated bodes to whch they have been assgned. We have developed a model of perceptual resoluton based on psychologcal theores of human percepton (Hll, 1999 and 2000). Hll s model predcts the level of detals at whch an agent wll perceve objects and ther propertes n the vrtual world. He appled hs model to synthetc helcopter plots n smulated mltary exercse. We extended the model to smulate many of the lmtatons of human percepton, both vsual and aural. 2.1 Vsual Percepton The vrtual human perceves dynamc objects, under the control of the smulator, by flterng updates (e.g., body locaton and orentaton, gaze locaton and orentaton, velocty, sze, and color) that the smulator perodcally broadcasts. As shown n fgure 1, we lmted the vrtual human s smulated vsual percepton to 190 horzontal degrees and 90 vertcal degrees so that the vrtual human only gets updates that he s currently sensng through the feld of vew (FOV). When the vrtual human senses the objects n the FOV, t frst processes how salent each object s n the respect of sze, dstance, and color. We consder the computatonal model (othegger et al., 2004) to compute the vsual salence of each object that s After computng the vsual salences of the perceved objects, we appled a sgmod functon as a utlty functon that reduces the degree of salence of an object n the respect of angle dspartes between the vrtual human and the object. Then we classfed the levels of salences on those objects as hgh, medum, or low, dependng on where the objects s n the vrtual human s feld of vew and whether attenton s beng focused on t. 2.2 Aural Percepton Sde Vew Fgure 1. Vrtual Human s Functonal Vsual Feld (FVF) To model aural percepton, we estmate the sound pressure levels of objects n the envronment and compute ther ndvdual and cumulatve effects on each lstener based on the dstances and drectons of the sources. Ths enables the vrtual humans to perceve aural events nvolvng objects not n the vsual feld of vew. For example, when a vrtual human hears a vehcle s approachng from behnd, he can choose to look over hs shoulder to see who s comng. Another effect of modelng aural percepton s that some sound events can mask others. A helcopter flyng overhead can make t mpossble to hear someone speakng n normal tones a few feet away. The nose could then prompt the vrtual human to shout and could also prompt the addressee to cup hs ear to ndcate that he cannot hear. Gven a set of vsually or aurally perceved objects, the agent s perceptual model updates the attrbutes of objects that fall n the lmted sensory range. At any pont n tme, the vrtual human must recognze whch object s the most salent among those objects and draw hs focus of attenton on the object. The next secton descrbes our approach to

3 computng the salence of the objects n the feld of vew and the subsequent behavors assocated wth shftng the agent s gaze. 3. Computatonal Model of Perceptual Attenton To compute object salence and to control gaze behavors, we have developed a model called Dynamc Perceptual Attenton (DPA). Internally, DPA combnes objects selected by bottom-up and top-down perceptual processes wth a decson-theoretc perspectve and then selects the most salent object. Externally, DPA controls an emboded agent s gaze not only to exhbt ts current focus of attenton but also to update belefs (e.g., poston) of the selected object. That s, the emboded agent dynamcally decdes where to look, whch object to look for, and how long to attend to the object. TOP-DOW TOP-DOW PROCESS PROCESS BOTTOM-UP BOTTOM-UP PROCESS PROCESS Prorty (objp) Concern (objc) Desred Goal Informaton (objdgi) Current Goal Informaton (objcgi) constant (k) World coordnates & veloctes BEEFIT COST movng gaze to the object Reward object Fgure 3. The nformaton flow of the DPA module Motor Control gaze Fgure 2. A snapshot of the MRE smulaton 3.1 Decson-Theoretc Control One of the consequences of modelng percepton wth lmted sensory nputs s that t creates uncertanty on each perceved object. For nstance, f an object that s beng tracked moves out of an agent s feld of vew, the perceptual attenton model ncreases the uncertanty level of the target nformaton of the object that a vrtual human tres to observe. To llustrate ths dea, consder the screen snapshot of the MRE smulaton shown n fgure 2. An njured boy s beng attended to by hs mother and a medc. A sergeant s conversng wth a human partcpant. Snce the mother, the boy, and the medc are out of the vsual feld of vew of the sergeant whle the sergeant s conversng wth the human, the sergeant s uncertanty levels about each of these characters wll ncrease wth tme. The nformaton flow of the DPA module s shown n fgure 3. Top-down and bottom-up processes provde nformaton to the DPA module n the form of tuples composed as follows: tuple = objp, objc, objdgi, objcgi, k where, objp : prorty of the tuple objc : concern of the tuple objdgi : desred goal nformaton of the tuple objcgi : current goal nformaton of the tuple k : constant for the tuple The prorty attrbute, objp, s used to ndcate the absolute mportance of an object, whereas the concern attrbute, objc, s used to ndcate a conflct between the desred goal nformaton (objdgi) and the current certanty of nformaton (objcgi). For nstance, even f a person s gven a hgh prorty task, he may not be concerned about montorng objects assocated wth the task f the task s gong well, resultng n less frequent observatons. If the task runs nto some dffcultes, he wll ncrease hs concern for the task, resultng n more frequent observatons. By consderng both attrbutes (.e., prorty and concern), our vrtual humans compute the benefts of attendng to objects. Informaton certanty s one of factors that help the vrtual human decde whch object t has to focus on. To deal wth certantes of the perceved objects, we have chosen to take a decson theoretc approach to computng the perceptual costs and benefts of shftng the focus of perceptual attenton of the perceved objects. In the next two sectons, we wll descrbe how to compute the perceptual costs and benefts of shftng the focus of perceptual attenton. The expected cost s computed by calculatng the perceptual cost of shftng the gaze to the selected object. The expected beneft s computed by consderng the value of acqurng accurate nformaton about the selected object. Once a decson has been made, DPA shfts the vrtual human s gaze to focus hs perceptual attenton on the object that has the hghest reward. 3.2 Computng the Beneft To compute the beneft of focusng perceptual attenton on an object requres the estmated values of object-based nformaton certanty. We consder object-based nformaton certanty as a key factor n computng the beneft of shftng the focus of attenton to the object. The term, object-based nformaton certanty, s used here to descrbe the level of nformaton certanty of an object rendered n the agent s mental mage of a vrtual world.

4 Humans determne the desred goal nformaton certanty of perceved objects (objdgi) based on ther subjectve preferences or predcton and then make efforts to mantan the current certanty of nformaton (objcgi) wthn a specfc range of objdgi, defned as the nformaton certanty tolerance boundary (ICTB). Informaton certanty s dynamc both n space and tme. If (objcgi) s out of ICTB, we actvate one of two knds of EEDs: the EED for observaton or the EED for nhbton. The EED for observaton s actvated f objcgi goes below ICTB lower. The EED of nhbton s actvated as objcgi goes over ICTB upper. Accordng to Klen s account of the nhbton of return (Klen, 2000), too much nformaton can be a bad thng. By modelng the nhbton of return, perceptual attenton wll not permanently focus on the most actve salent nformaton but wll ncrease the chances of dvertng perceptual attenton to less salent nformaton. The orthogonal process model between nformaton certanty and the EEDs of observaton and nhbton s shown n fgure 4. eed for Observaton HIGH eed for Inhbton 0.0 HIGH 0.0 Informaton Certanty Fgure 4. The nterrelaton of Informaton Certanty and eed The desred goal nformaton certanty (objdgi) s determned by the prorty attrbute (objp). The nformaton certanty tolerance boundary s set by the concern attrbute (objc). The hgher the concern attrbute s, the narrower the length of the boundary s. The current goal nformaton certanty of the target object (objcgi) s set by top-down and bottom-up processes. If a vrtual human cannot retreve any nformaton certanty of the target from top-down and bottom-up processes, t sets objcgi to 0. After the values for objcgi and nformaton certanty tolerance boundary are set, the vrtual human computes the EED for observaton or for nhbton on each tuple as follows:! EED(tuple ) =! # 1.0 " objp " exp 0 $ objp " exp f objcgi > ICTB upper f ICTB lower!objcgi!ictb upper f objcgi < ICTB lower where, "#= objcgi - ICTB upper and $#= ICTB lower - objcgi The EED on tuple s used as a force that produces a beneft of dvertng perceptual attenton nto tuple. The beneft s computed as follows: 2 EED( tuple ) BEEFIT ( tuple ) = 2 Once BEEFIT(tuple ) s computed, t wll used wth COST(tuple ) to compute the REWARD(tuple ). 3.3 Computng the Cost Even f the beneft of drawng attenton to one object s hgher than the benefts of attendng to others, the vrtual human should not automatcally select that object as the best one snce the cost of shftng the focus of attenton must also be consdered. To compute the cost of shftng perceptual attenton from one object to another, we consder two sets of factors: physcal and socal. Physcal factors nclude the degrees of head and eye movements and dstance effcency. Socal factors ndcate the relatve costs of perceptual gaze shfts n socal nteracton. For nstance, t may be rude to look away when someone s speakng (hgh cost of shft), yet t may be very mportant to attend to an unexpected or potentally dangerous event (hgh cost not to shft). 3.4 Shftng Perceptual Attenton Wth the beneft and two sets of cost factors of each tuple, we compute REWARD(tuple ) as follows: REWARD ( tuple ) = BEEFIT ( tuple )! COST ( tuple ) After calculatng REWARD(tuple) of all tuples, the vrtual human selects a tuple that has the hghest REWARD. If the selected tuple s holdng the current focus of perceptual attenton, the vrtual human wll keep focus on t. If not, t wll dvert ts perceptual attenton to the tuple havng the hghest REWARD. The duraton of a gaze at an object affects the nformaton certanty level. Whle a vrtual human gazes at an object obj (.e., overt montorng), objcgi ncreases. Lkewse, whle obj s montored only n the vrtual human s memory and projecton (.e., covert montorng), objcgi decreases. Covert montorng wll cause the certanty of nformaton to decay over tme. 4. Implementaton n MRE Scenaro When our scenaro starts, a smulated army vehcle carres a human partcpant (leutenant) to an accdent ste where an Army vehcle has crashed nto a cvlan car, njurng a

5 boy. The partcpant then takes on the task of drectng the troops to rescue the boy by nteractng wth our three emboded conversatonal agents the sergeant (SGT), the mother, and the medc. We controlled the sergeant s gaze movements wth DPA. The sergeant s ntally lookng at the boy to update the boy s health status wth nformaton. In a typcal example of nteracton wth the system, the leutenant starts wth a general nqury as to what s gong on, Sergeant, what happened here? Snce ths nqury s gven as an aural event, the aural percepton flters the aural event and then gves a tuple for the event to DPA. When DPA gets ths tuple from the aural percepton, DPA shfts the sergeant s perceptual attenton, whch currently attends to the boy, to react to the aural event. As the result of the shft of perceptual attenton, the sergeant recognzes that the leutenant made an nqury. Then, the sergeant nternally processes the nqury. As a result of consderng both perceptual objects (the boy and the leutenant) the sergeant turns from the boy and faces the leutenant, answerng, There was an accdent. Ths woman and her son came from the sde street and our drver ddn t see them. The Leutenant contnues by askng Who s hurt?, and the sergeant reples The boy and our drver. ow when the Leutenant asks How bad s the boy hurt?, rather than answerng drectly, the sergeant defers to the medc, who has better knowledge of such thngs, and drects hm to answer, by lookng at hm and callng hs name Tucc? Lookng up the medc answers. The boy has crtcal njures. Sr we need to get a medevac n here ASAP. The leutenant decdes to call for the medcal evacuaton helcopter as requested and secures the local area. Then the leutenant commands the sergeant to execute the task of settng up a landng zone (LZ) so that the helcopter can safely land. When the sergeant starts executng the secure-landng zone task, the sergeant contacts the 3 rd squad n order to dspatch the squad to the LZ. The nteracton between the nformaton certanty of poston of the 3 rd squad and the beneft of observng the nformaton s shown later wth graphs and trees. The sergeant ntally knows the poston of the 3 rd squad and the locaton of the LZ. When the sergeant contemplates executon of the task (secure-lz), he tres to gan hgh-level nformaton certanty of the spatal nformaton of the 3 rd squad, who wll be dspatched to the LZ to secure t. The nformaton graph on the task s shown as follows: eed for Observaton eed for Inhbton rd -squad-secure-lz Snce the squad s not n the area for securng the LZ, the sergeant s DPA module determnes there s a beneft derved by observng the current spatal nformaton of the squad. ext, the sergeant contacts and then commands the squad forward to the LZ. After he observes that the squad s movng toward the LZ, he reduces the slant of the curve snce he gets hopes of achevng the task that may be gven from the emoton module. The changed nformaton graph on the task s shown as follows: eed for Observaton eed for Inhbton rd -squad-secure-lz After the LZ s secured by the squad whom the sergeant hghly trusts, he does not need to mantan the nformaton status that the LZ s secure wth one hundred percent certanty but can lower the prorty of the nformaton (.e., lower the desred certanty of nformaton and expand the tolerance boundary). Wth ths shft n prorty, allows the sergeant to observe, search, or track other nformaton. The changed nformaton graph on the task s shown as follows:

6 eed for Observaton eed for Observaton LT: IS The LZ SECURE? 0.0 eed for Inhbton eed for Inhbton rd -squad-secure-lz Whle the sergeant observes, searches, or tracks other objects, the certanty of the nformaton of the securty of the LZ wll gradually decrease. The changed nformaton tree on the task s shown as follows: eed for Observaton eed for Inhbton rd -squad-secure-lz ext, the LT asks the sergeant, Is the LZ secure? Ths speech event ncreases the desred certanty of nformaton and makes the tolerance boundary narrow snce the sergeant wants to be very sure of the nformaton that he wll convey to hs superor offcer, the LT. The changed nformaton graph and tree on the task s shown as follows: rd -squard-secure-lz Ths speech event changes the beneft level by the changng the attrbutes of the tuple(3 rd -squad-secure-lz). Ths, n turn, affects the sergeant s emotonal state by ncreasng the degree of dstress, suggests that he should update the belef of the status of securty of the LZ. As the result, the sergeant gazes at the landng zone to determne whether t s stll secured by the dspatched squad members, and he responds wth the status of the landng zone to the leutenant. Ths example llustrates the mportance of gaze n acqurng perceptual nformaton and montorng task performance whle embedded n the socal context of conversaton. Our am s to have the sergeant s behavor seem approprate wthn ths context, both n terms of behavng human-lke and usng perceptual gaze to medate between costs and benefts of nformaton updatng actons. 5. Related Work There are few models or frameworks that address the ssue of where and what an observer should look at n a gven tme. Fndlay and Walker (Fndlay and Walker, 1999) present a comprehensve psychologcal model of the nformaton flow routes and compettve pathways n saccade generaton. Ther model has not been mplemented as a computatonal system yet, but t served as a source of nspraton for aspects of the work descrbed n ths paper. There are a number of comprehensve computatonal models of perceptual attenton for vrtual humans. Chopra-Khullar and Badler (Chopra-Khullar and Badler, 2001) bult one of the most extensve models to date, a psychologcally motvated framework for generatng the vsual attendng behavors of an anmated human fgure. Ther mplementaton generates belevable anmaton behavors for a vrtual human performng a farly scrpted set of tasks, but t s not clear how the model would fare n a much more dynamc envronment where

7 the need to react to events n the world s much hgher than the vrtual world they descrbe. The model appears to fall nto the top-down attenton category, where gaze behavors are scheduled and placed n a queue. Cassell and Vlhjalmsson (Cassell and Vlhjalmsson, 1999) have used gaze as an mportant communcatve behavor n ther anmated characters. Ther anmated characters have several lmtatons: (1) the model does not operate n real tme, (2) the model only ncludes conversatonal gaze, and (3) the model does not nclude varablty due to emotonal state or ndvdual dfferences. Rckel and Johnson (Rckel and Johnson, 1999) also employ gaze n ther tutorng agent, STEVE, who looks at the student durng conversatonal nteracton, and looks at objects n the envronment when performng tasks or montorng the student. Ther man purpose of adoptng eye movements nto agents s to generate eye movements for non-verbal communcaton (e.g. turn-takng) that are controlled by top-down attenton. The general lmtaton of STEVE s that a gaze command typcally comes at the begnnng of a cogntve actvty, but s not updated durng that actvty. So, for example, f STEVE starts talkng to a person, he gazes at them. Then, f hs attenton s drawn to an acton n the envronment, he wll reman gazng at that acton untl somethng else causes a gaze command. Hll (Hll, 1999, 2000) appled a smulaton of attenton for a vrtual helcopter plot. The vrtual helcopter plot selectvely draw attenton to an object(s)/area(s) based on features of objects and ther prorty to tasks, and perceptual groupng of objects. However, the helcopter plot has no anmaton of head and eye movements. We extended Hll s model of perceptual resoluton based on psychologcal theores of human percepton. 6. Relatonshp to Socal Attenton Whle the model of perceptual attenton presented above handles many aspects of gaze behavor, there s another factor n the broader scope of attenton. Informaton certanty s just one of the motvatons for gaze, but nformaton can be acqured through other means than gaze, and gaze can be used for more than acqurng nformaton. In ths secton, we descrbe how these features can be added to the perceptual attenton model, presented above, for a more complete model of gaze and attenton n vrtual humans. In a socal settng, t s often mportant to use gaze to regulate the flow of conversaton, ncludng sgnals of turn-takng, and feedback. Some of ths can be modeled drectly as a concern for nformaton certanty, such as needng to look at an addressee whle speakng to get nformaton about whether that addressee s lstenng, understands, and agrees. Lkewse, lookng away from an addressee whle plannng speech could perhaps be modeled as nhbton of ths feedback nformaton when more cogntve facltes are needed for plannng the utterance. Some other factors are less easly modeled as relatng to nformaton, however. An alternatve reason for gaze averson by a speaker s that t makes t harder for an addressee to take the turn by speakng. Gaze also can be used as a form of non-verbal communcaton, e.g., to drect the gaze of others to an object, even when one does not need more nformaton oneself. Another ssue s that napproprate gaze or averson can send undesred sgnals about the attenton and respect of the speaker or addressee ths wll need to be fgured n to the cost model. In a socal stuaton, perceptual attenton may sometmes nteract wth socal and conversatonal attenton. In the example gven n Secton 4, above, we already descrbed how a queston about a proposton can change the desred certanty of nformaton. Conversaton can also be used to affect the actual certanty. For example, rather than lookng at the landng zone, the Sgt mght nstead rado to the squad and ask them about the securty. We then have three means of montorng: covert montorng though memory and nference about future projecton, overt perceptual gaze, and socal montorng through (perhaps prompted) reports of other agents. It may be dffcult to arbtrate between these sources of nformaton when they conflct. For nstance, one may remember the landng zone as secure and have no reason for thnkng t wll change. On the other hand, a verbal status report may conflct wth ths. If trust s suffcently hgh n the report s certanty (and the trustworthness of the reporter), one may choose to overrde the covert montorng wth ths nformaton. Another opton s to motvate a new gaze to arbtrate. Socal means may also change the relatve costs of perceptual gaze shfts. For nstance, t may be very rude to look away when someone s speakng (hgh cost of shft), yet t may be very mportant to attend to an unexpected or potentally dangerous event (hgh beneft to shft). Wth hgh utlty on ether end, the choce may be dffcult, and moreover potentally very costly ether way. One way around ths s to reduce the cost of the shft wth a socal acton, such as apologzng, or usng a non-verbal gesture ndcatng that the speaker should wat a moment. 7. Dscusson and Future Work The proposed computatonal model for controllng the focus of perceptual attenton for vrtual humans provdes the potental to support mult-party dalogues n a vrtual world. As we begn to ntegrate perceptual attenton nto mult-party, mult-conversatonal dalogue layers (Traum and Rckel, 2002), we have demonstrated that vrtual humans can respond dynamcally to events that are not relevant to the tasks and shft ther attenton among objects n the envronment and have gotten postve feedback to nformal demonstratons. The model we have descrbed here s stll a prototype that has to be tuned and tested n

8 a wder range crcumstances. In partcular, by ntegratng more robust and delberate language tasks wth the model we have descrbed n ths paper, we beleve we have made progress toward natural gaze behavors n emboded conversatonal agents. In addton, by ntegratng the concept of measurng the salence of a specfc class of spatal features wth the model, we beleve that ths model wll provde a large potental for generatng more reactve and realstc bottom-up attenton. 8. Acknowledgment The project or effort descrbed here has been sponsored by the U.S. Army Research, Development, and Engneerng Command (RDECOM). Statements and opnons expressed do not necessarly reflect the poston or the polcy of the Unted States Government, and no offcal endorsement should be nferred. 9. References J. Cassell and H. Vlhjalmsson: Fully Conversatonal Avatars: Makng Communcatve Behavors Autonomous Agents and Mult-Agent Systems, 2:45-64, Kluwer Academc Publshers, S. Chopra-Khullar and. Badler: Where to Look? Automatng Attendng Behavors of Vsual Human Characters Autonomous Agents and Mult-Agent Systems, 4(1-2), pp.9-23, T. Conde and D. Thalmann, An Artfcal Lfe Envronment for Autonomous Vrtual Agents wth mult-sensoral and mult-perceptve features, Computer Anmaton and Vrtual Worlds, Volume 15, Issue 3-4, John Wley, Courty, E. Marchand, and B. Arnald: A ew Applcaton for Salency Maps: Synthetc Vson of Autonomous Actors, IEEE Int. Conf. on Image Processng, ICIP 03, Barcelona, Span, Sep M. Garau, M. Slater, S. Bee, and M. A. Sasse, The mpact of eye gaze on communcaton usng humanod avatars, ACM SIGCHI, M. Glles and D. el: Eye Movements and Attenton for Behavoural Anmaton, n The Journal of Vsualzaton and Computer Anmaton. 13: pp R. Hll: Modelng Attenton n Vrtual Humans Proceedngs of the 8 th Conference on Computer Generated Forces and Behavoral Representaton, SISO, Orlando, Fla., 1999 R. Hll: Perceptual Attenton n Vrtual Humans: Toward Realstc and Belevable Gaze Behavors Proceedngs of the AAAI Fall Symposum on Smulatng Human Agents, pp.46-52, AAAI Press, Menlo Park, Calf., R. Hll, J. Gratch, S. Marsella, J. Rckel, W. Swartout, and D. Traum: Vrtual Humans n the Msson Rehearsal Exercse System. Künstlche Intellgenz (KI Journal). Specal ssue on Emboded Conversatonal Agents, L. Itt and C. Koch, Computatonal Modelng of Vsual Attenton, ature Revews euroscence, Vol. 2, o. 3, pp , Mar 2001 C. othegger, S. Wnter, and M. Raubal, Selecton of Salent Features for Route Drectons. Spatal Cognton and Computaton 4(2): , R. Klen, Inhbton of return, Trends n Cogntve Scences, 4, Moray: Desgnng for attenton Attenton: Selecton, Awareness, & Control, Oxford Press, A. ewell: Unfed Theores of Cognton Cambrdge, MA: Harvard Unversty Press, J. Rckel and W. Lews Johnson: Anmated Agents for Procedural Tranng n Vrtual Realty: Percepton, Cognton, and Motor Control Appled Artfcal Intellgence, 13: , 1999 W. Swartout, R. Hll, J. Gratch, W.L. Johnson, C. Kyrakaks, C. LaBore, R. Lndhem, S. Marsella, D. Mragla, B. Moore, J. More, J. Rckel, M. ThÚbaux, L. Tuch, R. Whtney and J. Douglas: Toward the holodeck: Integratng graphcs, sound, character and story In Proceedngs of 5 th Internatonal Conference on Autonomous Agents, D. Traum and J. Rckel: Emboded Agents for Multparty Dalogue n Immersve Vrtual Worlds, AAMAS 02, July 15-19, 2002, Bologna, Italy. J. Wolfe: Guded Search 2.0: A revsed model of vsual search Psychonomc Bulletn & Revew, 1 (2), pp , Author Bographes Youngjun Km s a Ph.D. canddacy at Unversty of Southern Calforna and works wth Dr. Randall W. Hll, Jr. at ICT. Hs research focuses on modelng perceptual attenton for vrtual humans Randall W. Hll, Jr. s the Drector of Appled Research and Transton at ICT and a research assstant professor of Computer Scence at Unversty of Southern Calforna. He earned hs M.S. and Ph.D. degrees n computer scence from the Unversty of Southern Calforna n 1987 and 1993, respectvely. Hs doctoral thess showed how ntellgent tutorng systems can be used n nteractve smulatons to assst students n learnng how to operate complex equpment. Davd R. Traum s a Research Scentst at ICT and a research assstant professor of Computer Scence at Unversty of Southern Calforna. He completed hs PhD n Computer scence at Unversty of Rochester n Hs research focuses on collaboraton and dalogue communcaton between agents, ncludng both human and artfcal agents.