Cognitive modeling for games and animation

Transcription

1 Give virtul chrcters n intellectul nd sensory boost to improve their chnces of survivl in nd control over their environments nd n Cognitive enhnced sense of physicl relity. Modeling for Gmes nd Animtion John Funge Cognitive modeling for gmes nd nimtion explores the provoctive but lrgely unchrted interfce between computer grphics nd rtificil intelligence. Tht interfce is now on the verge of explosive growth s new breed of highly utonomous, qusi-intelligent grphicl chrcters begins to populte the domins of production nimtion, gme development, nd multimedi content cretion, s well s distributed multiuser virtul worlds, e-commerce, nd other Web-enbled ctivities. The modeling of grphicl chrcters is multifceted endevor, progressing from geometric modeling t the bottom of the hierrchy, through intermedite-level physicsbsed modeling, up to behviorl modeling. My reserch hs sought to pioneer cognitive modeling s the hitherto bsent but substntive pex of the chrcter-modeling pyrmid (see Figure 1). Cognitive models go beyond behviorl models in tht they govern wht chrcter knows, how tht knowledge is cquired, nd how it cn be used to pln physicl nd sensing ctions. Cognitive models cn lso ply subsidiry roles in controlling cinemtogrphy nd lighting for computer gmes nd nimtion. Moreover, cognitive modeling ddresses chlleng- DIRECTING DICE" (CREATED BY STEPHEN CHENNEY WHILE AT THE UNIVERSITY OF CALIFORNIA, BERKELEY) 40 July 2000/Vol. 43, No. 7 COMMUNICATIONS OF THE ACM

2 COMMUNICATIONS OF THE ACM July 2000/Vol. 43, No. 7 41

3 Figure 1. Cognitive modeling is on top of the modeling hierrchy but works with the lower levels to crete visully compelling experience. To demonstrte the entire modeling hierrchy in one ppliction, Xioyun Tu nd I dpted n underse world [4, 11]. A geometric model cptures the form nd ppernce of the chrcters. A biomechnicl model cptures their ntomicl structure, including internl muscle ctutors, nd simultes the deformtion nd physicl dynmics of their bodies using physics model. A behviorl-control model implements simpler brin functions nd is responsible for motor, perception, nd low-level behvior control. The "merperson s" rective behvior system interprets intentions generted by the cognitive lyer, then genertes coordinted muscle ctions controlling locomotion by deforming the body to generte propulsion-inducing forces ginst the virtul wter. COGNITIVE MODELS BEHAVIOR BIOMECHANICS PHYSICS GEOMETRY 42 July 2000/Vol. 43, No. 7 COMMUNICATIONS OF THE ACM

4 ing problem closely relted to minstrem AI nd robotics reserch. Imgine virtul prehistoric world inhbited by Tyrnnosurus rex (T-rex) nd pck of Velocirptors (Rptors). Suppose tht, in smll numbers nd in open territory, the Rptors re no mtch for the T-rex. But pck of cunning Rptors conspires to fell their much lrger opponent. Through cognitive modeling, the Rptors htch strtegic pln n mbush. Bsed on their domin knowledge, they hve inferred tht the T-rex s size, her most importnt sset in open terrin, would hmper mneuverbility within nrrow pssge under stone rch. The leder of the pck plys the decoy, luring the unsuspecting opponent into the nrrow opening. Her pckmtes, ssuming positions ner both ends of the pssge, rush into it on commnd. Some Rptors jump on the T-rex, chomping on her bck while others bite her legs. Thus the pck overcomes the brute through strtegic plnning, coopertion, nd overwhelming numbers. This mbush scenrio is just one of the exciting possible pplictions of cognitive modeling. Before describing other pplictions, plese note the blloon over the T- rex s hed. Its contents represent the chrcter s own internl mentl model of its virtul world. It is this internl model I expressly refer to s cognitive model. Foundtionl work in behviorl modeling hs mde progress towrd self-nimting chrcters tht rect ppropritely usully, though not lwys, in the best interests of their own survivl to perceived environmentl stimuli. Without cognitive modeling, however, it would be difficult for both the gme developer nd the gme plyer to instruct these utonomous chrcters so they re ble to stisfy specific gols. Cognitive models do not function in vcuum. For exmple, the underse world in Figure 1 represents n ppliction in which ll lyers of the chrcter-modeling pyrmid must work coopertively to crete visully compelling experience. Therefore, much of my reserch dels with how cognitive modeling cn be perspicuously integrted into the modeling hierrchy. Even in self-contined complex, dynmic virtul worlds, life should pper s thrilling nd unpredictble to the chrcter s it does to the humn observer. instruction. This orgniztion is reminiscent of the clssic dictum knowledge + instruction = intelligent behvior from the field of AI tht (by seprting out knowledge from control) seeks to promote design modulrity. Domin (knowledge) specifiction involves dministering knowledge to the chrcter bout its world nd how tht world cn chnge. Chrcter instruction involves telling it how to try to behve within the world. These instructions cn involve detiled step-by-step directions or, lterntively, provide high-level gols, so the chrcter hs to work out for itself how to behve in order to chieve them. I refer to this high-level style of instruction s gol-directed behvior specifiction. The importnt middle ground between these two extremes step-by-step nd gol-directed instruction cn lso be exploited through the notion of complex ctions. As simple concrete exmple of cognitive modeling, I offer brief look t gol-directed specifiction pproch to synthesizing herding behvior. The exmple is from n ppliction in which the T-rex in Figure 1 utomticlly formultes plns for driving Rptors out of its volcnic territory through nrrow pssgewy into neighboring jungle territory [4, 5]. 1 The crnge in the figure demonstrtes why the Rptors hve good reson to fer the lrger, stronger T-rex should it come close. This is one piece of domin knowledge gme developer would need to give the T- rex bout its world nd bout the Rptors rective behvior (not unlike Crig Reynold s boids [10]). In totl, the gme developer needs to tell the T-rex the following four pieces of informtion: The Rptors re temporrily frightened if you pproch them; Frightened Rptors run wy from you; Unlrmed Rptors continue s they re; nd You cn t wlk through obstcles (see [4, 5] for the code I used to provide this nd other knowledge to the T-rex). Herding Behvior I decompose cognitive modeling into two relted subtsks: domin-knowledge specifiction nd chrcter 1 Given enough ptience, skill, nd ingenuity, skillful AI progrmmers could progrm step-by-step instructions for herding behvior. Using gol-directed specifiction llows them to do the sme thing with reltive ese. COMMUNICATIONS OF THE ACM July 2000/Vol. 43, No. 7 43

5 Figure 2. This sitution tree shows some of the min ides behind precondition xioms, effect xioms, nd complex ctions; see [4, 5, 8] for more complex ppliction domins. AN EFFECT AXIOM: occurrence move(d) results in position = djcent(p,d) when position = p N E S W SOME COMPLEX ACTIONS (i) while (position!= gol) [specifies whole tree!] pick(d) move(d) (ii) move(e); move(e) or move(s); pick(d) move(d); A PRECONDITION AXIOM: ction move(d) possible when c = djcent(position,d) && notmember(c,visited) && Free(c) A gol sitution Pruned by precondition conjunct: notmember(c) Pruned by precondition conjunct: Free(c) Pruned by complex ction (ii) Bredth-first serch of whole tree Bredth-first serch fter precondition Depth-first serch fter complex ction (ii) To get the T-rex to herd the Rptors towrd prticulr loction, the developer hs to give it the gol of getting more Rptors heding in the right direction thn re currently heding tht wy. This gol, long with the supplied domin knowledge, enbles the T-rex to pln its ctions s if it were smrt sheepdog. It utonomously devises collision-free pths to mneuver in nd round groups of Rptors in order to frighten them in the desired direction. This strtegy enbles the T-rex (which cn pln up to six moves hed of its current position) to quickly expel the unruly mob of Rptors from its territory. Longer-durtion plns degrde rel-time performnce nd re rrely useful, since the Rptors obstcle-voidnce routines men the second nd third ssumptions in its domin knowledge re only pproximtions of their true behvior. A better strtegy is dptive herding through periodic re-plnning. Semntics of Cognitive Modeling The sitution clculus cn be employed to provide simple, powerful, nd prticulrly elegnt semntics for cognitive modeling. This AI formlism, invented in the 1960s by John McCrthy of Stnford University, describes chnging worlds using sorted firstorder logic. From the gme developer s point of view, the underlying theory cn be hidden. To this end, I creted the Cognitive Modeling Lnguge (CML) to ct s high-level interction lnguge. CML s syntx employs descriptive keywords with precise mppings to the underlying forml semntics of the sitution clculus. Detils of the sitution clculus s more modern incrntions re well documented in numerous ppers nd books (such s [8]). A sitution represents snpshot of the stte of the world. Any property of the world tht cn chnge over time is known s fluent. Primitive ctions re the fundmentl instrument of chnge in the ontology. The sometimes-counterintuitive term primitive serves only to distinguish certin tomic ctions from complex compound ctions. The possibility of performing n ction in given sitution is specified by precondition xioms. Effect xioms give necessry conditions for fluent to tke on given vlue fter performing n ction. Unfortuntely, effect xioms do not necessrily prescribe wht remins unchnged when n ction is performed nd thus cn led to unexpected results. Enumerting ll the non-effects could, however, require the ddition of n exponentil number of frme xioms. This burdensome requirement would be pinstking nd error-prone; constntly considering ll of them would slow ny chrcter s rection time. A solution is to ssume the effect xioms enumerte ll 44 July 2000/Vol. 43, No. 7 COMMUNICATIONS OF THE ACM

6 possible wys the world cn chnge. In 1991, Ry Reiter of the University of Toronto showed how this ssumption cn be incorported through strightforwrd syntctic mnipultion of the user-supplied effect xioms to utomticlly generte set of successor stte xioms [9]. Sketch Plns with Complex Actions The ctions, effect xioms, nd preconditions I ve described cn be thought of s tree (see Figure 2). The nodes of the tree represent situtions, while effect xioms describe the chrcteristics of ech sitution. At the root of the tree is the initil sitution; ech pth through the tree represents possible sequence of ctions. The precondition xioms men tht some sequences of ctions re not possible. This winnowing of possible ctions is represented in the figure by the pruned-wy blck portion of the tree. If some situtions re desired gols, gme developer cn use conventionl logic progrmming pproch to utomticlly serch the tree for sequence of ctions to get to the gol. The green nodes in the figure represent gol situtions; vrious serch strtegies cn be used to come up with n pproprite pln. The problem with plotting long-rnge plns is tht the serch spce grows exponentilly in the depth of the tree. Much of the plnning literture hs sought to mitigte this problem with more sophisticted serch lgorithms, such s the well-known A* lgorithm, nd stochstic plnning techniques. A gme developer cn push the ide of pruning the tree further by using complex ctions to prune wy rbitrry subsets of the serch spce. How the chrcter hs been progrmmed to serch the remining spce is n importnt but independent problem for which ll previous work on plnning is pplicble. The right side of Figure 2 is n exmple of complex ction nd its corresponding effect of reducing the serch spce to the tree s blue region (see [4, 5, 8] for more intuitive exmples of complex ctions nd their definitions). The point I wnt to mke here is tht complex ctions provide convenient tool for encoding heuristic knowledge bout problem s nondeterministic behvior outline. By nondeterministic, I men multiple possibilities cn be covered in one instruction, not tht the behvior is rndom. This progrmming style llows mny behviors to be specified more nturlly, simply, nd succinctly t much higher level of bstrction thn would be possible otherwise. In generl, the serch spce of gme chrcter s options is still exponentil, but pruning with complex ctions llows the formultion of potentilly longer plns, yielding chrcters tht pper to the gme plyer lot more intelligent nd lot more fun nd entertining. Vrious reserch tems hve pplied AI techniques to produce inspiring results with nimted humns nd crtoon chrcters [1, 2, 6, 11] (see Tble 1). My own use of cognitive modeling is exemplified in three cse studies [4, 5]. The first is the dinosur-herding ppliction discussed erlier. The next, suggested by Eugene Fiume of the University of Toronto, pplies cognitive modeling to cinemtogrphy. One of my ims ws to show how seprting out the control Tble 1. Notble online resources relted to cognitive modeling. Web site URL Contins Bruce Blumberg Ppers, teching mterils Cognitive Robotics Group Golog source code, ppers, teching mterils John Funge Animtions, CML source code, ppers, tutorils Henry Kutz Ppers, source code, teching mterils John Lird i.eecs.umich.edu/people/lird Ppers, teching mterils informtion from the bckground domin knowledge mkes it esier to understnd nd mintin controllers. The resulting cmer controller is ostensibly rective, mking miniml use of plnning, but it demonstrtes tht cognitive modeling subsumes conventionl behviorl modeling s limiting cse. For exmple, the underse world cse study strted out s the brinchild of Demetri Terzopoulos, lso of the University of Toronto, from which we creted n elborte chrcter nimtion to demonstrte how complex ctions cn be used to crete n interctive story by giving chrcters loose script, or sketch pln. At runtime, the underse chrcter uses its bckground knowledge to utomticlly decide for itself how to fill in the necessry missing detils while still following the bsic plot. Integrting Cognitive Modeling For pplictions like robotics nd computer gmes involving interction with the rel world, it is importnt for progrmmers to be ble to del with chrcter s uncertinty bout its world. Even in self-contined virtul worlds, life should pper s COMMUNICATIONS OF THE ACM July 2000/Vol. 43, No. 7 45

7 Figure 3. Pnel (b) outlines my vision of n rchitecture to supercede the more trditionl one in (), thus necessitting wy to represent chrcter s uncertinty. Pnel (c) shows the bsic intuition behind my use of uncertinty intervls tht grow over time until sensing collpses them bck to the their true vlue. Pnel (d) is my two-lyer concrete instntition of the new rchitecture. SENSE Compile knowledge SENSE THINK GAME () THINK HARD! THINK REACT GAME (b) thrilling nd unpredictble to the chrcter s it does to the humn observer. Compre the excitement of wtching chrcter run for cover from flling stck of bricks to one tht ccurtely precomputes brick trjectories nd, relizing it is in no dnger, stnds round nonchlntly while bricks crsh down round it. On more prcticl note, the expense of performing multiple specultive high-fidelity forwrd simultions could esily be prohibitive. It usully mkes fr more sense for chrcter to decide wht to do using simplified cognitive model of its world, sense the outcome, nd perform follow-up ctions if things don t turn out s expected. The upper-left qudrnt of Figure 3 depicts the trditionl sense-think-ct cycle dvocted in much of the literture nd widely used in computer gmes nd nimtion. During every such cycle, the chrcter senses its world, decides wht (if ny) ction to perform next, then performs it. For noninterctive nimtion, this cycle works well nd is conceptully simple. Unfortuntely, for rel-time pplictions, including computer gmes, the cycle forces chrcters to mke split-second decisions bout wht my be highly complex situtions. Therefore, I propose the lterntive txonomy depicted on the figure s lower-left ACT More thinking = more cycles! ACT Speed USER REACTIVE SYSTEM Intervl vlues I speed Actul speed vlues Sensing ction Domin specifiction Behvior specifiction Sensory informtion Low-level commnds (c) (d) Cognitive Model 1) Preconditions for performing n ction 2) The effect tht performing n ction would hve on the virtul world 3) The initil stte of the virtul world REASONING ENGINE Time Informtion bout virtul world qudrnt the tight sense-rect-ct cycle vitl for creting lively nd rective chrcters. It lso llows more thoughtful delibertion to be spred over mny cycles. This new rchitecture involves mny chllenges. From n implementtion perspective, the delibertive behvior should be n independent process tht cn be suspended, interrupted, even borted, depending on other rel-time constrints nd the chnging stte of the world. Fortuntely, this process is reltively strightforwrd; complictions rise from deeper technicl issues. In prticulr, if the chrcter is thinking over period of time, there should be some wy to represent its incresing uncertinty bout its world. Previous pproches to the problem in AI proposed the use of possible worlds to represent wht chrcter knows nd doesn t know. Unfortuntely, if the ppliction includes set of reltionl fluents whose vlues my be lerned through sensing, the progrmmer hs no choice but to list potentilly exponentil number of initil possible worlds. Things get more complicted with functionl fluents whose rnge is the rel numbers, since we cnnot list the vst number of possible worlds ssocited with uncertinty bout their vlues. Therefore, I propound the prcticble lterntive of using intervls nd intervl rithmetic to represent nd reson bout uncertinty [5]. Specificlly, I introduce the notion of intervl-vlued epistemic (IVE) fluents to represent chrcter s uncertinty bout the true vlue of the vribles within its world. The intuition behind this pproch is in the top-right qudrnt of Figure 3 whereby sensing corresponds to nrrowing intervls. IVE fluents present no more implementtion difficulties thn previous versions of the sitution clculus tht could not ccommodte sensing, let lone noisy sensors. I ve lso proved correctness nd completeness results with respect to the previous possibleworlds pproch. 2 Armed with vible pproch to representing uncertinty, progrmmer cn go even further. For exmple, one problem with sensing fixed number of 2 IVE fluents represent uncertinty intervls bout time-dependent vribles. They do not represent nd re unrelted to time intervls of the sort used in the underlying semntics of vrious temporl logics. 46 July 2000/Vol. 43, No. 7 COMMUNICATIONS OF THE ACM

8 Figure 4. Sils T. Dog (left), whose behvior is defined using n ethologiclly inspired rchitecture for building utonomous nimted cretures (courtesy Bruce Blumberg, MIT Medi Lb). A Quke II screenshot (right) from the plyer s perspective, with the ssilnt controlled by the Sor AI rchitecture (courtesy Mike vn Lent nd John Lird, University of Michign); inset imge is the mp disply tool showing the mp the Sorbot hs lerned during its explortion. inputs t set frme rte, then re-plnning, is tht it is wsteful if previously sensed informtion is still usble. Worse, chrcter might not be re-plnning often enough t criticl times. A gme progrmmer would therefore like to be ble to crete chrcters tht sense synchronously nd re-pln only when necessry. The width of n IVE fluent mesures the degree of uncertinty, possibly indicting uncceptbly outdted informtion. The first concrete instntition of this rchitecture ws relized in [4, 5]. The bottom-right qudrnt of Figure 3 shows the system consisted of just two levels. The low-level rective-behvior system Xioyun Tu of the University of Toronto nd I used ws (with minor modifictions) the rtificil life simultor she developed (under the supervision of Demetri Terzopoulos) [12]. Lerning Sensing llows chrcter to cquire for itself knowledge bout the current stte of its world. Since one of the mjor bottlenecks in cognitive modeling is defining nd refining chrcter s domin knowledge, it would be extremely useful for gme developer, s well s gme plyer, if the chrcter could utomticlly cquire knowledge bout its world s dynmics. Such knowledge cquisition is studied in the field of mchine lerning. A chrcter should lso be ble to endevor to lern not only bout how its world behves but bout how other chrcters, including humn vtrs, behve in it. The chrcter could even seek some mesure of self improvement by lerning bout its own behvior. For exmple, I envisge hierrchy of resoning models (lower-left qudrnt of Figure 3) chrcter might use to ponder its world t incresing degrees of sophistiction. This hierrchy could include even post-gme-nlysis level to develop better strtegies for the next time the gme is plyed. Idelly, some mechnism should exist through which the knowledge obtined vi delibertion t higher level cn be compiled down into one of the underlying representtions. This process should eventully propgte ll the wy down to the lowest rective level where knowledge cn be represented s simple, fst-executing rules. Moreover, with the dvent of the Internet, there is no reson why ll resoning modes in prticulr gme hve to be on the sme mchine. A gme console might communicte with online processing centers tht utomticlly generte new behvior rules s needed. In contrst, progrmming chrcter to lern simple things bout its world is reltively strightforwrd. For exmple, it is strightforwrd to progrm chrcters to utonomously mp out ll the obstcles by exploring their world in preprocessing step. To help them cquire higher fidelity, knowledge reserchers hve turned to incresingly sophisticted mchine-lerning techniques. One notble pproch is bsed on the Sor AI rchitecture, generl cognitive rchitecture for developing systems exhibiting intelligent behvior [8]. 3 3 Historiclly, Sor stood for Stte, Opertor, And Result, becuse ll problem-solving in Sor is regrded s serch through problem spce in which n opertor is pplied to stte to get result, though it is no longer regrded s n cronym nd is no longer written in upper cse. COMMUNICATIONS OF THE ACM July 2000/Vol. 43, No. 7 47

9 Figure 5. A scene with hundreds of utonomous chrcters intercting (it took dys to simulte nd render [9]) running in rel time on Sony PlySttion 2 (courtesy Crig Reynolds, left, nd Eric Lrsen, right, both Sony Computer Entertinment Americ). A frme from reltime physicl simultion of the medievl flil wepon further demonstrtes the potentil of future gme consoles. (Both imges rendered using Gbor Ngy s rel-time renderer.) This pproch enbles chrcter to lern the knowledge it needs by first wtching n expert complete the given tsk. By wy of nlogy with motion cpture, this process is referred to s behvior cpture. It ws initilly designed for developing intelligent ir-combt gents for militry pplictions. More recently, it hs been pplied to number of computer gmes, including Quke II, producing dethmtch Sorbots, some prompted by voice commnds (see Figure 4). The most importnt topic of behvior-lerning involves n pproch inspired by ethology, rther thn the more trditionl AI outlook behind most other cognitive modeling work [3]. For exmple, Bruce Blumberg of the MIT Medi Lb nd his tem re building virtul dog to determine how closely its behvior cn be mde to resemble tht of rel dog. In prticulr, the tem wnts it to be ble to lern the kinds of things rel dogs re cpble of lerning. Moreover, they wnt to be ble to trin it using stndrd niml-trining technique clled clicker trining. The Future of Cognitive Modeling Cognitive reserchers hve only begun to embrce vision of the untpped synergy between AI nd computer grphics. In the next two to five yers, the potentil for communiction mong cognitively enbled chrcters should provide fertile ground for reserch into developing chrcters cpble of sophisticted coopertive group behviors. Nturlly, one of the key fctors fueling interest in such dvnced modeling techniques s cognitive nd physics-bsed modeling is the rpid pce of hrdwre development. I m especilly excited bout the emergence of powerful new gme consoles (see Figure 5) promising to invigorte ech lyer of the modeling hierrchy to yield chrcters with unprecedented levels of interctivity nd physicl relism. c References 1. Bdler, N., Phillips, C., nd Zeltzer, D. Simulting Humns. Oxford University Press, New York, Btes, J. The role of emotion in believble gents. Commun. ACM 37, 7 (July 1994), Blumberg, B. Old Tricks, New Dogs: Ethology nd Interctive Cretures. Ph.D. thesis, MIT Medi Lb, Cmbridge, Mss., Funge, J., Tu, X., nd Terzopoulos, D. Cognitive modeling: Knowledge, resoning, nd plnning for intelligent chrcters. In Proceedings of SIG- GRAPH 99 (Los Angeles, Aug. 8 13, 1999); see lso Funge, J. Representing knowledge within the sitution clculus using IVE fluents. J. Relible Comput. 5, 1 (1999), Funge, J. AI for Gmes nd Animtion: A Cognitive Modeling Approch. A.K. Peters, Ntick, Mss., Hyes-Roth, B., vn Gent, R., nd Huber, D. Acting in chrcter. In Creting Personlities for Synthetic Actors, R. Trppl nd P. Pett, Eds. Springer-Verlg, Berlin, vn Lent, M. nd Lird, J. Lerning Tsk Performnce Knowledge Through Observtion. Ph.D. thesis, Deprtment of Electricl Engineering nd Computer Science, University of Michign, Levesque, H., Reiter, R., Lespérnce, Y., Lin, F., nd Scherl, R. GOLOG: A logic progrmming lnguge for dynmic domins. J. Logic Progrm. 31, 1 3 (Apr. 6, 1997), Reiter, R. The frme problem in the sitution clculus: A simple solution (sometimes) nd completeness result for gol regression. In Artificil Intelligence nd Mthemticl Theory of Computtion: Ppers in Honor of John McCrthy, V. Lifschitz, ed. Acdemic Press, Sn Diego, 1991, Reynolds, C. Flocks, herds, nd schools: A distributed behviorl model. Comput. Grph. 21, 4 (1987), Mgnent-Thlmnn, N. nd Thlmnn, D. Synthetic Actors in Computer-generted Films. Springer-Verlg, Berlin, Tu, X nd Terzopoulos, D. Artificil fishes: Physics, locomotion, perception, behvior. In Proceedings of SIGGRAPH 94 (Orlndo, Fl., July 1994), 43 50; see lso Artificil nimls for computer nimtion: Biomechnics, locomotion, perception, nd behvior. Lect. Notes Comput. Sci. 1635, John Funge (funge@dgp.toronto.edu) is reserch scientist in Sony Computer Entertinment Americ in Foster City, Clif ACM /00/0700 $ July 2000/Vol. 43, No. 7 COMMUNICATIONS OF THE ACM