Learning and Vision Mobile Robotics Group Research Report

Transcription

1 Larning and Vision Mobil Robotics Group Rsarch Rport J. Andrad, F. Morno, R. Alquézar, J. Aranda, J. Climnt, A. Grau, E. Muñoz, F. Srratosa, E. Stafftti, J. Vrgés, T. Vidal, and A. Sanfliu Institut d Robòtica i Informàtica Industrial, UPC-CSIC Llorns Artigas 4-6, Barclona, Spain lrobots@iri.upc.s Abstract This articl prsnts th currnt trnds on whld mobil robotics bing pursud at IRI-UPC-CSIC. It includs an ovrviw of rcnt rsults producd in our group in a wid rang of aras, including robot localization, color invarianc, sgmntation, tracking, visual srvoing, and objct and fac rcognition. Kywords: robot localization, color invarianc, sgmntation, tracking, rcognition. 1 INTRODUCTION Th Larning and Vision Mobil Robotics Group, and intrdisciplinary tam of rsarchrs, is th product of a joint ffort btwn th Institut d Robòtica i Informàtica Industrial and th Dpartamnt d Enginyria d Sistmas, Automàtica i Informàtica Industrial at th Univrsitat Politècnica d Catalunya, and th Dpartamnt d Enginyria Informàtica i Matmàtiqus at th Univrsitat Rovira i Virgili. Hadd by Prof. A. Sanfliu, as of today, it mbracs 5 profssors, 2 posdoctoral associats, and 3 PhD studnts. Th group, consolidatd in 1996, has givn ris to 3 PhD thsis and 7 final yar projcts. Within th last 6 yars, th group has publishd 7 pr rviwd journal articls, 7 book chaptrs, 6 confrnc procding ditorials, and prsntd articls in ovr 40 intrnational confrncs(35indxdinsci),and15nationalconfrncs. Furthrmor, within th past two yars, th mobil robotics platforms dvlopd in our group hav bn portrayd numrous tims on liv and printd mdia [5, 7, 10, 19, 20, 21]. Financial support coms mainly from a continud st of projcts from th Intrministrial Council of Scinc and Tchnology (CICYT) TAP C04-03, TAP , and DPI CURRENT RESEARCH AREAS During th last yar, our fforts hav bn tailord at giving our mobil platforms th ability to navigat autonomusly in unknown structurd sttings. In this sns, w hav contributd nw insight in th classical simultanous localization and map building problm, from a control systms thory point of viw [1, 2, 6]. Furthrmor, w hav dvlopd nw fatur validation tchniqus that improv th robustnss of typical map building algorithms [3, 4]. Morovr, vry good rsults hav bn achivd in th tracking of subjcts undr varying illumination conditions and in cluttrd scns. On th on hand, w hav mastrd now th us of histogram basd tchniqus to color sgmntation and illumination normalization [8, 22, 23]. On th othr hand, w hav tstd diffrnt statistical stimation paradigms to track subjct candidats using not only color information but shap as wll [11, 12]. Most of our vido dmonstrations from last yar show rsults in this topic. With rspct to 3d objct rcognition and subjct idntification, w hav formalizd and validatd a compact rprsntation for a st of attributd graphs. W call this nw formulation a function dscribd graph, and it borrows th capability of probablisitic modlling from random graphs. FDG s ar th rsult of a long tim ffort in th sarch for a compact rprsntation of multipl viw modl dscriptors [13, 14, 15, 16, 17, 18]. In th following sctions w summariz th ky contributions from thr slctd publications [1, 11, 14]. Each of thm tackls vry diffrnt problms typically ncountrd in mobil robotics applications. 3 SIMULTANEOUS LOCALIZATION AND MAP BUILDING To univocally idntify landmarks from snsor data, w study svral landmark rprsntations,

2 and th mathmatical foundation ncssary to xtract th faturs that build thm from imags and lasr rang data. Th faturs xtractd from just on snsor may not suffic in th invariant charactrization of landmarks and objcts, pushing for th combination of information from multipl sourcs. Onc landmarks ar accuratly xtractd and idntifid, th scond part of th problm is to us ths obsrvations for th localization of th robot, as wll as th rfinmnt of th landmark location stimats. W considr robot motion and snsor obsrvations as stochastic procsss, and trat th problm from an stimation thortic point of viw, daling with nois by using probabilistic mthods. Th main drawback w ncountr is that currnt stimation tchniqus hav bn dvisd for static nvironmnts, and that thy lack robustnss in mor ralistic situations. To aid in thos situations in which landmark obsrvations might not b consistnt in tim, w propos a nw st of tmporal landmark quality functions, and show how by incorporating ths functions in th data association tsts, th ovrall stimation-thortic approach to map building and localization is improvd. Th basic ida consists on using th history of data association mismatchs for th computation of th liklihood of futur data association, togthr with th spatial compatibility tsts alrady availabl. Spcialattntionispaidinthatthrmovalof spurious landmarks from th map dos not violat th basic convrgnc proprtis of th localization and map building algorithms alrady dscribd in th litratur; namly, asymptotic convrgnc and full corrlation. W contribut also an in dpth analysis of th fully corrlatd modl to localization and map building from a control systms thory point of viw. Considring th fact that th Kalman filtr is nothing ls but an optimal obsrvr, w analyz th implications of having a stat vctor that is bing rvisd by fully corrlatd nois masurmnts. W nd up rvaling thortically and with xprimnts th strong limitations of using a fully corrlatd nois drivn stimation thortic approach to map building and localization in rlation to th total numbr of landmarks usd. Partial obsrvability hindrs full rconstructibility of th stat spac, making th final map stimat dpndant on th initial obsrvations, and dos not guarant convrgnc to a positiv dfinit covarianc matrix. Partial controllability on th othr hand, maks th filtr bliv aftr a numbr of itrations, that it has accurat stimats of th landmark stats, with thir corrsponding Kalman gains convrging to zro. That is, aftr a fw stps, innovations ar uslss. W show how to palliat th ffcts of full corrlation and partial controllability. Any map building and localization algorithm for mobil robotics that is to work in ral tim must b abl to rlat obsrvations and modl matchs in an xpditious way. Som of th landmark compatibility tsts ar computationally xpnsiv, and thir application has to b carfully dsignd. W touch upon th tim complxity issus of th various landmark compatibility tsts usd, and also on th dsirabl proprtis of our chosn map data structur. Furthrmor, w propos a sris of tasks that must b handld whn daling with landmark data association. From modl compatibility tsts, to sarch spac rduction and hypothsis formation, to th actual association of obsrvations and modls. Figur 1a shows som of th modl compatibility huristics dvisd for th validation of straight lins xtractd from lasr rang data into walls. Fram b shows data as xtractd from a lasr rang findr, and 2σ covarianc llipss around hypothsizd landmark stimats. Th third fram shows a virtual rality modl of th map constructd during a run of th algorithm. 4 FUSION OF COLOR AND SHAPE FOR OBJECT TRACKING UNDER VARYING ILLUMINATION Color rprsnts a visual fatur commonly usd for objct dtction and tracking systms, spcially in th fild of human-computr intraction. For such cass in which th nvironmnt is rlativly simpl, with controlld lighting conditions and an uncluttrd background, color can b considrd a robust cu. Th problm appars whn w ar daling with scns with varying illumination conditions and confusing background. Thus, an important challng for any color tracking systm to work in ral unconstraind nvironmnts, is th ability to accommodat variations in th amount of sourc light rflctd from th trackd surfac. Th choic of diffrnt color spacs lik HSL,normalizd color rgb, or th color spac (B G, G R, R + G + B), can giv som robustnss against varying illumination, highlights, intrrflctions or changs in surfac orintation for an analysis of diffrnt color spacs. But non of ths transfor-

3 s 0 20cm 5m s 0 s 5m 1 5 a) b) c) Figur 1: Concurrnt mobil robot localization and map building. a) Hypothsis sarch rang for walls xtractd from a lasr rang scan. b) Th blu dots indicat snsor raw data coming from a lasr rang findr. Th grn lins rprsnt walls infrrd from conscutiv radings. Th rd lins indicat th stimatd robot trajctory. c) Graphical rprsntation of th map built. o 1 s 2 2 mations is gnral nough to cop with arbitrary changs in illumination. Instad of sarching for color constancy, othr approachs try to adapt th color distribution ovr tim. On such tchniqu is to us Gaussian mixturs modls to stimat dnsitis of color, and undr th assumption that lighting conditions chang smoothly ovr tim, th modls ar rcursivly adaptd. Anothr option is to paramtriz th color distribution as a random vctor and to us scond ordr Markov modl to prdict th volution of th corrsponding color histogram. Ths tchniqus prform much bttr than th mr chang of color spac rprsntation, but hav th drawback that thy do not chck for th goodnss of th adaptation, which can still lad to failur. Th fusion of svral visual moduls using diffrnt critria offrs mor rliability than mthods that only us on fatur. As an xampl, systms that track in ral-tim an individual might modl th had of a prson by an llips and us intnsity gradints and color histograms to updat th had position ovr tim. In [12], color histograms ar fusd with strovision information in ordr to dynamically adapt th siz of th trackd had. Ths ral tim applications howvr ar constraind only to th tracking of lliptical shaps. A nw mthodology that addrsss th problms prsnt in th approachs dscribd abov, rsults in a robust tracking systm abl to cop with cluttrd scns and varying illumination conditions. Th fusion is don using th CONDENSATION algorithm that formulats multipl hypothsis about th stimation of th objct s color distribution and validats thm taking into account th contour information of th objct [9]. Four sts of squnc rsults ar summarizd in Figur 2 to illustrat th robustnss of our systm undr diffrnt conditions. In th first xprimnt w show how our systm is abl to accommodat color by applying it ovr a synthtic squnc of circls moving around and changing randomly its color. In th uppr lft fram of th figur th path of th color distributions for th trackd circl is shown. Th scond xprimnt is to track a colord rctangl. It has to b pointd out that in th prvious xprimnt w usd th RGB color spac, but in th prsnt and subsqunt xprimnts th color spac usd was th (B G, G R, R + G + B) inordrtoprovid robustnss to spcular higlights. Th last two xprimnts, corrspond to outdoor scns, whr although th chang in illumination conditions is limitd, thy ar usful to show that our mthod

4 Figur 2: Rsults of th 4 xprimnts. works with non-uniform shaps (third xprimnt of a batl tracking), and in cluttrd scnarios (fourth xprimnt of a snail tracking). and rlations (arcs). Du to th computational intractability of GRGs, causd by th difficulty in stimating and handling th high-ordr joint probability distribution, First-Ordr Random Graphs (FORGs) wr proposd for ral applications. Strong simplifications wr mad in FORGs to allow th us of random graphs in practical cass; mor prcisly, th following assumptions wr mad: a) th random vrtics ar mutually indpndnt; b) givn valus for th random vrtics, th random arcs ar indpndnt; c) th arcs ar indpndnt of th vrtics xcpt for th vrtics that thy connct. FDGs can b sn as a typ of simplification of th GRGs, diffrnt from FORGs, in which som structural constraints ar rcordd. A drawback of FORGs is that th strong assumptions about th statistical indpndnc of nods and arcs may lad to an xcssiv gnralisation of th sampl graphs whn synthsising a FORG. To allviat this waknss, a qualitativ information of th joint probabilitis of two nods is incorporatd into FDGs, thus improving th rprsntational powr of FORGs with a ngligibl incras of computational cost. 5 FUNCTION-DESCRIBED GRAPHS FOR MODELLING OBJECTS REPRESENTED BY SETS OF ATTRIBUTED GRAPHS A function-dscribd graph (FDG) is a modl that contains probabilistic and structural dscriptions of a st of attributd graphs (AGs) to maintain, to th most, th local faturs of th AGs that blong to th st and othr AGs that ar nar thm, as wll as to allow th rjction of th AGs that do not blong to it or ar far of thm. Lt us considr, as an xampl, th 3D-objct modlling and rcognition problm. Th basic ida is that only a singl FDG is synthsisd from th graphs that rprsnt svral viws of a 3D-objct. Thrfor, in th rcognition procss, only on comparison is ndd btwn ach modl rprsntd by an FDG and th unclassifid objct (viw of a 3Dobjct) rprsntd by a graph. Random graphs ar on th othr hand, on of th arlist approachs usd to rprsnt a st of AGs. In this approach, AGs ar xtndd to includ probabilistic information. Wong t al. first dfind th Gnral Random Graphs (GRGs) for modlling classs of pattrns dscribd by AGs through a joint probability spac of random variabls ranging ovr pattrn primitivs (vrtics) 5.1 APPLICATION OF FDGS FOR MODELLING AND RECOGNITION OF OBJECTS FDGs ar applid hr to 3D-objct rprsntation and rcognition. Th attribut of th vrtics is th avrag hu of th rgion (cyclic rang from 0 to 49) and th attribut of th dgs is th diffrnc btwn th colours of th two nighbouring rgions. W first prsnt an xprimntal validation of FDGs using artificial 3D-objcts in which th adjacncy graphs hav bn xtractd manually and aftrwards w prsnt a ral application on an imag databas in which th graphs hav bn xtractd automatically. Th advantags of th xprimntal validation ar that th rsults do not dpnd on th sgmntation procss and that w can us a suprvisd synthsis, sinc w know which vrtics of th AGs rprsnt th sam planar fac of th objct. Thus, w can discrn btwn th ffcts of computing a distanc masur using diffrnt valus of th costs on th 2nd-ordr rlations. In th ral application, w show th capacity of FDGs to kp th structural and smantic knowldg of an objct dspit th nois introducd by th sgmntation procss and an automatic synthsis.

5 Figur 3: 10 viws of 5 artificial objcts dsignd with a CAD program SUPERVISED SYNTHESIS ON ARTIFICIAL OBJECTS W dsignd fiv objcts using a CAD program (s Figur 3). Aftr that, w took fiv sts of viws from ths objcts and from ths viws w xtractd a total of 101 adjacncy graphs. To obtain th AGs of th tst st and of th rfrnc st, w modifid th attribut valus of th vrtics and arcs of th adjacncy graphs by adding zro-man Gausian nois with diffrnt variancs. Morovr, som vrtics and arcs whr insrtd and dltd randomly in som cass. Th FDGs wr synthsisd using th AGs that blongd to th sam 3D-objct and using th synthsis givn a common lablling from a st of AGs dscribd in. Tabl 1 shows th ratio of corrctnss for diffrnt lvls of nois and applying svral costs on th antagonisms and occurrncs. W s that th bsts rsults appar always whn w us modrat 2ndordr costs. Furthrmor, whn nois incrass, th rcognition ratio dcrass drastically whn w us high costs but thr is only a slight dcras whn w us modrat costs. Morovr, in Tabl 2 w compar th FDGs (with 2nd-ordr costs) to othr two mthods. Th FDG classifir always obtains bttr rsults than th 3-Narst Nighbours and th Random Graph classifirs. Th diffrnc of th ratio btwn FDGs and th othr two mthods incrass whn th nois also incrass. FORGs obtain bttr rsults than th 3-N.N. only whn th nois is high UNSUPERVISED SYNTHESIS ON REAL LIFE OBJECTS Imags wr xtractd from th databas COIL- 100 from Columbia Univrsity. It is composd of 100 isolatd objcts and for ach objct thr ar 72 viws (on viw ach 5 dgrs). Adjacncy graphs ar obtaind by color sgmntation. Figur 4 shows 20 objcts at angl 100 and thir sgmntd imags with th adjacncy graphs. Th tst st was composd by 36 viws pr objct (takn at th angls 0, 10, 20 and so on), whras th rfrnc st was composd by th 36 rmaining viws (takn at th angls 5, 15, 25 and so on). FDGs wr synthsisd automatically using th AGs in th rfrnc st that rprsnt th sam objct. Th mthod of incrmntal synthsis, in which th FDGs ar updatd whil nw AGs ar squntially prsntd, was applid. W mad 6 diffrnt xprimnts in which th numbr of FDGs that rprsnts ach 3D-objct varid. If th 3D-objct was rprsntd by only on FDG, th 36 AGs from th rfrnc st that rprsnt th 3D-objct wr usd to synthsis th FDG. If it was rprsntd by 2 FDGs, th 18 first and conscutiv AGs from th rfrnc st wr usd to synthsis on of th FDGs and th othr 18 AGs wr usd to synthsis th othr FDG. A similar mthod was usd for th othr xprimnts with 3, 4, 6 and 9 FDGs pr 3D-objct. Similarly to th prvious xprimntal rsults, th corrctnss is highr whn 2nd-ordr rlations ar usd with a modrat cost. Th bst rsult appars whn ach objct is rprsntd by 4 FDGs, that is, ach FDG rprsnts 90 dgrs of th 3Dobjct. Whn objcts ar rprsntd by 9 FDGs, ach FDG rprsnts 40 dgrs of th 3D-objct and 4 AGs pr FDG, thr is poor probabilistic knowldg and thrfor th costs on th vrtics and arcs ar coars. Morovr, whn objcts ar rprsntd by only 1 or 2 FDGs, thr ar too much spurious rgions (producd in th sgmntation procss) to kp th structural and smantic knowldg of th objct. Rfrncs [1] J. Andrad-Ctto. Environmnt Larning for Indoor Mobil Robots. PhD thsis, UPC, Barclona, Apr [2] J. Andrad-Ctto and A. Sanfliu. Concurrnt map building and localization on indoor dynamic nvironmnts. Int. J. Pattrn Rcogn., 16(3): , May [3] J. Andrad-Ctto and A. Sanfliu. Concurrnt map building and localization with landmark validation. In Proc. 16th IAPR Int. Conf. Pattrn Rcog., volum 2, pags , Qubc, Aug IEEE Comp. Soc. [4] J. Andrad-Ctto and A. Sanfliu. Concurrnt map building and localization with tmporal landmark validation. Accptd for prsntation in 2003 IEEE Intrnational Confrnc on Robotics and Automation, 2003.

6 Tabl 1: FDGs ratio of corrctnss. Num. Vrtics Ins & Dl Standard Dviation Cost Antag Cost Occu Modrat Modrat High Null Null High High High Modrat Null Null Modrat Null Null Tabl 2: FDGs (modrat 2nd ordr costs), FORGs and 3-NN ratio of corrctnss. Num Vrtics Ins. or Dl Standard Dviation FDGs (Modrat costs) Random Graphs (FORGs) N.N. (Edit Op. Distanc) [5] Canal Blau TV. Th IRI Robots in Canal Blau, May [6] A. Chca, J. Andrad, and A. Sanfliu. Construcció d maps pr robots móbils quipats amb snsors làsr d profunditat. Tchnical Rport IRI-DT-03-01, IRI, UPC, Mar [7] El Priódico d Catalunya. El Robot Doméstico Llama a la Purta, Apr [8] A. Grau, J. Climnt, F. Srratosa, and A. Sanfliu. Txprint: A nw algorithm to discriminat txturs structurally. In T. Calli, A. Amin, R. P. W. Duin, M. Kaml, and D. d Riddr, ditors, Proc. IAPR Int. Workshop Syntactical Structural Pattrn Rcog., volum 2396 of Lct. Nots Comp. Sci., pags , Windsor, Aug Springr-Vrlag. [9] M. Isard and A. Blak. Condnsation - conditional dnsity propagation for visual tracking. Int. J. Comput. Vision, 29(1):5 28, Aug [10] La Politècnica, Num 4. UPC. Criaturs Cibrnètiqus, Nov [11] F. Morno, J. Andrad-Ctto, and A. Sanfliu. Fusion of color and shap for objct tracking undr varying illumination. In F. J. Prals, A. J. C. Campilho, N. P. d la Blanca, and A. Sanfliu, ditors, Proc. 1st Ibrian Conf. Pattrn Rcog. Imag Anal., volum 2652 of Lct. Nots Comp. Sci., pags , Purto d Andratx, Jun Springr-Vrlag. [12] F. Morno, A. Tarrida, J. Andrad-Ctto, and A. Sanfliu. 3d ral-tim had tracking fusing color histograms and strovision. In Proc. 16th IAPR Int. Conf. Pattrn Rcog., volum 1, pags , Qubc, Aug IEEE Comp. Soc. [13] A. Sanfliu, R. Alquézar, J. Andrad, J. Climnt, F. Srratosa, and J. Vrgés. Graphbasd rprsntations and tchniqus for imag procssing and imag analysis. Pattrn Rcogn., 35: , Mar [14]F.Srratosa,R.Alquézar, and A. Sanfliu. Function-dscribd graphs for modlling objcts rprsntd by sts of attributd graphs. Pattrn Rcognition, volum 36, pags , [15] F. Srratosa, R. Alquézar, and A. Sanfliu. Estimating th joint probability distribution of random vrtics and arcs by mans of scond-ordr random graphs. In T. Calli, A. Amin, R. P. W. Duin, M. Kaml, and D. d Riddr, ditors, Proc. IAPR Int. Workshop Syntactical Structural Pattrn Rcog., volum 2396 of Lct. Nots Comp. Sci., pags , Windsor, Aug Springr-Vrlag. [16]F.Srratosa,R.Alquézar, and A. Sanfliu. Synthsis of function-dscribd graphs and clustring of attributd graphs. Int. J. Pattrn Rcogn., 16(6): , [17] F. Srratosa, A. Sanfliu, and R. Alquézar. Modlling and rcognizing 3d-objcts d-

7 Figur 4: Th 20 slctd objcts and th sgmntd imags with th AGs. scribd by multipl viws using functiondscribd graphs. In Proc. 16th IAPR Int. Conf. Pattrn Rcog., volum 2, pags , Qubc, Aug IEEE Comp. Soc. IAPR Int. Conf. Pattrn Rcog., pags , Qubc, Aug IEEE Comp. Soc. [18] E. Stafftti, A. Grau, F. Srratosa, and A. Sanfliu. Orintd matroids for shap rprsntation and indxing. In F. J. Prals, A. J. C. Campilho, N. P. d la Blanca, and A. Sanfliu, ditors, Proc. 1st Ibrian Conf. Pattrn Rcog. Imag Anal., volum 2652 of Lct. Nots Comp. Sci., pags , Purto d Andratx, Jun Springr- Vrlag. [19] Tlvisió d Catalunya. Robot Casolà, May [20] Tlvisión Española. Inform Smanal, Jan [21] UPC Libraris. Mor Than Just Machins, Jan [22] J. Vrgés-Llahí and A. Sanfliu. Colour constancy algorithm basd on th minimization of th distanc btwn colour histograms. In F. J. Prals, A. J. C. Campilho, N. P. d la Blanca, and A. Sanfliu, ditors, Proc. 1st Ibrian Conf. Pattrn Rcog. Imag Anal., volum 2652 of Lct. Nots Comp. Sci., pags , Purto d Andratx, Jun Springr-Vrlag. [23] J. Vrgés-Llahí, A. Tarrida, and A. Sanfliu. Nw approachs to colour histogram adaptation in fac tracking tasks. In Proc. 16th