SVM-based Discriminative Accumulation Scheme for Place Recognition

Transcription

1 SVM-based Discriminative Accumulation Scheme for Place Recognition Andrzej Pronobis CAS/CVAP, KTH Stockholm, Sweden Óscar Martínez Mozos AIS, University Of Freiburg Freiburg, Germany Barbara Caputo IDIAP, Martigny, Switzerland

2 Outline High-level, non-linear cue integration scheme Multi-sensory place recognition system Applied to mobile robot topological localization Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 2

3 Motivation Place Recognition The Corridor! Crucial for mobile autonomous agents Solution for typical problems with metric localization Loop closing, kidnapped robot problem Key element of topological and hybrid localization systems Allows to include semantics into space representations Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 3

4 Motivation Multimodal Cue Integration Camera Laser Scanner Range sensors Pros Robust to visual variations Data easy to process Cons Suffers from perceptual aliasing Purely metric information Visual sensors Pros Rich and descriptive Source of semantic information Cons Noisy More data to process Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 4

5 Motivation Multi-cue Place Recognition Distribution of errors made by single cue systems Visual Global Features Visual Local Features Laser Range Features Error Actua al Class Predicted Class Predicted Class Predicted Class How can we use multiple cues effectively? Can we learn these different patterns? Can we do it efficiently? OK Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 5

6 Contribution SVM-based Discriminative Accumulation Scheme High-level cue integration method Effectively and efficiently learns characteristics of different sensors and cues Multi-cue, multi-sensory place recognition system Employs two visual cues and laser range cues Robust to variations introduced by Illumination Everydayand long-term human activity Extensive evaluation in the domain of multi-sensory topological mobile robot localization Data collected over 6 months in a dynamic office environment Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 6

7 Support Vector Machines [Cristianini&Taylor 99] x 2 Input space Kernel K u 2 High-dimensional feature space φ( ) φ( ) φ( ) φ( ) φ( ) φ( ) φ( ) φ( ) φ( ) φ( ) φ( ) φ( ) φ( ) φ( ) φ( ) f(x) < 0 φ( ) φ( ) φ( ) φ( ) f(x) > 0 φ( ) φ( ) φ( ) x 1 M Discriminantfunction: f(x) = Σα i y i K(x i,x) + b i=1 Multi-class extensions: one-vs-one, one-vs-all, modified one-vs-all [Pronobis & Caputo 07] u 1 Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 7

8 Cue 1 Cue P SVM-DAS High Level Integration Input Data 1 0 Model 1 Model P Why high level? Outputs O 1 Confidence Outputs O P Cues are treated independently } Integration Function Models adapted to characteristics of each cue Misleading cues do not affect the others Problem is divided into sub-problems } Integrated Outputs O Σ Cue 1 Decision } Cue P Decision Final Decision Not all cues must always be present e.g. Confidence-based Cue Integration [Pronobis&Caputo 07] Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 8

9 Simple linear accumulation (G-DAS,[Pronobis&Caputo 07]) SVM-DAS SVM-DAS Integration Function O Σ = a 1 *O 1 + a 2 *O a P *O P Integrated output vector All outputs in one vector Output vector for cue no. P Multi-class SVM trained on labeled output vectors Labeled output vectors (V 1, y 1 ),, (V N, y N ) Opt. V = [O 1, O 2,, O P ] T Kernel determines the complexity (linear, non-linear) Final decision as in standard multi-class SVM Multi-class SVM model M O Σ = Σα i y i K(V i,v) + b i=1 Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 9

10 SVM-DAS vs. G-DAS G-DAS Simple, linear function Single weight for all outputs Parameters found by extensive search Integrates outputs of models of the same type SVM-DAS Complex(non-linear) function Each output treated separately Model inferred from training data by optimization algorithm Able to integrate outputs of different types of models Can give correct results even if all single cuesare wrong Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 10

11 The Place RecognitionSystem Overview Fully supervised approach [Pronobis et al ] Training: Kitchen Corridor Office Place Recognition System Recognition: Place Recognition System Office Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 11

12 The Place Recognition System Architecture Image Global Feature Extractor (CRFH) Features Classifier (SVM) Decision (CRFH) Local Feature Extractor (SIFT) Features Classifier (SVM) Decision (SIFT) Laser Scan Geometric Feature Extractor Features Classifier (SVM) (AdaBoost) Decision (Laser) Discriminative Cue Integration Final Decision Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 12

13 The Place Recognition System Global Visual Features High dimensional Composed Receptive Field Histograms (CRFH) [Linde & Lideberg 04] Input image L x (x,y,4) Histogram L(x,y,4) L y (x,y,4) Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 13

14 The Place Recognition System Local Visual Features Affine, scale-invariant DoGinterest-point detector [Rothganger et al. 06] and SIFT descriptor [Lowe 04] Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 14

15 The Place Recognition System Geometrical Laser-based Features d i d d (Σ d i ) / N # Gaps d > θ Minimum d Area Perimeter [Martínez Mozos et al. 07] with AdaBoost Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 15

16 Experimental Setup The IDOL2Database Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 16

17 Experimental Setup The IDOL2Database Five rooms of different funtionality One-person office Corridor Two-persons office Kitchen Printer area Three illumination settings over three weeks Cloudy Sunny Night Repeated after 6 months Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 17

18 Experimental Procedure Four sets of experiments Exp. 1 Stable illumination, close in time Exp.2 Varying illumination, close in time Exp.3 Stable illumination, distant in time Exp. 4 Varying illumination, distant in time Each set evaluates Four single-cue models SVM model trained on CRFH SVM model trained on SIFT SVM model trained on laser range features (L-SVM) AdaBoostmodel trained on laser range features (L-AB) Both cue integration schemes (G-DAS, SVM-DAS) Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 18

19 Results Comparison of Cue Integration Methods Varying illumination, distant in time Classification Rate Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 19

20 Results Single Cue VS Multiple Cues Similar ill., close in time Varyingill., distant in time Classification Rate Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 20

21 Results Single Cue VS Multiple Cues Similar ill., close in time Varyingill., distant in time Classification Rate Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 21

22 Results Single Cue VS Multiple Cues Similar ill., close in time Varyingill., distant in time Classification Rate Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 22

23 Results Single Cue VS Multiple Cues Similar ill., close in time Varyingill., distant in time Classification Rate Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 23

24 Results Single Cue VS Multiple Cues Similar ill., close in time Varyingill., distant in time Classification Rate Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 24

25 Results Single Cue VS Multiple Cues Similar ill., close in time Varyingill., distant in time Classification Rate Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 25

26 Results Single Cue VS Multiple Cues Similar ill., close in time Varyingill., distant in time Classification Rate Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 26

27 Results Confidence-based Cue Integration Drawback:more cues = more computations % % % % % % % % Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 27

28 Summary Conclusions (Multi-sensory) cue integration increases robustness More needed than weighted summation SVM-DAS: a flexible, effective, and efficient solution Multi-sensory discriminative place recognition: a robust base for topological localization Ongoing and Future Work Temporal and spatial cue integration Place categorization Topological localization and semantic labeling system Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 28

29 Thank you Contact: The IDOL2database:

30 Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 30

31 Results Stable illumination, close in time (E. 1) Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 31

32 Results Varyingillumination, close in time (E. 2) Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 32

33 Results Stable illumination, distant in time (E. 3) Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 33

34 Results Varyingillumination, distant in time (E. 4) Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 34

35 Results Comparison of Cue Integration Methods Varying illumination, distant in time Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 35

36 Confidence Estimation and Hypotheses Ranking Confidence information and hypotheses ranking derived from distances between samples and hyperplanes Solution based on the one-against-all principle Scores: Best hypothesis: Confidence and order of hypotheses is derived from V j Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 36

37 CRFH VS SIFT CRFH SIFT CRFH SIFT PeopleBot Cloudy -> PeopleBot Night PeopleBot Cloudy -> PowerBot Night Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 37

38 G-DAS Generalization of DAS [Nilsback & Caputo 04] Scores generated by several classifiers (P) accumulated through weighted summation: 1 2 a 1 * V 1, V 2,..., V M + a 2 * V 1, V 2,..., V M a P * V 1, V 2,..., V M P = V 1, V 2,..., V M Σ Can give correct results even if all classifiers are wrong Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 38

39 Confidence-based Cue Integration Integrationmost effective for cases of low confidence Using all cues can be expensive and unnecessary Solution: extract and use additional cues only when confidence is not satisfactory G-DAS results CRFH results Stable illumination conditions Recognition across platforms Andrzej Pronobis, Óscar Martínez Mozos, and Barbara Caputo. SVM-based Discriminative Accumulation Scheme for Place Recognition. 39