An Attentional Framework for 3D Object Discovery
|
|
|
- Neil Rose
- 5 years ago
- Views:
Transcription
1 An Attentional Framework for 3D Object Discovery Germán Martín García and Simone Frintrop Cognitive Vision Group Institute of Computer Science III University of Bonn, Germany
2 Saliency Computation Saliency Computation Difference Center Input Image Surround Saliency map Simone Frintrop 2
3 Attention vs Saliency Why visual attention is more than saliency computation: Attention is a process that operates over time Look at one region after the other explore a scene by eye movements (saccades) examine the painting freely assess the ages of the characters The gaze is triggered by bottom-up (saliency) AND top-down cues Simone Frintrop 3 [Yarbus 1967; Images: Sasha Archibald, after Ilya Repin, An Unexpected Visitor, 1884)]
4 Visual attention Tasks to be solved by visual attention systems: Where to direct the attention at (saliency + top-down cues) When to switch How to prevent returning to already visited locations Get new frame Saliency computation Detect Proto-objects no Inhibit Proto-object yes Fixated long enough? Simone Frintrop 4
5 Visual attention Tasks to be solved by visual attention systems: Where to direct the attention suppression at (saliency of processing + top-down of cues) locations When to switch and objects that had recently been the How to prevent returning focus to already of attention visited locations encourages orienting toward novelty Get new frame Inhibition of return (IOR) [Posner 1984] Saliency computation Detect Proto-objects happens in spatial coordinates and not in retinotopic coordinates no Inhibit Proto-object yes Fixated long enough? Simone Frintrop 5
6 Inhibition of Return Many systems simply compute the saliency map and ignore IOR Inhibition of return in computational attention systems: Inhibition of return [Itti,Koch,Niebur PAMI 1998] Feature 1 Max Finder Saliency map... Feature n Trajectory of FOAs Input image If realized, usually by setting values to zero in saliency map. In temporal sequences, if camera positions and/or objects change location, this is not sufficient! It is necessary to remember which regions/objects have already been attended not in image but in spatial coordinates Simone Frintrop 6
7 3D Object Discovery We propose an architecture for object discovery that operates in 3D (on RGB-D data) Object information is accumulated over time and 3D object models are built based on information from current and previous time steps 3D voxels are tagged by inhibition of return flags that indicate if a region was recently attended Get new frame Saliency computation Detect Proto-objects no Inhibit Proto-object yes Fixated long enough? 3D Map update (object information + IOR flags) Simone Frintrop 7
![of the annual meeting of Cognitive Sciences (CogSci), 2013 (to appear)]](/docs-images/81/83534921/images/8-2.jpg "[G. Martín-García & S.")
8 3D Object Discovery Color image Germán Martín García Depth image 3D Object Map [G. Martín-García & S. Frintrop, Proc. of the annual meeting of Cognitive Sciences (CogSci), 2013 (to appear)] [G. Martín-García & S. Frintrop, German Journal of Artificial Intelligence, 2013 (to appear)] 8
9 3D Object Discovery Color image Germán Martín García Depth image What are these proto-objects? [G. Martín-García & S. Frintrop, Proc. of the annual meeting of Cognitive Sciences (CogSci), 2013 (to appear)] [G. Martín-García & S. Frintrop, German Journal of Artificial Intelligence, 2013 (to appear)] 9
10 What are proto-objects? The triadic architecture of Rensink (2000): Proto-objects: Proto-objects are object hypotheses built from simple features Volatile structures with limited coherence in space and time (Collections of) protoobjects are the target of attention ( the focused attention system acts as a hand to grab protoobjects ) Simone Frintrop 10 R. A. Rensink, The Dynamic Representation of Scenes, Visual Cognition (7), 2000
![Itti/Koch [Itti et al, PAMI 1998] (image pyramids, feature channels, conspicuity maps, saliency](/docs-images/81/83534921/images/11-1.jpg "map) BITS [Klein/Frintrop ICCV 2011]: CoDi [Klein/Frintrop DAGM 2012]: Discrete distributions")
11 Saliency Detection For saliency computation, use information-theoretic saliency: Dominik Klein Represent center and surround regions by probability distributions Otherwise structure similar to Itti/Koch [Itti et al, PAMI 1998] (image pyramids, feature channels, conspicuity maps, saliency map) BITS [Klein/Frintrop ICCV 2011]: CoDi [Klein/Frintrop DAGM 2012]: Discrete distributions compare with Kullback-Leibler Divergence Continuous distributions (Gaussians) compare with Wasserstein distance W 2 Surround Center C++ code available at: Simone Frintrop 11
![[Klein/Frintrop, ICCV 2011]](/docs-images/81/83534921/images/12-1.jpg "AC10: [Achanta/Süsstrunk,")
![2010] Simone Frintrop invt:](/docs-images/81/83534921/images/12-2.jpg "[Itti et al.")
![, PAMI 1998] HZ08:](/docs-images/81/83534921/images/12-3.jpg "[Hou/Zhang, 2008] 12 ST:")
12 MSRA Salient Object Database BITS: [Klein/Frintrop, ICCV 2011] AC10: [Achanta/Süsstrunk, 2010] Simone Frintrop invt: [Itti et al., PAMI 1998] HZ08: [Hou/Zhang, 2008] 12 ST: [Walther/Koch, 2006] AIM: [Bruce/Tsotsos, 2009]
13 Saliency performance on MSRA Salient Object Database [Klein/Frintrop, DAGM, 2012] Simone Frintrop 13
14 From saliency to proto-objects We have now a saliency map: To obtain proto-objects, we follow the two-stage processing strategy of the HVS: Pre-attentive: parallel processing to find region of interest Attentive: serial processing, one region of interest at the time processing becomes selective. Simone Frintrop [Neisser, Cognitive Psychology, 1967] 14 [Treisman, Cognitive Psychology, 1985]
![Refine shape of blob with GrabCut segmentation [Rother 2004] (OpenCV implementation) saliency values serve to initialize](/docs-images/81/83534921/images/15-4.jpg "pixel likelihoods [G. Martín-García & S. Frintrop, Proc.")
15 From saliency to proto-objects Generate proto objects: Pre-attentive (parallel) stage: 1. compute the saliency map 2. threshold the saliency map 3. find connected components (discard too big and too small objects) 4. rank by average saliency 5. Inhibit already attended regions Attentive stage: Frame t 1. Pick one salient blob & fit a rectangle around IOR information from frames 1 to t-1 2. Refine shape of blob with GrabCut segmentation [Rother 2004] (OpenCV implementation) saliency values serve to initialize pixel likelihoods [G. Martín-García & S. Frintrop, Proc. of the annual meeting of Cognitive Sciences (CogSci), 2013 (to appear)] [G. Martín-García & S. Frintrop, German Journal of Artificial Intelligence, 2013 (to appear)] 15
16 Proto-Objects Now, we have a segmented proto-object for the currently attended region Examples: [G. Martín-García & S. Frintrop, Proc. of the annual meeting of Cognitive Sciences (CogSci), 2013 (to appear)] [G. Martín-García & S. Frintrop, German Journal of Artificial Intelligence, 2013 (to appear)] 16
17 Salient Object Discovery in 3D Original image Saliency map Object hypotheses [G. Martín-García & S. Frintrop, Proc. of the annual meeting of Cognitive Sciences (CogSci), 2013 (to appear)] [G. Martín-García & S. Frintrop, German Journal of Artificial Intelligence, 2013 (to appear)]
![of the annual meeting of Cognitive Sciences (CogSci), 2013 (to appear)] [G.](/docs-images/81/83534921/images/18-3.jpg "Martín-García & S.")
18 Salient Object Discovery in 3D Color image Depth image That s was all in 2D. Now we come to 3D [G. Martín-García & S. Frintrop, Proc. of the annual meeting of Cognitive Sciences (CogSci), 2013 (to appear)] [G. Martín-García & S. Frintrop, German Journal of Artificial Intelligence, 2013 (to appear)] 18
![with the Kinectfusion algorithm [Newcomb 2011].](/docs-images/81/83534921/images/19-1.jpg "It tracks the pose of the camera integrates the new")
![(CogSci), 2013 (to appear)] [G. Martín-García & S.](/docs-images/81/83534921/images/19-5.jpg "Frintrop, German Journal of Artificial Intelligence,")
19 Salient Object Discovery in 3D Color image From the depth data, incrementally built a 3D environment map with the Kinectfusion algorithm [Newcomb 2011]. It tracks the pose of the camera integrates the new measurement into the global model Then, project the proto-objects into the 3D scene Depth image 3D Object Map [G. Martín-García & S. Frintrop, Proc. of the annual meeting of Cognitive Sciences (CogSci), 2013 (to appear)] [G. Martín-García & S. Frintrop, German Journal of Artificial Intelligence, 2013 (to appear)] 19
20 Inhibition of Return in 3D Each voxel stores An IOR flag (active/not active) An IOR weight Increase IOR weight of attended voxels. If weight reaches threshold, set IOR flag active Decrease IOR weight of not attended voxels If weight reaches 0, set IOR flag inactive 20
21 Salient Object Discovery in 3D Color image Use knowledge from 3D map to improve object detection & segmentation raycast the object labels & inhibition flags to the current viewpoint Depth image constrain the segmentation step to not label pixels as foreground that belong to other objects Use inhibition flags to prevent selecting already attended blobs 3D Object Map [G. Martín-García & S. Frintrop, Proc. of the annual meeting of Cognitive Sciences (CogSci), 2013 (to appear)] [G. Martín-García & S. Frintrop, German Journal of Artificial Intelligence, 2013 (to appear)] 21
22 Results: Table top sequence Ground truth Detected Objects Simone Frintrop 22
23 Results: Table top sequence Ground truth Detected Objects Simone Frintrop 23
24 Salient Object Discovery in 3D [G. Martín-García & S. Frintrop, Proc. of the annual meeting of Cognitive Sciences (CogSci), 2013 (to appear)] [G. Martín-García & S. Frintrop, German Journal of Artificial Intelligence, 2013 (to appear)]
25 Results: Table top sequence Object Precision Recall Juice Bowl Cereals Apple Coffee Coffee Simone Frintrop 25
26 Results: Coffee machine sequence Ground truth Detected Objects Simone Frintrop
27 Results: Coffee machine sequence Ground truth Detected Objects Simone Frintrop 27
28 Results: Coffee machine sequence Object Precision Recall Note Y. cup W. cup Box Paper Sugar Cup Water Milk
29 3D Object Discovery Color image Take home message: Real-world attention requires inhibition of return in spatial coordinates Depth image Thank you for your attention! 3D Object Map [G. Martín-García & S. Frintrop, Proc. of the annual meeting of Cognitive Sciences (CogSci), 2013 (to appear)] [G. Martín-García & S. Frintrop, German Journal of Artificial Intelligence, 2013 (to appear)] 29
Computational Cognitive Science
Computational Cognitive Science Lecture 15: Visual Attention Chris Lucas (Slides adapted from Frank Keller s) School of Informatics University of Edinburgh clucas2@inf.ed.ac.uk 14 November 2017 1 / 28
Computational Cognitive Science. The Visual Processing Pipeline. The Visual Processing Pipeline. Lecture 15: Visual Attention.
Lecture 15: Visual Attention School of Informatics University of Edinburgh keller@inf.ed.ac.uk November 11, 2016 1 2 3 Reading: Itti et al. (1998). 1 2 When we view an image, we actually see this: The
Real-time computational attention model for dynamic scenes analysis
Computer Science Image and Interaction Laboratory Real-time computational attention model for dynamic scenes analysis Matthieu Perreira Da Silva Vincent Courboulay 19/04/2012 Photonics Europe 2012 Symposium,
Keywords- Saliency Visual Computational Model, Saliency Detection, Computer Vision, Saliency Map. Attention Bottle Neck
Volume 3, Issue 9, September 2013 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Visual Attention
VIDEO SALIENCY INCORPORATING SPATIOTEMPORAL CUES AND UNCERTAINTY WEIGHTING
VIDEO SALIENCY INCORPORATING SPATIOTEMPORAL CUES AND UNCERTAINTY WEIGHTING Yuming Fang, Zhou Wang 2, Weisi Lin School of Computer Engineering, Nanyang Technological University, Singapore 2 Department of
Goal-directed search with a top-down modulated computational attention system
Goal-directed search with a top-down modulated computational attention system Simone Frintrop 1, Gerriet Backer 2, and Erich Rome 1 1 Fraunhofer Institut für Autonome Intelligente Systeme (AIS), Schloss
A Computational Model of Saliency Depletion/Recovery Phenomena for the Salient Region Extraction of Videos
A Computational Model of Saliency Depletion/Recovery Phenomena for the Salient Region Extraction of Videos July 03, 2007 Media Information Laboratory NTT Communication Science Laboratories Nippon Telegraph
Natural Scene Statistics and Perception. W.S. Geisler
Natural Scene Statistics and Perception W.S. Geisler Some Important Visual Tasks Identification of objects and materials Navigation through the environment Estimation of motion trajectories and speeds
Computational Cognitive Science
Computational Cognitive Science Lecture 19: Contextual Guidance of Attention Chris Lucas (Slides adapted from Frank Keller s) School of Informatics University of Edinburgh clucas2@inf.ed.ac.uk 20 November
Validating the Visual Saliency Model
Validating the Visual Saliency Model Ali Alsam and Puneet Sharma Department of Informatics & e-learning (AITeL), Sør-Trøndelag University College (HiST), Trondheim, Norway er.puneetsharma@gmail.com Abstract.
On the implementation of Visual Attention Architectures
On the implementation of Visual Attention Architectures KONSTANTINOS RAPANTZIKOS AND NICOLAS TSAPATSOULIS DEPARTMENT OF ELECTRICAL AND COMPUTER ENGINEERING NATIONAL TECHNICAL UNIVERSITY OF ATHENS 9, IROON
Can Saliency Map Models Predict Human Egocentric Visual Attention?
Can Saliency Map Models Predict Human Egocentric Visual Attention? Kentaro Yamada 1, Yusuke Sugano 1, Takahiro Okabe 1 Yoichi Sato 1, Akihiro Sugimoto 2, and Kazuo Hiraki 3 1 The University of Tokyo, Tokyo,
Introduction to Computational Neuroscience
Introduction to Computational Neuroscience Lecture 11: Attention & Decision making Lesson Title 1 Introduction 2 Structure and Function of the NS 3 Windows to the Brain 4 Data analysis 5 Data analysis
Compound Effects of Top-down and Bottom-up Influences on Visual Attention During Action Recognition
Compound Effects of Top-down and Bottom-up Influences on Visual Attention During Action Recognition Bassam Khadhouri and Yiannis Demiris Department of Electrical and Electronic Engineering Imperial College
Visual Task Inference Using Hidden Markov Models
Visual Task Inference Using Hidden Markov Models Abstract It has been known for a long time that visual task, such as reading, counting and searching, greatly influences eye movement patterns. Perhaps
A Model of Saliency-Based Visual Attention for Rapid Scene Analysis
A Model of Saliency-Based Visual Attention for Rapid Scene Analysis Itti, L., Koch, C., Niebur, E. Presented by Russell Reinhart CS 674, Fall 2018 Presentation Overview Saliency concept and motivation
A Hierarchical Visual Saliency Model for Character Detection in Natural Scenes
A Hierarchical Visual Saliency Model for Character Detection in Natural Scenes Renwu Gao 1(B), Faisal Shafait 2, Seiichi Uchida 3, and Yaokai Feng 3 1 Information Sciene and Electrical Engineering, Kyushu
Experiences on Attention Direction through Manipulation of Salient Features
Experiences on Attention Direction through Manipulation of Salient Features Erick Mendez Graz University of Technology Dieter Schmalstieg Graz University of Technology Steven Feiner Columbia University
Motion Saliency Outweighs Other Low-level Features While Watching Videos
Motion Saliency Outweighs Other Low-level Features While Watching Videos Dwarikanath Mahapatra, Stefan Winkler and Shih-Cheng Yen Department of Electrical and Computer Engineering National University of
Video Saliency Detection via Dynamic Consistent Spatio- Temporal Attention Modelling
AAAI -13 July 16, 2013 Video Saliency Detection via Dynamic Consistent Spatio- Temporal Attention Modelling Sheng-hua ZHONG 1, Yan LIU 1, Feifei REN 1,2, Jinghuan ZHANG 2, Tongwei REN 3 1 Department of
HUMAN VISUAL PERCEPTION CONCEPTS AS MECHANISMS FOR SALIENCY DETECTION
HUMAN VISUAL PERCEPTION CONCEPTS AS MECHANISMS FOR SALIENCY DETECTION Oana Loredana BUZATU Gheorghe Asachi Technical University of Iasi, 11, Carol I Boulevard, 700506 Iasi, Romania lbuzatu@etti.tuiasi.ro
Computational Models of Visual Attention: Bottom-Up and Top-Down. By: Soheil Borhani
Computational Models of Visual Attention: Bottom-Up and Top-Down By: Soheil Borhani Neural Mechanisms for Visual Attention 1. Visual information enter the primary visual cortex via lateral geniculate nucleus
Neurally Inspired Mechanisms for the Dynamic Visual Attention Map Generation Task
Neurally Inspired Mechanisms for the Dynamic Visual Attention Map Generation Task Maria T. Lopez 1, Miguel A. Fernandez 1, Antonio Fernandez-Caballero 1, and Ana E. Delgado 2 Departamento de Informatica
Methods for comparing scanpaths and saliency maps: strengths and weaknesses
Methods for comparing scanpaths and saliency maps: strengths and weaknesses O. Le Meur olemeur@irisa.fr T. Baccino thierry.baccino@univ-paris8.fr Univ. of Rennes 1 http://www.irisa.fr/temics/staff/lemeur/
NEOCORTICAL CIRCUITS. specifications
NEOCORTICAL CIRCUITS specifications where are we coming from? human-based computing using typically human faculties associating words with images -> labels for image search locating objects in images ->
A context-dependent attention system for a social robot
A context-dependent attention system for a social robot Cynthia Breazeal and Brian Scassellati MIT Artificial Intelligence Lab 545 Technology Square Cambridge, MA 02139 U. S. A. Abstract This paper presents
The 29th Fuzzy System Symposium (Osaka, September 9-, 3) Color Feature Maps (BY, RG) Color Saliency Map Input Image (I) Linear Filtering and Gaussian
The 29th Fuzzy System Symposium (Osaka, September 9-, 3) A Fuzzy Inference Method Based on Saliency Map for Prediction Mao Wang, Yoichiro Maeda 2, Yasutake Takahashi Graduate School of Engineering, University
What we see is most likely to be what matters: Visual attention and applications
What we see is most likely to be what matters: Visual attention and applications O. Le Meur P. Le Callet olemeur@irisa.fr patrick.lecallet@univ-nantes.fr http://www.irisa.fr/temics/staff/lemeur/ November
Visual Attention Framework: Application to Event Analysis
VRIJE UNIVERSITEIT BRUSSEL FACULTY OF ENGINEERING Department of Electronics and Informatics (ETRO) Image Processing and Machine Vision Group (IRIS) Visual Attention Framework: Application to Event Analysis
MEMORABILITY OF NATURAL SCENES: THE ROLE OF ATTENTION
MEMORABILITY OF NATURAL SCENES: THE ROLE OF ATTENTION Matei Mancas University of Mons - UMONS, Belgium NumediArt Institute, 31, Bd. Dolez, Mons matei.mancas@umons.ac.be Olivier Le Meur University of Rennes
Object-based Saliency as a Predictor of Attention in Visual Tasks
Object-based Saliency as a Predictor of Attention in Visual Tasks Michal Dziemianko (m.dziemianko@sms.ed.ac.uk) Alasdair Clarke (a.clarke@ed.ac.uk) Frank Keller (keller@inf.ed.ac.uk) Institute for Language,
The Attraction of Visual Attention to Texts in Real-World Scenes
The Attraction of Visual Attention to Texts in Real-World Scenes Hsueh-Cheng Wang (hchengwang@gmail.com) Marc Pomplun (marc@cs.umb.edu) Department of Computer Science, University of Massachusetts at Boston,
Attention and Scene Understanding
Attention and Scene Understanding Vidhya Navalpakkam, Michael Arbib and Laurent Itti University of Southern California, Los Angeles Abstract Abstract: This paper presents a simplified, introductory view
A Model for Automatic Diagnostic of Road Signs Saliency
A Model for Automatic Diagnostic of Road Signs Saliency Ludovic Simon (1), Jean-Philippe Tarel (2), Roland Brémond (2) (1) Researcher-Engineer DREIF-CETE Ile-de-France, Dept. Mobility 12 rue Teisserenc
Evaluation of the Impetuses of Scan Path in Real Scene Searching
Evaluation of the Impetuses of Scan Path in Real Scene Searching Chen Chi, Laiyun Qing,Jun Miao, Xilin Chen Graduate University of Chinese Academy of Science,Beijing 0009, China. Key Laboratory of Intelligent
An Evaluation of Motion in Artificial Selective Attention
An Evaluation of Motion in Artificial Selective Attention Trent J. Williams Bruce A. Draper Colorado State University Computer Science Department Fort Collins, CO, U.S.A, 80523 E-mail: {trent, draper}@cs.colostate.edu
Computational modeling of visual attention and saliency in the Smart Playroom
Computational modeling of visual attention and saliency in the Smart Playroom Andrew Jones Department of Computer Science, Brown University Abstract The two canonical modes of human visual attention bottomup
GESTALT SALIENCY: SALIENT REGION DETECTION BASED ON GESTALT PRINCIPLES
GESTALT SALIENCY: SALIENT REGION DETECTION BASED ON GESTALT PRINCIPLES Jie Wu and Liqing Zhang MOE-Microsoft Laboratory for Intelligent Computing and Intelligent Systems Dept. of CSE, Shanghai Jiao Tong
Reading Assignments: Lecture 18: Visual Pre-Processing. Chapters TMB Brain Theory and Artificial Intelligence
Brain Theory and Artificial Intelligence Lecture 18: Visual Pre-Processing. Reading Assignments: Chapters TMB2 3.3. 1 Low-Level Processing Remember: Vision as a change in representation. At the low-level,
Change Blindness. The greater the lie, the greater the chance that it will be believed.
Change Blindness The greater the lie, the greater the chance that it will be believed. (kurt@kloover.com) Department of Computer Science Rochester Institute of Technology 1 Definitions Seeing: the use
Supplementary Material for submission 2147: Traditional Saliency Reloaded: A Good Old Model in New Shape
Sulementary Material for submission 247: Traditional Saliency Reloaded: A Good Old Model in New Shae Simone Frintro, Thomas Werner, and Germán M. García Institute of Comuter Science III Rheinische Friedrich-Wilhelms-Universität
Experimental evaluation of the accuracy of the second generation of Microsoft Kinect system, for using in stroke rehabilitation applications
Experimental evaluation of the accuracy of the second generation of Microsoft Kinect system, for using in stroke rehabilitation applications Mohammad Hossein Saadatzi 1 Home-based Stroke Rehabilitation
Goal-Directed Deployment of Attention in a Computational Model: A Study in Multiple-Object Tracking
Goal-Directed Deployment of Attention in a Computational Model: A Study in Multiple-Object Tracking Andrew Lovett (andrew.lovett.ctr@nrl.navy.mil) Will Bridewell (will.bridewell@nrl.navy.mil) Paul Bello
A Dynamical Systems Approach to Visual Attention Based on Saliency Maps
A Dynamical Systems Approach to Visual Attention Based on Saliency Maps Timothy Randall Rost E H U N I V E R S I T Y T O H F R G E D I N B U Master of Science School of Informatics University of Edinburgh
The synergy of top-down and bottom-up attention in complex task: going beyond saliency models.
The synergy of top-down and bottom-up attention in complex task: going beyond saliency models. Enkhbold Nyamsuren (e.nyamsuren@rug.nl) Niels A. Taatgen (n.a.taatgen@rug.nl) Department of Artificial Intelligence,
Object-Level Saliency Detection Combining the Contrast and Spatial Compactness Hypothesis
Object-Level Saliency Detection Combining the Contrast and Spatial Compactness Hypothesis Chi Zhang 1, Weiqiang Wang 1, 2, and Xiaoqian Liu 1 1 School of Computer and Control Engineering, University of
Recurrent Refinement for Visual Saliency Estimation in Surveillance Scenarios
2012 Ninth Conference on Computer and Robot Vision Recurrent Refinement for Visual Saliency Estimation in Surveillance Scenarios Neil D. B. Bruce*, Xun Shi*, and John K. Tsotsos Department of Computer
Advertisement Evaluation Based On Visual Attention Mechanism
2nd International Conference on Economics, Management Engineering and Education Technology (ICEMEET 2016) Advertisement Evaluation Based On Visual Attention Mechanism Yu Xiao1, 2, Peng Gan1, 2, Yuling
ELL 788 Computational Perception & Cognition July November 2015
ELL 788 Computational Perception & Cognition July November 2015 Module 8 Audio and Multimodal Attention Audio Scene Analysis Two-stage process Segmentation: decomposition to time-frequency segments Grouping
Novelty Is Not Always the Best Policy Inhibition of Return and Facilitation of Return as a Function of Visual Task
PSYCHOLOGICAL SCIENCE Research Article Novelty Is Not Always the Best Policy Inhibition of Return and Facilitation of Return as a Function of Visual Task Michael D. Dodd, 1 Stefan Van der Stigchel, 2 and
The Role of Top-down and Bottom-up Processes in Guiding Eye Movements during Visual Search
The Role of Top-down and Bottom-up Processes in Guiding Eye Movements during Visual Search Gregory J. Zelinsky, Wei Zhang, Bing Yu, Xin Chen, Dimitris Samaras Dept. of Psychology, Dept. of Computer Science
VENUS: A System for Novelty Detection in Video Streams with Learning
VENUS: A System for Novelty Detection in Video Streams with Learning Roger S. Gaborski, Vishal S. Vaingankar, Vineet S. Chaoji, Ankur M. Teredesai Laboratory for Applied Computing, Rochester Institute
Selective Attention. Modes of Control. Domains of Selection
The New Yorker (2/7/5) Selective Attention Perception and awareness are necessarily selective (cell phone while driving): attention gates access to awareness Selective attention is deployed via two modes
Inattentional Blindness in a Coupled Perceptual Cognitive System
Inattentional Blindness in a Coupled Perceptual Cognitive System Will Bridewell (will.bridewell@nrl.navy.mil) Paul F. Bello (paul.bello@nrl.navy.mil) U.S. Naval Research Laboratory Washington, DC 20375
IAT 814 Knowledge Visualization. Visual Attention. Lyn Bartram
IAT 814 Knowledge Visualization Visual Attention Lyn Bartram Why we care in an information-rich world, the wealth of information means a dearth of something else: a scarcity of whatever it is that information
I. INTRODUCTION VISUAL saliency, which is a term for the pop-out
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 27, NO. 6, JUNE 2016 1177 Spatiochromatic Context Modeling for Color Saliency Analysis Jun Zhang, Meng Wang, Member, IEEE, Shengping Zhang,
Modeling the Deployment of Spatial Attention
17 Chapter 3 Modeling the Deployment of Spatial Attention 3.1 Introduction When looking at a complex scene, our visual system is confronted with a large amount of visual information that needs to be broken
A LEARNING-BASED VISUAL SALIENCY FUSION MODEL FOR HIGH DYNAMIC RANGE VIDEO (LBVS-HDR)
A LEARNING-BASED VISUAL SALIENCY FUSION MODEL FOR HIGH DYNAMIC RANGE VIDEO (LBVS-HDR) Amin Banitalebi-Dehkordi1, Yuanyuan Dong1, Mahsa T. Pourazad1,2, and Panos Nasiopoulos1 1 2 ECE Department and ICICS
A Knowledge Driven Computational Visual Attention Model
www.ijcsi.org 134 A Knowledge Driven Computational Visual Attention Model Amudha J 1, Soman. K.P 2 and Padmakar Reddy. S 3 1 Department of computer science, Amrita School of Engineering Bangalore, Karnataka,
An Experimental Analysis of Saliency Detection with respect to Three Saliency Levels
An Experimental Analysis of Saliency Detection with respect to Three Saliency Levels Antonino Furnari, Giovanni Maria Farinella, Sebastiano Battiato {furnari,gfarinella,battiato}@dmi.unict.it Department
Increasing Spatial Competition Enhances Visual Prediction Learning
(2011). In A. Cangelosi, J. Triesch, I. Fasel, K. Rohlfing, F. Nori, P.-Y. Oudeyer, M. Schlesinger, Y. and Nagai (Eds.), Proceedings of the First Joint IEEE Conference on Development and Learning and on
(In)Attention and Visual Awareness IAT814
(In)Attention and Visual Awareness IAT814 Week 5 Lecture B 8.10.2009 Lyn Bartram lyn@sfu.ca SCHOOL OF INTERACTIVE ARTS + TECHNOLOGY [SIAT] WWW.SIAT.SFU.CA This is a useful topic Understand why you can
The discriminant center-surround hypothesis for bottom-up saliency
Appears in the Neural Information Processing Systems (NIPS) Conference, 27. The discriminant center-surround hypothesis for bottom-up saliency Dashan Gao Vijay Mahadevan Nuno Vasconcelos Department of
Novelty is not always the best policy: Inhibition of return and facilitation of return. as a function of visual task. Michael D.
IOR, task set 1 Running head: INHIBITION OF RETURN, TASK SET Novelty is not always the best policy: Inhibition of return and facilitation of return as a function of visual task Michael D. Dodd University
On the role of context in probabilistic models of visual saliency
1 On the role of context in probabilistic models of visual saliency Date Neil Bruce, Pierre Kornprobst NeuroMathComp Project Team, INRIA Sophia Antipolis, ENS Paris, UNSA, LJAD 2 Overview What is saliency?
Visual Attention. International Lecture Serie. Nicolas P. Rougier. INRIA National Institute for Research in Computer Science and Control
Visual Attention International Lecture Serie Nicolas P. Rougier ( ) INRIA National Institute for Research in Computer Science and Control National Institute of Informatics Tokyo, December 2, 2010 1 / 37
Actions in the Eye: Dynamic Gaze Datasets and Learnt Saliency Models for Visual Recognition
Actions in the Eye: Dynamic Gaze Datasets and Learnt Saliency Models for Visual Recognition Stefan Mathe, Cristian Sminchisescu Presented by Mit Shah Motivation Current Computer Vision Annotations subjectively
Salient Object Detection in Videos Based on SPATIO-Temporal Saliency Maps and Colour Features
Salient Object Detection in Videos Based on SPATIO-Temporal Saliency Maps and Colour Features U.Swamy Kumar PG Scholar Department of ECE, K.S.R.M College of Engineering (Autonomous), Kadapa. ABSTRACT Salient
Object detection in natural scenes by feedback
In: H.H. Būlthoff et al. (eds.), Biologically Motivated Computer Vision. Lecture Notes in Computer Science. Berlin, Heidelberg, New York: Springer Verlag, 398-407, 2002. c Springer-Verlag Object detection
A Neural Network Architecture for.
A Neural Network Architecture for Self-Organization of Object Understanding D. Heinke, H.-M. Gross Technical University of Ilmenau, Division of Neuroinformatics 98684 Ilmenau, Germany e-mail: dietmar@informatik.tu-ilmenau.de
Finding Saliency in Noisy Images
Finding Saliency in Noisy Images Chelhwon Kim and Peyman Milanfar Electrical Engineering Department, University of California, Santa Cruz, CA, USA ABSTRACT Recently, many computational saliency models
Visual Perception. Agenda. Visual perception. CS Information Visualization January 20, 2011 John Stasko. Pre-attentive processing Color Etc.
Topic Notes Visual Perception CS 7450 - Information Visualization January 20, 2011 John Stasko Agenda Visual perception Pre-attentive processing Color Etc. Spring 2011 CS 7450 2 1 Semiotics The study of
Dynamic Eye Movement Datasets and Learnt Saliency Models for Visual Action Recognition
Dynamic Eye Movement Datasets and Learnt Saliency Models for Visual Action Recognition Stefan Mathe 1,3 and Cristian Sminchisescu 2,1 1 Institute of Mathematics of the Romanian Academy (IMAR) 2 Faculty
Adding Shape to Saliency: A Computational Model of Shape Contrast
Adding Shape to Saliency: A Computational Model of Shape Contrast Yupei Chen 1, Chen-Ping Yu 2, Gregory Zelinsky 1,2 Department of Psychology 1, Department of Computer Science 2 Stony Brook University
Attending to Learn and Learning to Attend for a Social Robot
Attending to Learn and Learning to Attend for a Social Robot Lijin Aryananda Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology Cambridge, MA 02139, USA Email:
Control of Selective Visual Attention: Modeling the "Where" Pathway
Control of Selective Visual Attention: Modeling the "Where" Pathway Ernst Niebur Computation and Neural Systems 139-74 California Institute of Technology Christof Koch Computation and Neural Systems 139-74
Does scene context always facilitate retrieval of visual object representations?
Psychon Bull Rev (2011) 18:309 315 DOI 10.3758/s13423-010-0045-x Does scene context always facilitate retrieval of visual object representations? Ryoichi Nakashima & Kazuhiko Yokosawa Published online:
Attention Estimation by Simultaneous Observation of Viewer and View
Attention Estimation by Simultaneous Observation of Viewer and View Anup Doshi and Mohan M. Trivedi Computer Vision and Robotics Research Lab University of California, San Diego La Jolla, CA 92093-0434
Understanding eye movements in face recognition with hidden Markov model
Understanding eye movements in face recognition with hidden Markov model 1 Department of Psychology, The University of Hong Kong, Pokfulam Road, Hong Kong 2 Department of Computer Science, City University
Evaluating Visual Saliency Algorithms: Past, Present and Future
Journal of Imaging Science and Technology R 59(5): 050501-1 050501-17, 2015. c Society for Imaging Science and Technology 2015 Evaluating Visual Saliency Algorithms: Past, Present and Future Puneet Sharma
A Computer Vision Model for Visual-Object-Based Attention and Eye Movements
A Computer Vision Model for Visual-Object-Based Attention and Eye Movements Yaoru Sun a, Robert Fisher b, Fang Wang c and Herman Martins Gomes d a Nanjing University, China; Brain and Behavioural Sciences
Tumor cut segmentation for Blemish Cells Detection in Human Brain Based on Cellular Automata
Tumor cut segmentation for Blemish Cells Detection in Human Brain Based on Cellular Automata D.Mohanapriya 1 Department of Electronics and Communication Engineering, EBET Group of Institutions, Kangayam,
VISUAL search is necessary for rapid scene analysis
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 25, NO. 8, AUGUST 2016 3475 A Unified Framework for Salient Structure Detection by Contour-Guided Visual Search Kai-Fu Yang, Hui Li, Chao-Yi Li, and Yong-Jie
Cancer Cells Detection using OTSU Threshold Algorithm
Cancer Cells Detection using OTSU Threshold Algorithm Nalluri Sunny 1 Velagapudi Ramakrishna Siddhartha Engineering College Mithinti Srikanth 2 Velagapudi Ramakrishna Siddhartha Engineering College Kodali
Designing Caption Production Rules Based on Face, Text and Motion Detections
Designing Caption Production Rules Based on Face, Text and Motion Detections C. Chapdelaine *, M. Beaulieu, L. Gagnon R&D Department, Computer Research Institute of Montreal (CRIM), 550 Sherbrooke West,
intensities saliency map
Neuromorphic algorithms for computer vision and attention Florence Miau 1, Constantine Papageorgiou 2 and Laurent Itti 1 1 Department of Computer Science, University of Southern California, Los Angeles,
Learning to Predict Saliency on Face Images
Learning to Predict Saliency on Face Images Mai Xu, Yun Ren, Zulin Wang School of Electronic and Information Engineering, Beihang University, Beijing, 9, China MaiXu@buaa.edu.cn Abstract This paper proposes
Local Image Structures and Optic Flow Estimation
Local Image Structures and Optic Flow Estimation Sinan KALKAN 1, Dirk Calow 2, Florentin Wörgötter 1, Markus Lappe 2 and Norbert Krüger 3 1 Computational Neuroscience, Uni. of Stirling, Scotland; {sinan,worgott}@cn.stir.ac.uk
Computational Architectures in Biological Vision, USC, Spring 2001
Computational Architectures in Biological Vision, USC, Spring 2001 Lecture 11: Visual Illusions. Reading Assignments: None 1 What Can Illusions Teach Us? They exacerbate the failure modes of our visual
Top-down Attention Signals in Saliency 邓凝旖
Top-down Attention Signals in Saliency WORKS BY VIDHYA NAVALPAKKAM 邓凝旖 2014.11.10 Introduction of Vidhya Navalpakkam EDUCATION * Ph.D, Computer Science, Fall 2006, University of Southern California (USC),
State-of-the-Art in Visual Attention Modeling
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 35, NO. 1, JANUARY 2013 185 State-of-the-Art in Visual Attention Modeling Ali Borji, Member, IEEE, and Laurent Itti, Member, IEEE Abstract
Relative Influence of Bottom-up & Top-down Attention
Relative Influence of Bottom-up & Top-down Attention Matei Mancas 1 1 Engineering Faculty of Mons (FPMs) 31, Bd. Dolez, 7000 Mons, Belgium Matei.Mancas@fpms.ac.be Abstract. Attention and memory are very
Empirical Validation of the Saliency-based Model of Visual Attention
Electronic Letters on Computer Vision and Image Analysis 3(1):13-24, 2004 Empirical Validation of the Saliency-based Model of Visual Attention Nabil Ouerhani, Roman von Wartburg +, Heinz Hügli and René
FEATURE EXTRACTION USING GAZE OF PARTICIPANTS FOR CLASSIFYING GENDER OF PEDESTRIANS IN IMAGES
FEATURE EXTRACTION USING GAZE OF PARTICIPANTS FOR CLASSIFYING GENDER OF PEDESTRIANS IN IMAGES Riku Matsumoto, Hiroki Yoshimura, Masashi Nishiyama, and Yoshio Iwai Department of Information and Electronics,
PathGAN: Visual Scanpath Prediction with Generative Adversarial Networks
PathGAN: Visual Scanpath Prediction with Generative Adversarial Networks Marc Assens 1, Kevin McGuinness 1, Xavier Giro-i-Nieto 2, and Noel E. O Connor 1 1 Insight Centre for Data Analytic, Dublin City
5/20/2014. Leaving Andy Clark's safe shores: Scaling Predictive Processing to higher cognition. ANC sysmposium 19 May 2014
ANC sysmposium 19 May 2014 Lorentz workshop on HPI Leaving Andy Clark's safe shores: Scaling Predictive Processing to higher cognition Johan Kwisthout, Maria Otworowska, Harold Bekkering, Iris van Rooij
Attention. What is attention? Attention metaphors. Definitions of attention. Chapter 6. Attention as a mental process
What is attention? Attention Chapter 6 To drive a car you Use effort Sustain attention Orient to several locations Restrict attention Select particular objects Search for particular objects Respond with
Visual Thinking for Design Colin Ware
Visual Thinking for Design Colin Ware How much do we see? We do not have the entire visual world in conscious awareness We apprehend only a tiny fraction of information in our surrounding Just the right
Image-Based Estimation of Real Food Size for Accurate Food Calorie Estimation
Image-Based Estimation of Real Food Size for Accurate Food Calorie Estimation Takumi Ege, Yoshikazu Ando, Ryosuke Tanno, Wataru Shimoda and Keiji Yanai Department of Informatics, The University of Electro-Communications,
Attentional Selection for Action in Mobile Robots
7 Attentional Selection for Action in Mobile Robots Pilar Bachiller, Pablo Bustos and Luis J. Manso Universidad de Extremadura Spain Open Access Database www.i-techonline.com 1. Introduction During the