Natural Scene Categorization: from Humans to Computers

Size: px

Start display at page:

Download "Natural Scene Categorization: from Humans to Computers"

Daniella Crawford
5 years ago
Views:

1 Natural Scene Categorization: from Humans to Computers Li Fei-Fei Computer Science Dept. Princeton University

2 #1: natural scene categorization entails little attention (Rufin VanRullen, Pietro Perona, Christof Koch) Please type your An outdoor scene, I think. reminded me a a city... like walkingin description a park in new york or something. there seemed to be trees and a road and then this here: large skyscraper in the background. #2: what can we perceive within a glance of a scene? (Asha Iyer, Pietro #3: Bayesian graphical models for natural scene categorization and event recognition (Pietro Perona, Li-Jia Li) #4: decoding the neural representation of natural scene categories Eamon Caddigan, Dirk Walther, Diane Beck) VSS Natural Scene Symposium Li Fei-Fei

3 #1: natural scene categorization entails little attention Reference: Li et al. 2002; Fei-Fei et al VSS Natural Scene Symposium Li Fei-Fei

4 150 ms!! Thorpe, et al 1996

5 Treisman et al Attention binds features for recognition less attentional load more

6 Our question 1. How critical is attention in natural scene recognition? 2. How does this compare to other recognition tasks? less attentional load more

7 synthetic stimuli animals vehicle T Li et al. 2002

8 our finding less attentional load more Li et al. 2002

9 #2: what can we perceive within a glance of a scene? Reference: Fei-Fei et al. JoV 2007 VSS Natural Scene Symposium Li Fei-Fei

.. like walkingin a park in new york or something.

11 Stage I: Collect Image Description --- Illustration of 1 Trial Subject types freely what he/she saw in the image Please type your description here: An outdoor scene, I think. reminded me a a city... like walkingin a park in new york or something. there seemed to be trees and a road and then this large skyscraper in the background. time Mask onset: t = SOA 1 of 7 possible SOA s (msec): 27, 40, 53, 67, 80, 120, 500 Image onset: t = 0 msec

12 PT = 500ms PT = 27ms PT = 40ms subjects, subjects, images, images, responses responses PT = 67ms Fei-Fei et al. JoV, 2007

13 The attribute tree Fei-Fei et al. JoV, 2007

15 What s in a single glance? Fei-Fei et al. JoV, 2007

16 Scene level Fei-Fei et al. JoV, 2007

17 Object level Fei-Fei et al. JoV, 2007

18 (Social) Events Fei-Fei et al. JoV, 2007

19 Indoor vs. outdoor categorization Fei-Fei et al. JoV, 2007

20 #3: Bayesian graphical models for natural scene categorization and event recognition Reference: Fei-Fei et al. CVPR 2005, Li & Fei-Fei, submitted 2007 VSS Natural Scene Symposium Li Fei-Fei

21 from human vision we know living room beach city Fei-Fei et al natural scene categorization

22 model representation Image Bag of visual words Fei-Fei et al natural scene categorization

23 model representation & learning beach Latent Dirichlet Allocation (LDA) D c π Fei-Fei et al N z w natural scene categorization

24 evaluation & dataset Fei-Fei et al natural scene categorization

25 evaluation & dataset Our model Traditional texton model Fei-Fei et al natural scene categorization

26 evaluation & dataset Fei-Fei et al natural scene categorization

27 PT = 500ms Fei-Fei et al 2007 Event recognition

28 Li & Fei-Fei, submitted Event recognition

29 Li & Fei-Fei, submitted Event recognition

30 Li & Fei-Fei, submitted Event recognition

31 #4: decoding the neural representation of natural scene categories Reference: Et al. Fei-Fei, VSS 2007, HBM 2007 VSS Natural Scene Symposium Li Fei-Fei

32 Highways Mountains Buildings Industry Beaches Forests

33 Hubel & Wiesel 1961, Kanwisher et al. 1997

34 Epstein & Kanwisher, 1998

35 Tuesday May 15, Talk Session 9:00am # 865: Decoding distributed patterns of fmri activity associated with natural scene categories Eamon Caddigan, Dirk Bernhardt-Walther, Justas Birgiolas, Diane Beck & Li Fei-Fei

36 Summary Please type your description An outdoor scene, I think. reminded me a a city... like walkingin a park in new york or something. there seemed to be trees here: and a road and then this large skyscraper in the background. natural scene categorization entails little attention (Rufin VanRullen, Pietro Perona, Christof Koch) natural scene at a glance: gist of a scene includes much information on objects, scenes and beyond (Asha Iyer, Pietro Perona, Christof Koch) hierarchical graphical models for natural scene categorization and event recognition (Li-Jia Li) decoding the neural representation of natural scene categories from patterns associated with fmri activity (Diane Beck, Eamon Caddigan, Dirk Walther) VSS Natural Scene Symposium Li Fei-Fei

37 Thank you! VSS Natural Scene Symposium Li Fei-Fei

Natural Scene Categorization: from Humans to Computers

Natural Scene Categorization: from Humans to Computers Li Fei-Fei Beckman Institute, ECE, Psychology http://visionlab.ece.uiuc.edu #1: natural scene categorization entails little attention (Rufin VanRullen,