Expression Recognition. Mohammad Amanzadeh

Transcription

1 Expression Recognition Mohammad Amanzadeh

2 Body Movement Movement is one of the most basic human skills that used for communicating and interacting with the environment Although we have an intuitive understanding of our gestures, it is hard to explain their quality Gestures are collection of various actions for performing different tasks. But it does not explain How the tasks are performed

3 Expressive Movement Research Movement recognition is one of the most researched topics in the area of human computer interaction Most of the previous research focuses on the recognition of specific actions in movement Some recent work in this area tries to recognize the expressive aspects of movement to create a more natural representations of movement

4 MovingStories

5 Motion Capture 3D Motion Capture systems are widely used for collecting the movement data for gesture recognition Marker-based VS Marker-free systems (tradeoff between accuracy and user friendliness) Kinect has an infrared camera to understand the depth, and uses computer vision techniques on the RGB and depth-map inputs to track the position of body joints 20 body joints recognized with Microsoft Kinect adopted from

6 Laban Movement Analysis Laban Movement Analysis (LMA) is an experiential and analytical system developed by Rudolf Laban (1980) for expressing all types of the human movement. Four major components: Body: describes the structural and physical properties of the body while moving Effort: represents more subtle properties of the movement and answers how the body is moving Space: describes movement in terms of where the body moves in the environment Shape: describes the relationship and interaction of the body shape with its environments.

7 Basic Effort Factors Action has three inner stages, which are attention, intention, and decision LMA describes the effect of inner intention on movement with four Basic Effort Factors (BEFs) that represent the qualities of movement: Weight, Space, Time, and Flow Laban, R. (1980). The mastery of movement: a relaxed or forceful attitude towards weight, a pliant or lineal attitude towards space, a prolonging or shortening attitude towards time, and a liberating or withholding attitude towards flow. Laban Effort continuum Each BEF is expressed as a continuum between an indulging and fighting extreme that represent the opposite mental attitudes in performing an action

8 Basic Effort Actions Every human action carries a combination of indulging or fighting state of weight, space, and time efforts We can define 8 basic actions with a combination of the extreme quality of these BEFs The emphasis we put on the BEFs shows various type of actions Moving a piano (press, strong weight) vs. Ironing (Glide, light weight) Basic Effort Actions

9 Recognizing BEF/BEA We asked two individuals trained in LMA to assist us in recording eight Basic Effort Actions The raw motion capture data cannot be directly used for identifying the BEFs. The positional data will limit the system to the exact movement trajectories in 3D space Extracted the derivative of positional data with various orders to obtain information about the dynamics of motion, such as velocity and acceleration We also used different curvature measures to describe the angle of changes in position

10 Press Glide

11 Glide Press

12 Hidden Markov Model Hidden Markov Models (HMMs) have been widely used in modeling sequential data or signals With HMM we can describe the probability distribution of data while preserving its continuity It can identify regions of data that have similar distribution (similar mean and variance) For example a sequence can be described with 4 states, where each state has a specific distribution, and some probability for going from state x to state y Images adopted from MLSP (CS 598 PS) lecture notes by Paris Smaragdis

13 Hidden Markov Model * Images adopted from MLSP (CS 598 PS) lecture notes by Paris Smaragdis

14 Classifying BEFs Classify by comparing the likelihood of HMMs trained on indulging and fighting quality of a BEF

15 Movement Features

16 Testing on different performers

17 Limitations Low recognition scores, various reasons, mostly because of similarity of features Low recognition rate when trained on different person We require a more general model

18 Conducting Gestures Conducting gestures have well defined rules, and ideal for expression recognition Conductor directs an orchestra by conveying expressions and beat through specific gestures Recognizing Articulation Styles and Beat Pattern

19 Principle Component Analysis * Images adopted from MLSP (CS 598 PS) lecture notes by Paris Smaragdis

20 Beat Detection

21 Comparing Features 0.06 μ(acceleration) 0.04 μ(speed) Beat1 Beat2 Beat3 Beat4 BeatAvg Beat1 Beat2 Beat3 Beat4 BeatAvg Sostenuto60 Staccato60 Sostenuto120 Staccato120 Sostenuto60 Staccato60 Sostenuto120 Staccato σ²(acceleration) Beat1 Beat2 Beat3 Beat4 BeatAvg σ²(speed) Beat1 Beat2 Beat3 Beat4 BeatAvg Sostenuto60 Staccato60 Sostenuto120 Staccato120 Sostenuto60 Staccato60 Sostenuto120 Staccato120 Acceleration Speed