A Common Framework for Real-Time Emotion Recognition and Facial Action Unit Detection

Transcription

1 A Common Framework for Real-Time Emotion Recognition and Facial Action Unit Detection Tobias Gehrig and Hazım Kemal Ekenel Facial Image Processing and Analysis Group, Institute for Anthropomatics Karlsruhe Institute of Technology D Karlsruhe, P.O. Box 6980 Germany {tobias.gehrig, Abstract In this paper, we present a common framework for realtime action unit detection and emotion recognition that we have developed for the emotion recognition and action unit detection sub-challenges of the FG 2011 Facial Expression Recognition and Analysis Challenge. For these tasks we employed a local appearance-based face representation approach using discrete cosine transform, which has been shown to be very effective and robust for face recognition. Using these features, we trained multiple one-versusall support vector machine classifiers corresponding to the individual classes of the specific task. With this framework we achieve 24.2% and 7.6% absolute improvement over the overall baseline results on the emotion recognition and action unit detection sub-challenge, respectively. 1. Introduction Facial expressions are naturally used by humans as one way to communicate their emotions, opinions, intentions, and cognitive states with others and are therefore important in natural communication. However, in today s humancomputer interaction (HCI) scenarios this kind of information is mostly being neglected, in spite of the fact that it can be a much faster and more direct way than describing the affective state with words. To compensate for this neglect and improve HCI, recently, there have been many studies conducted on automatic facial expression analysis and the topic has become more and more popular. A wide range of applications [5] including, but not limited to, psychological studies, pain [2] or stress detection, online tutoring systems [19], and assistance systems for autistic persons [12] has also fueled this increasing interest in this topic. When we focus on applications in HCI scenarios, such as mobile service robots [20], or especially safety critical applications, such as drowsy driver detection [18], the runtime of an algorithm plays an important role, since the system should react in real-time without much latency. Facial expression analysis can be performed in different degrees of granularity. It can be either done by directly classifying the prototypic expressions (e.g. anger, fear, joy, surprise,... ) from face images or, with finer granularity, by detecting facial muscle activities. The latter is commonly described using the facial action coding system (FACS) [8], which defines action units (AU) corresponding to atomic facial muscle actions. Previous approaches to automatic expression analysis utilized, besides other feature representations, gabor wavelets [4], local binary patterns (LBP) [15] or the widely used active appearance models (AAM) to model the human face [11, 14]. Detailed surveys about works on automatic expressions analysis can be found in [9, 13, 16, 21]. In this study, we considered two main points while choosing our face representation method. The first main point was to be able to perform multiple face classification tasks, such as face recognition, gender and expression classification, with the same representation. The reason behind this is that if we can have a common framework that can provide us, for example, identity and gender information of the subject, this information can then be used to enhance the performance of the expression analysis system by employing person specific or gender specific expression analysis models, which has been shown in [14] to improve the performance. The second main point was to be able to have real-time processing capability. To achieve this, the feature extraction should be computed fast and at the same time the representation should be compact. These design choices led us to utilize the discrete cosine transform (DCT) for the representation. We employed a local appearancebased face representation approach, in which DCT is used to model local facial regions [7], and an ensemble of support vector machine (SVM) classifiers for the task of automatic facial expression analysis. We evaluate the approach 1

2 MCT-based face & eye detection eye-based alignment block-based DCT 10 Coeff. per block 1-vs-all SVM (RBF kernel) Figure 1. Overview of the common face processing and classification framework. within the emotion recognition and action unit detection sub-challenges of the FG 2011 Facial Expression Recognition and Analysis Challenge (FERA2011) [17] and compare our results to the baseline results, which were obtained using a local binary pattern (LBP)-based face representation [17]. We achieve 24.2% absolute improvement in terms of classification rate over the baseline results for the emotion recognition sub-challenge, and 7.6% absolute improvement in terms of F1-score for the sub-challenge on action unit detection using the same face representation. These results lead to a high rank and outperformed a range of other approaches in the FERA2011 emotion recognition and action unit detection sub-challenges [1]. In Section 2, we will describe the face representation framework, followed by a description of FERA2011, its dataset, tasks, and baseline system in Section 3. Then, we will present our proposed system for both tasks and discuss their results in Section 4. Finally, in Section 5, we will give conclusions and future research directions. 2. Methodology The face processing system is illustrated in Figure 1 and described in more detail in the following subsections Preprocessing The face and eyes are automatically detected using a modified census transformation (MCT) based face and eye detector [10]. Then the face image is aligned by an euclidean transform using the detected eye coordinates so that for all images the eyes are at the same position based on a given eye row and an interocular distance. This reduces the amount of variation in the feature space that is due to in-plane rotation, scale variations, and small pose changes. Finally, the image is converted to grayscale Local Appearance-based Face Representation From the preprocessed face image a local appearancebased face representation is retrieved. Such representations are known to provide more robustness against local appearance changes than holistic ones. This is due to the fact that the face representation of local approaches differs only in those regions, where changes occur, e.g. due to expression, local illumination, or occlusion, whereas for holistic approaches the whole face representation changes. Therefore, they are widely used in automatic face analysis, like face recognition [7]. Here, we retrieve these local regions by dividing the face image into equally sized, nonoverlapping blocks of N N pixels, as shown in Figure 1. The block size is chosen such, that it provides stationarity, simple transform complexity, and sufficient compression. These local blocks are then processed individually by the two-dimensional type-ii DCT, which provides a compact representation of the data similar to the Karhunen-Loeve transform (KLT), but with the advantage of being data independent. The DCT coefficients are extracted using zig-zag scanning and all coefficients but the first few are removed. To reduce the effect of illumination and balance the contribution of each coefficient, which normally has higher magnitude for small indices, the resulting coefficient vectors are normalized separately for each block, as proposed by Ekenel [7]. This is achieved by first dividing each coefficient by its standard deviation and normalizing the resulting local feature vector to unit norm. Finally, all local feature vectors are concatenated to form the overall feature vector Support Vector Machines (SVM) One of the most popular approaches of doing facial expression analysis is using SVMs [4, 11, 15, 17]. To prevent some attributes from dominating others, due to different numeric ranges, we normalize the feature vectors F = {f i }, before feeding them into the SVM, by making each attribute f i,j zero-mean and unit-variance over all feature vectors F. The normalization parameters are estimated on the training set and applied to the test set before classification. Here, we decided to use a SVM with a radial basis function (RBF) kernel, as proposed by [15], which transforms the features into a higher dimensional space, where it can be then linearly separated by a hyperplane. For both sub-challenges we utilize a single one-versusall classifier per class to be detected, which is trained on the occurances of the class as positive samples and occurances of samples of all the other classes as negative samples. For action unit detection, a threshold on the distance to the hyperplane serves us directly with the detection estimate for the particular AU in the current frame. For emotion recognition, additionally a model for probability estimates is trained for each classifier to make the classifier outputs more comparable. The decision for a frame is then achieved by selecting the class with the highest probability. Finally, majority voting over all frame decisions of a video gives us the emotion estimate for that video. 2

3 3. FG 2011 Facial Expression Recognition and Analysis Challenge (FERA2011) The presented approach is evaluated within the FG 2011 Facial Expression Recognition and Analysis Challenge (FERA2011) [17]. The aim of the challenge is to overcome the lack of standardized evaluation procedures and thus the low comparability in the previous literature on automatic facial expression analysis. In this section, we describe briefly the database used in FERA2011, its subchallenges, and the official baseline system. Detailed informations can be found in the challenge paper [17] Database The GEMEP-FERA database used for the FERA challenge is a subset derived from the GEneva Multimodal Emotion Portrayals (GEMEP) database [3]. It consists of recordings of 10 actors displaying a variety of expressions, while either uttering a meaningless phrase or the word Aaah. The movement of the actors faces varies from steady frontal to moving fast and wild with out-of-plane rotations. The sequences are between 1 and 4 seconds long. For the evaluation the dataset is provided as a strictly divided training and test set. The training set contains recordings of 7 actors, of which 3 are also present in the test set together with 3 unseen persons. Thus, the system can be evaluated on subject-dependent and subject-independent data Emotion Recognition Sub-Challenge For the emotion recognition sub-challenge the task is to predict one of five discrete, mutually-exclusive emotion classes (anger, fear, joy, relief, and sadness) per video. The performance of each emotion classifier is measured by means of the classification rate, meaning the ratio between the number of correctly classified videos and the total number of videos of the corresponding emotion. The overall system performance is computed as the average over all individual emotion classification rates Action Unit Detection Sub-Challenge For the action unit (AU) detection sub-challenge the task is to detect the presence or absence of 12 AUs from the facial action coding system (FACS) (1, 2, 4, 6, 7, 10, 12, 15, 17, 18, 25, and 26) on a frame-by-frame basis. During speech, which was labeled as AD50, AU25 and AU26 were not labeled and thus these speech frames are neither used for the training of the AU25 and AU26 classifiers nor are they included in the computation of the scores. The performance of each AU detector is measured by means of the F1- measure and the overall system performance is computed as the average over all individual F1-scores Baseline system The organizers of FERA2011 provide baseline results on the GEMEP-FERA corpus to have a common ground for participants to compare their results to [17]. This baseline system first uses the output of the OpenCV face and eye detectors to align and scale the face image to pixels. The aligned face image is then split into blocks of pixels, for which histograms of uniform local binary patterns (ULBP) with 8 neighbors and radius 1 are generated. These histograms are then concatenated to build a 5900 dimensional feature vector. The dimension of this feature vector is then reduced by applying the principal component analysis (PCA) to it. Finally one-versus-all SVMs with RBF kernel are utilized to classify the data. For the emotion recognition task they perform the processing as described above on the whole face, followed by a majority voting over the estimates for each video. In case of the AU task, they instead detect upper face AUs on the upper half of the face image and lower face AUs on the lower half, respectively. Additionally, they filtered the training set such that for each video only one frame per AU combination was used for the training. For further details about the baseline system the interested reader is kindly referred to the challenge paper [17]. 4. Evaluation of the proposed system In this section, we describe in detail our proposed setup common to both of our systems for the two FERA2011 sub-challenges. This is followed by the setup and results of the emotion recognition and action unit detection subchallenge, respectively. Finally, the run-time is evaluated Common Setup In our proposed common framework, each frame of the video sequences is first processed by a face and eye detector. Afterwards, the face images are aligned with respect to the detected eye center locations so that the eye distance equals 31 pixels and the eyes are in the 26th row of the crop-out. The DCT is calculated for blocks of 8 8 pixels and only the first 10 coefficients are used. This leads to a feature vector of 800 dimensions. These parameters have shown to be the best in preliminary experiments on the GEMEP-FERA data using double cross-validation or a development and validation set splitting on the training data. Also, it was shown in [7] that a block size of 8 8 and keeping 10 coefficients gives the best compromise between recognition rate and data compression for face recognition. If no face is detected the frame is simply ignored. If no eyes are detected the frame is ignored for training, but for testing the unaligned face is processed as described above. For classification, we use the SVM implementation as provided by LIBSVM [6] using a RBF kernel. The best 3

4 Table 1. Confusion matrix for emotion recognition on the part of the test set with unknown persons (person independent). pred. \ truth Anger Fear Joy Relief Sadness Anger Fear Joy Relief Sadness values for the soft margin parameter C and kernel parameter γ are estimated by doing a grid search utilizing a 5-fold subject-independent cross-validation on the training data Emotion Recognition Sub-Challenge Based on this common setup we built a system for the emotion recognition sub-challenge, which will be described in detail in this subsection together with a discussion of the results on the GEMEP-FERA dataset Setup For the emotion recognition sub-challenge we use a single one-versus-all SVM classifier per emotion class. For each classifier a model for probability estimates is trained. For the training of such an emotion classifier we use all the frames of the videos labeled with the corresponding emotion as positive samples and all others as negative samples. The grid search is performed over C = 2 k and γ = 2 l with k = 3,..., 1 and l = 16,..., 7, respectively. In the classification stage each frame of the videos is first classified by all the emotion classifiers. The estimated emotion for that frame is then set to the emotion corresponding to the classifier with the highest probability. Since the task is to output one emotion estimate per video, we finally performed majority voting over all the emotion estimates of the frames for each video, like it is done in the baseline method [17]. In case the same number of estimates is returned by multiple classifiers, we select the emotion that is first in an alphabetical order Results The confusion matrix for the emotion task is shown in Table 1, Table 2 and Table 3 for the person independent and person specific portion of the test set as well as for the whole test set, respectively. The classification rates for our proposed DCT-based approach and the official LBP-based baseline are given in Table 4 1. We can see from the confusion matrix on the person specific portion in Table 2 that there is almost no confusion be- 1 The results slightly changed since the challenge submission, since we found a small bug in the meantime. The submitted system had 65.8%, 94.4%, and 77.3% person independent, person specific, and overall classification rate, respectively. Table 2. Confusion matrix for emotion recognition on the part of the test set with known persons (person specific). pred. \ truth Anger Fear Joy Relief Sadness Anger Fear Joy Relief Sadness Table 3. Confusion matrix for emotion recognition on the whole test set. pred. \ truth Anger Fear Joy Relief Sadness Anger Fear Joy Relief Sadness Table 4. Classification rates for emotion recognition using DCT and the LBP baseline [17]. Results are provided for the person specific (PS), person independent (PI), and overall partitions. DCT LBP Emotion PI PS Overall PI PS Overall anger 92.9% 100.0% 96.3% 86% 92% 89% fear 40.0% 90.0% 60.0% 7% 40% 20% joy 100.0% 100.0% 100.0% 70% 73% 71% relief 68.8% 100.0% 80.8% 31% 70% 46% sadness 40.0% 100.0% 64.0% 27% 90% 52% Average 68.3% 98.0% 80.2% 44% 73% 56% tween the emotions, which can also be seen from the person specific column of the classification results in Table 4, where 100% accuracy was achieved for all classes besides fear. This shows that the local appearance-based DCT face representation is very well suited for emotion classification on known subjects. On the person independent portion the results look different. Here, 100% accuracy is achieved only for joy, but compared to the person specific portion sadness is confused by a huge amount with anger. Also fear is almost equally confused with anger and joy. Looking at the overall performance, one can see that anger and joy seem to be relatively easy to distinguish from the other emotions, which can also be observed from the LBP-based baseline results, reproduced in Table 4. But emotions like fear and sadness, and to some extend also relief, are very challenging for both approaches, when the subject is unknown. This could also simply mean that subjects tend to express these emotions differently. But to get a statistically meaningful answer to that, one would have to evaluate on more data. It can be observed that for each of the portions the DCTbased approach gives around 24.2% absolute improvement over the LBP-based baseline approach. Furthermore a classification rate of 98% on the person specific test data portion shows that our approach is well suited especially for person specific emotion classification tasks. 4

5 4.3. Action Unit Detection Sub-Challenge In this subsection, we apply the same local appearancebased DCT face representation framework to the action unit detection sub-challenge and describe the setup, the differences to the emotion classification system as well as the results on the GEMEP-FERA dataset Setup For the action unit detection sub-challenge we also use a single one-versus-all SVM classifier per action unit. For the training of such an action unit classifier we select all the frames of the videos for which the corresponding action unit was labeled to be active as positive sample candidates and all others as negative sample candidates. Only for AU25 and AU26 we additionally remove all samples from the training set that are labeled with AD50 as being present. These sample candidates are then balanced by randomly removing samples from the class that is larger until we have the same amount of positive and negative samples. The grid search for the parameter estimation of the SVM is performed over C = k and γ = 2 l with k = 0,..., 31 and l = 15,..., 1, respectively. In the classification stage, each frame of the videos is independently classified by all the action unit classifiers and all action units for which the distance to the hyperplane is greater than 0 are set to be present in that frame. In case, frames have been ignored due to missing face detections, all action units are set to be inactive. Table 5. F1 scores for action unit detection using DCT and the LBP baseline [17]. Results are provided for the person specific (PS), person independent (PI), and overall partitions. Action DCT LBP Unit PI PS Overall PI PS Overall AU1 60.6% 30.7% 50.8% 63.3% 36.2% 56.7% AU2 52.0% 40.5% 47.9% 67.5% 40.0% 58.9% AU4 59.9% 52.6% 57.3% 13.3% 29.8% 19.2% AU6 82.5% 62.0% 76.1% 53.6% 25.5% 46.3% AU7 51.6% 57.1% 53.9% 49.3% 48.1% 48.9% AU % 53.9% 49.4% 44.5% 52.6% 47.9% AU % 82.9% 80.5% 76.9% 68.8% 74.2% AU % 22.8% 18.0% 8.2% 19.9% 13.3% AU % 24.8% 42.7% 37.8% 34.9% 36.9% AU % 26.8% 31.9% 12.6% 24.0% 17.6% AU % 76.5% 78.0% 79.6% 80.9% 80.2% AU % 40.8% 45.3% 37.1% 47.4% 41.5% Average 55.2% 47.6% 52.7% 45.3% 42.3% 45.1% Table 6. Two alternative forced choice (2AFC) score for action unit detection using DCT and the LBP baseline [17]. Results are provided for the person specific (PS), person independent (PI), and overall partitions. Action DCT LBP Unit PI PS Overall PI PS Overall AU1 52.6% 58.2% 58.2% 84.5% 61.3% 79.0% AU2 72.9% 64.7% 70.0% 81.8% 64.0% 76.7% AU4 51.8% 49.9% 51.1% 48.1% 60.7% 52.6% AU6 87.9% 81.5% 84.8% 69.0% 56.8% 65.7% AU7 68.5% 62.6% 66.5% 57.2% 53.0% 55.6% AU % 59.1% 53.4% 57.7% 62.7% 59.7% AU % 90.0% 84.2% 73.8% 70.0% 72.4% AU % 50.9% 49.8% 55.5% 56.7% 56.3% AU % 58.2% 60.7% 67.9% 66.1% 64.6% AU % 57.4% 70.7% 62.0% 59.9% 61.0% AU % 67.1% 66.9% 54.4% 66.9% 59.3% AU % 54.2% 59.4% 45.7% 55.5% 50.0% Average 65.3% 62.8% 64.6% 63.1% 61.1% 62.8% Results The F1 measures and two alternative forced choice (2AFC) scores for the action unit task using our proposed DCTbased approach and the official LBP-based baseline are shown in Table 5 and Table 6, respectively. 2 We can see in Table 5, that for AU6, AU12, and AU25 we achieve reasonable results, which shows that smiles are very well detectable. The poor performance for AU15, AU17, AU18, and AU26 could be due to the amount of data available for their training, since for these AUs we had only between 400 and 1000 samples, while for the other AUs there were around 1300 to 2700 samples available. But still our approach performs on most of them better than the baseline, which might indicate that using DCT allows you to have less training samples for a classifier to generalize. Since both systems show similar trends in performance, these results could be due to the distribution of the AUs in the dataset or simply a hint that these AUs are simply harder to detect in this dataset with these features. 2 The results slightly changed since the challenge submission, since we found a small bug in the meantime. The submitted system had 52.2% overall F1-score and 64% overall 2AFC. One interesting thing about both systems is that for about two third of the AUs the performance is better on the person independent test set portion. According to the organizers, this could be due to the distribution of AUs across subjects. Compared to the LBP-based baseline results, our approach achieves 7.6% absolute improvement in terms of the overall F1 score Runtime Our system has also proven to be very fast in classifying the test data. On an Intel Core i5-750 with 4 cores of 2.67 GHz each, it processed the 4733 frames of the action unit detection test set in approximately 3.35 minutes, which corresponds to a frame rate of around 23.6 frames per second. For the emotion recognition, the classification of the 7537 test frames took approximately 4.92 minutes, which corresponds to around 25.5 frames per second. Both of these timing tests include everything, from loading the videos from disk, over feature extraction to classifying the samples and finally saving the results back to disk. These results show that the system is real-time capable. 5

6 5. Conclusion and Future Work In this paper, we proposed a common framework for real-time action unit detection and emotion recognition using a local appearance-based DCT face representation and one-versus-all SVM classifiers. We evaluated the proposed approach within the FERA 2011 Challenge and compared its performance with the official LBP-based baseline results. We achieve 24.2% and 7.6% absolute improvement on the emotion recognition and action unit detection sub-challenge, respectively. This shows that the local appearance-based DCT face representation is well suited for emotion classification as well as action unit detection tasks. We also showed that the system runs in real-time. In the future, we plan to investigate the low performance of some of the action unit detectors as well as the generalization of the approach to other databases. In addition, incorporating time information and utilizing the locality information of the AUs, as well as some intelligent sample selection method for determining on which samples the detectors should be trained could improve the performance. 6. Acknowledgments This work is funded by the Concept for the Future of Karlsruhe Institute of Technology within the framework of the German Excellence Initiative. References [1] FERA2011 website. 2 [2] A. B. Ashraf, S. Lucey, J. F. Cohn, T. Chen, Z. Ambadar, K. M. Prkachin, and P. E. Solomon. The painful face Pain expression recognition using active appearance models. Image and Vision Computing, 27(12): , [3] T. Bänziger and K. R. Scherer. Introducing the Geneva Multimodal Emotion Portrayal (GEMEP) Corpus. In K. R. Scherer, T. Bänziger, and E. B. Roesch, editors, Blueprint for affective computing: A sourcebook, pages Oxford University Press, Oxford, England, [4] M. S. Bartlett, G. C. Littlewort, M. G. Frank, C. Lainscsek, I. R. Fasel, and J. R. Movellan. Automatic Recognition of Facial Actions in Spontaneous Expressions. Journal of Multimedia, , 2 [5] M. S. Bartlett and J. Whitehill. Automated facial expression measurement: Recent applications to basic research in human behavior, learning, and education. In A. Calder, G. Rhodes, J. V. Haxby, and M. H. Johnson, editors, Handbook of Face Perception. Oxford University Press, [6] C.-C. Chang and C.-J. Lin. LIBSVM: a library for support vector machines, Software available at http: // cjlin/libsvm. 3 [7] H. K. Ekenel. A Robust Face Recognition Algorithm for Real-World Applications. PhD thesis, Universität Karlsruhe (TH), Karlsruhe, Germany, Feb , 2, 3 [8] P. Ekman and W. V. Friesen. Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto, California, [9] B. Fasel and J. Luettin. Automatic facial expression analysis: a survey. Pattern Recognition, 36(1): , Jan [10] C. Küblbeck and A. Ernst. Face detection and tracking in video sequences using the modified census transformation. Image and Vision Computing, 24(6): , June [11] P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, and Z. Ambadar. The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression. In Proceedings of 3rd IEEE CVPR Workshop on CVPR for Human Communicative Behavior Analysis, , 2 [12] M. Madsen, R. el Kaliouby, M. Goodwin, and R. W. Picard. Technology for Just-In-Time In-Situ Learning of Facial Affect for Persons Diagnosed with an Autism Spectrum Disorder. In Proceedings of the 10th ACM Conference on Computers and Accessibility (ASSETS), [13] M. Pantic and L. Rothkrantz. Automatic analysis of facial expressions: The state of the art. IEEE Trans. on Pattern Analysis and Machine Intelligence, 22(12): , [14] Y. Saatci and C. Town. Cascaded Classification of Gender and Facial Expression using Active Appearance Models. In Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition (FGR 06), [15] C. Shan, S. Gong, and P. W. McOwan. Facial expression recognition based on Local Binary Patterns: A comprehensive study. Image and Vision Computing, 27(6):803, , 2 [16] Y.-I. Tian, T. Kanade, and J. Cohn. Recognizing action units for facial expression analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(2):97 115, [17] M. F. Valstar, B. Jiang, M. Méhu, M. Pantic, and K. Scherer. The First Facial Expression Recognition and Analysis Challenge. In Proc. of the IEEE International Conference on Automatic Face and Gesture Recognition, , 3, 4, 5 [18] E. Vural, M. Cetin, A. Ercil, G. Littlewort, M. Bartlett, and J. Movellan. Drowsy Driver Detection Through Facial Movement Analysis. In Proceedings of ICCV 2007 Workshop on Human Computer Interaction, [19] J. Whitehill, M. Bartlett, and J. Movellan. Automatic Facial Expression Recognition for Intelligent Tutoring Systems. In Proceedings of IEEE CVPR Workshop on CVPR for Human Communicative Behavior Analysis, [20] T. Wilhelm, H.-J. Böhme, and H.-M. Groß. Classification of Face Images for Gender, Age, Facial Expression, and Identity. In Proceedings of 15th International Conference on Artificial Neural Networks: Biological Inspirations (ICANN 2005), pages , [21] Z. Zeng, M. Pantic, G. I. Roisman, and T. S. Huang. A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(1):39 58,