AFFECTIVE COMMUNICATION FOR IMPLICIT HUMAN-MACHINE INTERACTION

AFFECTIVE COMMUNICATION FOR IMPLICIT HUMAN-MACHINE INTERACTION Pramila Rani Electrical Engineering and Computer Science Vanderbilt University Nashville, TN 37235 pramila.rani@vanderbilt.edu Nilanjan Sarkar Mechanical Engineering Electrical Engineering Vanderbilt University Nashville, TN 37235 nilanjan.sarkar@vanderbilt.edu Craig A. Smith Psychology and Human Development Vanderbilt University Nashville, TN 37235 craig.a.smith@vanderbilt.edu Julie A. Adams Electrical Engineering and Computer Science Vanderbilt University Nashville, TN 37235 julie.a.adams@vanderbilt.edu Abstract: A novel implicit communication framework in human-machine interaction that is sensitive to human affective states is presented in this paper. The focus is to achieve detection and recognition of human affect based on physiological signals. This involves building an affect recognition system that accepts as input various physiological parameters and predicts the probable related affective state. Both decision tree and fuzzy logic methodologies have been applied to this problem. This paper presents the results of the two methods and discusses their comparative merit. Three human subject experiments were designed and trials were conducted with six participants. The experimental results demonstrate the feasibility of the proposed implicit human-machine interaction framework. Keywords: Affective computing, physiological responses, human-machine interaction, decision trees, fuzzy logic 1 Introduction Recent advances in robotics and intelligent systems are expected to usher in a new era where smart autonomous systems would have significant impact on our daily lives. As machines and people begin to co-exist and cooperatively share a variety of tasks, the need for machines to understand humans becomes increasingly important. The latest scientific findings indicate that emotions play an essential role in rational decision-making, perception, learning, and various cognitive tasks [2]. Therefore, endowing machines with a degree of emotional intelligence should permit more meaningful and natural human-machine interaction. The current trend in humanmachine interaction requires explicit communication between humans and machines. Typically communication occurs via explicitly typed or spoken instructions. Such communication modes are effective in many applications. However, this interaction relies solely on explicit communication and ignores the potential benefits of implicit communication. Implicit communication, as defined for this work, is affective communication where the person s affective state is interpreted by the machine. Examples of affective states that we have initially looked into are frustration, stress, anxiety, engagement and fatigue. It is argued that if machines understand a person s affective state, humanmachine interaction may become more intuitive, smoother and more efficient [15][16]. The potential systems applications that detect a person s affective state are varied and numerous. For instance, if a computer tutor is able to detect that a student is bored, the tutor may offer more challenging problems in order to engage the student and increase his or her attention. Alternatively, if a robot aiding in rehabilitation tasks detects that a physical therapy patient is becoming frustrated, the robot can simplify the task to better suit the patient s abilities. Other application areas include developing smart affect-based interfaces for tele-home health care [10], remote monitoring of soldiers and firefighters health [14], intelligent toys for engaging and educating children, and personal assistive machines in hospitals, homes and offices. The enormous potential benefit of affect-sensitive machines has motivated us to develop a theoretical, computational, and experimental approach that draws on emerging results from affective computing, psychology, advanced control theory, and robotics. This approach has been used to develop an intelligent and versatile humanmachine interaction framework that is affect-sensitive and capable of addressing the affective needs of a human. The resulting system should revolutionize human-machine interaction, enhancing human performance in numerous spheres of society. This paper focuses on affect recognition based on peripheral physiological signals obtained from wearable biofeedback sensors. Specifically we focus on detecting and recognizing anxiety and incorporating this capability into a machine s decision making process while it interacts 0-7803-7952-7/03/$17.00 IEEE 2003

with humans. We chose anxiety as the relevant affective state to focus on because the correlation of anxiety with physiology is well established in psychophysiology literature. Additionally, detection and recognition of anxiety is expected to improve the understanding between humans and machines. We designed experiments based on cognitive task performance to elicit anxiety in the participants and then used the associated physiological signals to develop anxiety detection methodologies for implicit human-machine interaction. Figure 1 provides a schematic overview of the humanmachine interaction framework. The physiological signals of the person working with a machine were recorded. Signal processing and pattern recognition tools were used to identify and classify the probable affective state associated with those physiological signals. The machine decides the next course of action based on the affective state information along with other environmental inputs. In the presented work we have limited ourselves to the signal processing for detection and recognition of anxiety. Measurement of Physiological signals Human Machine Signal Processing for Affect Detection and Recognition Figure 1. System Overview We have developed two anxiety detection methodologies one based on decision tree learning and another on fuzzy logic. Each system was trained using the same data to extract anxiety patterns from the physiological parameters. Both methods were tested to determine the accuracy and reliability of each. The paper is organized as follows: Section 2 describes affect detection and the rationale behind employing physiological signals. The experimental design details are presented in Section 3 along with the limitations of affect recognition and the applied approach. Sections 4 and 5 describe the two affect recognition methodologies. Section 4 discusses the fuzzy logic based approach and Section 5 describes the decision tree based methodology. The results from both approaches are presented in Section 6. A comparison of the methods is provided in Section 7. Finally, Section 8 summarizes the contribution of the paper and provides important conclusions. 2 Affect Detection and Rationale Intelligent Decision making based on Affective States Affect detection involves perceiving or sensing emotional states. Affective states have potentially observable effects over a wide range of response systems, including facial expressions, vocal intonation, gestures, and physiological responses (such as cardiovascular, electrodermal responses, muscle tension, respiratory rate and amplitude [12]). These diverse response modalities represent a wealth of information that can be used to infer an individual s affective state. However, an attempt to examine all available types of observable information would be immensely complex, both theoretically and computationally. Physical expressions (facial expressions, vocal intonation) are culture, gender, and age dependent that complicates their analysis. Physiological signals are employed in this work as they are generally involuntary and represent objective data points. Moreover, they offer an avenue for recognizing affect that may be less obvious to humans but more suitable for computers. There exists evidence that the physiological associated with various affective states is differentiated and systematically organized. The physiological signals initially examined along with the parameters derived from each signal are described in Table 1. These signals were selected because they can be measured non-invasively and are relatively resistant to movement artifacts. Additionally, electrodermal, the various cardiovascular parameters, and jaw EMG are strong indicators of anxiety [3][9][20]. In general, it is expected that these indicators can be correlated with anxiety such that higher physiological levels can be associated with greater anxiety [17][20]. Physiological Response Cardiac Electrodermal Electromyogra phic Temperature Table 1. Physiological Indices. Parameters derived Sympathetic power Parasympathetic power Mean IBI Std. of IBI Mean BVP Std. of BVP Mean Pulse Transit Time Mean tonic level Slope of tonic Mean phasic Maximum phasic Rate of phasic Mean - corrugator Std. of corrugator Mean of masseter Std. of masseter Mean temperature Slope of temperature Acronym used Power sym Power parasym IBI mean IBI std BVP mean BVP std Transit mean Tonic mean Tonic slope Phasic mean Phasic max Phasic rate Cor mean Cor std Mas mean Mas std Temp mean Temp slope

Multiple parameters (as shown in Table 1) were derived for each physiological measure. Power sym is the power associated with the sympathetic nervous system of the heart (in the frequency band 0.04-0.15 Hz.). Power parasym is the power associated with the parasympathetic nervous system of the heart (in the frequency band 0.15-0.4 Hz.). InterBeat Interval (IBI) is the time in millisecond between two normal R waves in the ECG waveform. IBI mean and IBI std are the mean and standard deviation of this IBI. The details of signal processing techniques used to obtain these parameters from cardiac response can be found in [13]. Blood volume pulse (BVP) measures the blood pressure in the extremities. Pulse transit time (PTT) is the time taken from a reference time for the pulse pressure wave to travel to the periphery. Tonic skin conductance refers to the ongoing or the baseline level of skin conductance in the absence of any particular discrete environmental events. Phasic skin conductance refers to the event related changes that occur, caused by a momentary increase in skin conductance (resembling a peak). The EMG signal from corrugator supercilii muscle (eyebrow) captures frowning instances of a person and detects the tension in that region. The EMG signal from the masseter muscle (jaw) captures the muscle movements while clenching/tightening of jaws. These anxiety indicators were combined in a multivariate manner and fed into the affect recognizers. A person s affective state was then inferred from this rich array of information. The results are compared with the participant s self-reports that represent the participant s assessment of his or her affective state. Some of these signals, either in combination or individually, have been used by other researchers to detect affective states of a person who is deliberately expressing a given emotion while at rest [19]. However, our approach to detect affective states while people are at work, to our knowledge has not been explored before. Various methods of extracting physiological signatures exist but efforts to identify the exact signatures related to emotions, such as anger, fear, or sadness have not been successfully developed due to person-stereotypy and situation-stereotypy [9]. That is, within a given context, different individuals express the same emotion with different characteristic response patterns (person stereotypy). In a similar manner, across contexts the same individual may express the same emotion differentially, with different contexts causing characteristic responses (situation stereotypy). The novelty of the presented affectrecognition system is that it is both individual- and contextspecific in order to accommodate the differences encountered in emotional expression. It is expected that in the future with enough data and understanding, affect recognizers for a class of people can be developed. Several possible methods of understanding the affective patterns were explored. It was required to represent each individual s anxiety index (a number on the nine-point scale indicating subjective anxiety level) as an approximate function of the physiological responses (parameters derived from cardiac, electrodermal, electromyogram ). Two affect recognizers were developed, one based on fuzzy logic and another based on decision tree learning. The MATLAB s statistics and fuzzy logic toolboxes were used to obtain the decision trees and to design the fuzzy logic system 3 Experimental Design An experiment was designed to gather physiological data related to anxiety. The experiment included tasks designed to elicit desired emotional responses from the participants. The research participants were presented with a series of tasks in which task difficulty was systematically varied in order to produce within-person variability to their anxiety level. The hypothesis was that as task difficulty increased, so would the person s anxiety levels. Six individuals (four female, and two male) participated in the experiment. Their ages ranged from 18 to 54 years of age. The design was a fully within-subjects design in which each participant engaged in six experimental sessions over the course of which they were engaged in two versions of three problem-solving tasks. During each session participants worked on the relevant task for approximately one hour. The three tasks consisted of an anagram solving task, a math problem-solving task, and a sound discrimination task. The anagram task presented participants with sequences of anagrams of varying difficulty. This technique has been used successfully in the past to explore the relationships between both electrodermal and cardiovascular with task engagement. By manipulating the difficulty of anagrams an individual was presented to solve, it was easy to systematically produce various levels of task engagement, ranging from boredom (a long sequence of trivially easy anagrams), optimal engagement (somewhat challenging anagrams), to both high levels of anxiety and ultimately task disengagement (a sequence of highly difficult or unsolvable anagrams). The math problem-solving task was similar to the anagram task. The sound discrimination task consisted of one third of the trials in which the participant heard a sequence of three tones. The participants were asked to determine whether the first and third tones were the same or different. The chosen tasks involved a variety of problem-solving skills, but all fell within the predefined cognitive problemsolving context. Each task set included easy and difficult tasks. A long series of trivially easy problems were expected to produce low anxiety levels. The difficult

problem sequence was expected to produce increasing anxiety levels. A variety of physiological measures (as mentioned in Table 1) were continuously collected during each task. Each of these measures was monitored using wearable sensors of the Procomp+ system from Thought Technology Ltd. [18], which is a multi-modality 8-channel system for real-time monitoring. The various biofeedback sensors were placed on the participant s body and while he or she performed the tasks being presented on the computer, the physiological responses were measured in real time. The ECG sampling rate was 256 Hz and that the sampling rate for the remaining sensors was 32 Hz. The Procomp+ sensor data acquisition and processing was done in Matlab environment. The Procomp system was interfaced directly to a PC via serial connection. A pair of sensors was placed on the distal phalanges of the index and ring fingers of the non-dominant hand to detect the electrodermal. The digit skin temperature was monitored from a sensor placed on the distal phalange of the small finger of the same hand; and the relative pulse volume was monitored by a photoplethysmograph placed in the distal phalange of that hand s middle finger. The ECG was monitored using a two-sensor placement on the participants chest. EMG was monitored using bipolar placements of miniature sensors over the left brow (corrugator supercilli) and the jaw muscle (masseter). A participant wearing all sensors was unable to freely move around. However, the objective was to demonstrate the feasibility of affect detection and recognition using wearable sensors. Once the scientific objective is achieved and the advantages demonstrated miniaturizing the sensors and streamlining the connection can be studied. The participants periodically reported their perceived subjective emotional states. This information was collected using a battery of fourteen questions rated on nine-point Likert scales. An anxiety index was determined based upon the scales by combining the responses that assessed anxiety, overload, and calm. This index provides a selfassessment of the participant s anxiety level while performing the tasks. The index is referred to as the selfreported anxiety. The self-reports were used as reference points to link the objective physiological data to participants subjective anxiety levels. Each task sequence was subdivided into a series of discrete epochs that were bounded by the selfreported affective state assessments. These assessments occurred repeatedly for each task. The first assessment occurred seven minutes into the task and every two minutes after that. Each physiological parameter was computed separately for each task epoch. 4 Fuzzy Logic Fuzzy logic is based on the fuzzy set theory. Zadeh introduced the concept in the 1960s to model the vagueness of natural language [21]. Unlike Boolean theory that has clearly defined set boundaries, fuzzy set theory has classes with fuzzy boundaries. Elements can simultaneously have partial membership in more than one set. This allows fuzzy models to flexibly capture data uncertainty. Fuzzy set theoretic methods have been used extensively for pattern recognition [6][11]. 4.1 Fuzzy Logic for Affect Detection The transition from one physiological state to another is a gradual one. These states cannot be treated as classical sets, which either wholly include a given affect or exclude it. Even within the physiological response variables, one set merges into another and cannot be clearly distinguished from another. For instance, consider two affective states- a relaxed state and an anxious state. If classical sets are used, a person is either relaxed or anxious at a given instance, but not both. The transition from one set to another is rather abrupt and such transitions do not occur in real life. In such cases, fuzzy reasoning can emulate the human decision-making process [13]. 4.2 Methodology The fuzzy model design and implementation was: (1) The input and output variable membership function specifications: An n-input 1-output fuzzy logic system was designed and trained based upon the collected physiological data. The number of inputs is variable as the number varies between individuals. Eighteen physiological parameters were recorded and the n most significant attributes were used for the classification system design. A correlation matrix was constructed to correlate the physiological variables and the self-reported anxiety index. The chosen physiological parameters were based upon the significant correlations, those with values over 0.30 which is considered high in psychological research for affect recognition. (2) Input variable fuzzification: Fuzzification determined the degree to which the variables belonged to each appropriate fuzzy set via membership functions [6]. For example, the extent to which the sympathetic input variable showed anxiety, it could be fuzzified into three sets: low, medium, and high. The employed membership functions were Gaussian or sigmoidal. The choice was based upon the type of variation the variables showed relative to the change in the self-reported affective state. (3) Defining rule statements to relate input variables to the output: The experimental result coupled with the

participant s self-report was used to formulate the fuzzy system rule set. n inputs resulted in 3n rules. (4) Output aggregations (Fuzzy Inference): The rules had to be aggregated in order for decisions to be made. All the rule outputs were combined into a single fuzzy set via the aggregation process. The result was one fuzzy set for each output variable. After all rules were evaluated, the output of each rule was aggregated into a single fuzzy set whose membership function assigned a weighting for every output value. (5) Output variable defuzzification: The defuzzification process transformed the fuzzy set into a single number. The aggregation process resulted in a range of output values that were defuzzified. The defuzzification process usually uses methods such as- centroid, bisector, middle of maximum (the average of the maximum value of the output set), largest of maximum, smallest of maximum, and other such methods to obtain a single value from the output set. 5 Decision Tree Learning Decision tree learning is a frequently used inductive inference method. A Decision tree learning based Decision Support System (DSS) has been used successfully for several medical applications such as ECG classification and various pathological diagnostic systems [4][5]. Decision trees approximate discrete valued functions that adapt well to noisy data and are capable of learning disjunctive expressions. A decision tree takes as input a situation or an object characterized by a set of properties and outputs a decision. Each node corresponds to a test of one input attribute and the branches that emerge from that node are possible test result values. The terminal or leaf nodes represent the value of the decision that will be returned if that node is reached. 5.1 Decision Tree Learning for Affect Detection Determining a person s probable affective state from the physiological response resembles a classification problem where the various attributes are the physiological parameters and the target function is the anxiety index. The training set is noisy as the biofeedback sensors are very sensitive to movement artifact. Some corrupted physiological data were discarded, resulting in the missing attributes. Decision tree learning is well suited to handle the above-mentioned data discrepancies. 5.2 Creating a decision tree A regression tree was employed to create a decisionmaking system that classified the physiological data and predicted the probable affective state [1]. Regression trees consist of several nodes, each representing a question that determines if a predictor satisfies a given condition. The system proceeds to the next question or arrives at a fitted response value dependent upon the provided answer. The creation of a decision tree began by choosing the best attribute to split the examples. The best attribute is the one that changes the classification the most. Once the examples were split each outcome was a new decision tree-learning problem containing fewer examples. Four cases needed to be considered when solving this problem for a particular node. They are: Examples encompassing many different values: Use the best attribute from the remaining attributes to split the examples. All examples have the same value: Return the value of the examples. No examples remain: No such example has been observed; therefore the return value is the majority classification of the node s parents. There are some examples with different output values but no attributes remain to classify them. This implies that the data is noisy or impure. The majority output value is returned. Proceeding in this manner the shortest decision tree that generalized the training data examples was constructed. Two primary issues existed: Choosing the best attribute to split the examples at each stage. Avoiding data over fitting. Many different criteria could be defined for selecting the best split at each node. This work determined the split that reduced node impurity the most. The same criterion was employed for tree pruning in order to avoid over fitting. 6 Results Six data sets were collected (one for each participant.) The results presented herein are based on the data set from a single participant. The same data set was used for all testing. The data set was randomly partitioned into two sets. The first set was used for system training while the second set was used to develop the results. A fixed attribute set (Power sym, Power parasym, IBI mean, IBI std etc.) and an ordinal output value (the anxiety index) was determined for each trial. A decision tree was constructed that classified all training examples by sorting them down the tree from the root to the leaf nodes. Once the training was completed, the tree was validated with the test data set. The test data set included new examples that had not been employed during the training phase. In particular, the capability of the tree to

generalize unknown data and the reliability of the tree as an inference engine was tested. One thousand trials were performed in which training and testing sets were randomly chosen. The mean percentage error (henceforth called error) was computed based upon the testing set for each regression tree. Three cases were selected for the comparison between the regression tree results and the fuzzy logic classification results. The selected cases represented the trees corresponding to the minimum error, maximum error, and average error. A fuzzy logic classification system was designed for each selected regression tree. The fuzzy logic system training sets were identical to those employed for the regression tree training. The fuzzy system was fine tuned and employed to classify the test set data. Table 2 provides a comparison of the results for each regression tree corresponding with the results of the best fuzzy logic system for a particular human subject (Subject #2). Other subjects also showed similar characteristics. Table 2 Mean percentage error - both methodologies 1 Best Regression Tree 2 Average Regression Tree 3 Worst Regression Tree Mean Percentage Error Regression Tree Fuzzy Logic 9.10 % 14.68% 20.56% 15.90% 41.18% 15.04% As can be observed from Table 2, the fuzzy logic system provides fairly stable error values. The mean percentage error is in the 14-15% range. However, the decision tree methodology appears to be lowering the error value to 9.1%. The decision tree performance is highly dependent on the training data set. A poor training set can increase the error as high as 41% while a good training set can considerably reduce the error (9.1%). The average error using the decision tree methodology was higher (20.56%) than the fuzzy logic methodology (15.9%). When conducting classification experiments, the system performance can be evaluated by determining the number of correctly classified examples compared to the incorrectly classified examples. The result is an indication of the classification goodness. In order to obtain a better understanding of the results, it is useful to know which data classes are most often misplaced. A useful tool for analyzing classifier system results is the confusion matrix. The matrix provides information regarding the actual and predicted classes [7]. The matrix columns represent the predicted classes; therefore data belongs to a particular column if it is classified as belonging to the class. The rows represent the actual classes, and data is placed in a particular row if it belongs to the corresponding class. A perfect classification results in a matrix with 0's everywhere except on the diagonal. A cell with a high count that is not located on the diagonal indicates that the classification system is confused between the classes associated with the row and those associated with the column. The comparison of the confusion matrices produced by decision tree and fuzzy logic methodologies provides feedback regarding the accuracy of each method. Tables 3 and 4 provide the resulting confusion matrices from the decision tree and fuzzy logic classifications. The tables represent the first case from Table 2. They correspond to the best classification achieved using for both methodologies. Table 3. The best decision tree confusion matrix. Anxiety 1.0-3.0 (I) 3.1-4.0 (II) 4.1-9.0 (III) Index 1.0-3.0 (I) 6 1 0 3.1-4.0 (II) 0 5 0 4.1-6.0 (III) 0 0 1 Table 4. The best fuzzy logic confusion matrix. Anxiety 1.0-3.0 (I) 3.1-4.0 (II) 4.1-9.0 (III) Index 1.0-3.0 (I) 4 2 0 3.1-4.0 (II) 2 3 0 4.1-6.0 (III) 0 1 1 It can be observed that the decision tree classification has higher accuracy as most values are along the diagonal. The fuzzy logic classification matrix has values scattered along as well as away from the diagonal. It can be observed that for class I, all six instances are correctly classified by the decision tree whereas the fuzzy logic system correctly classifies only four instances with two instances were incorrectly classified as Class II. Similar results were found for Class II. The decision tree correctly classified five instances while fuzzy logic correctly classified three of the six instances. 7 Discussion Similar work in this area includes the ongoing research in the Affective Computing Lab at MIT [12][19]. One of their focuses is on methods for recognizing the discrete emotional states of a person who is deliberately expressing one of eight pure emotions. Our work differs from theirs in two significant ways. First, our work focuses on detecting and recognizing one emotion anxiety, instead of several discrete emotions. The objective is to detect and isolate this emotion along a continuous axis (on a scale of

0-9 where 0 indicates least anxiety and 9 indicates the maximum level of anxiety.) Second, instead of collecting data from actors deliberately expressing a set of emotions, this work collects data from participants engaged in real life cognitive computer-based tasks. The methods that have been used by the MIT group include-hidden Markov models, sequential floating forward search (SFFS) feature selection with K-nearest neighbors classification, fisher Projection (FP) on structured subsets of features with MAP classification, and a hybrid SFFS-FP method. The methods employed in our work are fuzzy logic and decision trees to extract the level of anxiety from a given set of physiological parameters. This is an alternative approach that emulates human reasoning and holds promise for online recognition of affective states. The two classification methods that we employed were significantly different in their approach and performance. The decision tree method uses an impurity function to split the examples at each node. The redundant attributes were never used to split the examples and were automatically eliminated. However in the fuzzy logic system, the important attributes were identified as having a significant correlation to the output. The other attributes were considered redundant and manually eliminated from further computations. Using a good data set for the decision tree training reduced the mean percentage error of the output values. A good training data set best represents the system characteristics. However, the mean percentage error based on the fuzzy logic method usually saturated at 14-15%. Using a randomly selected training data set did not change the mean percentage error calculated for the fuzzy logic. So while there was a large fluctuation from the best to the worst decision tree classification performance, the fuzzy logic classification performance remained fairly stable over various data sets. Over fitting was avoided by using an impurity function to prune the decision tree. No steps were taken with the fuzzy system to avoid over fitting. This resulted in higher error rates for the testing data than in the training data. Another important difference was that the regression tree could handle missing attributes but the same did not hold for the fuzzy logic system. The fuzzy logic classification was not possible without all the attributes from an example data set. The development of the decision trees was automated and relatively simple. The fuzzy logic classification process has not yet been automated and required manual fine-tuning of the membership functions. The above comparison clearly indicates that fuzzy logic could be a good candidate when the training data set was either not large or not accurately representative of the system or phenomenon being modeled. The reason being fuzzy logic gave fairly stable error values irrespective of the quality of the training data set. Decision tree classification, on the other hand, was capable of reducing the error to very low levels when good training data sets were used (a mean error percentage error of 5% was attained using decision tree for Subject #4, while fuzzy logic classification stabilized at 19% mean percentage error). However, bad training sets could result in large error values (as high as 40%). This prompted us to infer that fuzzy logic could be used in the initial testing phases when the data sets were few and noisy. But as the research progressed and data collection assumed bigger and better proportions, decision tree classification and prediction techniques could be used with confidence. 8 Conclusion and Future Work An approach to computer-based affect recognition that employs human peripheral physiological signals in concert with decision theoretic and fuzzy logic methodologies to determine an underlying affective state was proposed. This work was performed as a feasibility study of implicit human-computer interaction. It is believed that such interactions may have broad applications. This paper demonstrated the ability to detect human anxiety. Three evaluations of six participants have been completed. These evaluations were designed to elicit anxiety. The physiological parameters that contain useful information regarding the target affective state have been determined. Decision trees and fuzzy logic have been employed for data classification. The results of testing both systems were presented as well as the strengths and weakness of each method. The proposed approach is general and may be employed to detect other affective states (i.e. frustration, engagement, and boredom). Future work includes performing further physiological data collection evaluations and a comparison of these methods with other methodologies such as reinforcement learning, neuro-fuzzy techniques, genetic algorithms and Bayesian learning. It would also involve implementation of the closed loop feedback for the humanmachine interaction framework (as shown in Figure 1).The affective state detected by the machine would be used for intelligent human-machine interaction. Acknowledgements The work was partially supported by NSF grant IIS- 0107775, NASA grant NAS5-98051 (07600-100), and a Vanderbilt University Discovery grant. The authors also acknowledge the help from Dr. Leslie D. Kirby.

References [1] L. Breiman, J. H. Friedman, R. A. Olshen and C. J. Stone, Classification and Regression Trees, Chapman & Hall/CRC, 1984. [2] A. R. Damasio, Descartes' error, Grosset/Putnam, New York, 1994. [3] M. E. Dawson, A. M. Schell and D. L. Filion, The electrodermal system, in J. T. Cacioppo and L. G. Tassinary (eds.), Principles of psychophysiology: Physical, social, and inferential elements, Cambridge University Press, New York, pp. 295-324, 1990. [4] S. Downey and M. J, Russell, A Decision Tree Approach to Task Independent Speech Recognition, Proc. Inst Acoustics Autumn Conf on Speech and Hearing, Vol 14, No 6, pp. 181-188, 1992. [5] E. Gilpin, R. A. Plshen, H. Henning, and J. Ross, Risk Prediction After Myocardial Infarction: Comparison of Three Multivariate Technologies, Cardiology, Vol 70, pp. 73-84, 1983. [6] A. Kandel, Fuzzy Techniques in Pattern recognition, Wiley-Interscience, New York, 1982. [7] R. Kohavi and F. Provost, Glossary of Terms, Editorial for the Special Issue on Applications of Machine Learning and the Knowledge Discovery Process, Vol 30, No 2/3, Feb/March 1998. [8] P. Kokol, M. Mernik, J. Završnik, K. Kancler and I. Malèiæ, Decision Trees and Automatic Learning and Their Use in Cardiology, Journal of Medical Systems, Vol 9, No 4, pp. 201-206, 1994. [9] J. L. Lacey and B. C. Lacey, Verification and extension of the principle of autonomic responsestereotypy, American Journal of Psychology, Vol 71, No 1, pp. 50-73, 1958. [10] C. Lisetti, F. Nasoz, C. LeRouge, O. Ozyer, and K. Alvarez, Developing Multimodal Intelligent Affective Interfaces for Tele-Home Health Care, International Journal of Human-Computer Studies Special Issue on Applications of Affective Computing in Human-Computer Interaction. [11] S. K. Pal and D. Majumder, Fuzzy Mathematical Approach to Pattern Recognition, Wiley(Halsted Press), New York, 1986. [12] R. Picard, Affective Computing, The MIT Press, Cambridge, 1997. [13] P. Rani, J. Sims, R. Brackin, and N. Sarkar, Online Stress Detection using Psychophysiological Signal for Implicit Human-Robot Cooperation, Robotica, Vol. 20, No. 6, pp. 673-686, 2002. [14] M. V. Scanlon, Acoustic Sensor for Health Status Monitoring, Proceeding of IRIS Acoustic and Sensing, Vol. 2, pp. 205-222, 1998. [15] H. Simon, Motivational and Emotional Controls of Cognition, Models of Thought, pp. 29-38, Yale University Press, 1979. [16] A. Sloman and M. Croucher, Why Robots will Have Emotions, Proc. of the Seventh International Conference on Artificial Intelligence, pp. 197-202, 1981. [17] C. A. Smith, Dimensions of appraisal and physiological response in emotion, Journal of Personality and Social Psychology, Vol 56, No 3, pp.339-353, 1989. [18] Thought Technology Limited http://www.thoughttechnology.com/ [19] E. Vyzas and R. W. Picard, Offline and Online Recognition of Emotion Expression from Physiological Data, Proc. of the Workshop on Emotion-Based Agent Architectures at the Third International Conference on Autonomous Agents, Seattle, 1999. [20] R. A. Wright and L. D. Kirby, Effort determination of cardiovascular response: An integrative analysis with applications in social psychology, in M. Zanna (ed.), Advances in experimental social psychology, Vol 33, Academic Press, pp. 255-307, 2001. [21] L. Zadeh, Fuzzy Sets, Information and Control, vol. 8, No 3, pp. 338-353, 1965.