Reader s Emotion Prediction Based on Partitioned Latent Dirichlet Allocation Model

Transcription

1 Reader s Emotion Prediction Based on Partitioned Latent Dirichlet Allocation Model Ruifeng Xu, Chengtian Zou, Jun Xu Key Laboratory of Network Oriented Intelligent Computation, Shenzhen Graduate School, Harbin Institute of Technology, Shen Zhen, China Abstract Different from traditional emotion analysis which focuses on the identification of emotions from the text, this research aims to predict the reader s emotion for given text. Regarding reader emotion as the response to the text, emotion prediction may be transferred to a classification problem which classifies the text into the categories causing different emotions. In this study, we propose an emotion prediction approach based on Partitioned Latent Dirichlet Allocation (PLDA) model. Through providing the supervised information to the training process of LDA model, PLDA model associates the words from one certain type of emotion to one certain partition of topics. The outputs of PLDA model are used as the features of a multi-label classifier for predicating the reader s emotion. Evaluations on a large community emotion corpus show that PLDA model achieves much better performance compared to bag of words model and LDA model. Keywords: Reader s Emotion Prediction, LDA model 1. Introduction Recently, with the development of Web 2.0 and social network, the social community emotion analysis becomes more and more attractive. Many websites allow users voting their reactions to the news articles, which makes the study of reader s emotion predication feasible. Up to now, most researches in emotion analysis area focus on the understanding of writer s emotion in the text while few works on reader s emotion. These two types of emotion are not always the same, even in contrary, in some cases. Consequently, the emotion analysis techniques from writer s perspective cannot be applied to reader s emotion prediction directly. The existing works in this area are few and most of which are based on the single-label classification method [1][2][3][4], which classifies the document to only one category within several basic emotion categories. However, the single-label classification approach is conflict to the fact that community readers reaction always contains more than one dominant emotion types, which is observed from reader emotion voting results among many news website. Obviously, regarding the reader emotion prediction problem as a multi-label classification task is more reasonable. Many previous works on emotion analysis and predication are majorly based on supervised machine learning approach. Most of them represents a document as a Bag of Words (BOW) and uses words vector as the classification features. The studies mainly focus on choosing appropriate word vectors to represent the content of the document. Document frequency (DF) and chi-square statics (CHI)[5] and so on were adopted as the selecting metrics. However, these purely statistic-based methods usually generate a large dimension word vectors with much noises. Meanwhile, these vectors always cannot reflect the content of the text effectively as each terms is considered as independent. With the development of topic model such as Latent Dirichlet Allocation [6], some research use the topics instead of words to represent a document for predicating the reader s emotion. A better performance is achieved. However, there are still some disadvantages in the researches based on the LDA model. Since LDA employs an unsupervised learning method while the supervised information, i.e. readers voting information are not used. In this paper, we propose to apply Partitioned Latent Dirichlet Allocation (PLDA) model to topic modeling, which makes all of the words from one certain type of emotion only associated to one certain partition of topics through introducing the supervised information into the training process of LDA model. Then, we employ the output of PLDA as features to do the multi-label classification for predicting the reader s emotion. Evaluations on a large user-generated community emotion corpus show that the proposed method achieves a better performance compared to BOW model, LDA model and weighted LDA model. The rest of this paper is organized as follows. Section 2 reviews the related work. Section 3 presents the emotion prediction approach based on PLDA and multi-label classification. Evaluations are given in Section 4. Finally, Section 5 concludes. 2. Related Works This section reviews some of the related work on reader s emotion prediction. As our major approach is based on the

2 topic model, some of the important works related to this is also introduced. Finally, some method concerns with the multi-label classification is reviewed. 2.1 Reader s Emotion Prediction With the rapid increase of user labeled emotions on the news website, the study tries to analyze and predicate the reader s emotion of the news article becoming available. With the help of corpus from Yahoo! Kimo News, Lin et al.[7] used five different strategies to select features for predicting readers emotions. The employed features include bigram, Chinese words, news meta data and so on. Later, they employed PLM (Pairwise Loss Minimization) and EDR (Emotional Distribution Regression) algorithms to rank the reader emotions. Ye et al.[5] took document frequency(df), chi-square statistic(chi) and some other criteria such as POS tags as features and applies Multi-Label K-nearest Neighbor(MLKNN)[8] and RAkEL [9] algorithm to make the emotion prediction, respectively. The results show that RAkEL com-bined with the intersection of chi-square statistics and document frequency achieve better results. 2.2 Topic Modeling There are many approaches for modeling docu-ments topic. Among them the Latent Dirichlet Allocation(LDA) model has been applied to many areas such as information retrieval and text category for its flexibilities and completeness. As the traditional LDA model employs unsupervised machine-learning method, many studies try to adding the supervised information into the training process of LDA model, or some other improvement. Ramage et al.[10] proposed a Labeled LDA which could be consider as extension of both LDA and Multinomial Naive Bayes. They associate each user tag with topics generate from the model, and use this model to label individual words as well as providing interpretable snippets for document summarization. Zhu et al.[11] proposed a maximum entropy discrimination LDA topic model(medlda) which utilizes the max-margin principle in the training process as a supervised topic models. Wilson et al.[12] 2010 proves that of each term in the LDA model should be weighted and give a scheme for this. 2.3 Incorporate Topic Model to Emotion Prediction Recently, many studies of emotion prediction are based on the topic of text. Normally, they got a better performance. Lin et al.[13] proposed an unsupervised joint sentiment/topic model (JST) based on LDA which may deal with both sentiment and topic at the same time. Bao et al. [14][15] proposed a joint emotion-topic model to predict reader s emotion. The main idea of their model is to add another emotion layer into traditional LDA model which appends emotion generation step for the generation process of the model. However, all these models output only one emotion prediction. Xu et al.[16] gave every term different weights of topics to expand the LDA model. They used the Multi- Label k-nearest Neighbour (MLkNN) to do the emotion predication. The achieved performance is higher than BOW model and LDA model. 3. Reader Emotion Prediction Based on PLDA 3.1 Partitioned Latent Dirichlet Allocation Model LDA model is proposed by Blei et al.[6] It may be considered as a three-level hierarchical Bayesian model.after Griffiths et al.[17] brought a Dirichlet prior distribution on parameter, LDA changes into a complete generative model. For generating a word of a document, LDA firstly choose a topic from the polynomial distribution of the document and then choose a word from the polynomial distribution of the topic. The generating process is shown in fig1. A document is treated as a mixture of topics instead a bag Fig. 1: Framework of Latent Dirichlet Allocation topic model of words, while a topic is treated as a mixture of words. It is reasonable to treat a document in this way because reader s emotions are usually connected with certain types of topics or events. The two important parameters which shows the topics distribution of each document and words distribution of each topic could be estimated by many algorithms. Since the Gibbs sampling is shown simple and computationally efficient in the previous studies, it is adopted in this study. The detailed procedures of Gibbs sampling for our method is presented in Xu et al. [16]. During the parameter estimation process, terms from document are assigned to any topic without considering the category information of documents. The two parameters θ and β are then updated. The analysis on the topic distribution of a document shows that each topic takes a d D

3 certain proportion. However, a better topic model should have a large disparity between topics, and document should shows obvious disparity on every topics. More specifically, in the emotion prediction task, documents belong to one certain type of emotion should have a larger proportion on the topics which could be connected with that type of emotion. For this purpose, in our study, an improved model is proposed which takes the labeled emotion category of each document into account. In this way, the traditional unsupervised LDA model is transferred to a supervised one. The generate process of this model is shown in Fig 2. Where c j means the j th type of emotion, K is the Doc d Fig. 2: Framework of Latent Dirichlet Allocation topic model total count of topics, M is the total number of emotions categories. The only step different from the traditional LDA model is that when generating a word of one document, a certain category of emotions should be chosen first, then the topic chosen could only be from this emotion category. It can be observed that the topics Z are partitioned by the category of emotions. When deal with the input training documents with reader s emotion label, during each iteration of the parameter estimating process, each word of document is assigned to the topics which is connected to the certain type of emotion of this document. For example, suppose there are 80 topics and 8 emotion category in total. Thus, each emotion category is associated with 10 topics. The "touched" emotion is connected with topic 0-9, "sympathy" emotion is connected with topic 10-19, and so on. During the training process, all the words from "touched" document are distributed in topic 0-9 and the "sympathy" ones are distributed in topic 10-19, the probability for any other topics are 0. Thus, the documents with the same emotion label are distributed in the same topics while the documents with different emotion label can t affect each other. This makes the model just like constructed by 8 independent small partitions. In the prediction period, after input the document without emotion label, the model will output the topic distribution of document and word distribution of topic, the first one of which could be used to make the multi-label classification on reader s emotion. 3.2 Multi-label Classification As the distribution of one document on every topic has already been calculated by the model and the connection between topics and emotion types is known, the probability of the document belongs to one specific type of emotion could be calculated, as shown in equation 1: P (d i c j ) = (j+1) ( K /M) 1 k=(j 1) ( K /M) θ ik (1) where d i is the input document, c j means the j th type of emotion, K is the total count of topics, M is the total type count of emotions, θ ik means the probability of document i belonging to topic k. For each group of K /M topics which could be associated with one single emotion, the corresponding probabilities may be computed. Thus, M values which shows the probabilities of the document belongs to each type of emotions are obtained. Then, the M values are normalized. The one or several of them if bigger than 0.3, an empirical threshold, the emotion labels which they belong to will be set as the document s emotion predication labels. 4. Experiments and Discussions 4.1 Dataset and Evaluation Metrics A community-based reader votes corpus is used to evaluate the proposed method. The news articles and corresponding reader s emotion votes are collected form sina.com. We adopted eight types of emotion in total with the labels Touched, Empathy, Boredom, Anger, Amusement, Sadness, Surprise and Warmness. The votes of articles which are lower than empirical threshold are filtered out. Finally, 8,802 articles with 1,454,912 emotional votes are obtained in total. There are about 165 votes for each article on the average. The detailed statistics for each emotion category is shown in table 1. We determined the majority emotion categories based on the statistics of original votes, in which there are 5,745 articles with one emotion labels, 2,770 ones with two emotion labels and 287 ones with three labels. This means that there are totally about 35 percent of articles with more than one label, which shows that the multi-label classification is important. Table 2 shows the number of aricles for each combination

4 of emotion types. More information about this dataset are presented in[5]. Table 1: Number of news articles with different number of labels Emotions News Articles Votes Touched Empathy Boredom Anger Amusement Sadness Surprise Warmness Table 2: Number of news articles with different number of labels Labels Labels combination News Articles Anger Touched 1189 Amusement 1032 Surprise 440 Anger Amusement Empathy Sadness 394 Boredom Amusement 374 Anger Sadness 271 Boredom Anger Amusement 65 3 Empathy Anger Sadness 47 Touched Empathy Sadness 26 Boredom Amusement Surprise 65 In this study, the commonly used metrics for single-label classification are replaced by those for multi-label classification due to the inherent differences of the classification problem. Hamming Loss (HL): it measures the inconsistency between the predicted sets of labels and the actual over the examples. Hamming Loss (HL): it measures the inconsistency between the predicted sets of labels and the actual over the examples. One-error (OE): it evaluates how many times the topranked label is not in the set of relevant labels of the instance, and it could evolve into general classification error rate in the single-label classification. Coverage (COV): it measures in the ranked list, from the category of the top-ranked to start, how many labels are needed to cover all the relevant labels of the example. Ranking Loss (RL): it shows the probability that the irrelevant labels are ranked higher than the relevant. Average Precision (AVP): it reflects the average precision of the predicted labels. We take 75 percent of articles with their votes as training and the others as test set in the experiments. 4.2 Baseline Systems In the experiment, we adopted two groups of baselines. The multi-label classification algorithms, namely MLkNN and BR are combined with BOW model and regular LDA model, as one group of baselines. Another one is the MLkNN combined with regular LDA model and Weighted LDA(WLDA) which is presented in [16]. 4.3 Experiment Results and Discussions Firstly, the performance of emotion prediction based on multi-label classification algorithms and BOW model are evaluated. The classification features set differs in the dimensions of words chosen by DF or CHI criteria. The results are shown in table 3. It is observed that the incorporation of BR algorithm and Table 3: Performances of Emotion predication by multi-label classification and BOW model with different feature set Features Methods HL OE COV RL AVP 10,000DF BOW+MLkNN BOW+BR ,000CHI BOW+MLkNN BOW+BR ,000DF BOW+MLkNN BOW+BR ,000CHI BOW+MLkNN BOW+BR ,000DF BOW+MLkNN BOW+BR ,000CHI BOW+MLkNN BOW+BR DF BOW+MLkNN BOW+BR CHI BOW+MLkNN BOW+BR Table 4: Performances of Emotion predication by LDA and WLDA with different feature set Topic Methods HL OE COV RL AVP 20 LDA+MLkNN WLDA+MLkNN LDA+MLkNN WLDA+MLkNN LDA+MLkNN WLDA+MLkNN LDA+MLkNN WLDA+MLkNN LDA+MLkNN WLDA+MLkNN LDA+MLkNN WLDA+MLkNN LDA+MLkNN WLDA+MLkNN BOW model with the features selected by CHI-test achieves better results. The average precision is Meanwhile, it is shown that more features are helpful. In the second experiment, LDA and WLDA model

5 Table 5: Performance of PLDA model on multi-label predicting. Topic HL OE COV RL AVP are incorporated with MLkNN classification algorithm, respectively. The Document-topic distributions with different number of topics generated from the model are used as feature vectors. The achieved performances are shown in table 4. In this experiment, MLkNN with topic models achieve better performance compared with BOW model. WLDA combined with MLkNN under the situation of 30 topics achieved the best performance. These results shows the advantage of topic model over BOW model. This also shows that WLDA model performs better than regular LDA model in readers emotion predication. In the third experiment, the PLDA model with multi-label classification is adopted. Table 5 gives the performances. The experiment begin with 40 topics, and add 8 topics each time. When topic number is less than 100, the performance keeps on increasing with the topic number. As the topic number is more than 100, the performance stabilized at about AVP 88 percent. It is observed that the performance with PLDA is much better than the base line systems on any metrics. 5. Conclusion In this paper, we propose a new topic model for reader s emotion prediction problem by incorporating the supervised information into the regular LDA model and change it into a supervised machine-learning model. The evaluations on the dataset of a large-scale community reader emotion votes show that PLDA achieves a rather encouraged result. Education , Shenzhen Foundational Research Funding JCYJ , and Shenzhen Interna-tional Cooperation Research Funding GJHZ References [1] Z. Kozareva, B. Navarro, S. Vazquez, and A. Montoyo. UA-ZBSA: a headline emotion classification through web information. In Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007), pages , [2] F. R. Chaumartin. UPAR7: a knowledge-based system for headline sentiment tagging. In Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007), pages , [3] P. Katz, M. Singleton, and R. Wicentowski. SWAT-MP: the semeval systems for task 5 and task 14. In Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007), pages , [4] K. H. Y. Lin, C. Yang, and H. H. Chen. Emotion classification of online news articles from the reader s perspective. In Proceedings of Web Intelligence, pages , [5] L. Ye, R. F. Xu, and J. Xu. Emotion prediction of news articles from reader s perspective based on multi-label classification. In Proceedings of IWWIP [6] D. M. Blei, A. Y. Ng, and M. I. Jordan. Latent dirichlet allocation. The Journal of Machine Learning Research, 3: , [7] K. H. Y. Lin, and H. H. Chen. Ranking reader emotions using pairwise loss minimization and emotional distribution regression. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages , [8] M. L. Zhang, and Z. H. Zhou. A k-nearest neighbor based algorithm for multi-label classification. In Proceedings of the 1st IEEE International Conference on Granular Computing (GrC 2005), pages , [9] P. K. Bhowmick, A. Basu, and P. Mitra. Reader perspective emotion analysis in text through ensemble based multi-label classification framework. Computer and Information Science, 2(4):64-74, [10] D. Ramage, D. Hall, R. Nallapati and C. D. Manning. Labeled LDA: a supervised topic model for credit attribution in multi-labeled corpora. In Proceedings of the 2009 conference on Empirical Methods in Natural Language Processing, pages , [11] J. Zhu, A. Ahmed, et al. MedLDA: maximum margin supervised topic models for regression and classification. ACM, [12] A. T. Wilson and P. A. Chew. Term weighting schemes for Lantent Dirichlet Allocation. The 2010 Annual Conference of the North American Chapter of the ACL, pages , [13] C, Lin and Y. He. Joint sentiment/topic model for sentiment analysis. ACM, [14] S. Bao, S. Xu, et al. Joint emotion-topic modeling for social affective text mining. In Proceedings of the 2009 Ninth IEEE International Conference on Data Mining, IEEE Computer Society, pages , [15] S. Bao, S. Xu, et al. Mining social emotions from affective text. IEEE Transactions on Knowledge and Data Engineering, [16] Ruifeng Xu, Lu Ye, Jun Xu. Reader s Emotion Prediction Based on Weighted Latent Dirichlet Allocation and Multi-Label k-nearest Neighbor Model. Journal of Computational Information Systems 9: 1 (2013) 1 8 [17] T. L. Griffiths, and M. Steyvers. Finding scientific topics. In Proceedings of the National Academy of Sciences, 101(suppl. 1): , Acknowledgement This research is supported by MOE Specialized Research Fund for the Doctoral Program of Higher