Detecting Deception in On and Off-line Communications

Size: px

Start display at page:

Download "Detecting Deception in On and Off-line Communications"

Matilda Griffin
6 years ago
Views:

1 Detecting Deception in On and Off-line Communications Eliana Feasley and Wesley Tansey ABSTRACT As online services become increasingly prevalent, detection of deception in electronic communication becomes vital to maintaining the many communities built around mutual trust between strangers (e.g. OkCupid, Quora, Yelp). Automating the detection process via machine learning has shown promise in recent years. This paper presents the results of using support vector machines (SVMs) and probabilistic context-free grammars (PCFGs) to classify text as truthful or deceptive. Experiments are conducted on two wellknown datasets of truthful and detective texts. A novel corpus is also developed via an online interrogation game, yielding additional insights regarding detecting deception in online conversations. The results of these experiments suggest that SVMs are superior to PCFGs at classifying deceptive text, and that the classification of deceptive text is highly dependant upon domain. 1. INTRODUCTION The ability to automatically discern whether an individual is writing deceptively is of critical importance to numerous online companies. Question-answer sites rely on accurate responses to openended questions from users, rating sites depend on honest reviews from patrons of local businesses, and dating sites require sincerity in dating profiles and follow-up messages. For such companies, if the veracity of their content cannot be trusted, the value to the end user is effectively zero. Given the vast size of such data and the speed with which it is being generated, detecting deceptive writing necessitates an automated approach. Previous approaches have focused on either simplified algorithm [2, 1] or purely offline communication [3]. While such techniques are useful in some scenarios such as detecting fake reviews, they do not lend insight into interactive communication. As most websites include the ability to ask follow-up questions and comments, a system trained on a corpus of interrogation data may have more practical significance than one trained on a passive corpus. Most previous work to date has also employed only n-gram and part-ofspeech approaches to modeling deception, potentially missing crucial information captured in the structure of deceptive sentences. To address these shortcomings, we conducted a set of experi- Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Copyright 20XX ACM X-XXXXX-XX-X/XX/XX...$ ments on three corpora of deceptive text. We first replicate a subset of the experiments performed in [3] by training an SVM n-gram classifier on their opinion spam dataset. We then apply the same techniques on the dataset from [2] to achieve state-of-the-art performance detecting deceptive text on that corpus. We also gather and analyze a novel dataset of 250 samples of an online interrogation game we developed, representing the first known statisticallymeaningful collection of human evaluations in deception detection. Finally, we present a PCFG-based classification approach and test its effectiveness on all three datasets. We find that the PCFG model performs well in some scenarios but is outperformed by the SVM model in all cases. This paper makes the following novel contributions: 1. An analysis of the performance of an n-gram SVM classifier on the corpus found in [2]. 2. The Deception interrogation game and its associated dataset of 250 matches. 3. An analysis of the performance of human detectives in the Deception dataset. 4. A PCFG-based approach to deception detection and a comparison of its performance to SVMs. The remainder of this paper is organized as follows. Section 2 presents our problem domain and the algorithms we employ. Section 3 presents our experimental setup and describes the details of the datasets and the Deception game. Section 4 presents the results of our experiments and a brief discussion of insights gained. Section 5 describes related work in the area. Section 6 presents some interesting future directions for our work. Section 7 concludes with a summary of our findings. 2. ALGORITHMS The automated detection of deception requires a system capable of assessing an arbitrary piece of prose as truthful or deceitful. Machine learning classifiers such as SVMs are well suited for such tasks as each piece of text can be considered as being labeled with one of two classes, presenting a straight-forward classification problem. It may also be possible that a linguistic model such as a PCFG can perform well on this task by leveraging more finegrained knowledge about language. We next describe these two approaches and how they are used in our experiments. 2.1 SVMs To maintain consistency with previous work [3], we used the SVM light package with a linear kernel to train our SVM.

2 Deception is played with three players, each with a different role. Detective Create a game. Your goal is to pick a topic question, interrogate the two suspects, and determine which one is lying. Both suspects will try to convince you that they are telling the truth about the answer. You re allowed to ask them up to three questions in a game. Truth Teller Join a game and simply tell the truth. Answer each question honestly and convince the detective to believe you. Liar Join a game and convince the detective that you are the truth teller. Pretend that you actually said the main topic question response. Table 1: The Rules of Deception. 2.2 PCFGs For each corpus, we create a PCFG to model deceptive text (P CF G D ) and another to model truthful text (P CF G T ). This is based on the approach in [4]. Part-of-speech tagging is done via the Python Natural Language Toolkit 1 and the Stanford Parser 2 is used to train the PCFGs. In each dataset, we simply take the maximum of the two PCFGs as the classification. In the Deception dataset, however, the detective knows that one of the two individuals is lying. Thus, we evaluate several additional approaches that leverage this knowledge. Algorithm 2.2 lists the four comparative strategies we evaluate. 3. EXPERIMENTAL SETUP We next describe the details of our experiments, including the existing datasets we analyzed and the game we used to collect our own novel corpus. 3.1 Existing Corpora Two existing, previously-published datasets were leveraged in our experiments. First, the PsychLing dataset [2] contains a collection of five experiments, totaling x samples, where subjects were asked to perform various deceptive tasks such as writing false opinions and lying about their previous actions. Second, the OpSpam dataset [3] contains 800 samples of online hotel reviews, half of which are labeled deceitful. Both datasets were provided upon request by the authors of their respective papers. 3.2 The Deception Game To gather our third dataset, we created a web-based interrogation game called Deception 3. The rules of the game are presented to players as shown in Table 1. Using workers from Amazon Mechanical Turk 4 and odesk 5, and participants from various viral social media sites, we gathered over 800 games, over 280 of which were completed (every player answered all questions, and every detective asked all questions and Condition Probability P(R1) P(R2) P(R3) P(R2 R1) P(R3 R2) P(R3 R1, R2) P( R2 R1) P( R3 R2) P( R3 R1, R2) P(R3 switched) Table 2: Marginal and conditional probabilities for the detective guesses in the Deception game dataset. R# represents the interrogation round number; switched refers to the case when the detective switched their guess from round two. The results show that detectives get more accurate as the interrogation progresses, unless they start out on the wrong path. made judgements.) Due to the viral spreading of the game, some of these matches were not played in good faith or involved participants going off-track. Others involved participants asking questions with a yes or no answer (for instance, Have you seen The Avengers "), which leads to a high probability of both players having the same answer, and neither lying. To improve the quality of the dataset, all games were filtered out if they involved the players or questioner directing profanity at one another, or began with a yes-or-no question that was not elaborated on in the answer. After filtering, the Deception dataset contains 250 usable examples, including 1000 questions and 750 instances each of deception, truth-telling, and detective judgements Methodology For each dataset, we apply 5-fold cross-validation on all models and average the performance. All classification results are measured for precision, recall, and F-score. 4. EXPERIMENTAL RESULTS This section presents the results of the experiments on the three deception datasets: OpSpam, PsychLing, and Deception. As both OpSpam and PsychLing are previously-published datasets, we begin by presenting summary statistics about the Deception dataset. We then present the results of our SVM and PCFG experiments on each dataset. The section concludes with a brief discussion of the insights gained from the experiments. 4.1 Deception Statistics To establish a baseline for accuracy, we first measure the probabilities that a human detective will correctly guess the liar in a given round, shown in Table 2. After only one question, human judges perform only slightly better than random but are more than 58% accurate by the end of the final round. Intuitively, if a judge has correctly guessed the liar in a previous round, they are substantially more likely to guess correctly the next round. Conversely, guessing incorrectly appears to have a similar effect as guessing wrong previously is a strong indicator of making future incorrect judgements. Interestingly, if a detective changes their opinion before the third round, they are actually less likely to guess correctly than the marginal probability. 6 See playdeception.com/finished/vcvdelir for an example of a completed game.

3 Algorithm 1 The four comparative strategies used to evaluate PCFGs in the Deception dataset. Require: P CF G T, P CF G D function PCFG-SUM(response1, response2) score1 P CF G D (response1) + P CF G T (response2) score2 P CF G D (response2) + P CF G T (response1) function PCFG-DIFF(response1, response2) score1 P CF G D (response1) P CF G D (response2) score2 P CF G T (response1) P CF G T (response2) function PCFG-LIE(response1, response2) score1 P CF G D (response1) score2 P CF G D (response2) function PCFG-LIE(response1, response2) score1 P CF G T (response1) score2 P CF G T (response2) function MAXLENGTH(response1, response2) if Length(response1) Length(response2) then

4 Approach Accuracy PCFG Sum 48.6 PCFG Difference 55.3 PCFG T ruth 50.2 PCFG Lie 61.8 Table 5: Results on datasets using PCFGS. In addition to analyzing the content of the suspect responses, three descriptive features were also observed: deletions, response length, and time to completion. Table 3 shows the three descriptive features measured. Deletions were approximated by analyzing snapshots of the current suspect response at 1-second intervals and detecting when the text was shorter than it previously had been. The results suggest that deletions and time were not significantly different for liars and truth tellers, however response length appears to be a potential indicator. These results support previous findings [] that show liars create shorter responses due to higher cognitive load. 4.2 Classifier Comparison Table 4 presents the results of our analysis comparing SVMs and PCFGs on our three datasets. While the PCFG model outperforms the part-of-speech SVM, it slightly underperforms the n- gram SVMs in all three experiments. In the case of the Psych- Ling dataset, the unigram SVM attains the highest known accuracy to-date. Interestingly, the unigram SVM also had the highest accuracy on the Deception dataset, though in both cases the F-score was lower than the bigram and trigram models, respectively. Table 4.2 presents the results of the PCFG variant models on the Deception data. These models have higher accuracy as they are leveraging the knowledge that one of the suspects is lying and the other is telling the truth. It appears that the best performing model is thus the PCFG Lie algorithm. However, it is worth noting that the majority of this prediction accuracy comes from using the length function as a tie-breaker. Using purely length is sufficient to achieve 61.6% accuracy on the final guess. 4.3 Discussion Deception detection has a long way to go. People are good at lying and are well practiced in conveying the impression of telling the truth. Our methods are comparable to those of a human judge, but building a discriminator based on number of words in the total communication was more effective than either natural language processing or human judges. In certian domains, deceptian detection may be easier or harder - for instance, in the OpSpam domain, the authors of one set of documents are Mechanical Turkers while the author of the other set are people who can afford to stay in fivestar hotels in Chicago. The strong influence of domain is evident when comparing the most important features in the SVM model generated for each corpus (See Figure 6). In the bigram model for the PsychLing corpus, the SVM seems to be discriminating morebet In the Psych- Ling dataset, the SVM is discriminating more based on the prolife/pro-choice stance of the student than on any high-level linguistic traits, with phrases classically associated with being pro-life such as kill, murder, and life appearing as features for deception, while the indicators for truthfulness contain more personal pronouns and counterfactuals (the phrases that if and it would ). In contrast, the OpSpam dataset contains many occurances of personal words like i and my as deceptive features, while the truthful accounts contain references to concrete nouns - the hotel, the bathroom, the concierge. These indicators are nearly opposite. In the Deception dataset, there is no intuitive explanation for the differences between the datasets - and indeed, the model performed no better than chance. The PCFG models were not very effective for the Deception dataset. This is likely because the highly informal chat-based format, involving many fragments and one-word responses, is not very amenable to a grammar being generated. The SVMs may have been more effective because they were able to take the entire document into account, rather than one sentence - or fragment - at a time. 5. RELATED WORK Analyzing text to detect deception has received considerable attention in psycholinguistics, but has only recently become the subject of study in the machine learning field. As a result, a majority of previous works rely on bag-of-words analyses and attempt to draw correlations between categories of words and deceptive text. For instance, Newman et al. [2] conducted a set of experiments in which subjects were asked to either lie or tell the truth about a topic. The tasks include presenting an opinion on abortion, with participants either writing, speaking, or typing both their opinion and the reverse of their opinion (pro-life vs. pro-choice); participants describing friends they like and dislike either positively or negatively; and participants either lying or telling the truth under interrogation about whether or not they had stolen a one dollar bill from a book in the middle of a room. The resulting dataset was analyzed via the Linguistic Inquiry and Word Count (LIWC) tool, looking only at individual words and performing logistic regression. While the system slightly outperformed human judges, we have shown that n-gram SVMs are a superior model to both approaches on the same dataset. More directly related, Hancock et al. [1] investigate distinctions between attributes of deceptive communication in online, verbal, and written communication. Some participants were motivated to lie successfully via a lying pep talk detailing the ways in which lying is essential for a successful life, while others were merely told to lie. Transcripts of 242 text-based chats in which participants either lied or told the truth were analyzed. The results support previous work showing that liars are less likely to use first person singular pronouns and negatives, while being more likely to use third person singular. Similarly to Newman et al., no effort is made to go beyond surface-level statistical analysis via a bag-of-words model. Ott et al. [3] present an approach to detecting opinion spam in online hotel reviews. The authors gather the first large-scale (800 total), gold-standard dataset of truthful and deceptive positive reviews, with half being deceptive and half truthful. Four main experiments are conducted to determine the feasibility of detecting deception. First, three human judges attempt to classify each review; results indicate that humans are poor judges and only achieve 60% accuracy. Three automated approaches are then tried: genre identification, psycholinguistic features, and text categorization. Genre identification relies purely on part-of-speech tags to attempt to detect whether the two classes are stylistically different. Psycholinguistic features derived from LIWC are combined with an SVM model to generate a classifier. Finally, n-gram models are combined with SVM and Naive Bayes (NB) to generate another collection of text classifiers. Permutations of LIWC with n-grams are also explored. The results show that n-gram + LIWC categorization methods are the most effective at the task (nearly 90% accuracy), though only slightly (0.02%) more effective than a purely n-gram SVM model, while even the simple genre identification model statistically significantly outperforms humans. The authors show confirmation of previous studies that deceivers have difficulty with spatial

5 Deletions Words Seconds Suspect Mean Median Stdev p-value Truth Teller Liar Truth Teller Liar Truth Teller Liar Table 3: Summary statistics for the suspects in the Deception game dataset. The results support previous findings that liars write less due to a higher cognitive load. Approach Corpus Accuracy P R F TruthAcc LieAcc Unigram SV M OpSpam Bigram SV M OpSpam Trigram SV M OpSpam POS SV M OpSpam PCFG OpSpam Unigram SV M PsychLing Bigram SV M PsychLing Trigram SV M PsychLing POS SV M PsychLing PCFG PsychLing Unigram SV M Deception Bigram SV M Deception Trigram SV M Deception POS SV M Deception PCFG Deception Table 4: Results using n-gram and part-of-speech features with SVMs on the OpSpam and PsychLing datasets. PsychLing PsychLing OpSpam OpSpam Deception Deception that if on the i dont do with we had i think i believe the hotel i guess that she the bathroom it is their own the concierge yes i but i michigan ave of a i can is a dont really it would room was that it have to distance to to get i think michigan avenue i wasnt 0.82 and then 1.03 the area 0.72 i live 0.89 have the 1.04 east hotel 0.76 a great 0.93 to take 1.07 to do 0.76 to explain 1.04 killing a 1.09 the food 0.82 i like 1.08 should not 1.12 next time 0.82 to me 1.13 shouldnt be 1.16 when i 0.84 a lot 1.15 because it 1.23 chicago hotel 0.90 i would 1.27 is murder 1.26 and i 0.93 i get 1.33 a life 1.38 to anyone 0.94 think it 1.47 a human 1.67 my stay 0.97 the way 1.52 to kill 1.75 my husband 1.63 i can Table 6: Weights on features in the linear-kernel SVM. Low weights predict truth; high weights, deception.

6 and temporal reasoning by analyzing the highest weighted words in their final models and noting that truthful reviews contain words like location, on, and bathroom. This same analysis also seems to contradict previous findings by showing that deceptive reviewers use pronouns like I and my frequently. The authors conclude this shows a need to analyze deception within the context of the given task and that a global model is insufficient. We borrow heavily from Ott et al. in this paper, first replicating their results 7 and then extending the technique to the other two datasets. It is worth noting that while the OpSpam dataset is large, it is also an offline dataset that does not include the interrogative components of the Deception dataset. 6. LIMITATIONS AND FUTURE WORK The introduction of the Deception game yields several promising directions for future research. First, the data is not particularly high-quality as the popularity of the site attracted a wide range of players, resulting in a correspondingly wide range of games. To overcome this, we plan to organize a large-scale controlled experiment with undergraduate students. Second, we have not yet done an in-depth analysis of the questions asked by the detective. Such an analysis may reveal insights into which questions yield the most informative answers from the suspects. Similarly, we have captured snapshots of all the text entered by the suspects at every second, but only performed a very surface-level analysis of the typing patterns. It would also be interesting to determine the success of detectives when they cannot see the suspects typing in real-time as opposed to our current data in which they can. 7. CONCLUSION Much work has been done in both psycholinguistics and computational linguistics in identifying deception, but this work has not been fully synthesized in the past. This paper has presented the results of applying a PCFG-based approach to two previously published corpora as well as a novel third corpus. While PCFGs were able to perform well on some of the datasets, they were inferior to SVMs in all three experiments. 8. REFERENCES [1] J.T. Hancock, L.E. Curry, S. Goorha, and M. Woodworth. On lying and being lied to: A linguistic analysis of deception in computer-mediated communication. Discourse Processes, 45(1):1 23, [2] M.L. Newman, J.W. Pennebaker, D.S. Berry, and J.M. Richards. Lying words: Predicting deception from linguistic styles. Personality and Social Psychology Bulletin, 29(5):665, [3] M. Ott, Y. Choi, C. Cardie, and J.T. Hancock. Finding deceptive opinion spam by any stretch of the imagination. Arxiv preprint arxiv: , [4] Sindhu Raghavan, Adriana Kovashka, and Raymond Mooney. Authorship attribution using probabilistic context-free grammars. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL-2010), pages 38 42, We were unable to achieve the same level of performance using SVM light and a linear kernel. Nevertheless, we achieved sufficiently high results to allow for adequate comparison.

Motivation. Motivation. Motivation. Finding Deceptive Opinion Spam by Any Stretch of the Imagination

Motivation. Motivation. Motivation. Finding Deceptive Opinion Spam by Any Stretch of the Imagination Finding Deceptive Opinion Spam by Any Stretch of the Imagination Myle Ott, 1 Yejin Choi, 1 Claire Cardie, 1 and Jeff Hancock 2! Dept. of Computer Science, 1 Communication 2! Cornell University, Ithaca,