Get a Clue! Some Truths about Online Deception

Transcription

1 Get a Clue! Some Truths about Online Deception Cheryl Lynn Booth 1, Shuyuan Mary Ho 1 1 Florida State University, School of Information Abstract As text-based computer-mediated technologies have become more and more commonplace in communication, our exposure to potential online deceptions like phishing, and even identity theft has also been increased. In order to avoid, or at least mitigate, these risks, we must be able to discern the underlying intent of the text message sender and to recognize whether these messages are deceptive, or not. The difficulty in computer-mediated communication is that we generally have only written words to rely on. However, prior research has demonstrated that applying certain classification methodologies to certain observable linguistic features can provide important clues to detecting deception with reasonable accuracy. This poster describes a study of an interactive online game designed and developed on Florida State University campus that mimics interpersonal deception scenarios. We applied different analytical models to the data collected as a step towards developing an automated process for detecting deception in online communication. Keywords: Interpersonal deception theory; computer-mediated communication; language-action cues; human computer interaction. doi: /16504 Copyright: Copyright is held by the authors. Acknowledgements: The 2nd author acknowledges the National Science Foundation EAGER grant # , 09/01/13 08/31/15, the Florida Center for Cybersecurity (FC2) grant # O, 03/01/15 02/28/16, and the Florida State University Council for Research and Creativity Planning Grant #034138, 12/01/13 12/15/14. The authors wish to thank Professor Xiuwen Liu for his advice on adopting the support vector machines approach, and Muye Liu, Shashanka Timmarajus, and Aravind Hariharan for their computational analytical support and research participation. Contact: clb14h@my.fsu.edu, smho@fsu.edu. 1 Introduction The use of text-based, computer-mediated communication (CMC) is ubiquitous in our society today. It has given us new tools (i.e. , text/instant message, social media post) that have increased and enhanced the geographical scope, speed and convenience of our personal and professional communications. Unfortunately, while greatly facilitating communication, these same tools have also facilitated the variety of deceptive online communications we often learn of in the news. In short, these new tools have opened the door to new troubles including exposing users to new and/or increased risks in terms of online safety, security and privacy. Assuming text-based CMC will remain a fixture in the communication landscape for the foreseeable future, a key question to address is how to identify and avoid (or at least mitigate) these risks. This requires an understanding of how we evaluate and assess the truthfulness and trustworthiness of an individual with whom we are communicating online both in terms of assessing his/her identity, and the information exchanged. This, in turn, requires an understanding of the fundamental nature of deception and deceptive communication. One of the fundamental truths about deception is that, in general, people are bad at detecting it (Ekman & O'Sullivan, 1991). Moreover, although previous research has demonstrated that it is possible to detect deception in CMC with reasonable accuracy through certain observable linguistic features and cues, there has been comparatively little about how to translate these features and cues into an algorithm or code for a program that could automatically flag them for the user to inform his/her assessment. This paper attempts to address the foundational research question: Can we computationally capture a liar? In the following sections, we will first highlight some truths about deception. Then, we will briefly discuss our research design and data is captured, along with an analysis of the findings. Finally, we will conclude by answering our research question, and outline our potential future work.

2 2 Deception In Theory: Our Theoretical Foundation Beyond the fact that people are poor deception detectors, there are additional truths about deceptive communication that must be understood. First, deception is a relatively common occurrence, with at least one quarter of all communications being deceptive to some extent (Buller & Burgoon, 1996). Second, deception has generally been defined as a message knowingly transmitted by a sender to foster a false belief or conclusion by the receiver (Buller & Burgoon, 1996, p. 205). It should therefore be understood as a volitional, intentional act; mistakes or unintentional misstatements of fact do not constitute deception. Further, as suggested by Interpersonal Deception Theory (IDT), deception is akin to a game of chess: it is an iterative and strategic process on the part of all parties, wherein one party s behaviors influence or affect the responsive behaviors of the other throughout the exchange (Buller & Burgoon, 1996). From this perspective, deceptive communication can be understood to involve and implement the persuasive strategies employed by the sender to deliberately distort the message s/he wants to convey and thereby influence the beliefs, attitudes and behaviors... of the receiver (Miller et al., 1983, p. 99). Our ability to detect deception, whether in CMC or face-to-face (F2F) communication, depends on numerous factors, especially the availability of clues and cues to the receiver with which s/he can assess and evaluate the sender. These cues can be physical (i.e. body language, facial expression) (Ekman & Friesen, 1969), verbal (i.e. words written or spoken) (DePaulo & Kashy, 1998; DePaulo et al., 1996; DePaulo et al., 2003), or both. The main problem with detecting deception in CMC is that the physical cues present in F2F communication are absent in CMC, so the receiver must rely almost entirely on just the sender s words (i.e. verbal cues) in his/her assessment. In addition, the availability of other verbal and non-verbal cues/ clues is also reduced in CMC as against F2F communication. However, both IDT and social distance theory (DePaulo et al., 1996), suggest there are nonetheless certain language-action cues that is, the linguistic style, phrases and patterns in an actor s written expression (Ho et al., 2015a; Ho et al., 2016a; Ho et al., 2016b; Ho et al., 2015b) that can still be observed and can reveal deceptive intent in CMC. Essentially, it has been shown that a speaker s usage of words is indicative of behavioral intent (i.e. deceptive or non-deceptive) (Zhou & Zhang, 2008). Among the non-verbal, syntactical cues that have been studied are overall conciseness (vs. wordiness) and the internal consistency of detail in the communication (Zhou & Zhang, 2004; Zhou et al., 2003), and our own study specifically considers latency or time-lag (the length of time between one communicating partner asking a question and the other responding to it). Examples of some of the various verbal language-action cues are noted in Table 1 below. IDT in particular emphasizes both verbal and non-verbal language-action cues, positing that such cues in a deceiver s message reflect his/her strategic attempts to manipulate information and shape behavior (Burgoon et al., 1996). Social distance theory also emphasizes these cues, suggesting that, in order to avoid the social discomfort associated with lying, deceivers will separate or distance themselves from the person to whom they are lying and, accordingly, their language-action cues will reflect this (DePaulo et al., 1996). 3 Deception In Practice: Our Study We designed and developed an interactive online game ( Real or Spiel ), hosted on Google+ Hangout, that simulates a real-time interactive deception scenario through synchronous communication channels (specifically, text-chat) (Figure 1). Each game involves two players, who are placed in assigned pairings by the research team, and are then randomly assigned an outer role as either a speaker or a detector in each scenario. The speaker in each scenario is also randomly assigned an inner role either saint (truthful) or sinner (deceptive). The speaker establishes the ground truth at the beginning of each scenario by truthfully answering the underlying question for that scenario. The detector then asks probing questions to derive an answer to this underlying scenario question. The speaker answers these questions based on his/her assigned inner role, and then at the end of the scenario the detector tries to determine whether or not the speaker was being deceptive or truthful based on his/her responses. 3.1 Data Collection We collected data starting in Fall 2014 through Spring In total, forty participants were recruited and randomly paired to play twenty game sessions. Each session lasted approximately thirty minutes. Data from each game scenario/ chat exchange were collected and stored in the game system s MySQL database. Before processing the data, we corrected all spelling errors, spelled out all abbreviations, acronyms and chat terms in full, and ensured player role/scenario alignment. Once the data were cleaned and validated, we had 2,196 lines of text and a total word count of 7,271 in the final dataset that was processed using Linguistic Inquiry and Word Count (LIWC) analysis (Table 1). 2

3 Figure 1. Online Game Interface Table 1. LIWC Categories Used In Analysis 3.2 Machine Learning Deception The extracted data was further analyzed using several different approaches to assess the types of cues appeared to have the greatest predictive value for detecting deception. Ho et al. (2016b) reported the order of language-action cues in which they were found to be significant in predicting truthful versus deceptive statements. We further developed a number of models to examine the predictive power of language-action cues/ indicators in different combinations. Although the output of each modeling approach demonstrated that the cues we investigated had at least some predictive value, we found support-vector machine (SVM) 3

4 analysis generated some particularly intriguing results. We created SVM models using RBF (radial basis function) kernels, which are able to provide particularly high accuracy because the support vector machine parameters can be fine-tuned in RBF kernel modeling. Figure 2 illustrates a two-dimensional view of the SVM RBF kernel; this model yielded a decision boundary accuracy of 98% which is to say that this model, using two predictors (I/ time-lag and insight/time-lag), predicts deception accurately 98% of the time. Figure 3 illustrates a more complex, three-dimensional view of the SVM RBF kernel, combining all three variables, which also yielded a decision boundary accuracy of 98%. It should be noted that we report on combinations of only three of the eighteen indicators we studied (see Table 1), and that the specific cues used in the analyses depicted below were selected more-or-less randomly, for purposes of illustrating our approach. The same approach could be used with any combination of two or three cues from the list, although, of course, different combinations of variables will yield different levels of accuracy. It should also be noted that our data were essentially the initial training data, and have thus not yet been cross-validated by further data collection. Nonetheless, this provides a good picture of the results generated, and the type of modeling approach we applied. Figure 2. 2D SVM RBF kernel with 98% accuracy Figure 3. 3D SVM RBF kernel with 98% accuracy 4 Conclusion and Future Work From the above discussion, it should be clear that we can respond to our research question in the affirmative: Yes it is possible to computationally capture a liar. Our ultimate objective is to be able to use these results to derive an algorithm or code that can be used to develop an automated process or system 4

5 for deception detection. To this end, our future work will focus on fine-tuning the game design to capture some additional data points including not only ensuring that the detector s final guess as to the speaker s role is captured, but also gathering data from him/her as to what specific thing(s) in the chat/ exchange led him/her to make that particular guess. 5 References Buller, D. B., & Burgoon, J. K. (1996). Interpersonal deception theory. Communication Theory, 6, Burgoon, J. K., Buller, D. B., Ebesu, A. S., White, C. H., & Rockwell, P. A. (1996) Testing interpersonal deception theory: Effects of suspicion on communication behaviors and perceptions. Communication Theory, 6, DePaulo, B. M., & Kashy, D. A. (1998). Everyday lies in close and causal relationships. Journal of Personality and Social Pyschology, 74, DePaulo, B. M., Kashy, D. A., Kirkendol, S. E., Wyer, M. M., & Epstein, J. A. (1996). Lying in everyday life. Journal of Personality and Social Pyschology, 70, DePaulo, B. M., Lindsay, J. J., Malone, B. E., Muhlenbruck, L., Charlton, K., & Cooper, H. (2003). Cues to deception. Pyschological Bulletin, 129, Ekman, P., & Friesen, W. B. (1969). Nonverbal leakage and clues to deception. Psychiatry, 32, Ekman, P., & O'Sullivan, M. (1991). Who can catch a liar? American Phychologist, 46, Ho, S. M., Fu, H., Timmarajus, S. S., Booth, C., Baeg, J. H., & Liu, M. (2015a). Insider threat: Languageaction cues in group dynamics. SIGMIS-CPR'15, pp ACM, Newport Beach, CA. Ho, S. M., Hancock, J. T., Booth, C., Burmester, M., Liu, X., & Timmarajus, S. S. (2016a). Demystifying insider threat: Language-action cues in group dynamics. Hawaii International Conference on System Sciences (HICSS-49), pp IEEE, January 5-6, Kauai, Hawaii. Ho, S. M., Hancock, J. T., Booth, C., Liu, X., Liu, M., Timmarajus, S. S., & Burmester, M. (2016b). Real or Spiel? A decision tree approach for automated detection of deceptive language-action cues. Hawaii International Conference on System Sciences (HICSS-49), pp IEEE, January 5-8, Kauai, Hawaii. Ho, S. M., Hancock, J. T., Booth, C., Liu, X., Timmarajus, S. S., & Burmester, M. (2015b). Liar, Liar, IM on Fire: Deceptive language-action cues in spontaneous online communication. IEEE International Conference on Intelligence and Security Informatics, pp IEEE, Baltimore, MD. Miller, G. R., Deturck, M. A., & Kalbfleisch, P. J. (1983). Self-monitoring, rehearsal, and deceptive communication. Human Communication Research, 10, Zhou, L., Twitchell, D. P., Qin, T., Burgoon, J. K., & Nunamaker Jr., J. F. (2003). An exploratory study into deception detection in text-based computer-mediated communication. HICSS'03, pp. IEEE, Hawaii. Zhou, L., & Zhang, D. (2004). Can online behavior unveil a deceiver? HICSS, pp. IEEE Press, Jan. 5-8, Hilton Waikoloa Village Big Island, Hawaii. Zhou, L., & Zhang, D. (2008). Following linguistic footprints: Automatic deception detection in online communication. Communications of the ACM, 51,