Sensing Behavioral Change over Time: Using Within-Person Variability Features from Mobile Sensing to Predict Personality Traits

Transcription

1 141 Sensing Behavioral Change over Time: Using Within-Person Variability Features from Mobile Sensing to Predict Personality Traits WEICHEN WANG, Dartmouth College, USA GABRIELLA M. HARARI, Stanford University, USA RUI WANG, Dartmouth College, USA SANDRINE R. MÜLLER, University of Cambridge, United Kingdom SHAYAN MIRJAFARI, KIZITO MASABA, and ANDREW T. CAMPBELL, Dartmouth College, USA Personality traits describe individual differences in patterns of thinking, feeling, and behaving ( between-person variability). But individuals also show changes in their own patterns over time ( within-person variability). Existing approaches to measuring within-person variability typically rely on self-report methods that do not account for fine-grained behavior change patterns (e.g., hour-by-hour). In this paper, we use passive sensing data from mobile phones to examine the extent to which within-person variability in behavioral patterns can predict self-reported personality traits. Data were collected from 646 college students who participated in a self-tracking assignment for 14 days. To measure variability in behavior, we focused on 5 sensed behaviors (ambient audio amplitude, exposure to human voice, physical activity, phone usage, and location data) and computed 4 within-person variability features (simple standard deviation, circadian rhythm, regularity index, and flexible regularity index). We identified a number of significant correlations between the within-person variability features and the self-reported personality traits. Finally, we designed a model to predict the personality traits from the within-person variability features. Our results show that we can predict personality traits with good accuracy. The resulting predictions correlate with self-reported personality traits in the range of r = 0.32, MAE = 0.45 (for Openness in ios users) to r = 0.69, MAE = 0.55 (for Extraversion in Android users). Our results suggest that within-person variability features from smartphone data has potential for passive personality assessment. CCS Concepts: Human-centered computing Ubiquitous and mobile computing; Applied computing Psychology; Additional Key Words and Phrases: Mobile Sensing, Personality Traits, Within-Person Variability ACM Reference Format: Weichen Wang, Gabriella M. Harari, Rui Wang, Sandrine R. Müller, Shayan Mirjafari, Kizito Masaba, and Andrew T. Campbell Sensing Behavioral Change over Time: Using Within-Person Variability Features from Mobile Sensing to Predict Personality Traits. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 3, Article 141 (September 2018), 21 pages. The research reported in this article is supported the National Science Foundation (NSF) Award BCS Authors addresses: Weichen Wang, Dartmouth College, Computer Science, Hanover, NH, 03755, USA, wang@cs.dartmouth.edu; Gabriella M. Harari, Stanford University, Department of Communication, Stanford, CA, 94305, USA, gharari@stanford.edu; Rui Wang, Dartmouth College, Computer Science, Hanover, NH, 03755, USA, rui.wang.gr@dartmouth.edu; Sandrine R. Müller, University of Cambridge, Department of Psychology, Cambridge, United Kingdom, srm77@cam.ac.uk; Shayan Mirjafari, shayan@cs.dartmouth.edu; Kizito Masaba, kizito.masaba.gr@ dartmouth.edu; Andrew T. Campbell, Dartmouth College, Computer Science, Hanover, NH, 03755, USA, campbell@cs.dartmouth.edu. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org Association for Computing Machinery /2018/9-ART141 $

2 141:2 W. Wang et al. 1 INTRODUCTION Personality psychology focuses on examining individual differences in people s thoughts, feelings, and behaviors. Compared to the amount of research on people s thoughts and feelings, considerably less research has examined how people behave in the context of everyday life (e.g., daily levels of physical activity, sociability, places visited). Traditionally, research examining individual differences has focused on between-person variability in mean levels of such behaviors. For example, people who are more extroverted can be characterized by higher mean levels of talkativeness, compared with people who are less extroverted. However, people also vary in the extent to which their own behavioral patterns change over time, which is referred to as within-person variability. For example, a person who is described as being more extroverted, may show great variability (e.g., socializing little during the week, but a great amount during the weekend) or stability (e.g., socializing a similar amount every day of the week) in their behavior patterns when considered over time. Past research has shown within-person variability to be linked to various psychological characteristics, such as a person s affective states [35], mental well-being [47] and personality trait ratings [19]. However, past research often relied on a person s capacity to accurately recall their daily experiences retrospectively [52, 53], a task that is challenging and time-consuming. Individuals may also intentionally under-report or over-estimate some of their behaviors [64]. Fortunately, smartphone sensing methods [39] are set to overcome these barriers by unobtrusively measuring behavioral patterns continuously over time and thereby allowing us to understand the fine-grained within-person variability in behavioral patterns. In this paper, we present a new approach to capture within-person variability in behaviors using mobile sensing with the goal of assessing and predicting self-reported personality traits. We use the Big Five model [34], which describes 5 major personality trait dimensions: Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism. Openness is a personality trait that describes the extent to which a person is imaginative and insightful. Conscientousness is a trait that describes the extent to which a person is thoughtful, shows impulse control, and engages in goal-directed behaviors. Extraversion is a trait that describes the extent to which a person is excitable, social, talkative, and exhibits emotional expressiveness. Agreeableness is a trait that describes the extent to which a person is trusting, altruistic, kind, and engages in prosocial behaviors. Neuroticism is a trait that describes the extent to which a person is moody and emotionally unstable. To examine how within-person variability in behavioral patterns are related to trait ratings, we consider the following everyday behaviors inferred using mobile sensing [27]: social interactions (i.e., how much a person socializes), physical activity and mobility (i.e., how physically active they are, and how many different places they spend time in), daily activities (i.e., how often they use their phone), and situational information (i.e., noisiness of their environment). To quantify within-person variability in lifestyle behaviors, we compute the following within-person variability metrics for each of the four everyday behaviors: standard deviation, circadian rhythm, regularity index, and flexible regularity index. We examine the connections between within-person variability metrics and personality traits; specifically, we pose the following broad research question: To what extent do day-to-day behavioral patterns of stability and change reveal a person s personality traits? To answer this research question, we collected self-reported personality trait ratings as our measure of ground truth, along with the following sensing data as our measures of lifestyle behaviors: ambient audio amplitude levels (which indicates how quiet or noisy the environment is), exposure to human voice (relating to how social the user is), physical activity, phone usage and location data from 646 students using their Android and iphones at the University of Texas at Austin (UTA) over a course of up to 2 weeks. The contributions of the paper are as follows: We demonstrate for the first time how within-person variability patterns collected passively by a comprehensive cross-platform (i.e., Android and ios) mobile sensing app can be used to predict personality traits. Specifically, we measure and assess everyday behaviors including social interactions, physical movement,

3 Sensing Behavioral Change over Time: Using Within-Person Variability Features from Mobile :3 daily activity and situational information. Furthermore, we propose using the following measures of withinperson variability: standard deviation, circadian rhythm, regularity index, and flexible regularity index. These measures capture behavioral variability from different perspectives (these measures are explained in more detail in Section 3). We identify a number of important associations between the within-person variability features and selfreported personality traits. Furthermore, we predict personality traits solely based on within-person behavior change features. Our results show that our proposed personality prediction model based on withinperson variability features provides good estimation of personality traits, particularly for extraversion and agreeableness. For example, for Android users the leave-one-out model for predicting the extraversion trait achieves 0.55 of MAE, which is 0.24 (30%) lower than the average baseline and 0.5 (48%) lower than the random baseline; for ios users the leave-one-out model for predicting the extraversion trait achieves 0.61 of MAE, which is 0.11 (15%) lower than the average baseline and 0.39 (39%) lower than the random baseline. To the best of our knowledge, we are the first to explore how within-person variability patterns can be used to predict personality traits using features derived from mobile sensing. Our results pave the way for future research on this psychological topic. The structure of the paper is as follows. First, we present related work on personality and mobile sensing research in Section 2, followed by a detailed description of the within-person variability measurements in Section 3. We describe the sensing system, study design, and the dataset in Section 4. Following that, we discuss the correlations between the within-person variability features and personality traits in Section 5.1. In Section 5.2, we present the results of using the within-person variability features to regress on and predict personality scores. We discuss limitations of our methods in Section 6. Finally, we present concluding remarks and the implications in Section 7. 2 RELATED WORK In recent years, mobile sensing has demonstrated its potential as a tool for tracking and modeling human behavior [28, 29, 51, 62]. Equipped with unobtrusive sensors, smartphones can collect continuous sensing data that reveal individuals behavioral patterns and psychological states over long periods of time. For example, several studies have used smartphone sensing to continuously assess people s mental health [5, 39, 58]. The StudentLife study [68] investigated the relationship between many types of smartphone data (e.g., conversation, sleep, activity, and co-location) and mental health outcomes (e.g., depression, stress, loneliness, and flourishing) in Dartmouth students during an academic term. Using the same dataset, Harari et al. [26] analyzed the changes of students activity and sociability behaviors over a term via the accelerometer and microphone sensors. Ben-Zeev et al. [6] used mobile phones to collect passive sensing data from smartphones and find schizophrenia relapse signals in location, activity, and exposure to conversation prior to patients experiencing relapses. Saeb et al. [62] reported depressive symptom severity correlates with mobility patterns and phone usage derived from smartphone data. They replicated their findings using the StudentLife [68] dataset [61]. Canzian and Musolesi [13] proposed a location routine index computed from smartphone location data, which was predictive depression severity. Abdullah et al. [1] reported using location features computed from smartphone data, such as distance traveled, conversation frequency, and non-stationary duration, to infer the social rhythm metric (SRM) [53] score, a widely used lifestyle regularity metric. Several other studies have focused on inferring stability or variability in lifestyle behavioral patterns inferred from smartphone sensing data. For example, Abdullah et al. [2] computed daily rhythms related to sleep to measure well-being. Saeb et al [62] explored using circadian rhythm inferred from smartphone location data to assess depressive symptom severity. Ghandeharioun et al. [21] computed a sleep regularity index (SRI) using accelerometer data and show that SRI differs significantly between days with good and poor mental health.

4 141:4 W. Wang et al. Mehtrotra and Musolesi [51] proposed a movement digital biomarker for monitoring emotional state, which measures the similarity between the sequences of visited places in a day. There has also been an increasing amount of research focused on predicting personality traits from digital media data. Researchers have sought to predict personality using social network structures and interactions, showing that the behavioral data collected from social media platforms such as Facebook [44, 54, 66, 72] or Twitter [22, 23, 57] can be informative in the prediction of people s personalities. Youyou et al. [72] showed that using Facebook Likes, a computer can predict participants personality more accurately than their Facebook friends. Moreover, the computer-made personality judgments had higher external validity when predicting life outcomes ( e.g., substance use, political attitudes, and physical health). Park et al. [54] built a predictive model of personality based on 66,732 Facebook users written language. The predicted personality scores correlate with the ground truth. Golbeck et al. [22] shows that Twitter users language use, sentiment, and Twitter use can be used to predict personality traits. However, these methods usually require access to extensive information about people s online social networks. Researchers have also utilized the information from mobile phones in personality assessment studies. Early studies tried to find relationships between personality traits and phone communications (i.e., phone calls and text messages) [11, 14, 16]. For instance, Montjoye et al. [15] used standard mobile phone logs (i.e., calls and texts) to predict users personality. Staiano et al. [63] collected call logs and Bluetooth proximity data from 53 subjects over 8 weeks to build a call network and a Bluetooth proximity network and used network characteristics to predict users personality traits. A recent study [67] explored using the StudentLife dataset, which comprises Wi-Fi, GPS, Bluetooth, accelerometer, and Piazza usage data to assess personality, finding significant correlations between behavior features Wi-Fi location based behavior features and personality traits. However, much of the aforementioned work used behavioral features focused on capturing between-person variability in behavioral patterns. In this paper, we focus on capturing and assessing within-person variability features from objective sensing behaviors, and show how these features can predict personality traits of smartphone users. 3 WITHIN-PERSON VARIABILITY MEASUREMENTS In what follows, we describe the passive smartphone sensing measures of behavior that we used to quantify behavioral variability. We first describe the behavioral data collected (social interactions, movement and mobility, daily activity, and situational information), then we introduce the metrics that we used to quantify within-person variability (Standard Deviation, Circadian Rhythm, Regularity Index, Flexible Regularity Index). 3.1 Behaviors Inferred from Passive Smartphone Sensing In what follows, we present the four aspects of daily behaviors that are captured through passive sensing using smartphones. We extend the StudentLife Android app [1, 40, 68, 70], which was originally used to capture students behaviors during a term and port it to the Apple ios platform. The app measures daily social interactions, movement and mobility, daily activities and situational information by continuously collecting audio amplitude, ambient voice, participants physical activities, lock/unlock events and location coordinates. Social interaction. We consider inferred ambient voice labels as a proxy for social interactions (i.e., being around conversation). We implement a conversation classifier on the phone to infer whether or not a 32ms audio frame is human voice. The voice classifier is implemented using a duty-cycled audio sensor that continuously runs on the smartphone. The voice classifier is the most energy consuming module in the app. To save energy, we set the duty cycle to be 1 minute on and 3 minutes off. It has been show in [68] that using this duty cycle we can achieve a balance between accuracy and resource usage. Further more, to preserve participant privacy, only the labels from the classifier are kept in the dataset and no raw human voices or speech content is recorded. Our classifier uses privacy preserving features [58, 68] and first determines if the frame contains speech and if so a higher level

5 Sensing Behavioral Change over Time: Using Within-Person Variability Features from Mobile :5 conversation classifier determines if there are sufficient speech frames to indicate the start and later the end of a conversation. The frequency and duration of conversations are stored on the phone and uploaded for analysis. Note that the conversation classifier does not use speaker identification for privacy reasons and therefore only indicates if the user is in the presence of conversation rather than being an active speaker. Therefore, we consider our conversation inferences a proxy for social engagement. The speech/conversation classifier has been validated in several previous studies [6, 58, 68, 70]. Movement and mobility. The phone provides location data that allows us to understand users movement and mobility patterns. We find a user s significant places (i.e., those places where users spent significant time) during the day and their associated dwell times (when the user arrives and leaves a location) by clustering the sampled coordinates during a day using density-based spatial clustering of applications with noise (DBSCAN) [18]. The DBSCAN algorithm groups the points that are close to each other and computes the center of the cluster. The center of the cluster is considered a significant location. Daily activity. We consider two kinds of daily activity: physical activity and phone activity (i.e., phone usage). Personality traits are hypothesized to exert influence on physical activity through a health-behavior model [48, 60, 71]. Our app obtains activity inferences (i.e., stationary, walking, running, cycling, in vehicle) from the Android activity recognition API [24] and ios Core Motion [32]. We compute the sedentary duration within every hour of the day using the phone s physical activity inferences. Another aspect of daily activity we consider as providing signal is phone usage. An increasing number of researchers have shown that smartphone usage reflects psychological well-being [10, 11, 36, 41, 45]. We compute the number of phone lock/unlock events and phone unlock duration to estimate the phone usage. Situational information. We use ambient sound from the phone as a proxy for contextual information about the environment of users. Previous work [50] shows that participants personality traits are related to the quotidian manifestations derived from the sampled snippets of ambient sounds of users immediate environment. We periodically collect sound levels to measure the ambient sound environment. For privacy reasons we do not store any raw audio data. Rather, we compute the average sound amplitude over a period of one second so that the audio can not be reconstructed. 3.2 Within-Person Variability Metrics Standard deviation is one of the simplest and most common approaches to measuring within-person variability. However, there may be other behavioural change metrics that better capture the patterns of change and stability in peoples everyday lives. These approaches could be more meaningful in better understanding within-person variability and its relationship to personality. To study this, we compute a number of variability measures including: the simple standard deviation, the circadian rhythm [2, 13, 61, 62], the regularity index and the flexible regularity index. We use these measures to assess the within-person variability of each behavior inferred using passive sensing. We first partition a day s data into 24 one-hour periods and process the sensor data in an hourly fashion. For example, consider ambient audio amplitude, we compute the mean audio amplitude for each one-hour period; for voice, we compute the amount of conversation duration measured in one-hour periods; for physical activity, we compute the sedentary duration across each hour period; for the phone usage, we compute the number of phone unlock events registered during each hour period. We do not preprocess location data. Next, we compute the four within-person variability metrics discussed above. Table 1 summarizes the within-person variability features used in this study. In what follows, we describe each metric in detail. Standard deviations [STD] measure the variance in daily behaviors. We compute the STD over three epochs during a 24-hour period: day time (9am 6pm), evening (6pm-12am), and night (12am-9am) across all days of the week. Because people are likely to have different behavioral patterns during weekdays in comparison to

6 141:6 W. Wang et al. weekends, we compute the STD for the three epochs (i.e. day, evening, night) for weekdays data only. We do not, however, compute the STD for weekends because of limited amounts of weekend data. Circadian rhythm [CR][2, 13, 61, 62] measures the strength with which a user follows a 24-hour rhythm in behaviors [61]. Humans have a biological clock that optimizes the physiology and behavior of organisms, hormonal secretion and mood [37]. However, people differ in their circadian rhythms. For example, previous studies have shown individual differences in the morning and evening related to personality [3, 59]. We compute the CR across the study period for hourly behavioral data using spectrum analysis. Specifically, we first use the least-squares spectral analysis [56] to transform the behavioral sensing data (i.e., physical activity, phone activity ambient sound, ambient voice) from the time domain to the frequency domain. We then compute the ratio of energy that fall into the 24 ± 0.5 h period (which corresponds to 2π/(24 ± 0.5) = (0.2565, )) over the total spectrum energy in the 24 ± 12 h period (which corresponds to 2π/(24 ± 12) = (0.1745, )) as the CR: CR = 2π /(23.5) 2π /(24.5) psd(x)dx 2π /(12) 2π /(36) psd(x)dx (1) where psd(x) denotes the power spectral density at frequency bin x. The raw hourly signal is first aligned to zero mean before performing spectral analysis. Fig. 1 shows an example of the computed CR for ambient sound. The plots show the raw hourly ambient sound amplitude and the corresponding power spectrum for two participants selected from the study. The raw data show that participant (a) has a more pronounced 24-hour cycle for ambient sound than (b). As such, the CR value for participant (a) is whereas the CR value for participant (b) is For the mobility data, we use the steps described in [61] to compute the CR of mobility: we first generate the CR for the latitude and longitude values then combine them through CR = log(cr lat + CR lonд ). (a) Participant with high circadian rhythm value (b) Participant with low circadian rhythm value Fig. 1. Circadian Rhythm analysis using audio amplitude sensing data from two participants. Regularity index [RI] assesses the difference between the same hours across two different days. We first rescale the behavioral data for each participant to [ 0.5, 0.5], where -0.5 corresponds to the minimum value in the origin data and 0.5 corresponds to the maximum value. The product of two rescaled values is positive if the original

7 Sensing Behavioral Change over Time: Using Within-Person Variability Features from Mobile :7 values are close and negative if they are not similar. Subsequently, we define the regularity between day a and b as: (a,b) S, RI a,b = T f (xt a )f (xt b )/T (2) where S is the set of two-day pairs, a,b are the two days in a two-day pair, T = 24hours, and x a t is the rescaled value in hour t of day a. We compute the mobility RI differently because unlike other behavioral data mobility data is nominal (i.e., a mobility data point represent a location). We therefore compare whether or not a user is at the same place in two days. The mobility RI is formally defined as: t=1 t 1 T loc RI_loc a,b = д(ct a,ct b )/T loc (3) where a,b represent two different days, t represent the time window in a day (a window lasts 10 minutes), T loc is the available number of overlapped time windows in both days, ct a is the significant location id time t in the day a, and д(m, n) indicate whether or not m = n. A higher RI score indicates that the user visited similar places around a similar time of day in two given days. We compute the average and range of the RI values from every possible pair within the following sets: (1) weekdays vs weekends, (2) within weekdays, (3) within all days. Flexible regularity index [FRI] is an edit distance based (or Levenshtein distance [42]) measure to assess the difference between two days differently. An edit distance quantifies how dissimilar two strings are to one another by counting the minimum number of operations needed to transform one string to the other. Such operations include removing, inserting, or substituting one character in the string. Different operations may have different weights. The edit distance of the behavioral data in two days reveals how similar the behaviors in two days are. A lower edit distance (i.e., lower FRI) between different two days indicates more similar behaviors. We compute the FRI as follows. First, we transform the behavioral sensing data into strings. Specifically, for behavioral data other than mobility, we label a one-hour chuck as a if the mean sensor reading in this hour is within the bottom 25 percentile of all data from this user; c if the mean is within the top 25 percentile; and b if the mean is between the bottom 25 percentile and top 25 percentile. For mobility, we use the significant location id to generate the mobility string in day. We define the weights for each operation as shown in Table 2. 4 DATA COLLECTION AND PROCESSING We collected a dataset from 646 students at the University of Texas at Austin (UTA). The participants were enrolled in an online introductory psychology class across two semesters. As part of a course assignment, participants could self-track their lifestyle behaviors in exchange for personalized feedback using a tracking method of their choice: a mobile sensing app, -based surveys, or a handwritten journal. Here we focus on the data collected from students who elected to use the mobile sensing app. Participants installed the data collection app on their phones and were asked to participate for at least seven days. Participants were able to participate for up to fourteen days. Among the 646 participants, 117 used Android phones and 529 used iphones. All participants complete the Big Five personality trait questionnaire [34] at the start of the study period, which serves as the ground truth for personality traits in this study. Fig. 2 shows the overall system and study design. The complete system included the sensing app and cloud and was based on an earlier version of the StudentLife system [68]. We continuously collected behavioral passive sensing data from participants Android phones and iphones. The data was then automatically uploaded to our

8 141:8 W. Wang et al. Table 1. Description of the features computed Feature category Standard Deviation (STD) Circadian Rhythm (CR) Features computed Description of feature Sensing data std_night_all STD on the data at night across all days std_day_all STD on the data during day time across all days Ambient sound Ambient voice std_evening_all STD on the data in the evening across all days Physical activity Phone activity std_night_weekday STD on the data at night across weekdays std_day_weekday STD on the data during day time across weekdays std_evening_weekday STD on the data at night across weekdays circadian_all Circadian rhythm from data across all days Ambient sound circadian_weekday Circadian rhythm from data across weekends Voice labels Physical activity Phone activity Location Regularity Index (RI) Flexible Regularity Index (FRI) ri_weekday_vs_weekend_avg ri_weekday_avg ri_all_avg ri_weekday_vs_weekend_range ri_weekday_range ri_all_range fri_weekday_vs_weekend_avg fri_weekday_avg fri_all_avg fri_weekday_vs_weekend_range fri_weekday_range fri_all_range Measures the average hour-by-hour similarity between weekdays and weekends Measures the average hour-by-hour similarity across weekdays Measures the average hour-by-hour similarity across all days Measures the range (i.e., the difference of the most similar pair and the most distinguished pair) of hour-by-hour similarity between weekdays and weekends Measures the range of hour-by-hour similarity between weekdays Measures the range of hour-by-hour similarity across all days All the definitions are parallel to the RI. The FRI measures the edit distance between two days, which allows the hours to be slightly shifted when doing the hour-by-hour comparison with reasonable penalty. Ambient sound Ambient voice Physical activity Phone activity Location Ambient sound Ambient voice Physical activity Phone activity Location secure server. The server processed the data, and generated personalized web-pages of feedback reports, which were sent to students via during the study. Those reports included personalized visualizations of the tracked behaviors as well as class average charts for comparison. In what follows, we discuss how we processed the data in detail.

9 Sensing Behavioral Change over Time: Using Within-Person Variability Features from Mobile :9 Table 2. Cost definition in FRI calculation Operation Cost sensing data other than location location sequence insertion 1 1 removal 1 1 substitution if letters are adjacent (e.g., a and b ) others Personalized feedback Webpage generated automatically based on the data of the previous day 1 Sensing physical activity ambient sound ambient voice phone usage location 2 Upload when phone is being powered and connected to Internet 4 Generate regularity features Big Five traits openness conscientiousness extraversion agreeableness neuroticism standard deviation circadian rhythm regularity index flexible regularity index weekdays all days weekdays vs weekends 5 Analysis Is personality associated with within-person variability? Can we predict personality purely from within-person variability? Fig. 2. System and study design. 4.1 Data Inclusion Criteria Data quality is crucial for analysis. Missing data across a day will adversely affect the accuracy of the withinperson variability features. Therefore, we exclude days with less than 19 hours of sensing data. The 19-hour threshold is based on previous studies [68, 70], which balances the need for data quality and quantity. We also exclude participants who have less than 7 days of usable data (more than 19 hours of sensing data). 159 out of the 646 participants satisfy our data inclusion criteria and included in our analysis. Among them, 70 are Android users and 89 are iphone users. The high data exclusion rate is mainly due to participants running the app for less than 14 days. Other factors include the phone being powered down, turning location off, and stopping the app.

10 141:10 W. Wang et al. 4.2 Big Five Personality Ground truth We use the self-reported Big Five Inventory (BFI) [34] scores as our personality ground truth. The BFI measures the personality traits: openness, conscientiousness, extraversion, agreeableness and neuroticism. Fig. 3 shows the distributions of the Big Five scores of the included 159 participants. The distributions show that the values for all five traits approximate a normal distribution in this sample. Table 3 shows the mean and standard deviation of the five personality trait scores. Fig. 3. Histograms of the Big Five scores. The X axis displays the value of the score, which ranges from 1 (lowest) to 5 (highest). The Y axis shows the number of individuals that fall into the specific score bins. The three rows show the distribution of the scores for all participants (first row), the Android (second row) and the ios (third row) users. Table 3. Distributions of the personality ground truth Big Five trait Mean (std) Android mean (std) ios mean (std) t-test p value Openness 3.54 (0.62) 3.61 (0.61) 3.48 (0.62) 0.21 Conscientiousness 3.45 (0.65) 3.31 (0.63) 3.57 (0.65) 0.01 Extraversion 2.99 (0.90) 2.97 (0.92) 3.00 (0.88) 0.83 Agreeableness 3.74 (0.63) 3.70 (0.63) 3.76 (0.63) 0.63 Neuroticism 3.03 (0.79) 3.08 (0.79) 3.00 (0.80) 0.50 The mean of the trait scores are close to a score of 3 (the middle of the 1-5 range). The agreeableness score is close to 4, which is the highest mean of the trait scores, followed by openness, conscientiousness, neuroticism

11 Sensing Behavioral Change over Time: Using Within-Person Variability Features from Mobile :11 and extraversion. For most of the personality traits, minor differences are of small to negligible effect size. This is inline with a recent study [25] among a large multi-national (N = 1081) and a German-speaking sample (N = 2438). However, in our data Android users seem to be less conscientious than ios users (t-test p = 0.01), which is in contrast to the findings in [25]. The reason could be the population and age: unlike a large multi-national user group with various occupations, our study participants are college students from one university. In addition, our population of users is smaller. 5 ASSESSING PERSONALITY USING WITHIN-PERSON VARIABILITY MEASURES In what follows, we present the results from association analysis and prediction of personality traits using passive sensing data from smartphones. We first extract all the within-person variability features and report linear correlations between the features and the self-reported personality scores. Then, we create a personality prediction model and analyze its prediction performance using only within-person variability features. 5.1 Association Analysis We use the bivariate linear mixed model [49] to assess the relationship between the within-person variability features and Big Five personality traits. In our study, the sensing data come from two clusters: Android and ios. There exist potential differences between two systems (e.g., both Google and Apple have their own physical activity classifiers and audio software development kits). Therefore, the sensed behavioral patterns are not independent. Linear mixed models are an extension of simple linear models to allow both fixed and random effects, and are particularly useful when there is non-independence in the data. Our association results are presented in Table 4. In order to address the multiple comparisons problem, we apply the Benjamini-Hochberg procedure (BH) [8, 9] to control the false discovery rate (FDR) in our exploratory regression analysis. The multiple comparisons problem arise when multiple simultaneous statistical tests are involved in the analysis, which may lead to erroneous discoveries. We present associations with p < 0.05 and mark associations that have FDR < 0.1 and FDR < In what follows, we discuss our results as they relate to the personality traits. Openness. We find that four within-person variability features are positively associated with the openness trait. They are as follows: (1) the deviation in ambient sound on weekday (i.e. Monday-Friday) evenings (6pm-12am); (2) the deviation in physical activity in the evening across all days (i.e. Monday-Sunday); (3) the deviation in physical activity in the evening of weekdays; and finally (4) the range of the regularity index in ambient sound during weekdays. Our results indicate that participants who have various activities during the weekday evenings are more likely to be more open to new experiences. On several weekdays they stay in similar environments at the same hour of each day. However, on some other weekdays they spend time in very different environments (as captured by ambient sound). We find that two within-person variability features are negatively associated with the openness trait. They are (1) the deviation in ambient voice between 9am - 6pm across all days; and (2) the deviation in ambient voice between 9am - 6pm on weekdays. Assuming ambient voice is a proxy for social interaction, our results indicate that people who have changing patterns in interaction with others during the daytime are likely to be less open to new experiences. Conscientiousness. We find that four within-person variability features are positively associated with the conscientiousness trait. These are (1) the average of the flexible regularity index in ambient voice on weekdays; (2) the circadian rhythm in phone usage across all days; (3) the circadian rhythm in phone usage on weekdays; and (4) the flexible regularity index in physical activity. Our results indicate that participants who spend weekdays engaged in more regular social interactions (with hours slightly shifted), and who show more regular phone

12 141:12 W. Wang et al. usage during the day, and have regular patterns of physical activity during weekdays are more likely to have good impulse control and goal-directed behaviors. We find that five within-person variability features are negatively associated with the conscientiousness trait. These are (1) the deviation of exposure to ambient voice on weekday evenings; (2) the regularity index in locations between weekdays and weekends; (3) the regularity index in locations across all days; (4) the flexible regularity index in locations between weekdays and weekends; and finally (5) the flexible regularity index in locations across all days. This leads us to believe that individuals who have unstable social interactions during weekday evenings, and more overlap between weekdays and weekends in terms of their location routines are likely to be less conscientious. Extraversion. We find that five within-person variability features are positively associated with the conscientiousness trait. These are (1) the circadian rhythm in ambient sound across all days; (2) the circadian rhythm in phone usage across all days; (3) the circadian rhythm in phone usage on weekdays; (4) the deviation in physical activity in the evening across all days; and finally (5) the deviation in physical activity in the evening on weekdays. This suggests that people who are more extraverted are, interestingly, more likely to follow a 24-hour rhythm with regards to the environments they spend time in (as captured by ambient sound) as well as their smartphone usage. These participants also tend to have various activities during the evening similar to the patterns observed for openness. In addition, we find that five within-person variability features are negatively associated with the conscientiousness trait. These are (1) the deviation in exposure to ambient voice during the daytime (9am-6pm) across all days; (2) the deviation in exposure to human voices during the evening (6pm-12am) across all days; (3) the deviation in exposure to human voices during the daytime (9am-6pm) on weekdays; (4) the deviation in exposure to human voices during the evening (6pm-12am) on weekdays; and finally (5) the deviation of phone usage in the evening periods across all days. These results tell us that more extroverted individuals are less likely to socialize in a changing pattern. Rather, they are more likely to maintain a stable pattern of interpersonal communication over the week, particularly in the evenings. Similarly, this also applies to their phone usage during evening periods. Agreeableness. We find a large number of within-person variability features are positively associated with the agreeableness trait. These are (1) the range of the flexible regularity index in ambient sound; (2) the range of the flexible regularity index in phone usage on weekdays; (3) the circadian rhythm in physical activity; (4) the flexible regular index in physical activity on weekdays; (5) the flexible regularity index in physical activity across all days; (6) the regularity index in physical activity on weekdays; (7) the range of regularity index; and finally (8) the flexible regularity index in locations on weekdays. These results indicate more agreeable participants are more likely to have to follow a 24-hour rhythm in physical activity. They also tend to have more regular physical activity patterns based on hour-by-hour comparisons between days. They are more likely to change their ambient sound environment and location routines for some weekdays, while they keep them unchanged on other days. In addition, we find a number of within-person variability features that are negatively associated with the agreeableness trait. These are (1) the deviation in phone usage during the night (12am-9am) on weekdays; (2) the deviation in physical activity during the night across all days; (3) the deviation in physical activity during the night period on weekdays; (4) the range of the flexible regularity index in physical activity on weekdays; and finally (5) the range of the regularity index in physical activity on weekdays within weekdays, between weekdays and weekends, and across all days. This seems to tell us that people who are agreeable show less deviation in the time they use their phone during the night. They maintain high regularity in physical activity during the study, as indicated by the smaller range and higher average of their measured regular index.

13 Sensing Behavioral Change over Time: Using Within-Person Variability Features from Mobile :13 Neuroticism. Neuroticism indicates moodiness and emotional instability. In our study, we do not see any within person variability features significantly associated with the neuroticism trait. To sum up, the association analysis supports our hypothesis that within-person behavior change patterns derived directly from smartphone sensing data are related to self-reported personality traits. Four of the five personality traits were associated with different types of within-person variability features. However, some features associated with multiple personality traits. For example, the higher deviation in physical activity during the evening period was associated with being more open to new experiences and being more extroverted; the higher regularity in physical activity on weekdays was associated with being more conscientious and more agreeable. Table 4. Big Five traits in related to regularity using mixed-effect model. Big Five trait Association Related within-person regularity features (N=159) (+) sound_std_evening_weekday, stationary_std_evening_all, openness stationary_std_evening_weekday, sound_ri_weekday_range (-) voice_std_day_all, voice_std_day_weekday (+) voice_fri_weekday_avg, lock_circadian_all, lock_circadian_weekday, conscientiousness stationary_fri_weekday_avg (-) voice_std_evening_weekday, location_ri_weekday_vs_weekend_avg, location_ri_all_avg, location_fri_weekday_vs_weekend_avg, location_fri_all_avg (+) sound_circadian_all, lock_circadian_all*, lock_circadian_weekday, extraversion stationary_std_evening_all, stationary_std_evening_weekday (-) voice_std_day_all, voice_std_evening_all**, voice_std_day_weekday, voice_std_evening_weekday**, lock_std_evening_all (+) sound_fri_all_range, lock_fri_weekday_range, stationary_circadian_all, agreeableness stationary_fri_weekday_avg, stationary_fri_all_avg, stationary_ri_weekday_avg, location_ri_weekday_range, location_fri_weekday_range (-) lock_std_night_weekday, stationary_std_night_all, stationary_std_night_weekday, stationary_fri_weekday_range, stationary_ri_weekday_vs_weekend_range, stationary_ri_weekday_range, stationary_ri_all_range (+) neuroticism (-) p < 0.05; bold p < 0.01, *FDR < 0.1, **FDR < Prediction Analysis We use Gradient Boosted Regression Trees (GBRT) [20, 55] to predict the self-reported Big Five personality scores. GBRT is an ensemble method that trains and combines several weak regression trees to make more accurate and robust predictions. It builds base estimators (i.e., regression trees) sequentially. Each estimator tries to reduce the bias of the previously combined estimators. By doing so, in each stage a new regression tree is trained on the negative gradient of the loss function. GBRT is less sensitive to outliers and robust to overfitting [17]. Another advantage inherited from the tree based model is that it computes feature importance measures, which can be used for feature selection. We have a total of 96 features (see Table 1) and a relatively small number of training examples (70 Android and 89 ios). We reduce the feature space dimensionality using the importance vector generated from GBRT. GBRT computes feature importance by averaging the number of times a particular feature is used for splitting a branch

14 141:14 W. Wang et al. across the ensemble trees. Higher values correspond to higher importance. We select features with a feature importance value higher than the mean importance iteratively. We repeat this process until we get no more than 9 features. Our heuristic of selecting 9 features is based on experiments in which we find we get higher training errors with a lower or higher threshold. We train and test models separately among the Android and ios users. We take this strategy on the basis of our observation that a model trained from the mixed dataset gives poorer predictions, even if we normalize the features separately among Android and ios users. We believe this is due to the differences (i.e., heterogeneities) between Android and ios devices, i.e. the accuracy of sensed behaviors may be influenced by the sensors and algorithms on different platforms. For example, activities are derived from the Google Activity Recognition API [24] on Android phones and from the ios Core motion API [32] on iphones; the values of sound amplitude in the same environment may be distinct between the two platforms; the conversation classifier may have potentially small differences because of the different platforms (e.g., different microphones and audio APIs). While we have designed and implemented our sensing algorithms on both platforms to take known differences between the ios and Android platforms into account there are still no widely accepted techniques for equalizing sensing data from these different platforms. Scaling the values to standard normal distributions separately on both platforms does not result in an ideal solution - it violates the observation that Android and ios users may have different behaviors and phone usage patterns [7, 31, 43, 46, 65]. Adding a binary feature DeviceType to control for the platform of the device, does not result in better performance when there are no significant differences for most of the personality traits (Table 3). Specifically, in GBRT the tree grows greedily in a top-down fashion using binary splits. For each tree node, the split minimizing the objective is chosen. The DeviceType may not be selected at the root of regression trees due to the similar means of ground truth; i.e., the personality trait scores in the two groups. Taking these challenges into account we opted to divide the data into Android and ios users. We train different models on these two groups. Usually, with a small sample set leave-one-out cross validation [30] would be the best option to show the performance of the personality trait prediction model. This technique is widely used in estimating the performance of the model from a small human-centered dataset in existing studies [4, 33, 69, 73]. Leave-one-out has low bias, because each fold uses almost the entire dataset as a training set [38]. However, as a result, the estimation is also very specific for this particular dataset. This could results in high variance compared to the same model s performance on new datasets. Therefore, we also use five-fold cross validation. For each personality trait, we use ten times five-fold cross validation and report the average. We compare our predictive models with two baseline models. The baseline 1 model takes the average of the scores as the predicted value. This is the most basic regression with only an intercept. The baseline 2 model randomly generates a sample from the already known distribution (i.e., the distribution of the personality trait scores) and uses it as the predicted value. We validate our prediction model using the Mean Absolute Error (MAE), the root mean squared error (RMSE), the Pearson correlation and the R-squared value. MAE and RMSE describe the bias of the predictions; the Pearson s r describes how well the predictions are associated with the ground truth; and the R-squared value measures the goodness of fit by indicating how much of the variance our model explains [12]. Table 5 shows the performance of the prediction model purely based on within-person variability features. The models perform better than the two baselines, and capture considerable variance of the original distribution. The predicted personality score is highly correlated with the ground truth. Our model works better in predicting the extraversion and agreeableness traits. For example, for Android users the leave-one-out model for predicting the extraversion trait achieves 0.55 of MAE, which is 0.24 (30%) lower than the average baseline model and 0.5 (48%) lower than the random baseline model. For ios users the leave-one-out model for predicting the extraversion trait achieves 0.61 of MAE, which is 0.11 (15%) lower than the average baseline model and 0.39 (39%) lower than