Good Vibrations: Designing and Evaluating a Ubiquitous Intervention for Reducing Digital Consumption

Transcription

1 1 Good Vibrations: Designing and Evaluating a Ubiquitous Intervention for Reducing Digital Consumption FABIAN OKEKE, Cornell Tech MICHAEL SOBOLEV, Technion and Cornell Tech NICOLA DELL, The Jacob s Institute, Cornell Tech DEBORAH ESTRIN, Cornell Tech This paper presents the design, implementation, and evaluation of a ubiquitous mobile intervention that aims to reduce the amount of time that people spend consuming digital content on their mobile devices. Specifically, we created an intervention that continuously monitors a person s mobile application usage and provides (1) realtime feedback that increases awareness of their personal usage habits, and (2) negative reinforcement, delivered via mobile phone vibration, that reduces the amount of time participants spend using an application. We evaluate our intervention through a three-week controlled study with 68 participants on Amazon s Mechanical Turk platform. Our findings show that negative reinforcement via vibration significantly reduced the amount of time that participants spent using a target application (Facebook), although the reduction did not persist after the vibration was removed. Insights from qualitative feedback also suggest that the system made participants more aware of their personal application usage habits. Taken together, our findings suggest that negative reinforcement delivered via mobile phone vibration is an effective strategy for reducing digital consumption. CCS Concepts: Human-centered computing Ubiquitous and mobile computing design and evaluation methods; Additional Key Words and Phrases: Social Media, Digital Awareness, Negative Reinforcement, Behavioral Intervention ACM Reference format: Fabian Okeke, Michael Sobolev, Nicola Dell, and Deborah Estrin Good Vibrations: Designing and Evaluating a Ubiquitous Intervention for Reducing Digital Consumption. 1, 1, Article 1 (January 2016), 18 pages. DOI: ACM acknowledges that this contribution was authored or co-authored by an employee, or contractor of the national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only. Permission to make digital or hard copies for personal or classroom use is granted. Copies must bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. To copy otherwise, distribute, republish, or post, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org ACM. XXXX-XX/2016/1-ART1 $15.00 DOI:

2 1:2 F.N. Okeke et al. 1 INTRODUCTION The amount of time that people spend consuming digital content has dramatically increased over the years because of the opportunity to use mobile devices in nearly every context and at nearly every moment. Popular mobile apps including Facebook, Instagram, YouTube, Whatsapp, Snapchat, Gmail, News, and more have become an integral part of people s everyday lives, helping them to share their thoughts, express their feelings, connect with friends and families, receive news updates, and enjoy many forms of digital entertainment. However, research has shown that the ability to engage with technology at any time also burdens people with the pressure of continual availability [1, 39], the need to constantly check in [4, 40, 44], and the ease of being able to procrastinate on work, studies, and personal goals [27, 41, 43, 52]. These negative effects cause many people to report a desire to reduce or limit their technology use [26]. To address this desire, prior research has suggested completely abandoning social media [3], limiting usage through personal productivity tools [46, 53, 54], refraining from smartphone ownership in favor of traditional feature phones [33], telephoning users to encourage them to reflect on their social media usage [60], and reminding people of their personal usage goals via pop-ups [23]. Our research contributes to this growing literature on digital consumption by investigating techniques that help people to reduce the amount of time they spend using applications on mobile devices. Specifically, we designed, implemented, and evaluated a mobile system that tracks daily usage of any desired application on an Android device (e.g., Facebook, YouTube, Whatsapp, Gmail, and Candy Crush) and provides users with two kinds of feedback: (1) non-invasive, realtime usage statistics that they can access at any time and that tells them how much time they have spent that day using the targeted application; and (2) negative reinforcement, delivered via mobile phone vibration, when they exceed a daily usage limit on the application of interest. We created and tested several preliminary strategies for determining a person s daily usage limit, including (1) setting a static fixed limit, (2) calculating the average usage time of a group of users, and (3) personalizing the usage limit according to each specific user s habits. We evaluated our intervention through a three-week controlled study with 68 participants recruited via Amazon s Mechanical Turk (mturk) platform. The study consisted of four experimental conditions that aimed to isolate the impact of providing negative reinforcement via vibration: the control condition received usage statistics without vibration; the rigid condition received usage statistics and vibration after exceeding a static fixed daily limit; the GroupAverage condition received usage statistics and vibration after their usage exceeded the average of a group of users; and the Personalized condition received usage statistics and vibration tailored to their personal habits. Participants in all four conditions took part in the study for a total of three weeks. The first week consisted of a baseline period, in which we gathered data regarding participants application usage habits and provided feedback via usage statistics without any vibration. The second week was the treatment period, in which participants received both usage statistics and vibration feedback according to the experimental condition to which they were assigned. The third week consisted of a follow-up period, in which participants still received usage statistics but any vibration feedback that the participant was receiving was removed, enabling us to understand what happens to participants behaviors when the intervention that they were receiving stopped. Although we implemented our system so that it is capable of tracking usage of any mobile application, we chose to specifically target Facebook as the application to monitor in our controlled experiment. We selected Facebook because it is the most dominant mobile application in the world [14] and prior research has shown that a large proportion (e.g., 90%) of people who are interested in limiting their consumption of digital content say that they would like to reduce the amount of time that they spend on Facebook [23].

3 Good Vibrations: Designing and Evaluating a Ubiquitous Intervention for Reducing Digital Consumption 1:3 At a high-level, findings from our controlled experiment suggest that providing feedback via usage statistics made participants more aware of their daily application usage. In addition, applying negative reinforcement via vibration significantly reduced the amount of time that participants spent using the targeted application, although the reduction in usage did not persist after the vibration was removed. Finally, although negative reinforcement via vibration was generally effective, we found no detectable differences between the different strategies that we used to calculate daily usage limits. This suggests that further research is necessary to pinpoint effective strategies for determining the exact usage limits that should be used to trigger negative reinforcement. In summary, our research makes the following specific contributions to the Ubicomp community: The design and implementation of a ubiquitous and scalable intervention that uses negative reinforcement, delivered via mobile device vibration, to reduce the amount of time that people spend consuming digital content on their mobile devices. Findings from a rigorous three-week controlled experiment with 68 participants that quantitatively and qualitatively demonstrate that negative reinforcement, delivered via mobile phone vibration, is able to significantly reduce the amount of time people spend using a targeted application (e.g., Facebook); Lessons learned from conducting a longitudinal study on Amazon s mturk platform that will be valuable for behavioral science, HCI, and Ubicomp researchers interested in using crowd-sourcing platforms for longitudinal field experiments. 2 RELATED WORK Our intervention aims to reduce people s consumption of digital content by providing them with two kinds of feedback: (1) realtime statistics that increases awareness of their personal usage habits, and (2) negative reinforcement, delivered via mobile phone vibration, that encourages people to spend less time using applications. We discuss relevant literature related to each of these two kinds of feedback, and discuss the behavioral psychology theories on which our work is based. 2.1 Increasing Awareness of Personal Usage Researchers have shown that self awareness, which refers to the process of being aware of one s current state, can lead to changes in people s performance or behavior [6]. Providing people with feedback can increase self-awareness by providing a yardstick that enables them to compare their current state to their goal state [28]. To increase self awareness, HCI and Ubicomp researchers have designed numerous self-tracking systems. For example, Ubifit uses a subtle, glanceable display in the background [11, 12]. Other systems provide web-based summary pages with details about personally tracked activities (e.g. RescueTime [46], Fitbit dashboard [19]), summary displays for digital distractions [54], and smartphone lock screen widgets [8]. Other solutions have focused on characterizing people s digital consumption using smartphone traces [18, 25, 49, 57, 59], including predicting a person s digital consumption [9, 24]. Our research complements this prior literature by creating a low-priority, persistent notification that displays realtime usage statistics. As such, it provides a continuous, non-intrusive way for participants to view up-to-date application usage statistics whenever they choose.

4 1:4 F.N. Okeke et al. 2.2 Reducing Digital Consumption Although consuming digital content provides numerous benefits to people, research has shown that technologies such as social media platforms may act as a source of considerable stress [20], particularly for students, who are often prolific social media users [27]. As a result, smartphone owners have demonstrated an interest in tools that help them reduce their digital consumption [34], both as an individual [56] and as a group [30, 31]. Several projects have also specifically studied reasons for people to reduce their social media usage (e.g. [3, 48, 51]). A growing number of interventions also aim to help people reduce the time that they spend consuming digital content. For example, TimeAware aims to help users reduce their overall distraction on desktop devices by providing feedback of digital usage framed as a positive or negative message [29]. Zhou et al. aimed to reduce social media usage by telephoning participants at random times of the day and asking them to provide an estimate of their social media usage since they woke up [60]. Although the findings from this study suggest that participants were more aware of their usage habits, they did not reduce the amount of time spent on social media. Perhaps most similar to our work is MyTime, a mobile intervention that aimed to reduce the time spent across multiple mobile applications by simultaneously applying three techniques: personal daily goal-setting by users, feedback provided in the notification bar, and a pop-up message when an app is used past a user-defined limit [23]. Our work builds on this growing body of work in a number of key ways. First, we apply vibration as negative reinforcement, which provides several advantages. Vibration is ubiquitously available on all mobile devices and, unlike a pop-up message that interrupts usage and can be dismissed, vibration can be actively applied while an application is being used and cannot be easily dismissed or closed by the user. Second, our intervention automatically collects usage statistics without requiring configuration or manual input from users. Furthermore, our solution provides continuous, realtime feedback that is always available for users to view without requiring them to visit a separate application or webpage to see their personal data. Finally, we provide findings from a rigorous, controlled experiment that specifically isolates the impact of negative reinforcement delivered via vibration. As such, we are able to conclusively demonstrate that negative reinforcement is an effective mechanism for reducing digital consumption. 2.3 Theory of Negative Reinforcement Our research is grounded in theory from behavioral psychology. Specifically, we turn to negative reinforcement [50], which is the process of strengthening a behavior or response through repetition by stopping, removing, or avoiding a negative outcome or aversive stimulus. This process involves learning over time based on personal experiences and provides a high potential for successful behavior change [15, 16]. One common example of negative reinforcement is response to a morning alarm. When a morning alarm blares, a heavy sleeper wakes up and pushes the snooze button in order to eliminate the aversive ringing sound. In the future, whenever the alarm rings, the snooze behavior is repeated to eliminate the aversive stimulus. Over time, the desire to avoid the ringing alarm leads to a sustained behavior of hitting snooze button. Another example is when a driver starts a car without putting on the seat belt. This leads to a state where the car continuously makes a beeping sound until the driver puts on the seat belt to stop the irritating sound. Then, when the driver enters the car in the future, the seat belt is worn to avoid the aversive beeping sound. Interventions that applied a similar approach were effective in reducing cheating in college exams and increasing enforcement of safety rules in factories and hospitals [15]. Inspired by the universality of this psychological phenomenon, our work studies how to incorporate negative reinforcement, in the form of mobile phone vibration, in an attempt to reduce social media usage. The next section describes the design and implementation of our intervention.

5 Good Vibrations: Designing and Evaluating a Ubiquitous Intervention for Reducing Digital Consumption 1:5 3 SYSTEM DESIGN AND IMPLEMENTATION The goal of our research is to invent and study techniques that help people to reduce the amount of time they spend consuming digital content on mobile devices. To achieve this goal, we designed and implemented an experimental system that runs on Android and that continuously monitors a person s mobile application usage to provide (1) realtime feedback that increases awareness of personal usage habits, and (2) negative reinforcement, delivered via mobile phone vibration, that reduces the amount of time participants spend using target application(s). The first step in designing our system was to create a method for tracking people s usage of specific applications. To achieve this, our system continuously monitors all foreground applications. Whenever an application of interest is being actively used, our system tracks this usage and updates the amount of time spent and number of times that the specific application was opened. Every three hours, our system sends all application usage logs and statistics to a remote server. At the end of each day, the usage statistics are reset back to zero. To provide participants with realtime feedback and usage statistics regarding their own behavior, we created a lightweight notification system that requires minimal modifications to participant devices. The feedback consists of a persistent status bar notification that tells the user how long a targeted application has been used and how many times it has been opened so far that day (see Figures 1 and 2). Our experimental system automatically updates the usage statistics every five seconds, thereby providing continuous feedback. Further, to make the system unintrusive, we assigned the status bar notification to be low-priority so that, although the notification is always present, the status bar must be swiped down for a user to view it. In addition, the notification is not visible when the phone screen is locked and notifications from other applications will be placed above it. This lightweight approach enables participants to view their usage whenever they choose without being intrusive. To implement negative reinforcement, our experimental system monitors how often an application of interest is used and vibrates when the total amount of time spent using the application exceeds a specified daily limit. We chose vibration as the feedback mechanism because it is considered private and subtle [22] and it is ubiquitously available on all mobile devices. As a target application is used, the system compares the current usage statistics to specified daily usage limits. When a user reaches their usage limit, the system causes their mobile device to vibrate every five seconds that the application of interest is actively used beyond the usage limit. The vibration stops immediately when the user navigates away from a targeted application to another application or when the user locks their phone screen (i.e., if the application is still open but the screen is locked). Thus, our system only monitors and records when a user is actively engaging with the target application. The strategies used to compute a user s daily usage limits can vary from choosing a static, fixed limit to computing personalized limits based on individual usage. The controlled experiment that we conducted to evaluate our intervention tested several different strategies for computing usage limits, described in detail in the next section of the paper. We designed and built our system so that it is capable of tracking all the applications on a user s mobile device and adding vibration feedback for any desired application(s). After designing and implementing the system, we conducted tests that monitored five different popular applications: Facebook, YouTube, WhatsApp, Candy Crush, and Gmail. The purpose of these tests was to confirm that the system was capable of functioning correctly and successfully tracking usage of any installed application. In addition, to make it easy for us to conduct controlled online experiments, we created and implemented a simple web dashboard that can be used to select the application(s) that should be monitored, assign participants to experimental conditions and usage limits, and keep track of ongoing user participation and incoming data. Figure 1 and Figure 2 show example notification tracking for Facebook usage and WhatsApp usage respectively.

6 1:6 F.N. Okeke et al. Fig. 1. Notification tracking Facebook usage. Facebook was opened five times for a total of 500 seconds. Fig. 2. Notification tracking WhatsApp usage. WhatsApp was opened eight times for a total of 720 seconds. Prior to evaluating our system through a controlled experiment, we performed a preliminary pilot test with 10 participants recruited from Amazon s Mechanical Turk (mturk) platform. Participants were asked to download, install, and use the application for a total of five days with Facebook selected as the target application. This pilot testing helped us to address a few technical issues, fix bugs, and confirm that the system was working as expected. At the end of the five day pilot test period, participants were compensated $5.00 for their time. The final version of our open source vibration application is freely available on Github1. 4 EXPERIMENT DESIGN To evaluate the impact of providing negative reinforcement via vibration, we conducted a controlled, three-week experiment with 68 workers recruited via Amazon s mturk platform. We hypothesized that receiving negative reinforcement via vibration would reduce participants usage of a target application. Although we implemented our system so that it is capable of tracking usage of any mobile application, we chose to specifically target Facebook as the application to monitor in our controlled experiment. We did not allow participants to choose their own target application(s) because we wanted all participants to experience the same study procedures. Thus, we limited our experiment to a single, globally prevalent application (Facebook) to avoid confounds that may result from allowing individual participants to choose a wide variety of different and potentially unknown, unreliable, or buggy applications. We chose Facebook as the target application because it is the most dominant mobile application in the world [14] and prior research has shown that a large proportion (e.g., 90%) of people who are interested in limiting their consumption of digital content say that they would like to reduce the amount of time spent using Facebook [23]. The rest of this section describes our experiment in detail. All study procedures were IRB approved prior to the study. 1 Vibration application available at

7 Good Vibrations: Designing and Evaluating a Ubiquitous Intervention for Reducing Digital Consumption 1:7 4.1 Participants We recruited study participants via Amazon s mturk platform. To be eligible for the study, participants needed to be an active Facebook user, defined as (1) having the Facebook application installed on their personal phone, and (2) using Facebook for at least five minutes per day. We established a participant s eligibility via a short pre-study survey that we administered via mturk. During this survey, we noticed that the majority of our prospective participants (approx. 70%) were located in India. Therefore, to minimize the potential for confounds due to differences in location, we chose to further restrict participation to workers who were located in India. We recruited a total of 68 participants based in India (53 males and 15 females) via mturk. Participants ranged in age from 19 to 40 years (average = 28.8, median = 29) and self-reported that they had worked on mturk between one month and six years. Participants reported a diverse range of Facebook usage habits, self-reporting that they opened the Facebook application between one and 30 times per day, and spent between five minutes and three hours per day using Facebook. Participants reported using Facebook to read the news, see their friends status updates, and share their thoughts, comments, and pictures. 4.2 Study Procedure Prior to being enrolled in the study, prospective participants completed a short survey that asked about their Facebook usage habits. Based on the results of this survey, participants that were eligible for the study (i.e., used Facebook for more than five minutes per day) were contacted via mturk and invited to enroll in the three-week study. Participants who responded to the invitation were directed to an experiment landing page that contained a description of the study and a consent form. After reading and accepting the consent form, participants completed a short demographic questionnaire. Next, participants downloaded and installed the experimental system on their personal phone. Any participants who did not also have the Facebook application installed on their phones received an error message and were removed from the study. Participants who successfully installed the experimental system and who had the Facebook application installed on their phone were then randomized into one of four experimental conditions that used different strategies for computing the circumstances under which vibration would occur: Control: participants in the control condition did not receive any vibration feedback regardless of how often they opened or how much time they spent using the target application. Rigid: this strategy set an aggressive limit on the amount of time participants could spend using the target application before they received negative reinforcement. We wanted the majority of participants to experience vibration regardless of their personal usage habits and so we chose a static limit that allowed participants to open the target application two times per day and spend 10 minutes per day using it before vibration began. GroupAverage: this strategy set each participant s usage limit according to the behavior of the broader participant population. Specifically, we computed the average usage for all participants in this condition for the first five days of the experiment. Then, we set each participant s usage limit to be 50% of the population s average usage. For example, if there were three users in the GroupAverage condition, with five-day averages of 50 seconds, 100 seconds, and 150 seconds respectively, then the group average would be 100 seconds, leading to a usage limit of 50 seconds for each participant. Personalized: this strategy set each participant s usage limit according to their own personal usage. We calculated each participant s average daily usage for the first five days of the study period, and set their personal

8 1:8 F.N. Okeke et al. usage limit to be 50% of their personal average. For example, if a participant used Facebook for an average of one hour daily and Facebook was opened ten times per day for the first five days of the study, then the participant s usage limit would be set at 30 minutes and five times per day. Participants in all four experimental conditions received the persistent, low-priority notification that provided them with realtime usage statistics regarding their daily usage of the target application. We chose to keep the notification consistent across all conditions for two reasons: (1) we wanted differences in vibration strategies to be the only difference across conditions; and (2) the experiment on-boarding process explained to participants that the study was about understanding their Facebook usage habits. By making all users aware of the purpose of the study, it was easier to isolate and test the effect of vibration on specific groups of participants. After being randomized into one of the four experimental conditions, each participant took part in the study for a total of three weeks. The first week of the study consisted of a baseline period, in which we gathered data regarding participants application usage habits and provided feedback via usage statistics, but no vibration. The second week of the study was the treatment period, in which participants received both usage statistics and vibration feedback according to the experimental condition to which they were assigned. The third week of the study consisted of a follow-up period, in which any vibration feedback that the participant was receiving was removed, although the system continued to monitor the participant s daily usage and provide usage statistics. The goal of the follow-up week was to assess the effect on participants behavior of removing negative reinforcement. At the end of three weeks, all participants completed an exit survey and were instructed to uninstall the experimental system from their devices. Each participant received $15 (approx rupees) for participating in the study and a lottery chance for an additional $5 bonus. For perspective, $1.00 can buy a 12-ounce cup of coffee in India and $5.00 is sufficient for a full meal. 4.3 Data Collection and Analysis Our experimental system was instrumented to record the number of times participants opened, and the amount of time spent actively using, the target application. In total, our system logged 42,389 server records for all participants across the entire three-week study period. At the start of the study, we enrolled a total of 106 participants in the four experimental conditions: Control (n=25), Rigid (n=26), GroupAverage (n=28), and Personalized (n=27). However, several participants did not complete the study, and so we restrict our analysis to only include participants who missed fewer than three days of the study. This led to a total of 68 participants used for our final analysis: Control (n=16), Rigid (n=19), GroupAverage (n=18), and Personalized (n=15). We analyze the impact of negative reinforcement via vibration through a number of outcome metrics that capture participant usage of the target application Facebook. Our primary metric is the amount of time spent using the application. To calculate this metric, we compute the total amount of time that each participant spent on Facebook for each day of the study. A secondary metric is the number of times the application is opened, which we compute by counting the number of times that each participant opened the target application on each day of the study. Across all participants in the experiment, the maximum time spent on Facebook was 4.7 hours per day and the maximum number of times Facebook was opened daily was 99. Our analyses capture the change in application usage for each participant between a one-week baseline period, where participants across all experimental conditions did not receive vibration, and a one-week treatment period in which participants received negative reinforcement via vibration according to the group to which they were assigned. We then further capture the change for each participant during the one-week follow-up period, in which vibration feedback was removed for all experimental conditions. Before performing our analyses, we evaluated the Kolmogorov-Smirnov [38] test

9 Good Vibrations: Designing and Evaluating a Ubiquitous Intervention for Reducing Digital Consumption 1:9 to assess the normality of our data and found that it was statistically significant for all of our outcome metrics. As a result, we used non-parametric Wilcoxon Signed-Rank [58] tests to analyze changes within experimental conditions across study periods. 5 EXPERIMENT RESULTS At a high level, our experimental results show that negative reinforcement, delivered via vibration, significantly reduced the amount of time that participants spent using the target application regardless of the strategy used to compute the usage limit. However, the reduction in usage did not persist after the vibration intervention was removed in the follow-up period. Findings from qualitative feedback that we gathered from participants shows that they found the notification bar statistics useful for keeping track of their own usage, and that they correctly perceived the vibration as providing feedback that encouraged them to spend less time using the application. The rest of this section describes these findings in detail. Negative reinforcement successfully reduces digital consumption of a target application. Our findings show that negative reinforcement, delivered via vibration, led to a significant reduction in the amount of time participants spent using the target application in this case, Facebook. Figure 3 shows the average application usage for participants in each experimental condition during each of the three study periods. As the graph shows, we observed that the amount of time spent on Facebook per day during the treatment period decreased by over 20% for each of the vibration conditions: Rigid (23% or 9 minutes), GroupAverage (25% or 10 minutes), and Personalized (23% or 9 minutes), but not for the Control condition. Table 1 summarizes the results of Wilcoxon Signed-Rank tests that we conducted to analyze, for each experimental condition, the differences between the average amount of time that participants spent using Facebook across the baseline and treatment periods. As Table 1 shows, participants in all three of the vibration conditions spent statistically significantly less time using the target application during the treatment period, when they were receiving negative reinforcement via vibration, than in the baseline period. This suggests that the vibration intervention successfully reduces participant usage of the target application regardless of the particular strategy used to compute the usage limit before vibration occurs. At a high-level, these findings validate recent research that uses a combination of alternative (non-vibration) mechanisms to deliver negative reinforcement [23]. However, our work goes beyond this prior work by conducting a rigorous controlled experiment to isolate that the reduction in application usage is a result of negative reinforcement delivered via vibration, and not other feedback mechanisms or social desirability bias [21]. Participants perceived that the vibration provided negative feedback. At the end of the experiment, participants were invited to complete an exit survey that provided us with qualitative information regarding their experiences during the study and opinions of the vibration application. We received a total of 50 survey responses: Control (n=10), Rigid (n=8), GroupAverage (n=18), and Personalized (n=14). Thus, of the 50 participants who completed our exit survey, 40 were in one of the three conditions that experienced vibration (the other ten were in the control condition). Another nine participants who were in the GroupAverage condition expressed that they never experienced vibration. This was likely because these participants application usage was below the limit computed for the group, which is not surprising because taking the average usage of the population as the limit automatically implies that participants who are at the lower end of the statistic will not use the application enough to experience vibration. Our qualitative analysis of how participants felt when they received vibration therefore focuses on responses from the 31 participants who did experience vibration.

10 1:10 F.N. Okeke et al. Fig. 3. Average time that participants in each condition spent on Facebook per day during each study period. Fig. 4. Average number of times that participants opened Facebook per day during each study period. Experiment Condition Effect Size Baseline-Treatment p-value Baseline-Treatment Effect Size Baseline-Followup p-value Baseline-Followup Control Rigid ** GroupAverage * Personalized * All treatment **** Table 1. Average time spent on Facebook per day: Summary of pairwise comparisons between baseline and treatment. p < 0.05, p < 0.01, p < The majority of participants (19/31) who received negative reinforcement via vibration said that it helped to increase awareness of their application usage. When asked what they did when the vibration started, 21 participants said that they stopped using the application after a few minutes, while the other ten said that they continued to use the application as normal. This finding supports our quantitative data, which shows decreased usage for participants in the vibration conditions. In addition to understanding how participant behavior changed when vibration started, we were also interested to understand how the vibration made them feel. The majority (26/31) said that the vibration was irritating, with roughly half (n=14) saying they found it mildly irritating, and the other half (n=12) saying that they found it very irritating. This finding suggests that participants correctly perceived the vibration to be providing negative feedback. Although the majority (17/26) of participants that found the vibration irritating said that it caused them to spend less time using the application, none of the participants said that they stopped using it completely. This indicates that participants aversion to the vibration was not strong enough to cause them to leave the platform completely. The statistics provided through the notification bar increased awareness of application usage. Of the 50 participants who completed our qualitative exit survey, 46 said that the usage statistics provided by the persistent notification bar increased their general awareness of how much time they spent using Facebook every day. This result validates findings from a recent study by Rooksby et al. [47] that showed how personally tracking screen time on digital devices helped to improve overall awareness, and further extends these findings by demonstrating that feedback on specific application usage also increases awareness of personal usage habits.

11 Good Vibrations: Designing and Evaluating a Ubiquitous Intervention for Reducing Digital Consumption 1:11 When asked how often they checked the statistics provided by the notification bar, participants self-reported as looking at the feedback an average of 11.5 times per day (median=8.5, mode=5). Moreover, many participants submitted written comments suggesting that they enjoyed having the ability to keep track of their daily usage, with several describing how the feedback made them more aware of how much they were using the application. As one participant said, Through the status bar, I realized that I was using Facebook much more than what I had thought. Several other participants explained that receiving the feedback saved them time by causing them to use Facebook less than they had before. Taken together, these findings suggest that, regardless of whether the participants experienced negative reinforcement via vibration, the usage statistics provided through the notification bar were useful for participants and increased their awareness of their personal usage habits. Negative reinforcement via vibration reduces application usage only when actively applied. The goal of the follow-up period of the study was to investigate if any behavior change that occurred as a result of the vibration feedback persisted when the vibration was removed. We hypothesized that, in the absence of negative reinforcement, participants would gradually return to their original levels of application usage. As shown in Figure 3, the amount of time that participants in each of the three vibration conditions spent using the target application increased between the treatment period, when they were experiencing vibration, and the follow-up period, when vibration was removed. We conducted Wilcoxon Signed-Rank tests to analyze, for each experiment condition, the differences between the time spent on Facebook in the baseline period and in the follow-up period. The results of the tests were non-significant for all experimental conditions, indicating that there was no detectable difference between participants usage of the application in the baseline and follow-up periods. This finding suggests that although receiving vibration effectively reduced application usage, the change in behavior was not sustained when the intervention was removed. Future studies that involve a larger sample size or longer study duration may have more statistical power to investigate the rate at which participants in each experimental condition gradually return to the usage levels observed in the baseline period after negative reinforcement is discontinued. There are no conclusive differences between the strategies used to calculate usage limits. In addition to understanding the overall impact of negative reinforcement via vibration, we were also interested in understanding if there were differences between the strategies used to compute the usage limits that would result in vibration. In particular, we hypothesized that personalizing the vibration to an individual participant s usage habits would be more effective and result in greater reductions in application usage. This hypothesis is based on prior work that suggests that personalization leads to more successful interventions by catering to each individual s context [17]. However, although our analysis shows a descriptive difference in effect sizes (see Table 1) with both the GroupAverage and Personalized conditions consistently having higher effect sizes than Rigid condition, these differences are not statistically significant. This result could be due to a number of factors. For example, in the Personalized condition, we set the usage limit to be 50% of the participant s daily usage, which may not be the correct ratio to use. We also calculate the personalized usage limit based on five days of data, which may not be sufficient for determining an appropriate limit. Finally, we may not have enough statistical power to detect a significant difference between the groups given the small sample size of each group. Nevertheless, these limitations suggest exciting opportunities for more explorations. Future studies with a larger sample size and a longer study period will have more statistical power to detect and provide conclusive evidence on the comparison between personalized and static usage limits, which could eventually lead to the discovery of appropriate strategies for personalizing the intervention to individual participants.

12 1:12 F.N. Okeke et al. Negative reinforcement does not reduce the number of times participants opened the application. In addition to measuring the total amount of time that participants used the target application in each experimental condition, we were interested in how negative reinforcement may affect the number of times participants opened the application. Figure 4 shows the average number of times that participants in each condition opened Facebook in each study period. The results of Wilcoxon Signed-Rank tests that we conducted to analyze, for each experimental condition, the differences between the average number of times participants opened Facebook in the baseline period and the treatment period were non-significant for all four experimental conditions, indicating that the vibration did not detectably reduce the number of times that participants opened the application. This finding could be explained by the fact that participants only feel the vibration after they have already opened the application. As a result, even though a participant may open the application and immediately close it when they feel the vibration, their behavior will still count as opening the application. We hypothesize that, to reduce the number of times that participants open the application, it would be more effective to, for example, create a prompt that, when participants try to open the application, checks that they really wish to do so. Although not in the scope of our experiment, we note that such alternative behavior change mechanisms provide rich opportunities for future research. 6 DISCUSSION The high-level goal of our research is to invent and study techniques that help people to reduce the amount of time they spend consuming digital content on their mobile devices. To achieve this goal, we offer a solution that is simple, ubiquitous, and based on sound theory the concept of negative reinforcement from the theory of operant conditioning in behavioral psychology [50]. Our experimental results show that using vibration as negative reinforcement does successfully lead to behavior change and reduce participants usage of a target application. As participants exceeded the computed usage limits in our experimental conditions, the continuous vibration that they felt created an aversive environment for the participant through constant irritation. To avoid this negative stimulus, participants sensitive to the vibration responded by leaving Facebook after a short time. However, this change in behavior did not persist when the vibration intervention was withdrawn, which suggests that achieving long-term behavior change may require participants to experience negative reinforcement for a long period of time (and perhaps indefinitely). In addition to overall application usage, negative reinforcement via vibration may also be a good strategy for achieving behavior change in specific contexts, such as when studying for an exam, during class time, when participating at a meeting that requires full focus, etc. In these kinds of scenarios, the vibration could serve as a reminder and help to nudge participants back to the task at hand. Although we found that negative reinforcement via vibration was generally successful at reducing digital consumption of a target application, our attempts to personalize participants usage limits did not reduce application usage significantly more than a fixed static limit. This suggests a need for future experimentation that focuses on different personalization strategies. For example, the system could perform adaptive personalization that regularly recomputes usage limits over time instead of the static personalization used in our experiment. In addition, the system could be designed to integrate both personal preferences, such as sleep time, wake-up time, or recreation time, as well as contextual awareness, such as location, time of day, weekday or weekend, vacation settings, and more. Determining how best to personalize or optimize the intervention so that it most effectively helps people to reduce their consumption of digital content is an exciting area for future work. There are a number of key differences between our work and recent research in HCI and Ubicomp that focuses on reducing digital consumption. For example, one major difference between our study and prior work (e.g., [23, 29])

13 Good Vibrations: Designing and Evaluating a Ubiquitous Intervention for Reducing Digital Consumption 1:13 is that our experiment did not explicitly target participants who self-identified as wanting to curb their digital consumption. We made this choice because we were interested in the effect that our intervention might have on a general population of participants. The fact that we observed significant reductions in digital consumption even with participants who did not explicitly identify as wanting to reduce their usage is promising, and we hypothesize that future experiments conducted with participants who are interested in reducing their digital consumption could result in an even larger effect. Another key difference between our study and existing productivity tools (e.g., [42, 46, 53, 54]) is that we provide automatic, non-invasive, realtime feedback. By contrast, many existing productivity tools require users to visit a separate website or open a standalone application that displays their usage data and provides feedback. However, research has shown that many of these existing applications suffer from low levels engagement because of the need for users to make an effort to check a separate website or application [10]. Our solution provides a persistent yet non-intrusive notification that is always easily accessible and that displays realtime usage statistics that continuously update. In providing this continuous feedback, our work contributes to existing literature that aims to improve user engagement through visual feedback [12, 29, 47]. In addition, the haptic feedback provided by the vibration does not require users to visit or look at a separate page or notification. Further, users are not able to dismiss the vibration in the same way that they could close or dismiss a pop-up or dialogue box. Instead, the vibration provides a continuous reminder that persists until they close or navigate away from the application. Finally, many prior solutions require that participants manually choose the applications that will be monitored, set their own personal goals regarding usage limits, and modify configurations (e.g., [23, 29]). By contrast, our intervention is administered centrally and automatically monitors application usage without the participant needing to do anything. We made this choice so that it was possible for us to conduct a controlled experiment in which all participants received exactly the same procedure. However, although our approach relieves users of the burden of manually choosing the applications to monitor and deciding their own daily usage limits, individual participants will undoubtedly want to use and track different applications, as well as customize the intervention to their personal usage and habits. Providing a lightweight, intuitive interface that enables participants to customize the intervention to their own needs is left as a topic for future research. 6.1 Lessons Learned from Conducting Longitudinal Studies via Crowdsourcing Platforms In recent years, many behavioral science, HCI, and Ubicomp researchers have conducted experiments involving short tasks that engage workers recruited via crowdsourcing platforms for a few minutes or hours [2, 5, 7, 32, 37]. Indeed, when researchers refer to longitudinal studies conducted on such platforms, they typically refer to studies in which participants are contacted at a later date as follow up to a previous study [13]. By contrast, our longitudinal experiment was conducted with 68 workers for three weeks. We used this approach because of the benefits of working with participants from online crowdsourcing platforms it is cheaper, less time-consuming, and participants are more diverse [45]. Throughout our recruitment cycle from pilot studies to the final three-week experiment we reached about 650 participants and eventually completed our experiment with 68 participants. We found it challenging to recruit participants, in part because of our unique requirements: we targeted only participants who actively used Facebook, who had Facebook pre-installed on their phones, used Android phones that were compatible with our experimental application, were based in India, and enrolled only once in our study. Given these stringent requirements, it would have been challenging to recruit as many participants using traditional, non-online recruitment methods. As such, using a crowdsourcing platform provided easier access to a larger participant