Using Mechanical Turk to recruit participants for alcohol Internet intervention research Experience from the conduct of four randomized controlled trials John Cunningham Centre for Addiction and Mental Health 1
Purpose To test the feasibility of recruiting for alcohol Internet intervention trials through MTurk Mechanical Turk is an online portal operated by Amazon requesters (individuals or businesses) can submit jobs for workers (more than half a million people who have signed up to complete jobs on MTurk) to complete for pay. Essentially anonymous MTurk used extensively in social sciences research Separate recruitments for 4 trials Trial 1: September, 2016; Trial 2: December, 2016; Trial 3: January, 2017; March, 2017 Participants can only take part in one recruitment 2
Methods Survey on people s drinking 18 years or older, weekly drinkers $1.50 payment, less than 15 minutes Identify participants from baseline surveys: AUDIT 8 or more; > 15 drinks/week (Trials 2-4) Invite to take part in 3- (or 6-) month fu ($10) Mention that some will be asked their impressions of additional materials 3
Trial 1: Check Your Drinking 423 (3 hours) participants randomized to be sent password to access CheckYourDrinking.net vs Nothing 14.3 AUDIT, 18.6 drinks/week, 13.5% ever treated 85% follow-up rate Only 38% of participants in intervention condition accessed the CYD AUDIT-C showed an impact of the being provided access to the intervention 4
Trial 2: Alcohol Help Centre Sent password (N = 1003; 7 days) to portal where randomized to access CheckYourDrinking.net vs AlcoholHelpCentre.net vs Nothing Only included in trial if they accessed the study portal (N = 511) 17.7 AUDIT, 28.7 drinks/week, 22.9% ever treated 82% 6-month follow-up rate No significant differences in outcomes 5
Trial 3: AlcooQuizz app 987 (9 days) randomized to be sent request to download Alcooquizz app vs Nothing Reply to incentive email to provide proof that downloaded app ($5; control condition also paid) 57% of participants in Alcooquizz condition sent email confirming download of the app (86% in control) 18.3 AUDIT, 29 drinks/week, 20% ever treated 70% 6-month follow-up rate No significant differences in outcomes 6
Trial 4: Hello Sunday Morning 878 (32 days) randomized to be sent request to access HelloSundayMorning.org vs CheckYourDrinking.net vs Nothing Reply to incentive email to provide proof that accessed ($5; control condition also paid) 49% of participants in HSM condition, 58% in CYD, 84% in control condition sent email 18.1 AUDIT, 28.6 drinks/week 66% 6-month follow-up rate No significant differences in outcomes 7
Summary Possible to recruit large numbers of hazardous drinkers and to follow them up Recruitment slows down after a few thousand No evidence of impact of intervention Is this due to sample, app, or length of follow-up? Methods needed to improve compliance with accessing the intervention Possible explanation for limited evidence of impact? Unusual sample generalizability of the results? Experienced survey completers (Median = 500) who are doing this for pay Useful sample during the development phase of an intervention? 8
Declarations Sources of funding Canada Research Chair in Addictions Canadian Institutes of Health Research No conflicts of interest to declare For more information, see: Cunningham, J. A., Godinho, A., & Kushnir, V. (2017). Can Amazon's Mechanical Turk be used to recruit participants for internet intervention trials? A pilot study involving a randomized controlled trial of a brief online intervention for hazardous alcohol use. Internet Interventions, 10, 12-16. Cunningham, J.A., Godinho, A., & Kushnir, V. (2017). Using Mechanical Turk to recruit participants for Internet intervention research: Experience from recruitment for four trials targeting hazardous alcohol consumption. BMC Medical Research Methodology. 7, 156. 9
Baseline survey recruitment Duration of recruitment Trial 1 Trial 2 Trial 3 Trial 4 3 hours 7 days 9 days 32 days Total # (N) 1252 4943 5412 5846 % removed 30.4 34.4 36.1 39.5 Final baseline 871 3244 3456 3536 Eligible + Agree 423 511 987 878 10