Author's response to reviews

Transcription

1 Author's response to reviews Title: The effect of two lottery-style incentives on response rates to postal questionnaires in a prospective cohort study in preschool children at high risk of asthma: a randomized trial Authors: Lonneke B. van der Mark (l.b.vandermark@amc.uva.nl) Karina E. van Wonderen (k.vanwonderen@amc.uva.nl) Jacob Mohrs (j.mohrs@amc.uva.nl) Patrick J.E. Bindels (p.bindels@erasmusmc.nl) Milo A. Puhan (mpuhan@jhsph.edu) Gerben ter Riet (g.terriet@amc.uva.nl) Version: 2 Date: 21 November 2012 Author's response to reviews: see over

2 Academic Medical Center University of Amsterdam Division of Clinical Methods & Public Health Department of General Practice L.B. van der Mark P.O. Box DD Amsterdam The Netherlands Phone: Fax: Date: 21 November 2012 Context: Revision of manuscript ; The effect of two lottery-style incentives on response rates to postal questionnaires in a prospective cohort study in preschool children at high risk of asthma: a randomized trial. Dear Editor, Thank you for the opportunity to revise our manuscript: The effect of two lottery-style incentives on response rates to postal questionnaires in a prospective cohort study in preschool children at high risk of asthma: a randomized trial. Below you ll find our point-by-point replies to the issues raised by the reviewers. Attached please find enclosed our revised version of the paper Reply to the reviewers comments: Reviewer: Siddharth Pratap (1) The major finding of the paper is that there are no significant improvements in the response rate with lottery-style on-monitory incentives. The authors comment on the statistically significant negative interaction on the main endpoint by saying "Since we could not think of an explanation, we decided to ignore it and analysed the trials separately." This exclusion of the negative interaction(and subsequent separation of trail analysis) needs to be further justified and explained in more detail. It may be benefited by analysis by an independent statistician.

3 We agree with the reviewer that a more detailed explanation why we made this decision is desirable, therefore we added our considerations to the discussion section, see below. Due to the factorial design, participants in one intervention group (VD) had the chance to win both the voucher(s) and a daytrip. For the main outcome, we assessed the presence of a synergistic effect between the two lotteries and, in line with the literature, estimated the regression coefficient of a dummy variable representing the interaction using logistic regression.(7) We expected to find either no interaction between the two lotteries or a synergistic effect. The latter may have been present since those eager to win vouchers would automatically be candidates for a daytrip and vice versa. However, we found evidence of an antagonistic effect (OR 0.55, 95%CI , p=0.048). The implication is that the effect of the two lotteries combined is less than that of the product of the two separate effects (multiplicative scale due to the logistic regression model). We could not think of a plausible mechanism for this to happen and therefore in a Bayesian way of thinking we decided that given the small prior likelihood of an antagonistic effect combined with moderately weak evidence (p=0.048) of antagonism, the posterior likelihood of the hypothesis that antagonism is present was small. Therefore we decided to analyse the two interventions separately.(7-10) It is important to realize that even if the daytrip effect might have been reduced as follow-up became longer after at least one deadline had been missed this does not imply a differential effect across the groups and cannot explain a negative interaction. References (7) Montgomery AA, Peters TJ, Little P: Design, analysis and presentation of factorial randomised controlled trials. BMC Med Res Methodol 2003 Nov 24;3:26. (8) Goodman SN: p values, hypothesis tests, and likelihood: implications for epidemiology of a neglected historical debate. Am J Epidemiol 1993 Mar 1;137(5): (9) Goodman SN: Toward evidence-based medical statistics. 1: The P value fallacy. Ann Intern Med 1999 Jun 15;130(12): (10) Mcgrayne SB. The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy. Yale University Press; (2) In the Strengths and Limitations Section, the authors state the following: "For example, it is imaginable that participating families with lower level of education or non-western ethnicity earn less and therefore be more susceptible to these interventions

4 than highly educated or western participants." 2b. One of the stratifications among groups was among Ethnicity of Parents (western / non-western parents / NA). This statement could be misinterpreted as western parents, a high level of education and high income are significantly associated; conversely, the implication is that lower income, non-western parents, and lower level of education are associated. The authors should make a more detailed statement and provide referenced studies as to why non-western parents are grouped into the "earn less" than western parents beyond "it is imaginable". We agree with the reviewer that we cannot make this statement without a reference and therefore we added two references of the Netherlands statistics, which confirms the association of non-western ethnicity and lower level of education with a lower income in the Netherlands. References (11) CBS, Statistics Netherlands 11)-l&HD= &HDR=T&STB=G3,G2,G (12) CBS, Statistics Netherlands Reviewer: Phil Edwards The authors suggest implications for further research that do not seem to follow directly from their study (e.g. evaluation of personalised follow up). We agree with the reviewer that implications for further research regularly follow directly from the study, and therefore we changed the text. Implications for further research There is a lack of evaluations of (non-)monetary incentive strategies to increase response rates or retention rates in longitudinal research. Increasing response rates possibly has a beneficial effect on job satisfaction for the research group, due to decreasing workload as a consequence of non-responders (telephone calls, reminders, etc.). In addition, narrower confidence intervals due to larger numbers and better cost effectiveness may be obtained. Our interventions did not lead to the hypothesized beneficial effects, but the method of a nested RCT in our cohort study may encourage others in longitudinal research to test different strategies to increase response rates. For instance, it would be interesting to evaluate non-lottery-style interventions, for example a gift voucher for everyone who returns the questionnaire.

5 (2) The manuscript appears to conform well to CONSORT guidelines, including a figure showing the flow of participants through the study. It would be very helpful for readers if either figure 2 (or alternatively text added to the results section) can make explicit the numbers who responded in each group, with the percentage that these represent (e.g. daytrip 248/384 (64.6%) vs. no daytrip 253/387 (65.4%)). We agree with the reviewer that the figure could be more detailed and instructive, adding the numbers of returned questionnaires per group per time point, and therefore we changed the figure, please find the new version attached. We hope to have answered all points raised by the reviewers satisfactorily and that you will consider our revised manuscript for publication in BMC Medical Research Methodology. On behalf of all co-authors, Yours sincerely, Lonneke van der Mark