Author's response to reviews Title:Developing the Moti-4 intervention, assessing its feasibility and pilot testing its effectiveness Authors: Hans B Dupont (h.dupont@mondriaan.eu) Paul Lemmens (p.lemmens@maastrichtuniversity.nl) Gerald Adriana (g.adriana@mondriaan.eu) Dike van de Mheen (vandemheen@ivo.nl) Nanne K de Vries (n.devries@maastrichtuniversity.nl) Version:2Date:10 March 2015 Author's response to reviews: see over
Maastricht, 2 March 2015 Victorino Silvestre Journal Editorial Office BioMed Central Dear Dr. Silvestre, We would like to thank you, Dr. Tsai and Dr. Schippers for the time and effort spent on reviewing our manuscript. Please find enclosed our revised manuscript entitled Mapping the Moti-4 intervention, assessing its feasibility and pilot testing its effectiveness by H. Dupont, P. Lemmens, G. Adriana, D. van de Mheen and N. de Vries, which we would like you to reconsider for publication in BMC Public Health. In this revised manuscript, we have systematically addressed all of the criticisms expressed by the reviewers. We feel that our revisions have led to a much improved manuscript. Below we describe in detail how we have addressed each criticism, giving our responses in italics. If there are any further questions, please let me know. Sincerely, Hans B. Dupont Mondriaan Prevention/Department of Health Promotion School for Public Health and Primary Care (CAPHRI) Maastricht University Maastricht, The Netherlands Correspondence to: J.B.H.M. Dupont Hoofd Preventie Mondriaan h.dupont@mondriaan.eu 1
Responses to Reviewer Comments on Mapping the Moti-4 intervention, assessing its feasibility and pre and post testing its effectiveness Reviewer 1 (Jennifer Tsai) Comments 1. The article needs to be proofread by a native English speaker. The manuscript has many phrases that are used incorrectly and many grammatical errors, which makes it hard to understand. The final version of this submission was proofread by a professional English language editor, resulting in a manuscript we hope is easier to understand. 2. There needs to be a better justification for why the Moti-4 is a better intervention than other previously identified cannabis/drug interventions. On page 4 (lines 73-82), the following text has been added: A general conclusion of the survey referred to above [7] was the need to fill the gap between more formal, high threshold treatment approaches and general prevention with brief interventions, targeting specific risk groups. There was no evidence-based intervention for this group available in the Netherlands. In order to fill this gap, a randomized controlled trial of an Australian two-session intervention called Adolescent Cannabis Check-Up (ACCU) [9] was recently replicated in a Dutch sample [10]. The ACCU was translated into Dutch as Weed-check. Contrary to the hypothesis, the Australian results [9] were not confirmed in the Netherlands [10]. We think that the strait-jacket character of the ACCU partly explains this discrepancy. The strict requirements in ACCU regarding fidelity to the protocol do not allow the interventionist to adapt and modify the intervention according to the client s individual needs, and thus limits its feasibility. One of the findings of our feasibility study, in which we interviewed professionals trained in the ACCU and the Moti-4, was that, in comparison with ACCU, the Moti-4 protocol gives professionals relative freedom to choose a tool in accordance with the level of the adolescent, the personal preference of the prevention worker and the local organisational framework (see p.8 & 9). This would seem to us to make Moti-4 more appropriate for the Dutch situation. 3. Description of the specific components of the Moti-4 intervention are hard to follow. It seems the 6 steps of Intervention Mapping is a great tool that helps explain the development of the program. However, the outcomes generated from each subsequent Intervention Mapping step is lost, and the specific component utilized in the pilot is unclear and needs to be explained. With these remarks in mind we have thoroughly edited the background chapter, focusing more on the real outcomes of the Intervention Mapping process: the manual, the protocol checklist of compulsory elements and the feasibility study. 2
4. There needs to be a clearer description of how the Moti-4 intervention was implemented in the pilot study. In what setting did this take place? How long did it take to implement all four sessions? Were the participants consented and compensated? Who implemented the curriculum?! We thank the reviewer for this critical observation. We have provided the following additions: On page 8 (ln. 182-188): In 2012, four prevention departments had adopted the intervention in the Dutch provinces of Brabant (Novadic Kentron), Limburg (Mondriaan and VVGi) and Overijssel (Tactus). Thirty experienced prevention workers (with a degree form a university of applied sciences) had been trained by the first author and an assistant to carry out Moti-4 for a pilot study. Previous training in motivational interviewing was a precondition for taking part. The training course involved going through the motions for the four sessions, the use of the tools, recruitment of the target group members, the theoretical background of Moti-4, and an explanation of the need for evaluative research. (p.11 line 272): Moti-4 was administered at the offices of the prevention agencies. Participants completed the four sessions within a month, with at least a one-week interval between the sessions. All participants signed an informed consent form before participating. They received vouchers as compensation for their time. On the recruitment of participants (p.7, line 174): Recruitment for the intervention focuses on finding vulnerable adolescents such as marginalized youths, truants, children of addicted parents, and youths attending special education. Adolescents are referred by their parents, by agencies for youth care and drop-outs, and by student counselors in the school system. 5. There seems to be a very large attrition rate (30%) that occurred between sessions. For a pilot study of 31 participants, this is a large drop out rate, and needs to be addressed. Statistical methods of how these individuals were handled during data analysis is warranted (e.g., did you get rid of them completely?). We excuse for this unclarity. We have replaced the text on page 8 (line 195) by: A major topic in the interviews was that of adolescents dropping out of the program. The attrition rate during the pilot study was estimated to be 30%; adolescents dropped out after the second, third or even fourth session. According to respondents, this might be explained by the severity and nature of the problems these youngsters were dealing with, and the fact that attending four sessions was too much of an effort for them. Several youngsters who were initially thought to be eligible were found not to have cannabis as their primary issue, but gaming, alcohol or the use of other substances. We therefore tightened our eligibility criteria, and found there was no more dropout between the sessions in the pilot study. All 31 participants took part in a pre- and post-test measurement. 3
6. Describe the scales used to measure the pre and post test questions. On p9 (line 231), the following description of the scale has been inserted: «A pilot study involving pre- and post-assessment was performed among non-treatment seeking adolescent cannabis users, using a self-report questionnaire consisting of seven items on socio-demographic data, including gender, living situation, level of education, nationality, and country of birth of mother and father; seven questions on cannabis use, alcohol and other drug use; and 24 items on the respondent s perceived behavioral control, social norm, attitude, and intentions. The primary outcome measure was the quantity of cannabis used, which was assessed by the question to estimate the amount of money spent a week (or if the respondent was growing cannabis themselves or getting it for free, they were asked to give an estimation). As a check on this question, the questionnaire also asked the respondent to estimate the number of joints (Dutch slang for cannabis cigarette) they had smoked. Since the potency of cannabis varies with the price, and the number of joints is not a reliable measure of use because the amount of cannabis rolled into a cigarette varies, the amount of money spent is the most reliable estimation in a self-report. The psychosocial determinants (i.e. attitude, perceived behavioral control, social influence, intention and action plans) of cannabis use were based on the I-Change model [25]. All determinants were assessed with items using five-point Likert answering scales, which were later combined into one variable for each determinant. Attitude was measured with eight items, four of which regarded the pros and four the cons of cannabis use. Perceived behavioral control was assessed by two questions (How difficult is it for you not to smoke cannabis?; How difficult is it for you to refuse a joint when a friend offers you one?). Social influence was assessed by one social modeling question, four questions on social norm and one question on perceived peer pressure. Three kinds of intentions were measured: the intention to use cannabis, the intention to quit and the intention to reduce cannabis consumption. Action plans were measured by three questions. 7. The introduction is confusing and too long. The manuscript starts by saying that cannabis use in Dutch adolescents has actually been decreasing in the past few years and that overall prevalence of cannabis in the Netherlands is low, which makes the intervention seem unwarranted (e.g., why a marijuana based intervention and not any other drug?). The authors can shorten the introduction and should set up the problem by discussing consequences associated with marijuana addiction sooner. We re sorry for the confusion here. Bearing the reviewer s remarks in mind, we thoroughly edited the background chapter and reduced it to produce a more balanced paper. 8. Limitations of the study s implications and changes made to the next round of Moti-4 implementation needs to be discussed in more detail. For example, why wasn t there any changes in motivation to smoke lesscannabis, but motivation to quit had changed significantly. Does this have to do with the 4
gaps in the intervention or are there other reasons that can explain this discrepancy (e.g., dependant upon stage of planned behavior)? To address this comment we inserted the following text in the discussion section: The experiences gained with the feasibility check and thepilot study have led to adjustments to the protocol eventually used for the RCT. Alternatives to the screening instruments (such as euro-adad and MATE-Y) have been added to the manual. We have tightened our eligibility criteria, and will also add tools that can be used for very low-skilled adolescents. In their recent edition of MI [27], Miller & Rollnick discouraged the use of the decisional balance. Nevertheless, it is regarded [27] as a potentially useful instrument when a client is in an early stage. Future versions of the Moti-4 handbook will take this into account. 9. The manuscript will benefit from more clarity and cohesiveness. We hope our adaptations provide a much improved version of the manuscript. Reviewer 2 (Gerard Schippers), Version:2Date:18 October 2014 : This paper presents the development, implementation, feasibility testing, and result measurement of a new brief motivational intervention for (problematic) cannabis users. General remarks: The paper is peculiar because it follows the general structure of a research paper, focussing on the outcomes of the intervention. It contains, however, a very large introductory section including an extensive report on the development of the instrument, the implementation in the Netherlands, and even a test of the feasibility by interviewing practitioners and clients. Because the study on the outcomes evaluation is rather limited, with a restricted pre-post design using untested instruments and outcome measures, the paper is somewhat out of balance. It would have been more appropriate to report separately on development, implementation, and feasibility testing. Although certainly exemplary and absolutely worthwhile while designing a new intervention and accounting for that, one could wonder whether the used arguments and background warrant an extensive report in a scientific journal, as is done. There are many reports on brief interventions (for alcohol ánd for cannabis) and this one does not add much new elements to it. The report on data on the feasibility and on the implementation, again, although worthwhile in itself for users and stakeholders, lacks a clear function within the context of this paper. This makes that the paper is out of balance. Where in the description of the intervention many details are provided, as to the steps to be taken in the sessions, and the items obligatory to discuss, none of this is presented in the empirical study (no manual fidelity is observed), the goals of the intervention are extensively explained in a matrix but only a few of them are included in the testing. As a major revision it is recommended that the paper is brought more in balance. 5
We addressed this reviewer s comments by thoroughly revising the background chapter and reducing it, to bring the paper more in balance. Minor essential points Title: Pre-post testing of effectiveness is impossible, other term should be used The title was changed to: Mapping the Moti-4 intervention, assessing its feasibility and pilot testing its effectiveness r.158-169: The section explaining for whom the intervention is meant might be better seen as part of Step 1: needs assessment than in Step 2: Behavioral and Learning Outcomes. We thank the reviewer for this observation. This section has been moved to the needs assessment section. r.193: on the decisional balance: Miller &Rollnick discourage this use in their recent edition of MI. The following text has been inserted in the discussion section: In their recent edition of MI [27], Miller & Rollnick discouraged the use of the decisional balance. Nevertheless, it is regarded [27] as a potentially useful instrument when a client is in an early stage. Future versions of the Moti-4 handbook will take this into account. r.292: Questioning whether participants diminished their drug use after the intervention, separate from the question whether they changed their motivation for drug use (in that order) is somewhat strange, since the two are highly related. Can behavior change without motivation change? We disagree with the reviewer here. Many behavior changes occur without a motivation change. Furthermore, the Transtheoretical Model of behavioural change, also known as the stages of change model (SOC, Proschaska & Diclimente, 1984) has been widely criticized in recent years (West, 2005, Etter & Sutton, 2002, Riemsma, Pattenden, Bridle, Sowden, Mather Watt. Walker, 2002, van Sluijs, van Poppel en van Mechelen, 2004). Some have even advocated a complete paradigm shift (West, 2005). r 279-283. The intervention Moti-4 does not seem to include an assessment for those participants who have a SUD and need to be referred for more intensive intervention. Argumentation for that is lacking. In what is now line 146 we have inserted the following sentence: If the prevention worker who does the screening experiences a sense of alarm and the adolescent is 6
assessed [30] to have a substance abuse disorder, he or she is referred to treatment as soon as possible. r 298-302. The instruments used are partly new, self-made and not tested. Discussion of their psychometric validity is lacking of very limited. (The amount of money spent on cannabis last week ; the operationalization of the motivational stage by three questions and the expectations regarding negative effects. It is not argumented why these instruments have been used. See our answer topoint 6 of the first reviewer r 314 Description of inclusion criteria and recruitment is very limited. For those who are not familiar with the activities of prevention workers in the Netherlands, this might be hard to understand. Were all participants in agreement with the characteristics described in r.161-169? We thank the reviewer for this critical observation, and have added some explanatory notes. On page 8: In 2012, four prevention departments had adopted the intervention in the Dutch provinces of Brabant (Novadic Kentron), Limburg (Mondriaan and VVGi) and Overijssel (Tactus). Thirty experienced prevention workers (with a degree form a university of applied sciences) had been trained by the first author and an assistant to carry out Moti-4 for a pilot study. Previous training in motivational interviewing was a precondition for taking part. The training course involved going through the motions for the four sessions, the use of the tools, recruitment of the target group members, the theoretical background of Moti-4, and an explanation of the need for evaluative research. (p.11 line 272): Moti-4 was administered at the offices of the prevention agencies. Participants completed the four sessions within a month, with at least a one-week interval between the sessions. All participants signed an informed consent form before participating. They received vouchers as compensation for their time. On the recruitment of participants (p.7, line 174): Recruitment for the intervention focuses on finding vulnerable adolescents such as marginalized youths, truants, children of addicted parents, and youths attending special education. Adolescents are referred by their parents, by agencies for youth care and drop-outs, and by student counselors in the school system. r.324 In the result section it is concluded that p<.05), a significant decrease was found for last week s frequency of use, but no significant change in the last month use indicator was seen. These seemingly contradictory results needs more explanation than is given. We apologize for the ambiguity here. As the intervention was post-tested after a week, asking about past-month use seems redundant, so we left it out in this version. Quality of written English:Needs some language corrections before being published 7
The final version of this submission was proofread by a professional English language editor, resulting in a manuscript we hope is easier to understand. 8