REGISTERED REPORTS AUTHOR AND REVIEWER GUIDELINES

Transcription

1 REGISTERED REPORTS AUTHOR AND REVIEWER GUIDELINES A Registered Reprt is a frm f empirical article ffered at Nature Human Behaviur in which the methds and prpsed analyses are pre-registered and reviewed prir t data cllectin. The frmat is ffered fr hypthesis-driven quantitative research with primary research data. High quality prtcls are prvisinally accepted fr publicatin befre data cllectin cmmences. This frmat is designed t minimize publicatin bias and research bias in hypthesis-driven research, while als allwing the flexibility t cnduct explratry (unregistered) analyses and reprt serendipitus findings. The review prcess fr Registered Reprts

2 Guidelines fr authrs The crnerstne f the Registered Reprts frmat is that a significant part f the manuscript will be assessed prir t data cllectin, with the highest quality submissins accepted in advance. Initial submissins will include a descriptin f the key research questin and backgrund literature, hyptheses, experimental prcedures, analysis pipeline, a statistical pwer analysis (r Bayesian equivalent), and pilt data (where applicable). Initial submissins will be triaged by the editrial team fr strength f scientific advance and suitability fr a brad, multidisciplinary audience. Thse that pass triage will then be sent fr indepth peer review (Stage 1). Fllwing review, the article will then be either rejected r accepted in principle fr publicatin. After in principle acceptance (IPA), the authrs will prceed t cnduct the study, adhering exactly t the peer-reviewed prcedures. When the study is cmplete the authrs will submit their finalised manuscript fr re-review (Stage 2) and will uplad their raw data, study materials, cmputer cde (if relevant) and labratry lg t a publicly accessible file-sharing service. Pending quality checks and a sensible interpretatin f the findings, the manuscript will be published regardless f the significance r directin f the results. Stage 1: Initial manuscript submissin and review Nature Human Behaviur aims t publish research f utstanding significance. Fr this reasn, the editrs select nly the mst scientifically prmising manuscripts fr in-depth peer review. Stage 1 submissins shuld include the manuscript (details belw) and a brief cver letter. Authrs are welcme t submit presubmissin enquires fr advice n the likely suitability f a study as a Registered Reprt. Hwever, please nte that we cannt cmmit t sending a manuscript fr indepth review until a cmplete Stage 1 submissin has been cnsidered. The cver letter shuld include: A brief scientific case fr cnsideratin. The jurnal aims t publish research that represents a significant scientific advance and is f relevance t a brad, multidisciplinary audience. High-value replicatin studies are welcme in additin t nvel studies. A statement cnfirming that all necessary supprt (e.g. funding, facilities) and apprvals (e.g. ethics) are in place fr the prpsed research. Nte that manuscripts will be generally cnsidered nly fr studies that are able t cmmence immediately; hwever authrs with alternative plans are encuraged t cntact the jurnal ffice fr advice. An anticipated timeline fr cmpleting the study if the initial submissin is accepted. A statement cnfirming that the authrs agree t share their raw data, any digital study materials, cmputer cde (if relevant), and labratry lg fr all published results. A statement cnfirming that if the authrs later withdraw their paper, they agree t the Jurnal publishing a shrt summary f the pre-registered study under a sectin Withdrawn Registratins. Manuscript preparatin guidelines Stage 1 Initial Stage 1 submissins shuld include the fllwing sectins: Intrductin A review f the relevant literature that mtivates the research questin and a full descriptin f the experimental aims and hyptheses. Please nte that fllwing IPA, the Intrductin sectin cannt be altered (see belw). Methds Full descriptin f prpsed sample characteristics, including criteria fr data

3 inclusin and exclusin (e.g. utlier extractin). Prcedures fr bjectively defining exclusin criteria due t technical errrs r fr any ther reasns must be specified, including details f hw and under what cnditins data wuld be replaced. A descriptin f experimental prcedures in sufficient detail t allw anther researcher t repeat the methdlgy exactly, withut requiring further infrmatin. These prcedures must be adhered t exactly in the subsequent experiments r any Stage 2 manuscript can be rejected. Prpsed analysis pipeline, including all preprcessing steps, and a precise descriptin f all planned analyses, including apprpriate crrectin fr multiple cmparisns. Any cvariates r regressrs must be stated. Where analysis decisins are cntingent n the utcme f prir analyses, these cntingencies must be specified and adhered t. Only pre-planned analyses can be reprted in the main Results sectin f Stage 2 submissins. Hwever, unplanned explratry analyses will be admissible in a separate sectin f the Results (see belw). Studies invlving Neyman-Pearsn inference must include a statistical pwer analysis. Estimated effect sizes shuld be justified with reference t the existing literature. Since publicatin bias verinflates published estimates f effect size, pwer analysis must be based n the lwest available r meaningful estimate f the effect size. Fr frequentist analysis plans, the a priri pwer must be 0.95 r higher fr all prpsed hypthesis tests. In the case f highly uncertain effect sizes, a variable sample size and interim data analysis is permissible but with inspectin pints stated in advance, apprpriate Type I errr crrectin fr peeking emplyed, and a final stpping rule fr data cllectin utlined. Methds invlving Bayesian hypthesis testing are encuraged. Fr studies invlving analyses with Bayes factrs, the predictins f the thery must be specified s that a Bayes factr can be calculated. Authrs shuld indicate what distributin will be used t represent the predictins f the thery and hw its parameters will be specified. Fr example, will yu use a unifrm up t sme specified maximum, r a nrmal/half-nrmal t represent a likely effect size, r a JZS/Cauchy with a specified scaling cnstant? Fr inference by Bayes factrs, authrs must be able t guarantee data cllectin until the Bayes factr is at least 10 times in favur f the experimental hypthesis ver the null hypthesis (r vice versa). Authrs with resurce limitatins are permitted t specify a maximum feasible sample size at which data cllectin must cease regardless f the Bayes factr; hwever t be eligible fr advance acceptance this number must be sufficiently large that incnclusive results at this sample size wuld nevertheless be an imprtant message fr the field. Fr further advice n Bayes factrs r Bayesian sampling methds, prspective authrs are encuraged t read this key article by Schönbrdt and Wagenmakers. Full descriptins must be prvided f any utcme-neutral criteria that must be met fr successful testing f the stated hyptheses. Such quality checks might include the absence f flr r ceiling effects in data distributins, psitive cntrls, r ther quality checks that are rthgnal t the experimental hyptheses. Timeline fr cmpletin f the study and prpsed resubmissin date if Stage 1 review is successful. Extensins t this deadline can be negtiated with the handling editr. Any descriptin f prspective methds r analysis plans shuld be written in future tense. Pilt Data Optinal. Can be included t establish prf f cncept, effect size estimatins, r feasibility f prpsed methds. Any pilt experiments will be published with the final versin f the manuscript and will be clearly distinguished frm data btained fr the pre-registered experiment(s).

4 Stage 1 submissins that are judged by the editrs t be f sufficient quality and scientific imprtance will be sent fr in-depth peer review. In cnsidering papers at the registratin stage, reviewers will be asked t assess: 1. The imprtance f the research questin(s) and relevance fr a brad, multidisciplinary audience. 2. The extent t which the prpsed study can satisfactrily answer the research questin(s). 3. The lgic, ratinale, and plausibility f the prpsed hyptheses. 4. The sundness and feasibility f the methdlgy and analysis pipeline (including statistical pwer analysis where apprpriate). 5. Whether the clarity and degree f methdlgical detail is sufficient t exactly replicate the prpsed experimental prcedures and analysis pipeline. 6. Whether the authrs have pre-specified sufficient utcme-neutral tests fr ensuring that the results btained are able t test the stated hyptheses, including psitive cntrls and quality checks. Fllwing Stage 1 peer review, manuscripts will be rejected utright, ffered the pprtunity t revise, r accepted. Prpsals that exceed the highest standards f imprtance and scientific rigur will be issued an in principle acceptance (IPA), indicating that the article will be published pending cmpletin f the apprved methds and analytic prcedures, passing f all pre-specified quality checks, and a defensible interpretatin f the results. Stage 1 prtcls are nt published fllwing IPA. Instead they are held in reserve by the jurnal and integrated int a single cmpleted article fllwing apprval f the final Stage 2 manuscript. Authrs are reminded that any deviatin frm the stated experimental prcedures, regardless f hw minr it may seem t the authrs, culd lead t rejectin f the manuscript at Stage 2. In cases where the pre-registered prtcl is altered after IPA due t unfreseen circumstances (e.g. change f equipment r unanticipated technical errr), the authrs must cnsult the editrs immediately fr advice, and prir t the cmpletin f data cllectin. Minr changes t the prtcl may be permitted accrding t editrial discretin. In such cases, IPA wuld be preserved and the deviatin reprted in the Stage 2 submissin. If the authrs wish t alter the experimental prcedures mre substantially fllwing IPA but still wish t publish their article as a Registered Reprt then the manuscript must be withdrawn and resubmitted as a new Stage 1 submissin. Nte that registered analyses must be undertaken, but additinal unregistered analyses can als be included in a final manuscript (see belw). Stage 2: Full manuscript review Once the study is cmplete, authrs prepare and resubmit their manuscript fr full review, with the fllwing additins: Submissin f raw data and labratry lg Raw data, any digital experimental materials (e.g. stimuli etc.), and cmputer cde (if relevant) must be made freely available in a public repsitry. Data files shuld be apprpriately time stamped t shw that data was cllected after IPA and nt befre. Other than pre-registered and apprved pilt data, n data acquired prir t the date f IPA is admissible in the Stage 2 submissin. Raw data must be accmpanied by guidance ntes, where required, t assist ther scientists in replicating the analysis pipeline. Authrs are als expected t uplad any relevant analysis scripts and ther experimental materials that wuld assist in replicatin. Supplementary figures, tables, r ther text (such as supplementary methds) shuld be included as standard supplementary infrmatin that accmpanies the paper (they can als be archived tgether with the data). The raw data itself shuld

5 be archived (see abve) rather than submitted t the jurnal as supplementary material. The authrs must cllectively certify in the resubmissin Cver Letter that all nnpilt data was cllected after the date f IPA. A basic labratry lg must als be prvided utlining the range f dates during which data cllectin tk place. This lg shuld be upladed t the same public archive as the data, with a link prvided t the lg in the resubmissin Cver Letter. Backgrund, Ratinale and Methds Apart frm minr stylistic revisins, the Intrductin cannt be altered frm the apprved Stage 1 submissin, and the stated hyptheses cannt be amended r appended. At Stage 2, any descriptin f the ratinale r prpsed methdlgy that was written in future tense within the Stage 1 manuscript shuld be changed t past tense. Any textual changes t the Intrductin r Methds (e.g. crrectin f typgraphic errrs) must be clearly marked in the Stage 2 submissin. Any relevant literature that appeared fllwing the date f IPA shuld be cvered in the Discussin. Results & Discussin The utcme f all registered analyses must be reprted in the manuscript, except in rare instances where a registered and apprved analysis is subsequently shwn t be lgically flawed r unfunded. In such cases, the authrs, reviewers, and editr must agree that a cllective errr f judgment was made and that the analysis is inapprpriate. In such cases the analysis wuld still be mentined in the Methds but mitted with justificatin frm the Results. It is reasnable that authrs may wish t include additinal analyses that were nt included in the registered submissin. Fr instance, a new analytic apprach might becme available between IPA and Stage 2 review, r a particularly interesting and unexpected finding may emerge. Such analyses are admissible but must be clearly justified in the text, apprpriately caveated, and reprted in a separate sectin f the Results titled Explratry analyses. Authrs shuld be careful nt t base their cnclusins entirely n the utcme f statistically significant pst hc analyses. Authrs reprting null hypthesis significance tests are required t reprt exact p values and effect sizes fr all inferential analyses. The resubmissin will mst likely be cnsidered by the same reviewers as in Stage 1, but culd als be assessed by new reviewers. In cnsidering papers at Stage 2, reviewers will be asked t decide: 1. Whether the data are able t test the authrs prpsed hyptheses by satisfying the apprved utcme-neutral cnditins (such as quality checks, psitive cntrls) 2. Whether the Intrductin, ratinale and stated hyptheses are the same as the apprved Stage 1 submissin (required) 3. Whether the authrs adhered precisely t the registered experimental prcedures 4. Whether any unregistered pst hc analyses added by the authrs are justified, methdlgically sund, and infrmative 5. Whether the authrs cnclusins are justified given the data Reviewers are infrmed that editrial decisins will nt be based n the perceived imprtance, nvelty r cnclusiveness f the results. Thus while reviewers are free t enter such cmments n the recrd, they will nt influence editrial decisins. Reviewers at Stage 2 may suggest that authrs reprt additinal pst hc tests n their data; hwever authrs are nt bliged t d s unless such tests are necessary t satisfy ne r mre f the Stage 2 review criteria. Manuscript withdrawal and Withdrawn Registratins

6 It is pssible that authrs with IPA may wish t withdraw their manuscript fllwing r during data cllectin. Pssible reasns culd include majr technical errr, an inability t cmplete the study due t ther unfreseen circumstances, r the desire t submit the results t a different jurnal. In all such cases, manuscripts can f curse be withdrawn at the authrs discretin. Hwever, the jurnal will publicly recrd each case in a sectin called Withdrawn Registratins. This sectin will include the authrs, prpsed title, the abstract frm the apprved Stage 1 submissin, and brief reasn(s) fr the failure t cmplete the study. Partial withdrawals are nt pssible; i.e. authrs cannt publish part f a registered study by selectively withdrawing ne f the planned experiments. Such cases must lead t withdrawal f the entire paper. Studies that are nt cmpleted by the agreed Stage 2 submissin deadline (which can be extended in negtiatin with the editrial ffice) will be cnsidered withdrawn and will be subject t a Withdrawn Registratin. Incremental Registratins Authrs may add experiments t apprved submissins. In such cases the apprved Stage 2 manuscript will be accepted fr publicatin, and authrs can prpse additinal experiments fr Stage 1 cnsideratin. Where these experiments extend the apprved submissin (as ppsed t being part f new submissins), the editrial team will seek t fast-track the review prcess. This ptin may be particularly apprpriate where an initial experiment reveals a majr serendipitus finding that warrants fllw-up within the same paper. In cases where an incremented submissin is rejected (at either Stage 1 r 2), authrs will retain the ptin f publishing the mst recently apprved versin f the manuscript. Fr further advice n specific scenaris fr incremental registratin, authrs are invited t cntact the editrial ffice.

7 Guidelines fr reviewers The review prcess fr Registered Reprts is divided int tw stages. At Stage 1, reviewers assess study prpsals befre data are cllected. At Stage 2, reviewers cnsider the full study, including results and interpretatin. Stage 1 manuscripts will include nly an Intrductin, Methds (including prpsed analyses), and Pilt Data (where applicable). In cnsidering papers at Stage 1, reviewers will be asked t assess: 1. The imprtance f the research questin(s) and the relevance f the research fr a brad, multidisciplinary audience. 2. The lgic, ratinale, and plausibility f the prpsed hyptheses. 3. The sundness and feasibility f the methdlgy and analysis pipeline (including statistical pwer analysis where apprpriate). 4. Whether the clarity and degree f methdlgical detail is sufficient t exactly replicate the prpsed experimental prcedures and analysis pipeline. 5. Whether the authrs have pre-specified sufficient utcme-neutral tests fr ensuring that the results btained are able t test the stated hyptheses, including psitive cntrls and quality checks. Fllwing Stage 1 peer review, manuscripts will be accepted, ffered the pprtunity t revise, r rejected utright. Manuscripts that pass peer review will be issued an in principle acceptance (IPA), indicating that the article will be published pending successful cmpletin f the study accrding t the pre-registered methds and analytic prcedures, as well as a defensible and evidence-based interpretatin f the results. Fllwing cmpletin f the study, authrs will cmplete the manuscript, including Results and Discussin sectins. These Stage 2 manuscripts will mre clsely resemble a regular article frmat. The manuscript will then be returned t the reviewers, wh will be asked t appraise: 1. Whether the data are able t test the authrs prpsed hyptheses by satisfying the apprved utcme-neutral cnditins (such as quality checks, psitive cntrls) 2. Whether the Intrductin, ratinale and stated hyptheses are the same as the apprved Stage 1 submissin (required) 3. Whether the authrs adhered precisely t the registered experimental prcedures 4. Whether any unregistered pst hc analyses added by the authrs are justified, methdlgically sund, and infrmative 5. Whether the authrs cnclusins are justified given the data Reviewers at Stage 2 may suggest that authrs reprt additinal pst hc tests n their data; hwever authrs are nt bliged t d s unless such tests are necessary t satisfy ne r mre f the Stage 2 review criteria. Please nte that editrial decisins will nt be based n the perceived imprtance, nvelty, r cnclusiveness f the results.