Please find our point-by-point response to the reviewers comments in the table below: Reviewer 1

Transcription

1 Please find our point-by-point response to the reviewers comments in the table below: Reviewer 1 Reviewer comment Author response Analysis of the evidence for the add-on treatment in UK fertility centres This is an interesting approach to highlight the lack of evidence for much of what is being offered to fertility patients from fertility clinics in the United Kingdom. I am sure that this is no worse than other centres in Europe, the US and in Australia and NZ. We thank the reviewer for her comments and agree with the comment. No action required The problem with the evidence in much of the literature is widespread confusion about the outcomes and the best way of reporting them. 1. The outcomes Many studies only report clinical pregnancy rate which is inadequate for a number of reasons but mostly because of the difference from live birth rate. About 5% of pregnancies are lost between 1st trimester and birth. Reference Clarke JF, van Rumste, MM, Farquhar CM, Johnson NP Mol BWM. Herbison P. Measuring outcomes in fertility trials - can we rely on clinical pregnancy rates? Fertil Steril Oct;94(5): Epub 2010 Jan 13.PMID: We agree with this point and have added on the issue of losses and the reference the following (page 6, line 4) Furthermore, the key patient-relevant outcome for assisted reproductive health should be live birth; however, guidance often refers to pregnancy rates to form recommendations, which is inadequate for a number of reasons, not least because about 5% of pregnancies are lost between the1st trimester and birth. i

2 2. The denominator The second problem is the denominator as per embryo transfer is common but many treatments don't get to the stage of an embryo transfer and the denominator is smaller and using a smaller denominator overinflates the size of the benefit. And some treatments reduce the number of embryos such as preimplantation genetic diagnosis (at least the 1.0 version and the jury is still out for the PGS 2.0 as no satisfactory RCT done yet) See reference. Jack Wilkinson1,2,*, Stephen A. Roberts1, Marian Showell3, Daniel R. Brison4, and Andy Vail1,2 No common denominator: a review of outcome measures in IVF RCTs. Human Reproduction, pp. 1 9, 2016 doi: /humrep/dew Many of the studies have undeclared COIs. But I don't have much evidence for this as they are not required to be disclosed. But with companies doing their own studies on new stuff such as GM-CSF and time lapse systems and ERA then there are fairly obvious opportunity for bias. Here is an example Eg Ziebe See Zeibe 2013 GM-CSF additive for human embryo culture media marketed as EmbryoGen by ORIGIO we made comments to the Journal about the analysis and conduct of the study which you could search for but no action. They have a forum page. There has been no response from the journal or the authors of the paper. We agree with this point but to not add complexity to the article we have focused on the live birth issue and have added the following (page 6. line 6): Adoption of the IMPRINT modifications to the CONSORT checklist, which aim to improve the reporting of infertility treatments, should be mandatory for clinical trials if they are to prove meaningful. Reference Improving the reporting of clinical trials of infertility treatments (IMPRINT): modifying the CONSORT statement. Hum Reprod 2014;29: We looked at this when assessing the quality of the evidence we found and overall the field seems full of undisclosed interests. In an area where services are mixed between NHS and profit-making provision, this is not surprising, but still very poor practice.

3 Re the gaps in the Cochrane reviews we are aware of several of these and in fact do have titles in progress for GMCSF, freeze all cycles, intralipid, the intravaginal incubator and ERA. It is just hard to keep up with all the new and slightly unusual interventions that keep on popping up. This paper is very useful and will help us set next years priorities for our group and reminds us that women need this information. We thank the reviewers for this comment and having a paper that is used for prioritisation is great news: No action required Regarding the scope. I am not sure that patients with recurrent miscarriage should be in here I know that are frequently seen in reproductive medicine clinics but there are a range of other providers and they don't fit the definition of infertility and the treatments are different from infertility treatments. We have clarified the title as the terminology of add-on in the title, which may have been providing some confusion. The title is now: Analysis of the evidence for interventions additional to IVF treatment in UK fertility centres. Also we have clarified this in the text: page 1, last paragraph A range of additional investigations and treatments are offered at UK fertility treatment centres beyond standard IVF that may incur extra costs, [ref bmjopen R1] which can range from as little as 50 for a single screening blood test to as much as 8,000 for egg freezing packages (see Box 1). All of the included treatments are therefore available from fertility centres as defined in our accompanying paper. A further area of confusion is that many treatment offered to recurrent miscarriage patients are also offered as IVF treatment. For example, for Intralipid, the review is in recurrent miscarriage patients, not those undergoing IVF. But we also found a case control study in IVF patients, which shows no effect and yet this treatment is offered to IVF patients. In addition many treatments are offered to IVF couples with miscarriage as part of the centres treatments; therefore it would be wrong of us to exclude these treatments. As an example, endometrial

4 scratching is supposed to be offered in only those with recurrent failure although it seems to be offered outside this indication. There was a IUI survey which show lack of adherence to NICE GL. Kim D, Child T, Farquhar C. Intrauterine insemination: a UK survey on the adherence to NICE clinical guidelines by fertility clinics. BMJ Open 2015;5:e doi: /bmjopen We have added the following to the discussion (page 8, line 3): Furthermore, clinics awareness and uptake of guidance needs improving: a recent survey of 46 UK clinics, licensed to provide IUI, found that whilst the majority of clinics were aware of NICE guidance only ten clinics reduced the number of IUI cycles or restricted its use according to NICE recommendations. ii Finally, your paper has highlighted a lot of Cochrane reviews and some of them are incorrect or are not the most up to date version. We have taken the opportunity to give you the correct version. We have updated the table and the references as per each of the points below.

5 Incorrect ref cited in paper Correct Cochrane ref Heineman 2014 Bontekoe S, Johnson N, Blake D. Adherence compounds in embryo transfer media for assisted reproductive technologies. Cochrane Database of Systematic Reviews 2014, Issue 2. Art. No.: CD DOI: / CD pub3. Results corrected, citations corrected. Siristatidis 2011 Siristatidis CS, Basios G, Pergialiotis V, Vogiatzi P. Aspirin for in vitro fertilisation. Cochrane Database of Systematic Reviews 2016, Issue 11. Art. No.: CD DOI: / CD pub4. Update Published today! Glujovsky 2012 Glujovsky D, Farquhar C, Quinteiro Retamar AM, Alvarez Sedo CR, Blake D. Cleavage stage versus blastocyst stage embryo transfer in assisted reproductive technology. Cochrane Database of Systematic Reviews 2016, Issue 6. Art. No.: CD DOI: / CD pub5. Updated in June 2016 Kwan 2014 (correct except BMJ version has epub reference) Kwan I, Bhattacharya S, Kang A, Woolner A. Monitoring of stimulated cycles in assisted reproduction (IVF and ICSI). Cochrane Database of Systematic Reviews 2014, Issue 8. Art. No.: CD DOI: / CD pub3. BMJ also cites non-cochrane review: Martins 2014 Van Peperstraten 2008 Proctor M, Johnson N, van Peperstraten AM, Phillipson G. Techniques for surgical retrieval of sperm prior to intra-cytoplasmic sperm injection (ICSI) for azoospermia. Cochrane Database of Systematic Reviews 2008, Issue 2. Art. No.: CD DOI: / CD pub3. BMJ also cites non-cochrane review: Abhyankar 2016

6 We also note that these Cochrane reviews were overlooked. Fresh versus frozen embryo transfer Wong KM, van Wely M, Van der Veen F, Repping S, Mastenbroek S. Fresh versus frozen embryo transfers for assisted reproduction (Protocol). Cochrane Database of Systematic Reviews 2014, Issue 7. Art. No.: CD DOI: / CD Modified natural cycle IVF (Gentle vs light) Allersma T, Farquhar C, Cantineau AEP. Natural cycle in vitro fertilisation (IVF) for subfertile couples. Cochrane Database of Systematic Reviews 2013, Issue 8. Art. No.: CD DOI: / CD pub2. PGS Twisk M, Mastenbroek S, van Wely M, Heineman MJ, Van der Veen F, Repping S. Preimplantation genetic screening for abnormal number of chromosomes (aneuploidies) in in vitro fertilization or intracytoplasmic sperm injection, Cochrane Database of Systematic Reviews 2006, Issue 1. Art. No.: CD DOI: / CD pub2.; last assessed as up-to-date 14 July Fresh vs frozen embryo transfer: This is a protocol for a review and there are no results. We have added it to the references, but it makes no difference to the results. Modified natural cycle IVF: We referred to the review in the table: We did find a Cochrane review on natural IVF (Allersma et al). We did miss that this review includes 2 trials of modified natural cycle we have therefore added this to the references and amended the figures to match. PGS: we have already included this review under PGS version 1 (reference 14 in the full text) A 2006 Cochrane review of preimplantation genetic screening (PGS) included 9 trials, and concluded PGS significantly lowered live birth rate. The reference however does not include the upto date assessment which we have included. We shall add this as personal communication in the text. The Cochrane review on PGS from 2010 needs updating but we are aware of only one study on the newer PGS techniques and it is not well done. The two arms have different embryo transfer policies and claim to reduce multiple pregnancies but then we know that single embryo transfer reduces multiple pregnancies anyway. So it achieved very little. Other minor comments Table 1 was a challenge to read. It was too wide for portrait. I couldn't read the right hand column completely. We formatted this in landscape as this facilitates reading. The pdf version also removed some of the margin and this table is designed only for the web.

7 There was no search date this needs to be added. I couldn t read to the bottom of box one We have added search dates to the text. We shall re-upload the box Reviewer: 2 This paper addresses an important and controversial issue. The majority of patients requiring IVF and other forms of assisted conception either have to or choose to fund their own treatment, and a number of adjuncts have been developed with variable quantity and quality of evidence, which must inevitably, also be paid for. Given the emotive nature of this treatment there is considerable potential for exploitation, intentional or unintentional. The authors of this paper are aiming to address the quality of evidence associated with add-ons, and to make suggestions as to how this problem could be addressed by the regulator. This is very welcome However, the paper loses some of its power due to the inclusion of treatments that fall outside of the scope of "add-ons". Thank you. The methodology has led them to five interventions for which there may be some evidence of improvement in live birth rates. One of these is intrauterine insemination in a natural cycle. This is an alternative treatment, rather than an add-on, and usually not one applied to the population who require IVF, and certainly not one intended to increase live birth rate. We agree with this point but it is an intervention that is available from some private fertility clinics. Costs vary, but usually range from around 500 to 1,000 for each cycle of treatment. To make this clear we have added these costs to the Box 1 example of costs. In addition, we have cited the survey on IUI (in response to reviewer 1) which shows most clinics are aware of NICE guidance in restricting its use but do not currently apply it

8 Ovulation induction and cycle monitoring, which is included in the overall list of add-ons, is similarly a different treatment modality, and not an Add-on. Box 1, the example of costs, includes percutaneous epididymal sperm aspiration/ testicular pserm extraction. Again, this is not an add-on as such, but an essential element of treatment for men with obstructive azoospermia or testicular failure. It attracts an additional cost, but only for those for whom it is essential. This is the same issue as above and previous point. We have amended the title and included in the text the following: A range of additional investigations and treatments are offered at UK fertility treatment centres beyond standard IVF that may incur extra costs, [ref bmjopen r1] which can range from as little as 50 for a single screening blood test to as much as 8,000 for egg freezing packages (see Box 1). To be clear, if they are essential (an important point that we agree with) then the question is why are these treatments so expensive and routinely offered on clinics websites? Of note in the text we do make the following point: NICE guidance is generally clear about the populations to which the recommendations apply. For example, sperm cryopreservation should be offered to men and adolescent boys who are preparing for medical treatment for cancer that is likely to make them infertile. Similarly, oocyte or embryo cryopreservation should be offered to those women preparing for medical treatment for cancer that is likely to make them infertile, and the routine measurement of thyroid function should not be offered to the general population but should be confined to women with symptoms of thyroid disease. It is, however, important that we do review the evidence for these treatments, as we are aware that there is intervention creep in that in the private setting treatments are available and offered beyond their supposed treatment population. One of the best examples of this is egg freezing

9 Hysteroscopy (page 3, line 34 onwards) is a diagnostic procedure. NICE guidance runs as follows: Women should not be offered hysteroscopy on its own as part of the initial investigation unless clinically indicated because the effectiveness of surgical treatment of uterine abnormalities on improving pregnancy rates has not been established. [2004] Hysteroscopy is not intended to increase pregnancy rates in the sense of other addons. There may be some dispute with regard to the significance of uterine abnormalities, but it still does not fit the definition of an add-on. It is a quite separate procedure from the IVF, and is usually part of the process of understanding why natural conception has not taken place. Other treatments included in the list, which are not intended to increase live birth rate, but serve another practical purpose, include ovarian tissue freezing, a means of fertility preservation, similarly egg, embryo and sperm freezing, and vitirification, which is merely an alternative method of cryopreservation. Sperm freezing is essential in the use of donated sperm, for screening purposes, and may also be used where the male partner is likely to have difficulties providing a sample on the day of treatment, not to increase the live birth rate. We cite the guidance in the table and we agree with the point, and we state the following in the text: NICE guidance also clearly states recommendations on interventions that should not be offered: assisted hatching and hysteroscopy are not recommended because they have not been shown to improve pregnancy rates. This is how clinics refer to hysteroscopy ( itself ): There is some evidence that this procedure itself, or performing an endometrial scratch, can increase implantation rates." Therefore clinics are marketing this procedure as potentially increasing pregnancy rates and currently there is confusion about what is offered by which clinics to whom, which is one of the points of our piece. As for sperm freezing the indication for this is not clear: it is available for instance in some clinics if the quality of your sperm is deteriorating. and you may also want to consider semen freezing if your partner is having IUI or IVF treatment. It is therefore important to recognise that these treatments are being offered in the context of improving live birth rates, are being charged for, and therefore it is important we evaluate their effectiveness. See also next point. Pre-implantation genetic diagnosis (PGD), as opposed to pre-implantation genetic screening (PGS), is intended to avoid the birth of children with serious genetic diseases/conditions, not to improve the live birth rates. PGD is stated as intended for couples with genetic defects. But clinics state their success rates for PGD based on success in terms of babies born. For example: We are the largest and most successful unit in the UK for PGD. We have had

10 over 600 babies born as a result of this treatment." And also PGD is marketed to non-affected couples: More recently, PGD technology has been applied to the IVF embryos of infertile couples undergoing IVF treatment in the hope that the identification of chromosomally normal embryos for transfer will improve IVF success rates. This also suggests indication creep. Moreover: PGD may be especially worthwhile in women with a history of miscarriage, failed IVF cycles and in those of an older maternal age. We therefore consider again it is important for the public to know the benefits, the harms and the costs of these treatments as they are currently marketed. We consider the clarification of the title goes some way to addressing this point but we have also added the following to the text: (Page 7, paragraph 4) There is an incoherent framework as to which treatments should be offered in which populations, the potential for indication creep is therefore significant: expanding treatment use from people who have clearly benefited, to those where the evidence is much more shaky or non-existent. Finally, the mention of the ovarian tumours as a risk relating to ICSI (page 6, 47-50) should be clarified, as this is identified by NICE as being potentially associated with IVF and ICSI, and not specific to ICSI, as the paper implies. Amended.

11 For this paper to have the intended impact, the authors need to decide whether to include all procedures for which centres are charging a fee, other than IVF, or whether as the title suggests, they are going to stick to add-ons, specifically those for which claims of increased live birth rates are made. The message they seek to purvey is undoubtedly both important and sound when the argument is narrowed down. However, in its current form it could easily be dismissed by those it targets as having been written by those with no understanding of the field or the technology (something that would actually be a strength if the paper were rationalised.) The conclusions with regard to possible actions by NICE and the HFEA are sound, and anticipate a move by the HFEA to return to live births as a headline figure as opposed to pregnancy rates. For the sake of patients this subject should be aired, and I hope the authors will understand that my comments are intended to strengthen their argument. We started from a viewpoint of assessing all those treatments that are on offer and make claims of enhanced benefit: Our linked BMJ Open paper reports that: Many fertility centres in the UK offer a range of treatments in addition to standard IVF procedures, and for many of these interventions claims of benefit are made. In most cases the claims are not quantified and evidence is not cited to support the claims. Importantly we found 276 claims of benefit relating to 41 different fertility interventions made by 60 of the 74 centres. We therefore firstly consider that this paper in BMJ open should be clearly linked to from this present analysis - particularly the tables and figures. A considerable number of treatments are currently offered, with evidence of indication creep, and that the public should be made aware of all of the evidence about these treatments. Our approach is sound because we approach the evidence from the perspective of what is on offer, not where the evidence exists. To not do this would therefore be a significant mistake. Reviewer 1 backs this up when she refers to how useful it will be to prioritise the upcoming Cochrane reviews. In terms of a perceived lack of expertise - we are experts in evidence synthesis and also have expertise in device regulation. According to the HFEA, as GPs we are the go-to point of advice. There are significant problems in this field that we have not encountered previously. For example, guidelines for reporting live birth rates have only appeared in the last 2 years and there are multiple ways of referring to the same treatment modality. The lack of evidence for treatments offered in the UK is deeply worrying, and treatments that should not be offered - and are not recommended by NICE - are currently commonly available.

12 In our linked paper we also point out that The situation in countries without such regulatory authorities as the HFEA may be worse. We consider our approach is important as it highlight the worrying lack of recognition of the discordance between the private and public sector, and the disturbing trend for offering more and more treatments to couples with infertility problems. For example, over the past 3 years, the number of PGS cases in the USA has increased by >30% per year, resulting in the application of PGS in about 8% of all IVF treatments (Brown, 2014). One company claims on its website to have performed over one quarter of a million clinical PGS biopsies with their array platform alone (Website Blue Gnome, 2014). We therefore note this point but consider we should not weaken the approach by taking out treatments that are currently available to patients seeking help with reproduction in the UK. We hope that as a consequence of our work there will be some coherence in a forward looking policy amongst IVF experts. i Clarke JF, van Rumste, MM, Farquhar CM, Johnson NP Mol BWM. Herbison P. Measuring outcomes in fertility trials - can we rely on clinical pregnancy rates? Fertil Steril Oct;94(5): Epub 2010 Jan 13.PMID: ii Kim D, Child T, Farquhar C. Intrauterine insemination: a UK survey on the adherence to NICE clinical guidelines by fertility clinics. BMJ Open 2015;5:e doi: /bmjopen