BMJ - Decision on Manuscript ID BMJ

Transcription

1 BMJ - Decision on Manuscript ID BMJ

2 Body: 11-Aug-2017 Dear Dr. Wallis, Manuscript ID BMJ entitled "A comparison of post-operative outcomes among patients treated by male and female surgeons" Thank you for sending us your paper. We sent it for external peer review and discussed it at our manuscript committee meeting. There is a high level of interest in this topic. We would like to proceed with the paper but request that you revise it, as explained below, before we make a final decision. We are looking forward to reading the revised version. Very truly yours, Elizabeth Loder, MD, MPH eloder@bmj.com *** PLEASE NOTE: This is a two-step process. After clicking on the link, you will be directed to a webpage to confirm. *** a **Report from The BMJ s manuscript committee meeting** These comments are an attempt to summarise the discussions at the manuscript meeting. They are not an exact transcript. Decision: Put points Present at the meeting: Jose Merino (chair); Jamie Kirkham (statistician); Georg Roeggla; Wim Weber; Tiago Villanueva; Daoxin Yin; Sophie Cook; John Fletcher; Elizabeth Loder * We thought this was an interesting and reasonable research question. There is reason to believe that outcomes might be different between male and female surgeons for a number of reasons. The RQ does not lend itself to a clinical trial, so observational data are the best we have. * Our main concern is the present interpretation of the data. Our statistician noted, and other editors agreed, that "the conclusion that the composite and death rates are lower if surgeries are performed by females is largely based on weak statistical significance, noting that nearly all the confidence intervals are 0.99 or thereabouts. In fact when the full data is utilised (Supp Table 3), nearly all CI s tightly surround or are close to 1 and the difference in the single outcome death disappears." * You do conclude in the paper that the difference is small, but many editors still felt that the tone of the paper exaggerated the results. We thought you should do more to emphasise the small difference and acknowledge more vigorously that perhaps this is a chance finding one reviewer also comments that the outcome cannot be truly assigned to a single person performing the surgery, as surgeries involve a team, including the post-op care. This seems a valid point, which isn t considered given the outcome of interest is 30-day mortality.

3 * We feel strongly that a subgroup analysis looking at emergent vs elective surgery would be of great importance as a way of understanding what might be going on. * You might also provide more information about the degree to which Canadians have a choice about their surgeon. We were surprised to learn a number of years ago when we published another surgical paper, that in Sweden patients have no ability to choose their surgeon. One of our editors tells us this is also the case in Portugal. Is this true in Canada, or effectively true in rural areas perhaps, where there may be only a single surgeon? Our statistician had some additional points for consideration: 1) Please explain the purpose of the sample size calculation and what hypothesis this was based on in relation to potential differences in outcome between M/F surgeons. 2) The number of analyses in the stats analysis section and the unit of analysis is confusing. This section needs more attention. 3) He was not familiar with the term logit wing - do you mean link? 4) He notes you excluded repeat procedures. He comments "This is usually done as a convenience to simply the analysis which is acceptable, usually if the number of repeats is small. However, there were over 280k such exclusions which was substantial in this case. This is a potential further level of clustering. It would be interesting to know at the very least, some descriptive stats of these repeats in the context of this RQ. For example, what was the final outcome, were the repeats performed by the same surgeon or a different surgeon - was there any switching between M/F performing the repeat surgery?" Another editor noted that "An important issue is whether specific factors may affect outcome after surgery Is there a biological plausibility to explain any observed differences? Or differences in training and approach that differ by the sex of the surgeon? Are there real differences in the surgical approach and philosophy between men and women surgeons? Or are women surgeons more similar to surgeons who are men than to women internists? What is the underlying hypothesis that would explain these differences? The importance of having well developed hypotheses to justify these analyses to keep them from turning inot fishing expeditions was highlighted by this paper: See Transplant Proc Oct;42(8): doi: /j.transproceed Sign of the Zodiac as a predictor of survival for recipients of an allogeneic stem cell transplant for chronic myeloid leukaemia (CML): an artificial association. Szydlo RM1, Gabriel I, Olavarria E, Apperley J." Another editor had additional comments related to the interpretation of results: "The results have quite narrow confidence interval and this is usually looked on as a good thing! Here though the tiny difference just crawls into the realm of statistical significance and I agree with others that this is over interpreted. Could the small difference be due to confounding? They seem to have done a good job of guarding against confounding by surgeon characteristics. However, when it comes to patients they actually present good evidence why their results may be confounded. Although they have matched as far as they can, they have shown that women have a tendency to choose a female surgeon. Quite possible that people who choose a female surgeon are different in any number of other ways that are not measured and matched. There is a way out of this jam (if the sample size will stand it). Elective surgery may give patients a choice but this is unlikely for emergency surgery. If an analysis by elective vs emergency showed similar effects in each

4 group it would strengthen the assumption that confounding by patient characteristics was unlikely (to confound the results in this sub group there would have to be an association between characteristics of patients admitted when a female surgeon was on call vs a male surgeon, which seems a stretch). One of the reviewers worries about plastic surgery. The absolute mortality for plastic surgery would have to be very high to swing the overall figures... but worth asking at least I guess. Another editor commented "Interesting and provocative RQ. I do not think this is a baseless hypothesis: women are different from men. Apart from specific traits (more likely to follow guidelines, etc.), one could argue that the present generation of female surgeons had to work harder to get their position than their male counterparts, and are thus technically superior. But a composite endpoint in a broad range of surgical procedures (carpal tunnels vs. brain aneurysm surgery) is not so obviously clinically relevant. The differences are very small, and do not reach statistical significance in most comparisons. Confounding cannot be ruled out, as both the surgeons and the patients were different, and one can only adjust for so much. The results are similar to the JAMA Int Med paper, and these were probably also driven by residual confounding, as nicely explained in * On the sex/gender question, we prefer use of the term sex. First, please revise your paper to respond to all of the comments by the reviewers. Their reports are available at the end of this letter, below. Please also respond to the points above made by editors. We received a large number of reviews, probably reflecting the level of interest in this topic. Feel free to group similar comments and respond to them singly. In the case of conflicting recommendations from reviewers or editors, please decide what you think is the best way forward and justify that decision. We recognise that it is usually not possible to satisfy everyone. In your response please provide, point by point, your replies to the comments made by the reviewers and the editors, explaining how you have dealt with them in the paper. Comments from Reviewers Reviewer: 1 Recommendation: Comments: This study explores the difference in outcomes by patients treated by female and male surgeons. Patients treated by female surgeons had less complications than those treated by male surgeons in terms of complications, readmission and mortality within a 30-day, short-term post-operative window of operation. The sample analyzed consisted of a cohort of 104,630 patients who underwent 1 of 25 different surgical procedures between 2007 and This study examines an issue interesting to all patients and caregivers, as patients and caregivers want to be treated by a provider whose care will result in the least amount of complications, readmissions, and likelihood of death. However, this study fails to involve patients, or discuss the implications of the findings once explained to patients. Sharing the results with patients would bring up the questions: How would knowing this information influence your future decision-making and provider

5 selection? Will you continue to see your same provider, now that you know this information? If you had the choice of selecting a male or female surgeon after knowing the results of this study, which would you choose? Asking these questions to patients before and after explaining the 4% relative lower likelihood of composite outcome, and 12% lower likelihood of 30-day mortality when treated by female surgeons is critical to consider. These questions could be discussed when disseminating study results to patients, or, could have been polled in a focus group prior to data review for the study. Having this patient perspective to compare to data findings contextualizes and grounds the numbers in patient preference and decisions about care. There were far fewer female surgeons than male surgeons in this study, with 774 female to 2540 male. There is an exploration of the difficulties that female surgeons face when entering the medical field that explains this difference in the discussion portion; however, this is not brought up until nearly halfway through the article (15). This could and should certainly be mentioned sooner in order to help contextualize and explain the massive difference in the number of female-to-male surgeons sampled. Patients treated by female surgeons were more likely to be female and younger (11), which begs the question of whether this impacted health outcomes. I would have liked to see this aspect of the analysis and comparison explored further. Adolescents, and several other types of patients were excluded for numerous reasons (7). I would have also liked to see mention of transgender patients and whether or not they were included or excluded from this study. Additional Questions: Please enter your name: Charlie Blotner Job Title: MSW Health Practice Candidate Institution: University of Washington Reimbursement for attending a symposium?: No A fee for speaking?: Yes A fee for organising education?: Yes Funds for research?: No Funds for a member of staff?: No Fees for consulting?: Yes Have you in the past five years been employed by an organisation that may in any way gain or lose financially from the publication of this paper?: No Do you hold any stocks or shares in an organisation that may in any way gain or lose financially from the publication of this paper?: No If you have any competing interests <A HREF=' lists/declaration-competing-interests'target='_new'> (please see BMJ policy) </a>please declare them here: Stanford Medicine X Cure Forward Coalition of Compassionate Care California

6 Reviewer: 2 Recommendation: Comments: This work is deliberately provocative. However, better integration of data (interpretation of data) and main conclusions would make the message more focused and clear. Short term outcomes are hypothesized to involve technical (as well as cognitive) skills (stated in the What This Paper Adds, Section 1 and the Introduction). Are 30-day outcomes to be considered short term. There does not appear to be significant differences in complication rates even though there are differences in mortality, suggesting a differential failure to rescue component which is either due to cognitive skills or better hospital-based resources for dealing with complications. Was clustering at hospitals evaluated to see if the gender-based findings are robust? There did not seem to be a difference in complication or readmission rates but these results are reported differently throughout the paper. The actual findings seem to be: Patients treated by female surgeons were less likely to experience 30-day mortality (aor 0.88; 95% confidence interval 0.79 to 0.99, p=0.04), while there was no significant difference in readmissions or complications. and yet the What This Paper Adds section reports: This large, population-based matched-cohort analysis demonstrates that patients treated by female surgeons have a small but statistically significant decreased risk of death, complication or readmission in the 30 days following surgery. While the study is meant to accomplish a broader look, case mix may skew the results. For example, as shown in Figure 1, low numbers (of cases and of women surgeons) in some of the subspecialties may skew the results (noting wide confidence intervals). General surgery represents nearly 50% of cases and the impact of gender is absent (Figure 1). Miscellaneous considerations: Adjustment for emergency cases? Adjustment of case mix by gender (since the cases are of varying complexity and the question of whether gender is associated with practice type/pattern needs to be addressed)? Overall, this is a novel investigation. This work will be a source of controversy with clinicians and policymakers. The topic is of potential interest to general readers - does this work matter to clinicians, patients, teachers, or policymakers? Is a general journal the right place for it? Additional Questions: Please enter your name: Sandra L Wong Job Title: Professor of Surgery Institution: Dartmouth-Hitchcock Reimbursement for attending a symposium?: No

7 A fee for speaking?: No A fee for organising education?: No Funds for research?: No Funds for a member of staff?: No Fees for consulting?: No Have you in the past five years been employed by an organisation that may in any way gain or lose financially from the publication of this paper?: No Do you hold any stocks or shares in an organisation that may in any way gain or lose financially from the publication of this paper?: No If you have any competing interests <A HREF=' lists/declaration-competing-interests'target='_new'> (please see BMJ policy) </a>please declare them here: no Reviewer: 3 Recommendation: Comments: 1. The term "gender" is used throughout the manuscript. I think "sex" would be a better term. Do the authors know if they are categorizing surgeons by gender (social/cultural perception of sex), or by biological sex? 2. Can you provide more detail about what is meant by "1 of 25 surgical interventions, selected by multi-disciplinary consensus"? What is meant by multi disciplinary? 3. The overall findings (that women surgeons have better outcomes than men surgeons) appear largely due to strong subgroup effects, specfically plastic surgeons, and surgeons in practice years. If an outlier subgroup (eg plastic surgeons) is excluded in a sensitivity analysis, would the conclusions remain the same? If not, can the authors explain why this effect exists? The plastic surgery procedures (carpal tunnel release, reduction mammooplasty) do not seem to be procedures associated with adverse outcome risks as high as some of the other procedures in the study. However, the magnitude of the effect of surgeon sex on mortality is enormous (OR 0.23, 95% CI ). It does appear as if the overall effect is driven by a subgroup of plastic surgeons; this should be reflected in the overall conclusion of the paper. What about the plastic surgery procedures makes them so susceptible to a surgeon sex effect as compared with other types of surgery where surgeon sex seems to be unimportant? Additional Questions: Please enter your name: David Urbach Job Title: Professor Institution: Department of Surgery, University of Toronto Reimbursement for attending a symposium?: No A fee for speaking?: No

8 A fee for organising education?: No Funds for research?: No Funds for a member of staff?: No Fees for consulting?: No Have you in the past five years been employed by an organisation that may in any way gain or lose financially from the publication of this paper?: No Do you hold any stocks or shares in an organisation that may in any way gain or lose financially from the publication of this paper?: No If you have any competing interests <A HREF=' lists/declaration-competing-interests'target='_new'> (please see BMJ policy) </a>please declare them here: Reviewer: 4 Recommendation: Comments: This is a very well designed study coming from a large population analyzing of the differences on the effect of surgeon gender on postoperative outcomes. It is based on a large number of patients from a provincial database in Canada and therefore has a high level of accuracy. The authors are to congratulated on the completion of excellent matching despite the complexities, for a very thorough and complete statistical analysis and finally a well written manuscript. As a non-statistician, I cannot offer any criticisms of their analysis or the data provided. As a male surgeon, I would only bring up two points, both of which are somewhat addressed in the closing aspects of the discussion To refer to the relative differences in mortality and complication rate being modest, but yet having potential significant clinical implications is in my opinion as a bit of a stretch. A non-sophisticated look at the results would suggest that the difference in primary outcome was only 11.1 vs 11.6% and the incidence of death within 30 days was only 0.9 vs 1%. Therefore, are results simply reflective of the large number of patients included in this statistical analysis? Thus, I ask the question, for a difference to matter, it must not be a simply a statistical difference, but a true meaningful difference. I am not quite sure I can give a pass to the authors when they suggest that the results were as good as the internal medicine study and therefore it must be clinically relevant. Finally, a point again discussed by the authors in their to conclusions and implications was the concept of a team approach. Although the gender of the operating surgeon is certainly an important aspect, it must be considered that the surgeon is in fact just one member of the team and the operation is only a small component of patient care, particularly for a patient who dies after surgery. What do we know about the rest of the team? Were the operations inpatient or outpatient, emergent of elective? Certainly an academic or group practice would certainly have involvement of housestaff, fellows and more apt to have availability of a wide team of treating physicians. Since death was the single parameter that made a difference, I honestly wonder if the issue comes to down more to failure to rescue rather than the gender of the operating surgeon.

9 In conclusion, the authors have completed a well-done and interesting study which the results will certainly lead significant discussion. On the other hand, based on these statistics, I would suggest that the authors go back and repeat their analysis in a similar fashion based on whether the surgeon is right or left handed or other physical characteristics beyond gender to see if anything else makes a difference. Additional Questions: Please enter your name: Keith D. Lillemoe, MD Job Title: Chief of Surgery Institution: Massachusetts General Hospital Reimbursement for attending a symposium?: No A fee for speaking?: No A fee for organising education?: No Funds for research?: No Funds for a member of staff?: No Fees for consulting?: No Have you in the past five years been employed by an organisation that may in any way gain or lose financially from the publication of this paper?: No Do you hold any stocks or shares in an organisation that may in any way gain or lose financially from the publication of this paper?: No If you have any competing interests <A HREF=' lists/declaration-competing-interests'target='_new'> (please see BMJ policy) </a>please declare them here: Reviewer: 5 Recommendation: Comments: This article uses administrative data from Ontario ( ) to study the relationship between post-surgical mortality (defined as mortality within 30 days of surgery) between patients undergoing 1 of 25 surgeries by male vs female surgeons. The article uses as its primary motivation a recent study by Tsugawa et al. which demonstrated slightly lower 30-day mortality and readmissions among U.S. Medicare patients hospitalized with general medical conditions who were treated by female internists vs male internists (I was a co-author of that study). The Tsugawa et al. study used the quasi-randomization of patients to hospitalist physicians as a method to address the issue of selection bias that would otherwise confound such an analysis (i.e., patients treated by female physicians might otherwise be healthier). The current study finds similar relative reductions in mortality as the Tsugawa et al. study, except in the surgical population.

10 This is an interesting, timely, and well done paper. For full disclosure and given the open review process of the BMJ, I should note that my colleagues and I are conducting similar work in the U.S. population. I think that this paper could make a useful contribution to the BMJ. Below are a few areas which I think the authors can improve upon. 1. Right now, the paper is motivated more as a response to the Tsugawa et al. paper rather than an independent investigation into studying the outcomes and patterns of care of female vs male surgeons. For the paper to be timeless I would consider reframing why this issue should be studied. There is concern among female surgeons about inequitable treatment, pay, etc., part of which probably stems from an underlying concern in the field (either explicit or implicit) that the quality of care provided by female surgeons may be worse. There is even a social movement #ILookLikeASurgeon, which again probably stems from sentiment that the work of female surgeons is judged differently. I would consider reframing the paper in a bigger way. The above is only one suggestion. 2. The authors control for physician age, which would naturally be a confounder of physician sex because age may be independently correlated with surgical outcomes and is certainly correlated with sex (older surgeons are more likely to be male). I would highlight why controlling for age is important. 3. The main challenge with this paper, and others like it that I have worked on, is dealing with selection bias. Are patients treated by male and female doctors the same or do they differ on observable and unobservable characteristics? Most of our work in general medicine focuses on hospitalists or ED doctors to circumvent this issue (patients don t choose their hospitalist or ED doctor and vice versa). The current paper doesn t really address this issue. Matching is a solution for observable confounders but not unobservable ones. Indeed, Table 1 shows that patients treated by female surgeons are more likely to be female and are considerably younger. Are there ways that the authors can bolster the assertion that unobservable characteristics are unlikely to be important here? One way to do this might be to focus on emergent cases, i.e., cases where patients do not choose their surgeon and in which the ability of surgeons to choose their cases is lower. This might be employed by picking a set of high risk surgeries, or excluding specialties like OB-GYN which probably deal with more elective surgeries than emergent surgeries, or doing both approaches. This will be the key issue that readers of this study raise. 4. To the point above, what are unadjusted rates of mortality between male and female surgeons? How do these rates compare when you adjust only for type of surgery and physician age? It would be useful to understand how much does adjustment change the mortality relationship. 5. The authors focused on 30-day mortality to identify complications that could most plausibly be related to deficiencies in surgical care as opposed to outpatient care factors. But if patients are truly similar between surgeons of varying sex (i.e., if they are as good as randomly assigned), then looking at longer outcomes such as 90- and 180-day mortality would also make sense. It would add to the paper to be able to show that the mortality results persist for longer than 30 days. 6. Some additional discussion of alternative ways in which female surgeons may have better outcomes would be useful. Is this a technical skill issue? Is this about clinical acumen in identifying and managing complications from surgery quickly and appropriately (for this, studying failure to rescue measures in the claims data may be useful)? Is this about differences in risk-taking behavior, as has been suggested in studies outside of health care? 7. The authors should consider evaluating whether inpatient costs of care differ between male and female surgeons. Could it be that female surgeons are more likely to use consultants, order more tests, etc., that could improve outcomes? It s relatively easy to check this as a possibility. In our prior work we found no evidence of cost differences between male and female internists.

11 8. Could the authors better justify why they matched on surgeon volume? There is a potential issue with doing that. If female surgeons have lower volume and there is a volume-outcome relationship, then controlling for volume may eliminate some of the difference in outcomes that exists between male and female surgeons. In other words, a reasonable concern might be that any worse outcomes by female surgeons are masked by controlling for volume (which is the key driver of operative outcomes, perhaps). I don t suspect this to be the case, but something to check. Additional Questions: Please enter your name: Anupam Jena Job Title: Associate Professor Institution: Harvard Medical School Reimbursement for attending a symposium?: No A fee for speaking?: No A fee for organising education?: No Funds for research?: No Funds for a member of staff?: No Fees for consulting?: Yes Have you in the past five years been employed by an organisation that may in any way gain or lose financially from the publication of this paper?: No Do you hold any stocks or shares in an organisation that may in any way gain or lose financially from the publication of this paper?: No If you have any competing interests <A HREF=' lists/declaration-competing-interests'target='_new'> (please see BMJ policy) </a>please declare them here: I have received consulting fees unrelated to this work from Pfizer, Hill Rom, Bristol Myers Squibb, Vertex pharmaceuticals, Novartis pharmaceuticals, and Precision Health Economics, a health economics consultancy to the life sciences industry. Reviewer: 6 Recommendation: Comments: Overall a very interesting study, and I think an important contribution to the literature. Generally the authors have done an excellent job making sure that their methodology is sound and that their data demonstrate a meaningful difference, which as mentioned is sometimes hard to parse out from a very large dataset where significance is easy to find. One of the major concerns that I have is that there does not seem to be adjustment for time over the nearly 9 year period, there are most certainly changes in outcomes as surgical outcomes around the world have continued to improve. In particular, I would assume that the number of women surgeons has increased over time, meaning that their outcomes may disproportionally be modern outcomes. I am

12 also worried that patient comorbidity and severity of disease is not well documented. Have you adjusted for emergent and elective procedures? A consort diagram of your population would be helpful. Additional Questions: Please enter your name: Heather Yeo Job Title: Assistant Professor of Surgery and Healthcare Policy and Research Institution: NYP-WMC Reimbursement for attending a symposium?: No A fee for speaking?: No A fee for organising education?: No Funds for research?: No Funds for a member of staff?: No Fees for consulting?: No Have you in the past five years been employed by an organisation that may in any way gain or lose financially from the publication of this paper?: No Do you hold any stocks or shares in an organisation that may in any way gain or lose financially from the publication of this paper?: No If you have any competing interests <A HREF=' lists/declaration-competing-interests'target='_new'> (please see BMJ policy) </a>please declare them here: