EVIDENCE-BASED DERMATOLOGY: STUDY

Transcription

1 EVIDENCE-BASED DERMATOLOGY: STUDY SECTION EDITOR: MICHAEL BIGBY, MD; ASSISTANT SECTION EDITORS: OLIVIER CHOSIDOW, MD, PhD; ROBERT P. DELLAVALLE, MD, PhD, MSPH; DAIHUNG DO, MD; URBÀ GONZÁLEZ, MD, PhD; CATALIN M. POPESCU, MD, PhD; HYWEL WILLIAMS, MSc, PhD, FRCP The Reporting of Observational Research Studies in Dermatology Journals A Literature-Based Study Sinéad Langan, MD, MSc, PhD; Jochen Schmitt, MD, MPH; Pieter-Jan Coenraads, MD, PhD, MPH; Åke Svensson, MD, PhD; Erik von Elm, MD; Hywel Williams, MSc, PhD, FRCP; for the European Dermato-Epidemiology Network (EDEN) Objective: To assess the quality of reporting in observational studies in dermatology. Data Sources: Five dermatology journals the Archives of Dermatology, the British Journal of Dermatology, the Journal of the American Academy of Dermatology, the Journal of Investigative Dermatology, and Acta Dermato- Venereologica. Study Selection: Cohort, case-control, and crosssectional studies published as original articles during the period January 5 through December 7. Studies were identified with a literature search of PubMed combining the journal title and the term epidemiological studies (free text) and by hand searching all of the issues of each journal to identify relevant articles. Data Extraction: All articles were extracted by reviewers independently using standardized checklists based on the Strengthening the Reporting of Observational Studies in Epidemiology () recommendations. Data Synthesis: The number and proportion of reported items were analyzed for each article. The proportion of studies with good reporting for each item was also assessed. Results: A total of 138 articles were included and analyzed. Reporting quality was very mixed. Key areas that were infrequently reported included sample size calculations (n=1 [7%]), missing data (n=8 [6%]), losses to follow-up (n=17 [1%]), and statistical methods (n=19 [14%]). Only 13 studies (9%) explained the role of funders in the research. The quality of reporting was similar across study designs for critical questions with the exception of reporting of participant details, which was better reported in cohort studies (96%) compared with cross-sectional (8%) and case-control (7%) studies. Conclusions: It is difficult to judge the quality of dermatological research unless it is reported well. This study has identified a clear need to improve the quality of reporting of observational studies in dermatology. We recommend that dermatology journals adopt the criteria. Arch Dermatol. 1;146(5): Author Affiliations are listed at the end of this article. Group Information: Members of EDEN are listed at the end of this article. Section Editor s Note: The instrument was reviewed in a Commentary published in the Evidence-Based Dermatology section in the September 8 issue of the Archives of Dermatology (Nijsten T, Spuls P, Stern RS. : a beacon for observational studies. Arch Dermatol. 8; 144[9]:1-14). INTERPRETATION OF DATA FROM observational studies is often limited by poor quality of reporting. Poor reporting limits the assessment of a study s strengths and weaknesses and generalizability. 1 It also limits the use of observational data for secondary analyses. The clinical and scientific utility of research data may be lost in poorly reported studies. The Strengthening the Reporting of Observational Studies in Epidemiology () criteria have recently been developed to try and improve the quality of reporting of observational studies in medical research. The checklist aims to assess and improve the reporting of epidemiological studies, but it is not a measurement for the quality of the research itself. Poor reporting may also constitute an ethical problem if the necessary information is collected but is not described in research articles, thus reducing its usefulness. Similar initia- 534

2 tives for improving the quality of reporting of particular study types have already shown benefits in terms of the reporting of randomized controlled trials using the Consolidated Standards for Reporting Trials (CONSORT) statement. 3 From our collective experience in performing systematic reviews within the field of dermatology, we hypothesized that the quality of reporting of observational studies in dermatology was hit and miss, with many published articles failing to mention key aspects that would allow the reader to judge the validity of the study findings and conclusions. We set out to systematically assess the quality of reporting of observational studies in dermatology using the checklist as a guideline and to highlight specific areas that could be improved. METHODS A study protocol was devised prior to the study, and all methods were defined and piloted a priori. ELIGIBLE JOURNALS We selected the 4 journals with the highest impact factor (IF) in the category Dermatology in the Journal Citation Report 7 4 that have a section for the reporting of epidemiological studies, namely, the Archives of Dermatology (IF,.84), the British Journal of Dermatology (IF, 3.5), the Journal of the American Academy of Dermatology (IF,.9), and the Journal of Investigative Dermatology (IF, 4.83). Acta Dermato-Venereologica (IF, 1.93) was also included because this journal publishes a high proportion of epidemiological studies. The journal articles were reviewed over a 3-year period, from January 5 through December 7. ELIGIBLE STUDIES We included epidemiological studies that corresponded to a relevant checklist, namely, cohort, case-control, and cross-sectional studies published during the study period. Studies that were labeled as a randomized controlled trial (RCT) or laboratory research that were in fact observational epidemiological studies were included in this study under the appropriate category. EXCLUSION CRITERIA Epidemiological studies other than cohort, case-control, or crosssectional studies such as ecological studies were excluded. Studies in abstract format only were also excluded. Genetic epidemiological studies were not included because this study subtype has its own reporting guideline (Strengthening the Reporting of Genetic Associations [STREGA]). 5 STUDY SELECTION Studies were identified with a literature search of PubMed combining the journal title and the term epidemiological studies (free text) and by hand searching all of the issues of each journal to identify relevant articles. This process was carried out independently by reviewers (S.L. and J.S.), and disagreements were resolved by discussion and arbitration by a third person (H.W.) if needed. Articles were then numbered consecutively and randomly allocated to each reviewer using a random numbers table. DATA ABSTRACTION To determine whether further clarification was needed to define the questions when scoring items from the checklist, 3 articles representing each main study design underwent a pilot data abstraction by all reviewers (S.L., J.S., P.-J.C., and Å.S.) and by the European Dermato-Epidemiology Network (EDEN) steering group (1 members including S.L., J.S., P.-J.C., and Å.S.). Disagreements were resolved by discussion with all reviewers. Following the pilot phase, all articles were extracted by reviewers independently. Disagreements were resolved between pairs (S.L. and J.S.; Å.S. and P.-J.C.), and an independent arbitrator (H.W.) was involved for persisting differences. Relevant data from included articles was summarized in an Excel spreadsheet (Microsoft Corp, Redmond, Washington) containing information on the number and proportion of the items in the checklist that were reported against the checklist. REPORTING OF QUALITY OF STUDY REPORTING The checklists of items was operationalized into a series of questions for each study design, which can be answered yes, partly, no, unclear, or not applicable. One author (E.v.E.) was also involved in the development of the checklist. This guaranteed the consistency of this operationalization and of our research in general with the aims of the initiative. The checklists were piloted by all investigators on the 3 selected study types as previously described. Each investigator independently submitted their analysis to their paired author. A panel discussion by followed to resolve the differences in analyses and to obtain a consensus on how the responses are defined by individual investigators. A glossary of rules was developed throughout the study, which will help inform the initiative and other researchers. Briefly, a number of key decisions were made to translate the reporting items into data that could be assessed reliably. ANALYSIS The number and proportion of reported items ( yes responses) and not reported items (all responses except yes or not applicable ) were analyzed for each study. We then examined the median and range of reported items by study design. The proportion of studies with good reporting for each item was also assessed. Items that were not applicable were excluded from the analysis. A sensitivity analysis was carried out, in which partly responses were analyzed as yes responses to assess the effect on study conclusions. SELECTION OF THE MOST CRITICAL ITEMS To inform future studies, we believed it would be useful to identify a priori which questions within the checklist should be critical to complete when reporting observational studies. To achieve this, each author independently identified what they perceived to be 1 essential questions, and responses were then pooled to determine a critical list of items (Table 1). An item was considered essential if at least 4 of 5 reviewers (S.L., J.S., P.-J.C., Å.S., and H.W.) agreed on the item for inclusion, and these are given in Table. Ofthe items in the checklist, 4 were considered essential by the panel of authors prior to completion of the study. The 535

3 Table 1. Derivation of Critical List of Questions (Printed in Bold) and Rules Developed in Relation to Questions During the Study Original Item No. Description Rules Developed During the Study Title and Abstract 1 Is the design described adequately in the title or abstract? If the study design was not specifically stated, this was Does the abstract provide an informative summary of what was done and found? recorded as not being complete. For incorrectly labeled studies, the appropriate category of study design was identified and used for further analyses. No. of Reviewers Who Considered Item Critical Introduction Is the scientific background and rationale for the investigation 1 reported? 3 Are any prespecified hypotheses reported? Are the objectives reported? Methods 4 Are the key elements (ie, retrospective/prospective, cohort/cross-sectional) of the study design presented? 5 Are the settings reported? Studies that did not report the setting or locations but Are the locations reported? Are relevant dates including periods of recruitment reported? a referred readers to a previous publication were considered as inconsistent with complete reporting. If the dates of recruitment were recorded anywhere in the article (not necessarily in the Methods section), the corresponding item was rated as complete. Are relevant dates including periods of exposure reported? Are relevant dates including periods of follow-up reported? Are relevant dates including periods of data collection reported? 6 Are the eligibility criteria for participants described? 5 Are the sources of participants described? Are the methods of selection described? Are the methods of follow-up described? (only used for n=11) If it is a matched study, are the matching criteria and the numbers of exposed and unexposed described? (only used for n=3) 7 Are all outcomes described if applicable? 4 Are all exposures described if applicable? Are all predictors described if applicable? Are potential confounders described? We only recorded the authors as having reported confounders or effect modifiers in the Methods if these were specifically stated or, in the case of the latter, if they described the use of interaction terms or stratification. Are all effect modifiers described? Are diagnostic criteria described if applicable? 8 Are the sources of data and details of methods of measurement 1 given for each variable of interest? If there is more than 1 group, are the measurement methods comparable? 9 Was there any effort to address potential sources of bias? We recorded authors as having made attempts to address 3 sources of bias if they incorporated any tools to do this, eg, using standardized definitions or validated scoring systems. We also believed that addressing sources of bias should never be considered not applicable in an observational study. 1 Did they describe how the study size was determined? 1 11 Did they describe how quantitative variables were handled in the analysis? Did they describe which groupings were chosen for quantitative variables? Did they describe why quantitative groups were chosen? 1 Did they describe all statistical methods including those to deal with confounding? Did they describe methods to examine subgroups and interactions? Did they explain how missing data was addressed? Did they explain if applicable how losses to follow-up were addressed? Did they describe any sensitivity analysis? In relation to statistical methods, unless authors stated which confounders were adjusted for and why, this item was not recorded as complete. 5 4 (continued) 536

4 Table 1. Derivation of Critical List of Questions (Printed in Bold) and Rules Developed in Relation to Questions During the Study (continued) Original Item No. Description Rules Developed During the Study Results 13 Did they report the numbers of individuals at each stage of the study numbers potentially eligible, examined for eligibility, confirmed eligible, included in the study, and completed follow-up and were analyzed? a Did they give reasons for nonparticipation at each stage? a Did they use a flow diagram if appropriate? 14 Did they give the characteristics of study participants (eg, demographic, clinical, and social) and information on exposures and potential confounders? No. of Reviewers Who Considered Item Critical Did they indicate the number of participants with missing data for each variable of interest? Did they summarize follow-up time (average and total amount)? 15 Did they report numbers of outcome measures over time? Did they report summary measures over time? 16 Did they give unadjusted estimates and, if applicable, confounder-adjusted estimates and their precision (eg, 95% confidence interval)? 3 Did they detail which confounders were adjusted for and why they were included? Did they report category boundaries when continuous variables were categorized? Did they, if relevant, consider translating estimates of relative risk into absolute risk for a meaningful period? 17 Did they report on other analyses done, eg, analysis of subgroups or interactions? Did they do a sensitivity analysis? (only used for n=9) Discussion 18 Did they summarize key results with reference to study objectives? 19 Did they discuss the limitations of the study taking into account potential sources of bias or imprecision (including discussion of the magnitude of any potential sources of bias)? Did they give a cautious overall interpretation of results considering objectives, limitations, multiplicity of analyses, results from similar studies, and other relevant evidence? 1 Did they discuss the generalizability (external validity) of the study results? Other Information Did they give the source of the funding in the present study and, if applicable, for the original study on which the present article is based? Did they give the role of the funders in the present study and, if applicable, for the original study on which the present article is based? It was also our view that it was only appropriate to translate relative risk into absolute risk if there was convincing evidence of a causal association. Unless the role of the funders was specifically stated, we did not record this as complete. Abbreviations:, Strengthening the Reporting of Observational Studies in Epidemiology. a For studies based on disease registries or databases, a number of the checklist items are not applicable, eg, the dates of recruitment, numbers eligible at each stage of the study, reasons for nonparticipation, or flow diagrams. 1 items were as follows: question 1 (title and abstract), question 6 (detail of participants), question 7 (study variables), and question 1 (statistical methods). Items that none of the reviewers considered essential were question 11 (handling of quantitative variables) and question 17 (other analyses). RESULTS The search strategy identified 91 studies, of which 138 were relevant for this study (a list is provided in an online repository available at: eeden/eprojects/e88/index_ger.html). The Figure shows the flow diagram of included studies: 53 articles (38%) were cohort studies, 44 (3%) were case-control studies, and 41 (3%) were cross-sectional studies. PROPORTION OF REPORTED ITEMS PER ARTICLE The median (range) number of reported items per article was comparable across the different study designs. For cohort studies, the median number of reported items per ar- 537

5 Table. Checklist of Questions With Overall Scoring for Included Studies and Percent Complete Original Item No. Description Yes Partly No Unclear Not Applicable Title and Abstract 1 Is the design described adequately in the title or abstract? 1 (87) 13 (9) 3 () (1) 1 (87) 135 (98) 137 (99) 19 (93) 8 (6) 1 (1) 19 (93) Does the abstract provide an informative summary of what was done and found? No. (%) Individual Item Adequate Overall Item Adequate a Results of Sensitivity Analysis b Introduction Is the scientific background and rationale for the 134 (97) 4 (3) 134 (97) 134 (97) 138 (1) investigation reported? 3 Are any prespecified hypotheses reported? 1 (9) 4 (3) 11 (88) 1 (1) 1 (9) 13 (89) 133 (96) Are the objectives reported? 13 (89) 1 (7) 4 (3) 1 (1) 13 (89) Methods 4 Are the key elements (ie, retrospective/ prospective, 96 (7) (14) 19 (14) 3 () 96 (7) 96 (7) 116 (84) cohort/cross-sectional) of the study design presented? 5 Are the settings reported? 11 (81) 6 (4) 13 (9) 4 (3) 3 () 11 (83) 18 (78) 1 (88) Are the locations reported? 19 (79) 1 (7) 14 (1) (1) 3 () 19 (81) Are relevant dates including periods of recruitment 11 (73) 7 (5) 4 (17) 6 (4) 11 (77) reported? Are relevant dates including periods of exposure reported? 6 (45) 11 (8) 4 (17) 1 (1) 4 (9) 6 (63) Are relevant dates including periods of follow-up reported? 47 (34) 7 (5) 16 (1) 3 () 65 (47) 47 (64) Are relevant dates including periods of data collection 8 (59) 1 (7) 34 (5) 3 () 9 (7) 8 (64) reported? 6 Are the eligibility criteria for participants described? 116 (84) 13 (9) 6 (4) (1) 1 (1) 116 (85) 115 (83) 19 (93) Are the sources of participants described? 14 (9) 6 (4) 8 (6) 14 (9) Are the methods of selection described? 9 (67) 19 (14) (16) 3 () 1 (1) 9 (68) Are the methods of follow up described? (only used for 5 (51) 15 (15) 13 (13) 4 (4) 17 (17) 5 (6) n=11) If it is a matched study, are the matching criteria and the 3 (17) 7 (5) 18 (78) 3 (77) numbers of exposed and unexposed described? (only used for n=3) 7 Are all outcomes described if applicable? 1 (88) 7 (5) 1 (1) 4 (3) 4 (3) 1 (91) 73 (53) 114 (83) Are all exposures described if applicable? 88 (64) 15 (11) 6 (4) 8 () 88 (81) Are all predictors described if applicable? 58 (4) 1 (9) (16) (1) 43 (31) 58 (6) Are potential confounders described? 5 (36) 14 (1) 66 (48) 4 (3) 4 (3) 5 (37) Are all effect modifiers described? 3 () 1 (7) 74 (54) 8 (6) 16 (1) 3 (5) Are diagnostic criteria described if applicable? 69 (5) 13 (9) 38 (8) 1 (1) 17 (1) 69 (57) 8 Are the sources of data and details of methods of 15 (76) 3 (17) 9 (7) 1 (1) 15 (77) 118 (86) 13 (96) measurement given for each variable of interest? If there is more than 1 group, are the measurement methods 43 (31) 1 (1) (1) 5 (4) 87 (63) 43 (84) comparable? 9 Was there any effort to address potential sources of bias? 43 (31) 1 (9) 75 (54) 6 (4) (1) 43 (3) 43 (31) 55 (4) 1 Did they describe how the study size was determined? 1 (7) 15 (91) 3 () 1 (7) 1 (7) 1 (7) 11 Did they describe how quantitative variables were handled in 43 (31) 13 (9) 5 (38) 4 (3) 6 (19) 43 (38) 4 (36) 5 (45) the analysis? Did they describe which groupings were chosen for 37 (7) 6 (4) 51 (37) 4 (3) 4 (9) 37 (38) quantitative variables? Did they describe why quantitative groups were chosen? 11 (8) (1) 74 (54) 5 (4) 46 (33) 11 (1) 1 Did they describe all statistical methods including those to 8 (58) 3 (3) (16) 4 (3) 8 (6) 19 (14) 5 (19) deal with confounding? Did they describe methods to examine subgroups and (14) 5 (4) 67 (49) 1 (1) 44 (3) () interactions? Did they explain how missing data was addressed? 8 (6) (1) 14 (9) 1 (1) 3 () 8 (6) Did they explain if applicable how losses to follow-up were 17 (1) 7 (5) 74 (54) 1 (1) 39 (8) 17 (17) addressed? Did they describe any sensitivity analysis? 8 (6) 15 (91) 1 (1) 4 (3) 8 (6) (continued) ticle was 13 (7-1), and similarly, the median was 1 (7-17) for cross-sectional designs and 1 (4-18) for casecontrol studies. ITEMS USUALLY REPORTED Of the 138 studies, 137 (99%) reported key results in relation to study objectives (Table ). Similarly, 134 studies (97%) reported the scientific background and rationale for carrying out the study. Most studies (n=19 [93%]) provided a reasonable summary of the research in the abstract. The majority of studies reported the study objectives (n=13 [89%]), design (n=1 [87%]), and outcomes (n=1 [88%]). ITEMS NOT USUALLY REPORTED Sample size calculations were reported by only 1 (7%) of included studies. Other areas that were very poorly described relate to the issues of management of missing data 538

6 Table. Checklist of Questions With Overall Scoring for Included Studies and Percent Complete (continued) Original Item No. Description 13 Did they report the numbers of individuals at each stage of the study numbers potentially eligible, examined for eligibility, confirmed eligible, included in the study, and completed follow-up and were analyzed? No. (%) Individual Item Adequate Overall Item Adequate a Results of Sensitivity Analysis b Yes Partly No Unclear Not Applicable Results 54 (39) 3 (17) 46 (33) 15 (11) 54 (44) 3 (5) 43 (34) Did they give reasons for nonparticipation at each stage? 9 (6) 4 (3) 1 (74) 3 (17) 9 (8) Did they use a flow diagram if appropriate? 3 (17) 5 (4) 8 (58) (1) 8 () 3 (1) 14 Did they give the characteristics of study participants (eg, 77 (56) 31 () 9 (1) 1 (1) 77 (56) 6 (43) 9 (67) demographic, clinical, and social) and information on exposures and potential confounders? Did they indicate the number of participants with missing 11 (8) 13 (89) 4 (3) 11 (8) data for each variable of interest? Did they summarize follow-up time (average and total 1 (15) 31 () 86 (6) 1 (4) amount)? 15 Did they report numbers of outcome measures over time? 96 (7) 5 (4) 16 (1) (15) 96 (8) 98 (8) 14 (85) Did they report summary measures over time? 5 (5) 6 (6) 13 (13) 1 (1) 7 (8) 5 (71) 16 Did they give unadjusted estimates and, if applicable, confounder-adjusted estimates and their precision (eg, 95% CI)? 73 (53) 3 () 6 (19) 9 (6) 73 (57) 73 (54) 97 (7) Did they detail which confounders were adjusted for and 5 (36) 17 (1) 51 (37) (1) 18 (13) 5 (4) why they were included? Did they report category boundaries when continuous 61 (44) (1) 5 (18) 1 (1) 49 (46) 61 (69) variables were categorized? Did they, if relevant, consider translating estimates of relative 5 (4) 5 (18) 18 (78) 5 (17) risk into absolute risk for a meaningful period? 17 Did they report on other analyses done, eg, analysis of 37 (7) 13 (9) 68 (49) 1 (1) 9 (14) 37 (31) 39 (3) 51 (39) subgroups or interactions? Did they do a sensitivity analysis? (only used for n=87) 8 (6) 79 (57) 51 (4) 8 (9) Discussion 18 Did they summarize key results with reference to study objectives? 137 (99) 1 (1) 137 (99) 137 (99) 137 (99) 19 Did they discuss the limitations of the study taking into 76 (55) 31 () 3 () 1 (1) 76 (55) 76 (55) 17 (78) account potential sources of bias or imprecision (including discussion of the magnitude of any potential sources of bias)? Did they give a cautious overall interpretation of results 94 (68) 36 (6) 8 (6) 94 (68) 94 (68) 13 (94) considering objectives, limitations, multiplicity of analyses, results from similar studies, and other relevant evidence? 1 Did they discuss the generalizability (external validity) of the study results? 45 (33) 9 (1) 6 (45) 1 (1) 1 (1) 45 (33) 45 (33) 74 (54) Did they give the source of the funding in the present study and, if applicable, for the original study on which the present article is based? Did they give the role of the funders in the present study and, if applicable, for the original study on which the present article is based? Other Information 97 (7) 1 (1) 39 (8) 1 (1) 97 (71) 97 (7) 98 (7) 13 (9) (1) 83 (6) 4 (9) 13 (13) Abbreviation: CI, confidence interval. a An item was defined as adequate if 5% or more of its components were yes responses; not applicable components were excluded. b The analysis in the first footnote was repeated, but partly responses were combined with yes responses. (n=8 [6%]) and losses to follow-up (n=17 [1%]). In addition, infrequently addressed areas included the number of individuals at each stage of the study (n=54 [39%]) and the statistical methods (n=19 [14%]). Thirteen studies (9%) explained the role of funders in the research. REPORTING OF CRITICAL ITEMS Of those questions considered essential (Table ), questions 1 (98%) and 6 (83%) were frequently reported, whereas questions 7 (53%) and 1 (14%) were infrequently described. Examining the critical items by study design, item 1 was adequately completed by at least 95% of studies in all categories. Item 6 was reported in 96% of cohort studies compared with 8% of cross-sectional studies and 7% of case-control studies. Completion rates were similar for items 7 (5%-54%) and 1 (1%-18%) across all study designs. SENSITIVITY ANALYSIS Recoding partly responses as yes responses altered the proportion of completely reported items as given intable. Specifically, in relation to question 7 (study variables), this increased completion rates from 53% (n=73) to 83% (n=114). Similarly for question 14 (de- 539

7 91 Studies identified by search strategy 138 Studies included Figure. Flow diagram of included studies. Excluded studies 5 Reviews 48 Genetic studies 38 Case series 3 Experimental studies 14 Studies assessing outcome measures 9 Not observational 4 Intervention studies Safety studies 1 Cluster design tails of study participants), completion rate increased from 43% (n=6) to 67% (n=9). Significant increases were also observed for the questions 16 (reporting of crude and confounder adjusted estimates), 19 (study limitations), (cautious interpretation), and 1 (generalizability of the findings). COMMENT SUMMARY OF FINDINGS This study has highlighted areas where reporting of observational, epidemiological studies is good in the dermatology literature, along with other areas where improvements are required. In particular, there is a need for studies to improve their reporting of sample size calculations, statistical methods, and details of numbers and characteristics of participants. Sample size might be determined by any limited resources in time, resources, and eligible patients. We do not believe that researchers should be forced into post hoc sample size justifications, but they should say how they obtained the number of participants. Sample size estimates are useful to see what magnitude of effect the authors were looking for and also to see if there were problems recruiting the target sample owing to participant drop out. 6 Missing data and losses to follow-up are important potential sources of information and attrition bias. Two other critical areas of weakness included the description of study variables and statistical methods, both of which were identified a priori by the authors as being critical items in the reporting of observational studies. FINDINGS IN CONTEXT The quality of reporting of observational studies has been previously assessed in other fields of medical research 7,8 but not, to our knowledge, in dermatology. We only identified studies assessing the quality of reporting of observational studies using the criteria in ophthalmology and sexual health. Both specialties have identified similar problems with incomplete reporting, with particularly poor reporting of management of missing data and confounding. 9,1 Similar findings have also been shown in the dermatology literature in relation to the reporting of RCTs and the CONSORT guidelines In 1985, Bigby et al 13 reported (n=61 trials) similar findings in relation to reporting of RCTs, with only 3% of studies reporting power and 76% addressing losses to follow-up. The situation for dermatology RCTs appears to have improved following the introduction of CONSORT guidelines. 11,1,14-16 STRENGTHS AND LIMITATIONS OF THIS STUDY This is a novel study that has systematically highlighted important areas requiring improvement in the reporting of observational studies in dermatology. The results have direct relevance for readers and authors of epidemiological studies. The use of pilot methods to devise the study checklists, independent data extraction, and arbitration has helped improve the methodological rigor of our study. One of the key limitations is that it was not possible to formally assess interrater agreement because discussion between authors and the creation of rules were required to resolve these issues as the study identified new problems as it progressed. A degree of subjectivity is reflected by the findings of the sensitivity analysis. CLINICAL AND RESEARCH IMPLICATIONS This study has highlighted important deficiencies in the reporting of observational studies in dermatology journals. On the basis of these findings, we believe it would be useful for authors if dermatology journals adopted the use of the criteria to guide authors. Authors of epidemiological studies might also start using the guidance regardless of which dermatology journals they submit their article to. This study has defined a number of key rules that will be useful for researchers. These include the conclusion that settings and locations should always be defined even in the setting of referring readers to previous publications. Another important conclusion was that for studies based on disease registries or databases, a number of the checklist items are not applicable, for example, the dates of recruitment, numbers eligible at each stage of the study, reasons for nonparticipation, or flow diagrams. Full explanatory notes on the use of are on the Web site ( [accessed August 6, 9]), and a fuller list of other reporting guidelines are on the EQUATOR (Enhancing the Quality and Transparency of Health Research) Web site ( [accessed August 6, 9]). Use of is likely to lead to better quality reporting, as was found with the adoption of the CONSORT statement. 1 Better reporting is essential to maintain an undistorted scientific record, which can be used for synthesis of existing evidence, clinical decision making, and health policy. Further research could then review the quality of reporting to assess improvement after this important change. 54

8 Accepted for Publication: December 1, 9. Author Affiliations: Department of Dermatology, University of Nottingham, Nottingham, England (Drs Langan and Williams); Dermatology Department, Technical University Dresden, Dresden, Germany (Dr Schmitt); University Medical Centre, Groningen, the Netherlands (Dr Coenraads); Department of Dermatology, University Hospital Malmö, Malmö, Sweden (Dr Svensson); and Swiss Paraplegic Research, Nottwil, Switzerland, and German Cochrane Centre, University Medical Centre Freiburg, Freiburg, Germany (Dr von Elm). Correspondence: Sinéad Langan, PhD, Centre of Evidence-Based Dermatology, C Floor, South Block, Queen s Medical Centre, Nottingham NG UH, England (sinead.langan@nottingham.ac.uk). Author Contributions: Dr Langan had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Langan, Schmitt, Svensson, von Elm, and Williams. Acquisition of data: Langan, Schmitt, and Coenraads. Analysis and interpretation of data: Langan, Schmitt, Coenraads, Svensson, and von Elm. Drafting of the manuscript: Langan, Schmitt, and Svensson. Critical revision of the manuscript for important intellectual content: Coenraads, Svensson, von Elm, and Williams. Statistical analysis: Langan and von Elm. Study supervision: Svensson and Williams. Financial Disclosure: None reported. EDEN Group Members: Jan-Nico Bouwes Bavinck, MD, PhD; Pieter-Jan Coenraads, MD, PhD, MPH; Thomas Diepgen, MD, PhD; Peter Elsner, MD, PhD; Ignacio García- Doval, MD, MSc, PhD; Jean Jacques Grob, MD, PhD; Sinéad Langan, MD, MSc, PhD; Luigi Naldi, MD; Tamar Nijsten, MD, PhD; Jochen Schmitt, MD, MPH; Åke Svensson, MD, PhD; and Hywel Williams, MSc, PhD, FRCP. Additional Contributions: The EDEN steering group provided useful discussions and reviewed the study checklists. REFERENCES 1. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP; Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology () statement: guidelines for reporting observational studies. PLoS Med. 7;4(1):e96.. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP; Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology () statement: guidelines for reporting observational studies. Epidemiology. 7;18(6): Altman DG, Schulz KF, Moher D, et al; CONSORT GROUP (Consolidated Standards of Reporting Trials). The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med. 1;134(8): Institute of Scientific Information. Journal Citation Report 7. New York, NY: Thomson Reuters; von Elm E, Moher D, Little J; STREGA collaboration. Reporting genetic association studies: the STREGA statement. Lancet. 9;374(9684): Williams HC, Seed P. Inadequate size of negative clinical trials in dermatology. Br J Dermatol. 1993;18(3): Pocock SJ, Collier TJ, Dandreo KJ, et al. Issues in the reporting of epidemiological studies: a survey of recent practice. BMJ. 4;39(7471): Yoon U, Knobloch K. Quality of reporting in sports injury prevention abstracts according to the CONSORT and criteria an analysis of the World Congress of Sports Injury Prevention in 5 and 8 [published online July 6, 9]. Br J Sports Med. doi:1.1136/bjsm Fung AE, Palanki R, Bakri SJ, Depperschmidt E, Gibson A. Applying the CONSORT and statements to evaluate the reporting quality of neovascular agerelated macular degeneration studies. Ophthalmology. 9;116(): Müller M, Egger M. Strengthening the reporting of observational epidemiology () in sexual health. Sex Transm Infect. 9;85(3): Adetugbo K, Williams H. How well are randomized controlled trials reported in the dermatology literature? Arch Dermatol. ;136(3): Alvarez F, Meyer N, Gourraud PA, Paul C. CONSORT adoption and quality of reporting of randomized controlled trials: a systematic analysis in two dermatology journals. Br J Dermatol. 9;161(5): Bigby M, Stern RS, Bigby JA. An evaluation of method reporting and use in clinical trials in dermatology. Arch Dermatol. 1985;11(11): Begg C, Cho M, Eastwood S, et al. Improving the quality of reporting of randomized controlled trials: the CONSORT statement. JAMA. 1996;76(8): Rennie D. How to report randomized controlled trials: the CONSORT statement. JAMA. 1996;76(8): Moher D, Jones A, Lepage L; CONSORT Group (Consolitdated Standards for Reporting of Trials). Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. JAMA. 1;85 (15): EVIDENCE-BASED DERMATOLOGY: RESEARCH COMMENTARY Bleach Baths to Reduce Severity of Atopic Dermatitis Colonized by Staphylococcus Fiona E. Craig, MBBS, MRCP(UK); Emma V. Smith, MBChB, MRCP; Hywel C. Williams, MSc, PhD, FRCP; Departments of Dermatology, Aberdeen Royal Infirmary, Aberdeen, Scotland (Dr Craig), and Singleton Hospital, Swansea, Wales (Dr Smith); and Centre of Evidence-Based Dermatology, University of Nottingham, King s Meadow Campus, Nottingham, England (Dr Williams) Commentary on: Treatment of Staphylococcus aureus colonization in atopic dermatitis decreases disease severity Huang JT, Abrams M, Tiougan B, et al Pediatrics. 9;13(5):e88-e814 Question: DoessuppressionofStaphylococcusaureusgrowth with sodium hypochlorite (bleach) baths and intranasal mupirocin improve atopic dermatitis (AD) severity? Design: A 3-month, investigator-blinded, placebocontrolled, randomized controlled trial (RCT). Setting: Pediatric dermatology clinic at the Children s Memorial Hospital, Chicago, Illinois. Participants: Thirty-one participants aged 6 months to 17 years with moderate to severe AD and clinical signs of bacterial infection were randomized. Fifteen were assigned to receive active treatment and 16 to receive placebo. At baseline, the treatment group had higher mean Eczema 541