Positive-Outcome Bias and Other Limitations in the Outcome of Research Abstracts Submitted to a Scientific Meeting

Transcription

1 Positive-Outcome Bias and Other Limitations in the Outcome of Research Abstracts Submitted to a Scientific Meeting Michael L. Callaham, MD; Robert L. Wears, MD; Ellen J. Weber, MD; Christopher Barton, MD; Gary Young, MD Context. Studies with positive results are more likely to be published in biomedical journals than are studies with negative results. However, many studies submitted for consideration at scientific meetings are never published in full; bias in this setting is poorly studied. Objective. To identify features associated with the fate of research abstracts submitted to a scientific meeting. Design and Setting. Prospective observational cohort, with 5-year follow-up of all research submitted for consideration to the major annual 1991 US research meeting in the specialty of emergency medicine. Participants. All research abstracts submitted for consideration at the meeting for possible presentation. Main Outcome Measures. Characteristics associated with acceptance for presentation at the meeting and subsequent publication as a full manuscript. Results. A total of 492 research abstracts were submitted from programs in emergency medicine and other specialies affiliated with 13 US medical schools. A total of 179 (36%) were accepted for presentation and 214 (43%) were published in 44 journals. Of the 179 abstracts accepted for presentation, 111 studies were published. Scientific quality of abstracts or prestige of the journal in which the study was eventually published did not predict either of these outcomes. The best predictors (by logistic regression) of meeting acceptance were a subjective originality factor (odds ratio [OR], 2.7; 95% confidence interval [CI], ) and positive results (OR, 1.99; 95% CI, ), and, for publication, meeting acceptance (OR, 2.49; 95% CI, ) and large sample size (OR, 2.26; 95% CI, ). Forty-nine percent (241) of abstracts did not report on blinding, and 24% (118) did not report on randomization. Acceptance and publication were both more likely for positive outcomes (P =.3). Funnel plots showed the classic distribution of positive-outcome ( publication ) bias at each of the submission, acceptance, and publication phases. Meeting acceptance predicted publication with a sensitivity of only 51%, specificity of 71%, positive predictive value of 57%, and negative predictive value of 66%. Conclusions. Positive-outcome bias was evident when studies were submitted for consideration and was amplified in the selection of abstracts for both presentation and publication, neither of which was strongly related to study design or quality. JAMA. 1998;28: From the Division of Emergency Medicine, University of California, San Francisco (Drs Callaham, Weber, and Young); the Division of Emergency Medicine, University of Florida, Gainesville (Dr Wears); the Department of Emergency Medicine, University of North Carolina, Chapel Hill (Dr Barton); and the Department of Emergency Medicine, Highland Hospital, Alameda County Medical Center, Alameda, Calif (Dr Young). Dr Young is now with the Sacred Heart Medical Center, Eugene, Ore. Presented as a poster at the Third International Congress on Peer Review in Biomedical Publication, Prague, Czech Republic, September 19, Reprints: Michael L. Callaham, MD, Box 28, University of California, San Francisco, CA ( mlc@itsa.ucsf.edu). POSITIVE-OUTCOME (also known as publication ) bias refers to the fact that research with positive outcomes is much more likely to be published than that with negative outcomes. 1-4 Presentation of results in abstracts at scientific meetings is the first and often only publication for most biomedical research studies. 5 However, the abstract selection process for meetings rarely has been studied. We, therefore, examined all research submitted for presentation at a national meeting to determine if positive-outcome bias was present in this process and what characteristics determined successful subsequent publication in a peer review journal. METHODS The Society for Academic Emergency Medicine (SAEM) meeting is comparable to the meetings of 31 other societies of the Council of Academic Societies. 6 Abstracts with mandatory structured formats were submitted and, independent of our study (and similar to other specialty meetings), each submission was evaluated by 5 to 7 blinded members of the SAEM screening committee (selected for their relevant expertise) and ranked on a 5- point Likert scale. Selection was based on an average score, and no journal had right of first refusal. Four years after the 21st annual meeting of SAEM in 1991, all SAEM data were obtained by the authors and each submitted study was categorized according to design by a blinded Delphi panel. Since no established system exists for abstract classification, we modified a previously published approach. 7 In addition, the review panel ranked each study for scientific quality and originality ( newsworthiness ) on a Likert scale like the one previously validated. 8 Institutional review board approval was obtained, and a detailed description of the methods used is available from the authors. All authors names were searched in MEDLINE in late 1995 to determine if the study had been published in any listed journal. 9 For papers not found, the search was repeated in early 1996; if still not found, a questionnaire was sent to the authors and EMBASE and the Cochrane Collaboration databases were alsosearched. 9 Journalimpactfactorwas derived from the Science Citation Index for the year of publication The authors institutions were ranked according to a system 14,15 based on total dollars of National Institutes of Health (NIH) grant support. 254 JAMA, July 15, 1998 Vol 28, No. 3 Postive-Outcome Bias in Research Abstracts Callaham et al

2 Table 1. Characteristics Predicting Acceptance for Presentation in Main Group of 38 Submitted Studies* Variable Results Adjusted Odds Ratio (95% CI) Results Positive outcome 1.99 ( ) Originality score Low, medium, high 2.7 ( ) Quality score Low, medium, high 1.53 ( ) *Excludes surveys, simulations, and descriptions of methods. See Methods section for all variables tested. Higher odds ratios indicate greater probability of acceptance. CI indicates confidence interval. Logistic regression analysis vs null model 2 12 = 5.; P.1. Copas-le Cessie-van Houwelingen-Hosmer-Lemeshow goodness-of-fit test; P =.86. There is no standardized definition of positive results. 16 We used one of the more common definitions that results were positive if the studied variable produced positive (beneficial) results. 1,3,17,18 Some authors have defined positive results as those reporting statistically significant results (regardless of direction), 4,5,16,19-21 so we also used this definition. We performed the major logistic regression analysis on those studies in which the subjects were either humans oranimals,andthedesignwasaprospective interventional trial, a prospective observational study, or a retrospective observational study. We also separately examined prospective studies with controls, excluding retrospective studies. We used a general iterative model-building strategy, as suggested by Hosmer and Lemeshow 22 and Harrell, 23 assessed for goodness of fit and subjected to bootstrap validation using S-Plus version 4., release 3 (Mathsoft Inc, Seattle, Wash) and Harrell extensions ( stat.cmu.edu). Correlation coefficients were calculated using JMP software for Macintosh (SAS Institute, Inc, Cary, NC). We calculated the effect size of all interventions for the subgroup of prospective studies with controls in the usual fashion, except that SDs were not available. RESULTS Five hundred research abstracts were submitted to the SAEM selection committee from a total of 144 institutions, 13 of which had formal US medical school affiliations. Eight duplicates were deleted, leaving 492 abstracts as the basis of our study. The submitting schools averaged an NIH funding rank of 55 (of all US medical schools) (95% confidence interval [CI], 49-62). Thirty percent of studies did not state a hypothesis, and 49% did not report on blinding, 24% on randomization, and 74% onexclusioncriteria. Seventy-sixpercent of the studies were conducted on humans, 1% on animals, and the remainder on othermodels. Twenty-ninepercentofthe studies were retrospective, 27% prospective observational with control groups, and 26% prospective interventional trials. Sixty-six percent of all submitted interventional trials had positive outcome by our initial definition, 83% by the positive P value definition, and 8% by effect size. Respective figures for all submitted observational prospective studies were 7%, 92%, and 8%. Three hundred eighty studies met the criteriaforlogisticregression(see Methods ). Most measures of scientific merit did not predict the decision to accept an abstract for presentation. Instead, this decision was most strongly related to positive results and the reviewers subjective originality score, while controlling for institutional funding, study design, randomization, blinding, controls, exclusion criteria, and sample size (Table 1). Results were similar for the subgroup of 166 prospective studies with control groups except that sample size greater than 5 was also predictive (odds ratio [OR], 2.4; 95% CI, ) and similarly controlled. Full regression results are available from the authors. One hundred seventy-nine(36%) of the submitted abstracts were accepted for presentation at the meeting (11 reports onteachingmethodswereexcluded). The SAEMcommitteescoresdeterminingacceptance correlated best with our subjective quality scale (R =.57) and originality factor (R =.49). Two hundred fourteen (43%) of the 492 studies submitted were published, an average of 18 months after presentation, in 44 journals with impact factors ranging from.23 to A follow-up questionnaire to authors of unpublishedpaperswasreturnedby226authors and identified 21 publications not found in MEDLINE. 9 One hundred four (49%) of the 214 studies ultimately published were rejected for presentation at the meeting. The mean impact factor of the publishing journal did not differ for those papers rejected for the meeting vs those accepted (1.48 vs 1.19; P =.47), nor did time to publication. One hundred forty-seven studies (7%) were published in emergency medicine specialty journals. The remaining studies were published in 39 other journals, including American Journal of Public Health, Annals of Internal Medicine, JAMA, The New England Journal of Medicine, Pediatrics, and Stroke. Of all the studies published after the meeting, 38% had been published 1 year later, Table 2. Characteristics Predicting Peer-Reviewed Publication of a Manuscript in Main Group of 38 Submitted Studies* Variable Results Adjusted Odds Ratio (95% CI) No. of subjects ( ) for meeting vs rejected 2.49 ( ) *Excludes surveys, simulations, and descriptions of methods. See Methods section for all variables tested. Higher odds ratios indicate greater probability of publication. CI indicates confidence interval. Logistic regression analysis vs null model 2 13 = 4.4; P.1. Copas-le Cessie-van Houwelingen-Hosmer-Lemeshow goodness-of-fit test; P = % in 2 years, 88% in 3 years, and 95% in 4 years. Publication of a full manuscript in the main group of 38 studies was related most strongly to abstract acceptance and sample size, controlling for the previous variables by logistic regression (Table 2). Results in the 166 prospective studies with control groups were identical. Results of these analyses did not differ using either of the 2 definitions of positive outcome ( Methods ). Data allowing calculation of effect size was reported in only 122 (66%) of 186 prospective studies. Positive-outcome bias was evident for all studies submitted before any screening (Figure). Submissions with less positive effect size were then disproportionately rejected for presentation. At the level of publication of full manuscripts, the same bias again appeared. A funnel plot (Figure) shows the absence of expected negative effect sizes at low sample size, which is the hallmark of positive-outcome bias. Numerical testing confirms the funnel plots. The mean effect size of all submitted papers was.71 (95% CI,.4-1.1). The mean effect size of papers accepted for the meeting was.92 vs.45 for those rejected. The mean effect size of papers eventually published was.96 vs.45 for those never published (P =.3, Kruskal- Wallis analysis of variance). Effect size contributed much more to acceptance or publication than study sample size. LIMITATIONS Our study was limited to 1 specialty, but 13 medical schools contributed and the publication rate was comparable with 31 other academic society meetings. 6 Research from this meeting was published in 44 journals, 39 of them outside this specialty. The emergency medicine literature, the abstracts we studied, and the general medicine literature are identical in the proportion of studies with positive outcomes. 1 Effect size data were available for only a minority of studies, and most studies were not randomized controlled trials. These subgroups might not be representative of JAMA, July 15, 1998 Vol 28, No. 3 Postive-Outcome Bias in Research Abstracts Callaham et al 255

3 Rejected all 492 abstracts, but the results between groups were similar and consistent. COMMENT Presentation of scientific studies at meetings is an important part of the dissemination of knowledge, but half of these studies appear only as abstracts and never undergo any other peer review. 5 Whether the abbreviated peer review used to select abstracts for meetings actually identifies scientific merit is unknown, yet abstracts are cited as often as fully published papers. 24 We reviewed all submitted research, not just studies accepted for presentation, assessing those characteristics previously suggested to predict publication. 2-5,14,15 Our results show that acceptance of an abstract for presentation at the meeting was not strongly related to study design, methods, sample size, or even a subjective quality score. Instead, a subjective originality factor and presence of positive results best predicted acceptance (ORs, 2.7 and 1.99, respectively), regardless of study design. Publication as a full manuscript was best predicted by whether the abstract had been accepted at the meeting (OR, 2.49) and large sample size (OR, 2.26), again independent of study design or scientific quality. Positive studies were preferentially accepted during both the Published Not Published Funnel plot of all 122 prospective studies submitted to the meeting that reported effect sizes (see text). Studies subsequently accepted for presentation at the meeting are plotted separately from those rejected. Inset graph similarly shows results for all submitted research with effect-size data at the journal publication level. The absence of the left (negative) base of the inverted funnel suggests the presence of selection bias in favor of positive outcomes. acceptance and publication decisions (P =.3), which is illustrated in the funnel plots (Figure). Positive-outcome bias has been documented previously in publication of full journal articles, but not in detail at the meeting acceptance level. 3,4,18 Four studies examined, in limited ways, the publication of studies after acceptance for presentationatmeetings. 5,25-27 Abstracts submitted on the single subject of gestational exposure to cocaine demonstrated positive-outcome bias in acceptance. 28 The impact of sample size and positive outcome on acceptance and publication was reported for cancer abstracts. 24 Our study examined all submitted research from a broad cross-section of institutions, with subsequent publication in a broad variety of journals. Positiveeffect (or publication) bias was already present when studies were first submitted for consideration (Figure). Presumably this was due to authors who did not complete or submit smaller studies with negative effects, perhaps after experiencing a tradition of publication bias by meeting selection committees and scientific journals. The selection process for presentation at the meeting further increased this positive-outcome bias (Figure). Logistic regression showed that an intangible originality ( newsworthiness ) factor and positive outcome were more strongly associated with acceptance than traditional measures of scientific quality, such as study design, randomization, sample size, and blinding(tables 1 and 2). Positive-outcome bias appeared again in the selection process for publication in a journal. Full publication was best predicted by acceptance at the meeting and study size, rather than study methods or quality. Our results confirm a smaller study of the cancer literature, which did not control for scientific quality. 24 A number of potential solutions, such as trials registries, have been proposed to remedy positive-outcome bias. 21,29,3 We offer one more solution: that all studies submitted to scientific meetings be published as abstracts, indicating whether or not they were chosen to be presented. This might encourage researchers to submit studies with negative findings, and readers and researchers could more easily identify the entire spectrum of research. Journals might adopt a similar practice, publishing the abstracts of all submitted manuscripts. Despite the mandatory structured format, 49% of SAEM abstracts failed to report adequately about blinding, 74% about exclusion criteria, 24% about randomization, and 14% about sample size. Perhaps because these deficiencies made the merit of the research difficult to evaluate, acceptance for presentation at the meeting predicted publication as a full manuscript with a sensitivity of only 51%, a specificity of 71%, a positive predictive value of 57%, and a negative predictive value of 66%. We wish to thank Mary Ann Schropp, executive director of the SAEM, for her generous help with data, and John Gallagher, MD, for sharing the institutional research funding data. References 1. Moscati R, Jehle D, Ellis D, Fiorello A, Landi M. Positive-outcome bias: comparison of emergency medicine and general medicine literatures. Acad Emerg Med. 1994;1: Dickersin K, Chan S, Chalmers TC, Sacks HS, Smith H Jr. Publication bias and clinical trials. Control Clin Trials. 1987;8: Dickersin K, Min Y-I, Meinert CL. Factors influencing publication of research results: follow-up of applications submitted to two institutional review boards. JAMA. 1992;267: EasterbrookPJ, BerlinJA, GopalanR, Matthews DR. Publication bias in clinical research. Lancet. 1991;337: Scherer RW, Dickersin K, Langenberg P. Full publication of results initially presented in abstracts: a meta-analysis. JAMA. 1994;272: Wuerz R, Holliman J. Attendance and research abstract activity at the 1993 annual meetings of the academic medical societies. Acad Emerg Med. 1994; 1:A JAMA, July 15, 1998 Vol 28, No. 3 Postive-Outcome Bias in Research Abstracts Callaham et al

4 7. Chalmers T, Smith H, Blackburn B, et al. A method for assessing the quality of a randomized controlled trial. Control Clin Trials. 1981;2: Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol. 1991;44: Weber EJ, Callaham ML, Wears RL, Barton C, Young G. Unpublished research from a medical specialty meeting: why investigators fail to publish. JAMA. 1998;28: Garfield E. SCI Journal Citation Reports: A Information Inc; Garfield E. SCI Journal Citation Reports: A Information Inc; Garfield E. SCI Journal Citation Reports: A ISI Database; Philadelphia, Pa: Institute for Science Information Inc; Garfield E. SCI Journal Citation Reports: A Information Inc; Gallagher E, Goldfrank L, Anderson G, et al. Current status of academic emergency medicine within academic medicine in the United States. Acad Emerg Med. 1994;1: Garfunkel JM, Ulshen MH, Hamrick HJ, Lawson EE. Effect of institutional prestige on reviewers recommendations and editorial decisions. JAMA. 1994;272: Olson CM. Publication bias. Acad Emerg Med. 1994;1: Simes RJ. Publication bias: the case for an international registry of clinical trials. J Clin Oncol. 1986;4: DickersinK, MinYI. NIHclinicaltrialsandpublication bias. Online J Curr Clin Trials [serial online]. 1993;doc Rennie D, Flanagin A. Publication bias: the triumph of hope over experience. JAMA. 1992;267: Berlin JA. Will publication bias vanish in the age of online journals? Online J Curr Clin Trials [serial online]. 1992;doc Begg CB, Berlin JA. Publication bias and dissemination of clinical research. J Natl Cancer Inst. 1989;81: Hosmer D, Lemeshow S. Applied Logistic Regression.NewYork,NY:JohnWiley&SonsInc; Harrell FE. Predicting Outcomes: Applied Survival Analysis and Logistic Regression. Charlottesville: Dept of Health Evaluation Sciences, School of Medicine, University of Virginia; de Bellefeuille C, Morrison C, Tannock I. The fate of abstracts submitted to a cancer meeting: factors which influence presentation and subsequent publication. Ann Oncol. 1992;3: Goldman L, Loscalzo A. Fate of cardiology research originally published in abstract form. N Engl J Med. 198;33: Meranze J, Ellison N, Greenhow D. Publications reseulting from anesthesia meeting abstracts. Anesth Analg. 1982;61: McCormick M, Holmes J. Publication of research presented at the pediatric meetings. AJDC. 1985;139: Koren G, Graham K, Shear H, Einarson T. Bias against the null hypothesis: the reproductive hazards of cocaine. Lancet. 1989;2: Newcombe RG. Towards a reduction in publication bias. BMJ. 1987;295: Dickersin K. The existence of publication bias and risk factors for its occurrence. JAMA. 199;263: Weintraub WH. Are published manuscripts representative of the surgical meeting abstracts? an objective appraisal. J Pediatr Surg. 1987;22: Unpublished Research From a Medical Specialty Meeting Why Investigators Fail to Publish Ellen J. Weber, MD; Michael L. Callaham, MD; Robert L. Wears, MD; Christopher Barton, MD; Gary Young, MD Context. It is not known whether peer review of research abstracts submitted to scientific meetings influences subsequent attempts at publication. Objective. To determine why research submitted to a scientific meeting is not subsequently published. We hypothesized that authors of abstracts rejected by a meeting are less likely to pursue publication than those whose abstracts are accepted, regardless of research quality. Design and Participants. Blinded review of abstracts submitted to a medical specialty meeting in 1991 and not published as full manuscripts within 5 years. In 1996, authors of 266 unpublished studies were asked to complete questionnaires. Main Outcome Measures. Submission of a full manuscript to a journal between 1991 and 1996; failure to submit a manuscript to a journal because the investigator believed it would not be accepted for publication. Results. A total of 223 (84%) of the unpublished investigators returned the questionnaire. Only 44 (2%) had submitted manuscripts to a journal. Manuscript submission was not associated with abstract quality (odds ratio [OR], 1.16; 95% confidence interval [CI], ), positive results (OR,.75; 95% CI, ), or other study characteristics. Having an abstract accepted for presentation at the meeting weakly predicted submission of a manuscript to a journal (OR, 1.88; 95% CI, ). Authors of accepted abstracts were significantly less likely to believe a journal would not publish their manuscript than were authors of rejected abstracts (OR,.23; 95% CI,.1-.61). Conclusions. Study characteristics do not predict attempts to publish research submitted to a scientific meeting. Investigators whose research is rejected by a meeting are pessimistic about chances for publication and may make less effort to publish. JAMA. 1998;28: UNDERREPORTING OF research is a well-recognized problem with serious implications for clinical practice. 1 Most unpublished research is never submitted to a journal for consideration. 2-6 Previous investigations suggest researchers are more likely to attempt to publish studies with positive outcomes (publication bias). 2-4 Much of the research submitted to scientific meetings is never published, 5-12 yet it is not known whether acceptance or rejection of the abstract From the Division of Emergency Medicine (Drs Weber, Callaham, and Young), University of California, San Francisco; the Division of Emergency Medicine, University of Florida Health Center, Jacksonville (Dr Wears); the Department of Emergency Medicine, University of North Carolina, Chapel Hill (Dr Barton); and the Department of Emergency Medicine, Highland Hospital, Alameda County Medical Center, Oakland, Calif (Dr Young). Dr Young is now with Sacred Heart Medical Center, Eugene, Ore. Presented at the Third International Congress on Peer Review in Biomedical Publication, Prague, Czech Republic, September 19, Corresponding author: Ellen J. Weber, MD, Division of Emergency Medicine, University of California, San Francisco, Box 28, San Francisco, CA ( weber@itsa.ucsf.edu). Reprints not available from the authors. JAMA, July 15, 1998 Vol 28, No. 3 Why Investigators Fail to Publish Weber et al 257

5 cents. We agree that the methods used in our study do not provide the most accurate assessment of vigorous physical activity in children. While use of measurement devices such as heart rate recording or accelerometers is preferable, such measurements are not practical in a large national survey. Although the accuracy of self-report of vigorous activity from children does warrant examination, it is unlikely that most parents could provide data that were more reliable. The subjective assessmentofvigorousphysicalactivitybyachildmayalsodifferfrom what parents consider vigorous, especially among overweight children. We continue to stress that all health care professionals should create opportunities to encourage youth to minimize sedentary behaviors and to participate in physical activity that may be continued for a lifetime of healthy living. Ross E. Andersen, PhD The Johns Hopkins University School of Medicine Baltimore, Md Carlos Crespo, DrPH, MS American University Washington, DC Susan J. Bartlett, PhD Johns Hopkins University School of Medicine Michael Pratt, MD, MPH Centers for Disease Control and Prevention Atlanta, Ga 1. Goran MI, Shewchuk R, Gower BA, Nagy TR, Carpenter WH, Johnson RK. Longitudinal changes in fatness in white children: no effect of childhood energy expenditure. Am J Clin Nutr. 1998;67: Dietz WH, Gortmaker SL. Do we fatten our children at the television set? Pediatrics. 1985;75: The JAMA Patient Page To the Editor. Over the past few decades, the demand for consumer health-related information has increased dramatically. Easy-to-understand health-related information correlates with increased patient satisfaction. 1-4 Patient education is both a science and an art. I was pleased to see the JAMA Patient Page, 5 and I find it useful in educating patients. I have evaluated many patient-oriented computer software programs and databases, and I found the Patient Page to be complete, accurate, and useful. However, in future issues, extra attention should be given to the type size and sentence spacing. 6 Type size is crucial to readability; at least 12-point type is generally recommended, especially for older patients. Adequate spacing between letters, lines, and paragraphs also enhances readability. Typically, text spacing should have no more than 3 kerning (space between letters), while sentence spacing should have 2- to 4-point leading (space between lines). By enhancing readability, the JAMA Patient Page would increase the benefit to the elderly, the fastest-growing segment of the population. Candy Tsourounis, PharmD University of California, San Francisco School of Pharmacy 1. Stein MD, Fleishman J, Mor V, Dresser M. Factors associated with patient satisfaction among symptomatic HIV-infected persons. Med Care. 1993;31: Schauffler HH, Rodriguez T, Milstein A. Health education and patient satisfaction. J Fam Pract. 1996;42: Robbins JA, Bertakis KD, Helms LJ, Azari R, Callahan EJ, Creten DA. The influence of physician practice behaviors on patient satisfaction. Fam Med. 1993;25: Laine C, Davidoff F, Lewis CE, et al. Important elements of outpatient care: comparison of patients and physicians opinions. Ann Intern Med. 1996;125: Glass RM, Molter J, Hwang MY. Providing a tool for physicians to educate patients: the JAMA Patient Page. JAMA. 1998;279: US Department of Health and Human Services. Prescription information program launched. Federal Register. 1997;62:551. In Reply. We appreciate the feedback about the JAMA Patient Page and hope that the page is being well used for its intended purposes to help facilitate communication between patients and health care professionals and to educate patients about medical issues. We are attempting to provide accurate, helpful medical information while staying within the constraints of a single page. With the space limitation, it is an ongoing challenge to determine what information to include that would be helpful to patients. Making the information readable and understandable, while also being thorough and accurate, are major goals in producing the page on a weekly basis. Currently, the main-body text of the Patient Page is set in 1/1 type, and the secondary text (symptoms, diagnosis, treatment, etc) is set in 9/1 type, which is consistent with the rest of the text in JAMA. There is no kerning (condensing the space between the letters of words) on the page. We make every effort to enhance readability through design, use of white space, and pull-out information. Although we would like to increase the size of the font, doing so would seriously limit the amount of information we can include on the page. JAMA plans to conduct focus groups to see how useful the Patient Page has been for medical professionals and patients. We will explore issues of readability, value of the information, and ways to improve the page. After more comprehensive feedback about the page, we will consider whether changes need to be made to improve the overall quality of the Patient Page. Mi Young Hwang, MSJ Writer, JAMA Patient Page Richard M. Glass, MD Deputy Editor, JAMA CORRECTION Incorrect Figure. In the Peer Review article entitled Positive- Outcome Bias and Other Limitations in the Outcome of Research Abstracts Submitted to a Scientific Meeting, published in the July 15, 1998, issue of THE JOURNAL (1998;28: ), the Figure was printed with the incorrect inset graph. The inset graph should be a linear plot of the studies accepted for presentation, not a log axis plot. The Figure is correct as printed below. Rejected Published Not Published Funnel plot of all 122 prospective studies submitted to the meeting that reported effect sizes (see text). Studies subsequently accepted for presentation at the meeting are plotted separately from those rejected. Inset graph similarly shows results for all submitted research with effect-size data at the journal publication level. The absence of the left (negative) base of the inverted funnel region suggests the presence of selection bias in favor of positive outcomes JAMA, October 14, 1998 Vol 28, No. 14 Letters