Statistical Consideration for Bilateral Cases in Orthopaedic Research

Transcription

1 1732 COPYRIGHT Ó 2010 BY THE JOURNAL OF BONE AND JOINT SURGERY, INCORPORATED Statistical Consieration for Bilateral Cases in Orthopaeic Research By Moon Seok Park, MD, Sung Ju Kim, MS, Chin Youb Chung, MD, In Ho Choi, MD, Sang Hyeong Lee, MD, an Kyoung Min Lee, MD Investigation performe at Seoul National University Bunang Hospital, Sungnam, Kyungki, South Korea Backgroun: Statistical inepenence means that one observation is not affecte by another; however, the principle of statistical inepenence is violate if left an right-sie measures within a subject are consiere to be inepenent, because they are usually correlate an can affect each other. The purpose of the present stuy was to analyze the violation of statistical inepenence in recent orthopaeic research papers an to emonstrate the effect of statistical analysis that consiere the ata epenency within a subject. Methos: First, all original articles that ha been publishe in The Journal of Bone an Joint Surgery (American Volume) over a two-year perio were evaluate. The analysis was esigne to ientify articles that inclue bilateral cases an possible violations of statistical inepenence. Secon, a emonstrative logistic regression without consieration of statistical inepenence was performe an was compare with a statistical analysis that consiereata epenency within a subject. Raiographs of 1200 hips in 600 patients were use to examine the ifferences in terms of os ratios (with 95% confience intervals) of the risk factors for hip osteoarthritis. Results: Four hunre an eighty-six original articles were reviewe, an 151 articles (incluing forty-one articles involving the hip, thirty-four involving the knee, twenty-one involving the foot or ankle, nineteen involving the shouler, ten involving the han or wrist, nine involving the elbow, an seventeen involving other structures) were consiere to inclue bilateral cases. Of the 486 articles that were reviewe, 120 articles (25%) (incluing thirty-six articles involving the hip, twenty-six involving the knee, fifteen involving the foot or ankle, fourteen involving the shouler, seven involving the elbow, six involving the han or wrist, an sixteen involving other structures) were foun to have possibly violate statistical inepenence. Demonstrative statistical analysis showe that logistic regression was not robust to the violation of statistical inepenence. The 95% confience intervals of the os ratios for the risk factors showe narrower ranges (1.13 to 2.68 times) when ata epenency within a subject was not consiere. Conclusions: Researchers nee to consier statistical inepenence when performing statistical analysis, particularly in stuies involving bilateral cases. If ata epenency within a subject is not consiere, stuies involving bilateral cases can bias results, epening on the context of those stuies. Given 1-7 the tren towar evience-base meicine, meical researchers commonly use statistical analyses to emonstrate stuy hypotheses objectively. However, one of the most important assumptions in statistical analysis when comparing inepenent groups is that the observations are inepenent of each other 8. Nevertheless, investigators in orthopaeic research frequently encounter cases of bilateral involvement in which the left an right sies are possibly correlate with each other. The principle of statistical inepenence coul be violate if, in these cases, the left an rightsie measures are consiere to be inepenent espite their correlation 9. By efinition, statistical inepenence means that each observation is not affecte by another observation. The effects of bilateral cases have raise concerns in orthopaeic research, epening on the hypothesis. First, if a stuy inclues bilateral cases, researchers may be face with a ilemma as to whether to analyze the joints or the patients. Joint-base analysis coul cause the emographic ata to be uplicate, an patient-base analysis is likely to miss the clinical implications associate with unilaterality an bilaterality. Secon, the inclusion of both joints in statistical analysis violates the unerlying principle of statistical ine- Disclosure: The authors i not receive any outsie funing or grants in support of their research for or preparation of this work. Neither they nor a member of their immeiate families receive payments or other benefits or a commitment or agreement to provie such benefits from a commercial entity. J Bone Joint Surg Am. 2010;92: oi: /jbjs.i.00724

2 1733 penence an coul bias the stuy results by exaggerating the levels of significance an narrowing the confience intervals if the joints in same iniviuals are significantly relate to each other 10,11. In the present stuy, we hypothesize that a consierable amount of orthopaeic research oes not properly consier the principle of statistical inepenence in bilateral cases. Although reports may mention that right an left sies are closely correlate, this correlation between right an left sies shoul be accounte for when analyzing ata 9, Therefore, the purpose of the present stuy was to assess the possible violation of statistical inepenence in orthopaeic research involving bilateral cases an to emonstrate the effect of statistical analysis that consiers the ata epenency within a subject. Materials an Methos The present stuy was approve by the institutional review boar at our institute. It consiste of two parts: (1) a journal review an (2) a emonstration of the effects of the statistical analysis consiering the ata epenency within a subject. The journal review was performe by two orthopaeic surgeons (M.S.P. an K.M.L.) to assess consierations of bilaterality when performing statistical analysis in orthopaeic research. Articles publishe in The Journal of Bone an Joint Surgery (American Volume) (JBJS-Am) were analyze. The articles that were inclue met the following criteria: (1) publication between January 2007 an December 2008 an (2) the presentation of original research regaring either the upper or lower extremity. The articles that were exclue inclue (1) case reports, (2) Current Concepts Reviews, (3) eitorials, (4) Instructional Course Lectures, (5) basic-science stuies, (6) economic stuies, (7) meta-analyses, an (8) stuies of spinal iseases. For each article that met the above criteria, we recore the number of subjects (persons) an cases (limbs, joints, muscles, an so on) along with the specific boy parts involve (hips, knees, shoulers, an so on). The articles were categorize accoring to criteria establishe among three of the authors (M.S.P., S.H.L., an K.M.L.). Articles with the same number of subjects an cases were classifie as unilateral stuies, an articles with ifferent numbers of subjects an cases were classifie as potentially bilateral stuies an were analyze further. The articles in which the subject or case numbers were not inicate in the text were assigne to Category 1 ( possible bilateral cases, but not clarifie in the text ). The statistical methos of the remaining stuies were then reviewe further in the Materials an Methos section an were assigne to Category 2 ( bilateral cases consiere to be inepenent cases in the statistical analysis ), Category 3 ( bilateral cases with one representative measurement [average or one selecte sie] inclue in the statistical analysis ), Category 4 ( bilateral cases separate from unilateral cases ), Category 5 ( bilateral cases statistically consiere in terms of correlations between right an left sies ), Category 6 ( bilateral cases not subjecte to statistical analysis [i.e., only escriptive analysis performe]), or Category 7 ( bilateral cases in which the contralateral sies were use as controls ). Of these seven categories, Categories 1 an 2 were consiere to possibly violate the principle of statistical inepenence an Categories 3 through 7 were consiere soun in this respect. The secon part of this stuy involve examining the os ratios (with 95% confience intervals) of the risk factors for hip osteoarthritis in elerly patients. Six hunre patients with an age of more than sixty-five years who ha been evaluate with anteroposterior weight-bearing hip raiographs at our hospital from September 2005 to August 2006 were selecte consecutively. Patients with total joint arthroplasty, previous osseous surgery, or severe joint estruction ue to infection, trauma, or tumor were exclue. On the hip raiographs, the joint space with was assesse in three areas: (1) the lateral en of the subchonral sclerotic line, (2) a vertical line from the center of the femoral hea, an (3) the meial en of the weight-bearing surface near the fovea. The minimum joint space with was selecte as the smallest of these three measurements or as another measurement if the minimum joint space with was foun outsie the above three areas. Hip osteoarthritis was efine as a minimum joint space with of 2.5 mm The emographic ata, incluing age, sex, an boy-mass inex, were recore as caniate risk factors for hip osteoarthritis. After setting the cutoff values for the continuous variables (seventy years for age an 25 kg/m 2 for boymass inex) with coing, the os ratio (an 95% confience interval) of each caniate risk factor for hip osteoarthritis was calculate with use of logistic regression, a generalize estimating equation 17, an multilevel moeling 18,19, the latter two of which consiere the correlation between the right an left hips. The os ratios (with 95% confience intervals) were calculate for all 600 patients as well as for 300, 100, an fifty patients who were selecte ranomly by means of computergenerate ranom numbers with use of a uniform istribution. All raiographic images were acquireigitally with use of a picture archiving an communication system (PACS) (IM- PAX; Agfa, Mortsel, Belgium). All raiographic measurements were carrie out with use of PACS software by a single orthopaeic surgeon (K.M.L.) who was unaware of the clinical information. For this part of the stuy, caniate risk factors were coe as binary ata accoring to the cutoff values. The os ratios of each caniate risk factor were calculate with use of logistic regression, the generalize estimating equation moel, an multilevel moeling an are presente with the 95% confience intervals. The level of significance was set at p < Source of Funing There was no external funing source relate to this investigation. Results Four hunre an eighty-six original articles were foun in JBJS-Am uring the inex perio. Of these, we exclue

3 1734 TABLE I Categories of the 351 Original Articles Inclue in Present Stuy Category No. of Articles Boy Parts Articles containing only unilateral cases knee, 27 hip, 36 shouler, 23 elbow, 17 foot/ankle, 12 wrist/han, an 44 others Articles containing bilateral cases hip, 34 knee, 21 foot/ankle, 19 shouler, 9 elbow, 10 wrist/han, an 17 others Articles possibly violating statistical inepenence hip, 26 knee, 15 foot/ankle, 14 shouler, 7 elbow, 6 wrist/han, an 16 others Not clarifie 33 Bilateral cases analyze as inepenent 87 Articles not violating statistical inepenence 31 8 knee, 6 foot/ankle, 5 hip, 5 shouler, 2 elbow, 4 wrist/ han, an 1 other One representative measurement 6 Unilateral cases separate from bilateral cases 4 Statistically consiere for correlation between 6 right an left sies No statistical analysis 10 Other sie as matche control articles concerning spinal iseases, meta-analyses, economic stuies, an basic-science stuies. Of the remaining 351 articles (incluing seventy-five articles on the knee, sixty-eight articles on the hip, fifty-five articles on the shouler, thirtyeight articles on the foot or ankle, thirty-two articles on the elbow, twenty-two articles on the han an wrist, an sixty-one other articles regaring multiple sites or specific bones an tenons), 200 articles inclue only unilateral cases. Finally, 151 articles (forty-one articles on the hip, thirty-four articles on the knee, twenty-one articles on the foot or ankle, nineteen articles on the shouler, ten articles on the han or wrist, nine articles on the elbow, an seventeen other articles regaring TABLE II Change in the Confience Interval with Various Subject Numbers Accoring to Statistical Metho Use* Logistic Regression* Generalize Estimating Equation Multilevel Moel Risk Factors Os 95% Confience Interval (A) Os 95% Confience Interval (B) (B/A) Os 95% Confience Interval (C) (C/A) 600 persons, 1200 joints Age to to to Boy-mass inex to to to Sex to to to persons, 600 joints Age to to to Boy-mass inex to to to Sex to to to persons, 200 joints Age to to to Boy-mass inex to to to Sex to to to persons, 100 joints Age to to to Boy-mass inex to to to Sex to to to *Logistic regression was performe assuming that the left an right hips are inepenent. The generalize estimating equation an the multilevel moel were use to consier the correlation between both hips.

4 1735 Fig. 1 Bar graph showing the proportion of bilateral cases in each stuy that possibly violate the principle of statistical inepenence. In the graph, the x axis represents the percentage of bilateral cases in each article that possibly violate statistical inepenence. multiple sites or specific bones an tenons) were classifie as potentially bilateral stuies. Thirty-three stuies i not clarify the numbers of subjects or cases. Eighty-seven stuies analyze sies as inepenent cases. Six stuies analyze representative ata from a single subject by averaging or selecting one sie. Four stuies separate unilateral cases from bilateral cases an analyze the ata separately. Six stuies consiere the correlation between right an left-sie ata in the statistical analysis. Ten stuies i not involve a statistical analysis, an five stuies use contralateral sies as matche controls uring statistical analysis. Therefore, of the 486 original articles, 120 articles (thirty-six articles on the hip, twenty-six articles on the knee, fifteen articles on the foot or ankle, fourteen articles on the shouler, seven articles on the elbow, six articles on the han or wrist, an sixteen other articles) were consiere to have possibly violate statistical inepenence (Table I); in 77% of these stuies, <30% of the total number of cases were bilateral (Fig. 1). Raiographs of the hip for the 258 male an 342 female patients were retrieve. The mean age of the 600 consecutive patients selecte for raiographic measurements was 71.4 ± 5.0 years. The mean boy-mass inex was 24.4 ± 3.2 kg/m 2. The overall prevalence of hip osteoarthritis was 13.3%. The prevalence of unilateral an bilateral hip osteoarthritis was 9.8% an 3.5%, respectively. The correlation of joint space with (in millimeters) between right an left hips within a subject was When correlation between the right an left hips was not consiere (that is, when the ata were evaluate with logistic regression), the 95% confience intervals of the os ratios of each caniate risk factor were 1.13 to 2.68 times narrower than the ranges when correlation of the right an left hips was consiere (that is, when the ata were evaluate with the generalize estimating equation an multilevel moeling). The ifference in the with of the 95% confience intervals between logistic regression an generalize estimating equation (or multilevel moeling) appeare to ecrease with increasing number of cases (Table II); however, this result nees to be investigate further to etermine its significance. Discussion Aconsierable proportion (25%) of original orthopaeic research articles in one meical journal were foun to have potentially violate the principle of statistical inepenence as they i not consier the possible ata epenency within a subject. In our stuy of the prevalence of osteoarthritis, the risk factors for hip osteoarthritis showe narrower 95% confience intervals for the os ratios when the correlation between the right an left hip was not consiere. This fining suggests that

5 1736 ignoring the ata epenency within a subject coul bias the stuy results. The aim of this secon part of the stuy was not to examine the prevalence of or risk factors for hip osteoarthritis but to emonstrate the effects of correlation between the two sies on statistical analysis. These results shoul be iscusse in terms of their statistical aspects an not their orthopaeic aspects. Before the implications of this stuy are aresse, its limitations shoul be iscusse. First, in our analysis, the stuies that i not clarify the number of cases an subjects were consiere to have violate statistical inepenence, which coul have exaggerate the number of stuies that actually violate the principle of statistical inepenence. Secon, we evaluate articles from only one orthopaeic surgery journal. Thir, the secon part of this stuy, which emonstrate the effect of the metho of statistical analysis an consiereata epenency within a subject, coul not represent all clinical or research situations. Therefore, the issue of statistical inepenence shoul be assesse on the basis of the specific research hypothesis being teste. Fourth, the secon part of the stuy was performe in the specific setting of 100% bilateral cases with a correlation coefficient of 0.761, with specific numbers of subjects (fifty, 100, 300, an 600). We believe that the egree of the correlation, the proportion of bilateral cases, an the number of cases all can affect the stuy results. Consequently, these results may not be generalizable; however, we believe that the present stuy has shown the effect of violating the principle of statistical inepenence. The use of bilateral cases in orthopaeic research has several ifferent implications. First, statistical power an comparability coul be enhance if one sie is assigne to a stuy group an the other sie is assigne to a matche control group, because oing so coul eliminate or reuce the effect of some confouning variables. Secon, mixing subjects an cases can result in a isparity between subject-base results an joint-base results, sometimes leaing to absur conclusions. For instance, Anersen escribe a trial that resulte in the eaths of 22% of patients but only 16% of legs 20. Thir, the inclusion of unilateral an bilateral cases in the same group coul violate the principle of statistical inepenence. If this principle is violate, the calculate p values or 95% confience intervals may be spurious, within-group variabilities may be unerestimate, an sample sizes may be inaccurately estimate 21. Despite the fact that several stuies have raise these concerns 9,11,13, ata epenency has not been affore sufficient attention. One quarter of the articles that were examine in the present stuy may have violate the principle of statistical inepenence, an various numbers of cases an percentages of bilateral cases were reporte in these articles. It is possible that both the percentage of bilateral cases an the number of stuy subjects can affect the robustness of the statistical analysis. However, this hypothesis was not teste in the present stuy. Aitional stuy with use of a Monte Carlo simulation may ientify the number of cases an the portion of bilateral cases that woul be allowable so as not to introuce unacceptable bias. Several methos have been propose, such as hierarchical moeling (also calle multilevel moeling) an the use of a generalize estimating equation 17 to ajust for any epenency within subjects. These methos are known to improve the statistical power of a stuy an provie guiance regaring the interpretation of site-specific results. We believe that investigators can obtain more precise statistical results by consiering statistical inepenence in stuies involving bilateral cases. n NOTE: The authors thank Mi Seon Ryu for ata collection. Moon Seok Park, MD Chin Youb Chung, MD Sang Hyeong Lee, MD Kyoung Min Lee, MD Department of Orthopaeic Surgery, Seoul National University Bunang Hospital, 300 Gumi-Dong, Bunang-Gu, Sungnam, Kyungki , South Korea. aress for K.M. Lee: oasis100@empal.com Sung Ju Kim, MS Health Economics an Outcome Research, Market Access, Novartis Korea, Namaemun 5th Street, Chung-Gu, Seoul , South Korea In Ho Choi, MD Department of Orthopaeic Surgery, Seoul National University Chilren s Hospital, 28 Yongun-Dong, Chongno-Gu, Seoul , South Korea References 1. Akai M. Evience-base meicine for orthopeic practice. J Orthop Sci. 2002; 7: Barbier O, Hoogmartens M. Evience-base meicine in orthopaeics. Acta Orthop Belg. 2004;70: Bernstein J. Evience-base meicine. J Am Aca Orthop Surg. 2004;12: Bhanari M, Giannouis PV. Evience-base meicine: what it is an what it is not. Injury. 2006;37: Fisher CG, Woo KB. Introuction to an techniques of evience-base meicine. Spine (Phila Pa 1976). 2007;32(19 Suppl):S Hurwitz S. Evience-base meicine in orthopaeic surgery a way to the future. Iowa Orthop J. 2003;23: Kuhn JE, Dunn WR, Spinler KP. Evience-base meicine for orthopeic surgeons. J Knee Surg. 2005;18: Nasser R. Deciing whether there is statistical inepenence or not? J Math Stat. 2007;3: Zhang Y, Glynn RJ, Felson DT. Musculoskeletal isease research: shoul we analyze the joint or the person? J Rheumatol. 1996;23: Glynn RJ, Rosner B. Comparison of alternative regression moels for paire binary ata. Stat Me. 1994;13:

6 Sauerlan S, Lefering R, Bayer-Sanow T, Brüser P, Neugebauer EA. Fingers, hans or patients? The concept of inepenent observations. J Han Surg Br. 2003;28: Glynn RJ, Rosner B. Accounting for the correlation between fellow eyes in regression analysis. Arch Ophthalmol. 1992;110: Bryant D, Havey TC, Roberts R, Guyatt G. How many patients? How many limbs? Analysis of patients or limbs in the orthopaeic literature: a systematic review. J Bone Joint Surg Am. 2006;88: Lau EM, Lin F, Lam D, Silman A, Croft P. Hip osteoarthritis anysplasia in Chinese men. Ann Rheum Dis. 1995;54: Ali-Gombe A, Croft PR, Silman AJ. Osteoarthritis of the hip an acetabular ysplasia in Nigerian men. J Rheumatol. 1996;23: Ingvarsson T, Hägglun G, Lohmaner LS. Prevalence of hip osteoarthritis in Icelan. Ann Rheum Dis. 1999;58: Liang K-Y, Zeger SL. Longituinal ata analysis using generalize linear moels. Biometrika. 1986;73: Hox JJ. Multilevel analysis: techniques an applications. Mahwah, NJ: Lawrence Erlbaum Associates; Kreft I, e Leeuw J. Introucing multilevel moeling. Lonon: SAGE; Anersen B. Methoological errors in meical research. Oxfor: Blackwell Science; Altman DG, Blan JM. Statistics notes. Units of analysis. BMJ. 1997; 314:1874.