rheumatism Official Journal of the American Rheumatism Association

Transcription

1 arthritis and rheumatism Official Journal of the American Rheumatism Association ASSESSMENT OF RADIOLOGIC PROGRESSION IN RHEUMATOID ARTHRITIS A Randomized, Controlled Trial JAMES F. FRIES, DANIEL A. BLOCH, JOHN T. SHARP, DENNIS J. McSHANE, PATRICIA SPITZ, GILBERT B. BLUHM, DEBORAH FORRESTER, HARRY GENANT, PHILIP GOFTON, STEVEN RICHMAN, BARBARA WEISSMAN, and FREDERICK WOLFE Radiologic assessment of progressive joint destruction in rheumatoid arthritis is generally considered to be the ultimate standard for evaluation of treatment. We compared alternative radiologic techniques by performing a randomized, controlled trial in which hand films of rheumatoid arthritis patients were read by several skilled observers. The number of joints evaluated (34 versus 18) was found to make relatively little difference, but the number of readers and their experience level was critical. Films should be read in pairs. Supported in part by a grant from Sandoz Pharmaceuticals and in part by NIH grant AM to the American Rheumatism Association Medical Information System. James F. Fries, MD: Department of Medicine, Stanford University School of Medicine, Stanford, California; Daniel A. Bloch, PhD: Department of Medicine, Stanford University School of Medicine; John T. Sharp, MD: Department of Medicine, Joe and Betty Alpert Arthritis Center, Denver, Colorado; Dennis J. McShane, MD: Department of Medicine, Stanford University School of Medicine; Patricia Spitz, RN, MS: Department of Medicine, Stanford University School of Medicine; Gilbert B. Bluhm, MD: Department of Rheumatology, Henry Ford Hospital, Detroit, Michigan; Deborah Forrester, MD: University of Southern California Medical Center, Los Angeles; Harry Genant, MD: Department of Radiology, University of California Medical Center, San Francisco; Philip Gofton, MD: Department of Medicine, University of British Columbia, Vancouver, British Columbia, Canada; Steven Richman, MD: Montclair Radiologic Association, Montclair, New Jersey; Barbara Weissman, MD: Department of Radiology, Brigham and Women s Hospital, Boston, Massachusetts; Frederick Wolfe, MD: The Arthritis Center, Wichita, Kansas. Address reprint requests to Dr. James Fries, HRP Building, Room 109C, Stanford University, Stanford, CA Submitted for publication April 12, 1985; accepted in revised form June 17, Joint space narrowing and erosion scores were shown to contribute independent information. Use of recommended techniques can reduce the number of patients required and, thus, can reduce the cost of a clinical trial by more than half and can substantially increase the sensitivity and efficiency of a trial. Therefore, critical selection of the method of assessing study endpoint is of great importance. Radiologic assessment serves as an objective standard for evaluation of the progression of rheumatoid arthritis. Therapies are evaluated on the basis of their effectiveness at retarding radiologic progression, yet studies on the use of radiographs to measure disease progression have not been consistent with one another with regard to methods and procedures. At the Beaver Creek Conference on Radiologic Assessment of the Rheumatoid Arthritis Patient (Beaver Creek, CO, June 28-July 1, 1984), the authors met for the purpose of beginning to develop a consensus on radiologic assessment methods. Review of the literature (1-18) and performance of preliminary studies (19) identified areas of common agreement, as well as unresolved areas. These studies have led to general agreement that absence or presence and degree of erosions and joint space narrowing are the critical observations, that the hands and wrists are the most appropriate sites, and that a single posteroanterior (PA) film is as useful as multiple views, including oblique views. Further, the Beaver Arthritis and Rheumatism, Vol. 29, No. 1 (January 1986) 1

2 2 FRIES ET AL Creek Conference group believed that reading fine detail (standard) radiographs with a hand lens was equivalent to reading magnified (microfocal) radiographs (16). Specialized techniques (botle scan, nuclear magnetic resonance) were considered to be too unproven, expensive, or unavailable to serve as clinical trial standards at this time. Several issues remained unresolved. There was little agreement about the number of joints or which specific joints to read. Some authorities recommend joints and others recommend as few as 18 for evaluation of erosions. A reading which includes only 6 sites has been considered adequate for assessment of joint space narrowing. There is no consensus about the relative importance of erosions versus joint space narrowing scores. The preferred method of scoring erosions is in dispute; some authorities propose a simple global score for each joint (from 0 to 4, 5, or 6), while others favor counting marginal erosions (4 areas per joint) individually and summing those points for a total score. Still others have advocated counting individual erosioins and assigning a size score (1 to 4) to each specific erosion before totaling. While most authorities now favor reading of paired films (beforelafter or afterhefore) simultaneously, with films in random sequence, the superiority of this technique has never been established, and some investigators still recommend separate readings in which the before and after films for one individual are randomly scattered among films from multiple individuals. This study was designed to compare alternative techniques for scoring radiographs of the hands of individuals with rheumatoid arthritis. The goal was to define i&n optimal technique or techniques which could be used in developing more sensitive and less expensive clinical trials. A randomized, controlled trial with multiple readers was designed to provide statistically valid ainswers to the following questions: 1. What is the relative importance of erosion and joint space narrowing scores? 2. How does interobserver agreement and testretest reliability vary with different scoring techniques? 3. Can readers accurately identify and agree on progression of disease? 4. How much does prior scoring experience or specialty (rheumatologist or radiologist) affect reading skill? 5. How many joint areas should be assessed? Table 1. Methods of scoring radiographs Strategy I ( size and count ) Strategy I1 ( global ) Erosion count, 34 joints Global erosion score, 34 joints Weighted erosion count, 34 joints Joint space narrowing, 36 joints Joint space narrowing, 36 joints Global erosion score, 18 joints Erosion count, 18 joints Joint space narrowing, 6 sites Weighted erosion count, 18 joints Joint space narrowing, 6 sites 6. Do radiologic scores correlate with other variables such as disease duration and erythrocyte sedimentation rates (external validity)? 7. How does the number of readers affect the reliability and validity? 8. Should films be read separately or in pairs? 9. What are the effects of reading techniques on the power requirements, and thus the costs, of clinical trials? METHODS Previously recommended reading techniques were divided into 2 general strhtegies and subdivided into scoring methods (Table l), defined by their erosion assessmeht technique. Strategy I (size ahd count) requires specific evaluation of individual erosions, while strategy I1 (globkl) uses ohly an overall assessment of each joint. Within the size and count strategy there are 2 further scoring distinctions. An erosion count is a simple sum of erosions in the assessed joints, while a weighted erosion count also incorporates the size of each erosion (from 1-4, measured with a template). All of the erosion assessment methods were scored both for 34 joints (comprehensive) and for 18 joints (selective). Each strategy evaluates joint space narrowing in the same manner, scored both for 36 joints and for 6 sites. Eight readers were recruited; 4 were rheumatologists and 4 were radiologists. All had at least 12 years experience in interpretation of rheumatic disease patients radiographs and in teaching these techniques. Four readers (2 rheumatologists and 2 radiologists) had extensive experience in scoring radiographs in clinical trials and are hereinafter referred to as trained. A ninth experienced reader (JTS) selected 24 pairs of PA view films of both hands from candidate films, using the following criteria: 1) standard radiographic technique erhployed; 2) duration between paired films of months; 3) films represent a range of disease severity; 4) diagnosis of definite or classic rheurnatoid arthritis (20); and 5) no obvious clues to determine film sequence (e.g., a missing digit). All patients had definite or classic rheumatoid arthritis. Readers were stratified for balance (rheumatologists versus radiologists, and scoring experience) and randomly assigned to read 24 pairs of films first by strategy I or first by strategy 11. Readers recorded observations on specially designed forms, scoring 34 areas for erosions and 36 for joint

3 RADIOLOGIC ASSESSMENT IN RA 3 Table 2. Inter-reader agreement, by different techniques (average correlation among readers)* Strategy I Strategy 11 ERO-34 WTERO-34 JSN-36 ERO-18 WTERO-18 JSN-6 GLOB-34 JSN-36 GLOB-I8 JSN-6 Raw data First films of pairs Last films of pairs Progression: all readers Progression: trained readers Ranked data First films of pairs Last films of pairs Progression: all readers Progression: trained readers * ERO-34 = erosion count, 34joints; WTERO-34 = weighted erosion count, 34 joints; JSN-36 = joint space narrowing, 36 joints: ERO-18 = erosion count, 18 joints; WTERO-18 = weighted erosion count, 18joints; JSN-6 = joint space narrowing, 6 sites: GLOB-34 = global erosion score, 34 joints; GLOB-I8 = global erosion score, 18 joints. space narrowing.* Readers were also asked to rate the member of the film pair showing more severe involvement as an overall assessment, not based on specific erosion or joint space narrowing scores. Immediately after reading, forms were collected, scrutinized for completeness, and all ambiguities were clarified by the reader. Two graders independently tabulated the forms for each of the scoring methods, and all disagreements were resolved by a third arbitrator. The 34 joint areas rated, 17 on each hand, included the 5 metacarpophalangeal (MCP) joints, 4 proximal interphalangeal (PIP) joints, thumb, first carpometacarpal, radiocarpal joints, and wrist bones (scored circumferentially): scaphoid, lunate, triquetrum, trapezium, and ulnar styloid. For the 18-joint rating, only the 4 PIP joints, the 4 MCP joints, and the ulnar styloid of each hand were read. The thumb was excluded. Joint space narrowing was scored for 36 joints or for only 6 sites (in which case, the worst PIP joint, worst MCP joint, and worst radiocarpal joint of each hand were the sites used). All values were entered into an IBM PC/XT microcomputer and verified. There were 8 reading stations, each station contained 3 pairs of films. All 8 readers scored all 24 pairs of films using both strategies (I and 11) in multiple reading sessions. Readers remained at each station for 30 minutes, and no reading session lasted more than 2% hours. On the first conference day, 4 readers scored all 24 films by strategy 11, and 4 scored all films by strategy I. On the second day, the readers who scored using strategy I switched to strategy 11, and vice versa. At the beginning of each day, readers read practice films for 30 minutes, with an instructor who was experienced in the reading strategy that they would use that day. Films were read as pairs, in random sequence order, on doubleview boxes. Magnifying hand lenses were used for difficult assessments. At the end of each day, each reader reread 3 pairs of randomly assigned films. The association of radiologic scores with other markers of disease was determined using laboratory, disease, and demographic values *Copies of scoring forms can be obtained from the American Rheumatism Association Medical Information System (ARAMIS), 701 Welch Road, Suite 3301, Palo Alto, CA 94304, or can be requested with reprints. available in detail from American Rheumatism Association Medical Information System data banks. RESULTS Relative importance of erosions and joint space narrowing. Readers assessment of the relative importance of erosions and joint space narrowing in judging disease progression seen on radiographs was noted using an analog scale of Erosions were judged to be more important in determining progression 34% of the time, joint space narrowing was judged to be more important 3 1% of the time, and the 2 were judged to be equally important 35% of the time. Correlation coefficients between erosion scores and narrowing scores ranged from only , suggesting that it would be preferable to use scoring techniques in which both scores are combined. Inter-reader agreement. Agreement between readers was evaluated by use of the intra-class correlation coefficient both on the raw data and on ranked data for each scoring technique (21,22). The intra-class correlation coefficient can be calculated using the intra-reader versus inter-reader variation with oneway analysis of variance. This correlation coefficient was used to assess concordance among readers for first films, second films, and progression scores. As shown in Table 2, for ranked data, the last (in time) films showed consistently higher concordance than the first in terms of inter-reader agreement, which is not surprising since it is easier to rank films that have a wider spread of severity. The correlations for progression were lower, as expected, since a progression score is the difference between 2 variables. The progression correlations, however, are the most important to clinical trial design, since such studies consist of analysis of paired differences.

4 4 FRIES ET AL Table 3. Inter-reader agreement, by different combinations of joint space narrowing and erosion evaluations (average correlation among readers)* Combination ERO-341 WTERO-341 ERO-181 WTERO-181 GLOB-341 GLOB-18/ JSN-36 JSN-36 JSNd JSNd JSN-36 JSNd Raw data First films of pairs Last films of pairs Progression: all readers Progression: trained readers Ranked data First films of pairs Last films of pairs Progression: all readers Progression: trained readers * See Table 2 for definitions. Ranked and raw data provide different views of reader concordance. The validity of inferences drawn from the intra-class correlation coefficient based on raw data depends not only on readers scoring independently of each other (which they did), but also that they xore with the same precision (which they did not). 13y using ranked data, each reader s mean rank and variances become equal, and therefore, one can accept these correlations at face value. However, if the ra.w data results are markedly lower than the ranked data results, then there are marked differences among: the readers in the sensitivity of the readings. What 11s desired most is high agreement with raw data. Thus, scoring techniques in which the raw data and ranked data both result in relatively high average correlation coefficients are of special interest. With raw data, first film scores had consistently higher concordance than second film scores. Films showing greater disease involvement, more frequent among the second films, produced higher reader variability, leading to reduced concordance. These data showeid relatively little difference between techniques. With ranked data, there was little difference in concordance between comprehensive and selective numbers of joints (34 versus 18). Similarly, comprehensiv~e reading of joint space narrowing (36 joints), including some joint areas that are difficult to assess, had an average correlation among readers comparable with thiat found using selective joints (6 joints). The inter-reader agreements among radiologists and thlose among rheumatologists were separately calculated, and no differences were found. However, the most experienced readers ( trained ) had consistently higher agreement. Combining joint space narrowing and erosion scores by simple addition provided high inter-reader agreement (Table 3). Both raw data and ranked data correlations were improved by such combinations. Relating disease progression to film sequence. Serial radiographs of rheumatoid arthritis patients will show progression over time in the majority of instances, regardless of therapy employed, although some films may show the disease to be stable and a few may actually show improvement (19). This provides an opportunity to test each scoring method against an external standard, the correct identification of the last (in time) film as showing the worst disease. Table 4 demonstrates these comparisons among different scoring techniques. Paired t-tests taking order into account (as well as McNemar s chi-square test) revealed statistically significant differences between weighted erosion count-34 joints and weighted erosion count-18 joints (WTERO-18) (P < 0.01) and between erosion count-34 joints (ERO-34) and erosion count-18 joints (ERO-18) (P < 0.02). The smallest number of errors (i.e., not identifying the correct film sequence) was seen when a larger number of joints was evaluated, although the absolute differences were relatively small. These data include all judgments by all readers. For joint space narrowing, the same scoring methods were used for strategy I and strategy 11; note that these independent assessments gave essentially identical results. Use of the larger ( comprehensive ) number of joints (36 joints) improved accuracy (P < 0.001, Student s paired t-test). Films from 7 patients were prejudged (JTS) as showing slight or no progression. These films were scored by the majority of readers as showing either no progression or less disease involvement on the second films. When these 7 film pairs were excluded, all methods showed improvement, and again, more complex scoring techniques were more sensitive in identi-

5 RADIOLOGIC ASSESSMENT IN RA 5 Table 4. Identification of correct time sequence of films (8 readers, 24 films)* Separate scoring techniques Strategy I ERO-34 WTERO-34 JSN-36 ERO- 18 WTERO-18. JSN-6 Strategy I1 G L 0 B JSN-36 GLOB-I8 JSN-6 Average Correct Tie Wrong Score I Combined scoring techniques ERO-34/ JSN-36 WTERO-34/ JSN ERO- 18/ JSN WTERO-18/ JSN GLOB-341 JSN-36.~ ~ GLOB-IS/ JSN Average * Score = number correct + one-half the number of ties. See Table 2 for definitions. fying true time sequence of films. Joint space narrowing-36 joints (JSN-36) remained significantly more accurate than joint space narrowing-6 sites (JSN-6) (P < 0.02). Paired t-tests, however, revealed no significant differences between erosion scoring techniques when these 7 film pairs were excluded. Readers additionally were asked to make qualitative assessments of the worst film. On these assessments they were correct 86% of the time, incorrect 6% of the time, and 8% of the time, they indicated no progression. Of the 6% incorrect assessments, 87% were from the 7 pairs of films that were most difficult to assess. The most striking finding, as shown in Table 4, was the marked increase in accuracy when erosion and narrowing assessments were combined, yielding excellent accuracy with even the most simple scoring techniques (for example, the ombination of WTERO-18 and JSN-6). External validity. Table 5 provides summary statistics for 6 external (nonradiographic) variables recorded for all 24 patients during the period between radiographs. These variables (erythrocyte sedimenta- tion rate [ESR], number of active joints, duration of disease, American Rheumatism Association [ARA] functional class, rheumatoid factor titer, and physician s global estimate of disease activity on a scale) have been commonly used as endpoints in clinical trials in rheumatoid arthritis. Each has been criticized for conceptual weaknesses, for high variability, and for an inconsistent relationship with the disease process (23,24). ESR, for example, varies greatly from one visit to the next. For these analyses, we utilized patient averages for multiple values to provide greater stability. Latex fixation titers were transformed into log values, and the highest titer recorded between radiographic evaluations was utilized. By adding 8-reader averages for all scoring techniques, the film pairs were classified into 3 groups: 9 which showed the most progression over the interval, 9 which showed less progression, and 6 which initially showed serious involvement and continued to show this. Patients with the most radiologic progression had ESRs >50 mm/hour, in contrast to a value of approximately 20 mm/hour in those showing less progression. ARA functional class was highest in patients with the greatest progression, and these patients also exhibited the highest rheumatoid factor titers and greatest disease activity. These results demonstrate the association of radiologic scores with other measures of disease activity (external validity). Test-retest reliability. Test-retest reliability of a reader s ability to replicate his or her previous results after an average time interval of 6 hours was evaluated. The data were analyzed by paired t-test for first films, second films, and progression scores (Table 6). The r scores are correlation coefficients; the Student s I scores identify systematic differences on rereading. If the t score is significantly different from 0, then the results from rereading are not interchangeable with the first reading results. In the ideal situation, r will be close to unity, and t will be close to 0. Few statistically significant differences on retest for progression of disease were noted, except in global erosion score-34 joints (GLOB-34) and GLOB- 34 in combination with JSN-36. For second films, ERO-34, ERO-18, GLOB-34, and ERO-18 added to JSN-6 scores did not show a strong degree of consistency on retest. The JSN-36 score did not show strong agreement on retest for either first or second films. Of the combined techniques, ERO-34 plus JSN-36, global erosion score-18 joints (GLOB-18) plus JSN-6, and WTERO-18 plus JSN-6 had consistently good retest results (Table 7).

6 6 FRIES ET AL Table 5. Clinical variables and radiologic changes Variable Erythrocyte sedimentation rate Mean' 2 SE of patient averages No. of observations No. of active joints Mean * SE, first film Mean t SE, second film Disease duration (years), mean t SE, first film American Rheumatism Association functional class Mean t SE of patient averages No. of observations Rheumatoid factor (log titer), mean t SE of greatest log Global disease activity Mean t SE, first film Mean 2 SE. second film Radiologic progression Marked (9 Minimal (9 Stable (6 film pairs) film pairs) film pairs) 53.4 t t ? ? t t t t t t t t t t ? ? t t ? t t 1.4 The effect of multiple readers. With the Spearman-Brown prophecy statistic, one can calculate the intra-class correlation coefficients of the mean of k (1, 2,..., 8) reader progression scores. Marked increase in reliability is obtained for all techniques by increasing the number of readers, with correlation coefficients ranging from for a single reader, increasing to a range of for 8 readers. Using standard statistical methodology (25), the number of subjects and readers required for any of the scoring techniques can be calculated, given explicitly defined study assumpl.ions. Increasing the number of readers markedly reducled the number of subjects required for study, as shown in Figure 1. Paired versus separate readings. To evaluate the sensitivity of reading the film pairs simultaneously versus reading the 2 films at separate times, scores derived from the first (in time) films from the first reading, paired with the second films from the retest reading, and vice versa, were studied. Readers' accuracy at detecting true time sequence was compared by using scores of paired readings versus these separate scores. Accuracy was assessed further by calculating the relative variability of progression scores. Paired readings were, on average, 13% more accurate at detecting the true time sequence of films than were separate readings. On average, progression scores were more variable (standard deviations 11.2% Table 6. Test-retest reliabilitv. individual scoring techniaues Correlation (r) and tz3 (t) values* Correlation (r) and t- (t) values* ERO-34 WTERO-34 ERO-18 WTERO-18 GLOB-34 GLOB-18 JSN-36 JSN-6 First films r t Second films I t 1.18 r t 2.40t $ t Progression r t * Subscripts on the t statistic refer to the number of degrees of freedom. See Table 2 for definitions. t P < 0.03, paired second reading versus first reading. $ P < 0.001, paired second reading versus first reading. 5 P < 0.05, paired second reading versus first reading.

7 I II ~ RADIOLOGIC ASSESSMENT IN RA 7 Table 7. Test-retest reliability, combined scoring techniques Correlation (r) and ft1 tt) values* ERO-34/ WTERO-34/ EPO-18/ WTERO-18/ GLOB-34/ GLOB-18/ JSN-36 JSN-36 JSN-6 JSN-6 JSN-36 JSN-6 First films r t t 2.45$ $ 0.64 Second films r t Progression $ r t t * Subscript on the t statistic refers to the number of degrees of freedom. See Table 2 for definitions. t P < 0.05, paired second reading versus first reading. $ P < 0.03, paired second reading versus first reading. higher) for separate readings than for paired readings. These results are to be expected, since a reader cannot compare specific erosions or joint space narrowing for increases or decreases when the films are not read together. Films should be read in pairs. Power requirements and the cost of a randomized clinical trial. A clinical trial in rheumatoid arthritis may compare radiologic progression of disease between a treatment group and a control group. To perform such a study, the number of patients (n) in the v) I- z w a W 3 2 ASSUMPTIONS: a= p=.05 TEST DRUG REDUCES X-RAY PROGRESSION BY ONE.HALF FROM PILOT STUDY RESULTS 170 I I OJSNB ~ER034 owtero18 AER034 + JSNBE OWTERO18 + JSNBmWTERO18 + JSNB I (trained) control and treatment groups and the number of readers scoring radiographs (k) should be determined, to provide a specified level of efficacy (alpha level) with a power of (1 - beta). It is also desirable to choose a scoring method which will minimize the cost of the clinical trial. We have derived formulas for the number of readers and patients required to minimize the cost of a randomized clinical trial, when the cost (C) is derived by C = al + (a2 + 2a3n)k + 2a4n where al == the overhead cost of the clinical trial, a2 == the average cost to train a reader to read radiographs by a particular scoring technique, a3 = the amount a reader is paid to score a pair of radiographs, n = the number of patients in either the treatment or the control group, k = the number of readers, and a4 = the average cost per subject in the clinical trial, assumed to be equal for treatment and control patients. Figure 1 presents the relationship between the number of readers and the number of patients required, assuming a study design in which both alpha and beta are set at The desired difference be- L I I I I tween the study and control groups is assumed to be one-half the mean progression scores obtained in this NUMBER OF READERS Figure 1. Number Of patients required versus number of readers required in order to achieve study reliability. JSN6 = joint space narrowing, 6 sites; ER034 = erosion count, 34 joints; WTER018 = weighted erosion count. 18 ioints: JSN36 = ioint mace narrowine. Y 36 joints. pilot study. Based on these assumptions, the number of patients required could be as high as 160 in each group when there is a single reader evaluating joint space narrowing only, or as low as 50 patients in each group when there are 9 highly trained readers utilizing

8 8 FRIES ET AL ASSUMPTIONS: n =8=.05 TIEST DRUG REDUCES X-RAY PROGRESSION BY ONE-HALF FROM PILOT STUDY RESULTS. TIRAINING COST = $I,OOO/READER. READING COST = $30/READING PATIENT COSTS (over 2 years) = $3, l l l 0 JSNG WTEROl8 o WTEROlI+JSNG - 8 WTEROlI+JSNG (trained) 0 Point of lowest - cost NUMBER OF READERS Figure 2. Study costs versus number of readers. JSN6 =joint space narrowing, 6 sites; WTER018 = weighted erosion count, 18 joints. one of the optimal combinations of techniques. The number of patients required will vary depending on the choice of alpha and beta levels, the length of the trial, and the clinical significance level, but the shape and relative position of the curves will vary little. Figure 2 illustrates the above effects on the total cost of a study, assuming a2 = $1,000, a3 = $30, and a4 = $3,000, reasonably typical costs for recent studies. Study costs as high as $700,000-$900,000 using single methods and a small number of readers can be reduced to as low as $350,000 by the use of 3 or more highly trained readers using the most cost-efficient methods.* DISCUSSION Evaluation of radiologic progression in rheumatoid arthritis has become a gold standard for estimating: the therapeutic efficacy of gold itself and of other agents (such as methotrexate or penicillamine) believed to favorably alter the natural history of the disease. While radiologic changes do not have the ultimate significance of the newer outcome assessments which measure function, pain, and patient satisfaction (24), they represent an observable biologic endpoint resulting from inflammation and enzymatic *A technical report detailing the power and cost calculations is available upon request from ARAMIS. - - degradation of cartilage and subchondral bone, they probably precede functional loss, they are relatively independent of the multiple personal and social forces which affect disability, and they may be objectively assessed. Each of the several tests used to rate the different scoring methods contributes important information to the overall evaluation. Test-retest reliability is necessary since a method can hardly be optimal if the same observer cannot reliably reproduce his or her observations. If there is good interobserver agreement with a technique, then one may expect that any comparable reader is likely to perform similarly. Identification of correct time sequence is in some ways the most important test since it is objective and is a practical test of ability to determine progression. However, these time sequence results must be considered in a broader context, since the second radiograph is not necessarily the worst, more comprehensive scoring methods have higher scores and fewer ties, and in a future study comparing 2 drugs, the magnitude of the change, not merely its direction, will be investigated. A number of important conclusions emerge. Erosions and joint space narrowing contribute different information, and both should be evaluated. There is little difference between global erosion estimates and weighted erosion counting. The unweighted erosion counting technique is less efficient. The use of trained, experienced readers is critically important for accurate identification of disease progression. Specialty affiliation (rheumatologist or radiologist) does not appear to predict skill in reading of radiographs. Evaluating either a more comprehensive or a more selective number of joints makes relatively little difference in either reliability or validity. The validity of measuring progression of disease radiologically was confirmed by relating progression scores with clinical variables. Averaging the scores of 3 or more readers greatly increases the reliability of progression scores. Films should be read in pairs rather than separately. With optimal reading techniques, the required number of patients in a study can be greatly reduced; such reading techniques can reduce costs by at least onehalf and can achieve high study power with reasonable numbers of patients. From the several comparisons presented, the use of a simple combination of scores of joint space narrowing in selected sites (JSN-6) plus either the weighted erosion count on selected joints (WTERO- 18) or the global, overall erosion score on selected joints (GLOB- 18) is recommended for clinical trials using radiographic progression as an endpoint. These

9 RADIOLOGIC ASSESSMENT IN RA 9 scoring techniques resulted in good performance on all tests. Additionally, evaluation of 18 joints, rather than reading these same areas as a subset of 34 joints (as in this study), should yield both tirne savings and increased accuracy. Using the weighted erosion count instead of the global erosion score provides the advantage of having a written record of which specific erosions were judged to progress, and by how much. This enables more sophisticated secondary analysis. Studies of the effect of drugs in altering the radiologic natural history of rheumatoid arthritis often have been controversial and have produced inconclusive findings. The cost of conclusive studies, measured in the hundreds of thousands of dollars, has made such studies infrequent and difficult. These problems are common to a number of areas of medicine, where results of coronary arteriograms, pulmonary angiograms, computerized tomography, or other measures which require subjective judgment serve as endpoints. Careful attention to measurement of the study endpoint, use of multiple observers to increase reliability, and carefully chosen scoring methods can greatly increase the sensitivity and efficiency of such studies. ACKNOWLEDGMENTS The authors greatly appreciate statistical contributions made by Lincoln E. Moses, secretarial support from Ginny Wielgps, and logistical help from Mindy Tharan. REFERENCES 1. Sharp JT, Lidsky MD, Collins LC, Moreland J: Methods of scoring the progression of radiologic changes in rheumatoid arthritis: correlation of radiologic, clinical and laboratory abnormalities. Arthritis Rheum 14: , Genant HK: Methods of assessing radiographic change in rheumatoid arthritis. Am J Med 76:35-47, I Smith DW, Bluhm GB: Rater reliability in reading PA films of hands for bone and cartilage changes in rheumatoid arthritis. Eur J Rheumatol Inflamm 5: , Sigler JW, Bluhm GB, Duncan H, Sharp JT, Ensign DC, McCrum WR: Gold salts in the treatment of rheumatoid arthritis. Ann Intern Med 80:21-26, Gofton JP, O Brien WM: Effects of auranofin on the radiological progression of joint erosion in rheumatoid arthritis. J Rheumatol (suppl) 8: 16S172, Krane SM: Joint erosion in rheumatoid arthritis. Arthritis Rheum 17: , Lidsky MD, Sharp JT, Billings S: Double-blind study of cyclophosphamide in rheumatoid arthritis. Arthritis Rheum 16: , Mattingly PC, Matheson JA, Dickson RA: The distribution of radiological joint damage in the rheumatoid hand. Rheumatol Rehabil 18: , Luukkainen R, Isomaki H, Kajander A: Effect of gold treatment on the progression of erosions in RA patients. Scand J Rheumatol6: , Jalava S, Reunanen K: Healing of erosions in rheumatoid arthritis. Scand J Rheumatol 11:97-100, Duncan H, Mathews CHE, Crouch MM, Parfitt AM: Bone erosion in rheumatoid arthritis. Henry Ford Hosp Med J 26: Scatt DL, Grindulis KA, Struthers GR, Coulton BL, Popert AJ, Bacon PA: Progression of radiological changes in rheumatoid arthritis. Ann Rheum Dis 43:&17, Cooperating Clinics Committee of the American Rheumatism Association: A controlled trial of cyclophosphamide in rheumatoid arthritis. N Engl J Med 283: , Bunch TW, O Duffy JD: Disease modifying drugs for progressive rheumatoid arthritis. Mayo Clin Proc 55: , Multicentre Trial Group: Controlled trial of D(-) penicillamine in severe rheumatoid arthritis. Lancet 1 : , Genant HK, Doi D, Mall JC: Optical versus radiographic magnification for fine-detail skeletal radiography. Invest Radio1 10: , Kellgren JH: Radiological signs of rheumatoid arthritis: a study of observer differences in the reading of hand films. Ann Rheum Dis 15:55-60, Larsen A: Radiological grading of rheumatoid artht-itis. Scand J Rheumatol 2: , Sharp JT, Bluhm GB, Brook A, Brower AC, Corbett M, Decker JL, Genant HK, Gofton JP, Goodman N, Larson A, Lidsky MD, Pussila P, Weinstein AS, Weissman BN, Young DY: Reproducibility of multipleobserver scoring of radiologic abnormalities in the hands and wrists of patients with rheumatoid arthritis. Arthritis Rheum 28: 16-24, Ropes MW, Bennett GA, Cobb S, Jacox R, Jessar RA: 1958 revision of diagnostic criteria for rheumatoid arthritis. Bull Rheum Dis 9: , Kendall MC, Stuart A: The Advanced Theory of Statistics. Vol. 2. New York, Hafner Publishing Company, 1967, pp Winer BJ: Statistical Principles in Experimental Design. New York, McGraw-Hill, 1971, pp Bombardier C, Tugwell P, Sinclair A, Dok C, Anderson G, Buchanan W: Preference for endpoint measures in clinical trials: results of structural workshops. J Rheumatol9: , Fries JF: Toward an understanding of patient outcome measurement. Arthritis Rheum 26: , Sokal RR, Rohlf FJ: Biometry. San Francisco, WH Freeman, 1969, pp