Reliability of dermoscopy in the assessment of patients with Raynaud s phenomenon

Transcription

1 RHEUMATOLOGY Rheumatology 2010;49: doi: /rheumatology/kep408 Advance Access publication 22 December 2009 Original article Reliability of dermoscopy in the assessment of patients with Raynaud s phenomenon Tonia L. Moore 1, Christopher Roberts 2, Andrea K. Murray 3, Ingrid Helbling 4 and Ariane L. Herrick 3 Abstract Objective. Few rheumatologists have access to wide field or video capillaroscopy (abnormal capillaries being highly predictive of CTD), and a key question is whether they should, therefore, purchase a dermatoscope. The aim of this study was to estimate the inter- and intra-observer variability of dermoscopy (magnification 10) among rheumatologists with little or no experience of the technique. Good reliability is a necessary prerequisite for a test to be a valid clinical or research tool. Methods. Dermoscopy was performed in all 10 nail folds from 16 subjects with a range of capillary normality/abnormality. The 160 nail fold images thus acquired were made into two PowerPoint presentations, each of 80 images with 16 duplicate slides. Each participating rheumatologist graded one of the sets of 96 images, grading scale (0 3): normal, mildly abnormal/ suspicious, definitely abnormal, grossly abnormal capillaries or unclassifiable when capillaries could not be adequately identified. Data from both presentations were pooled for analysis. Results. Twenty-eight rheumatologists participated in the study. For the decision as to whether an image could be classified or not, the inter- and intra-observer k-coefficients were 0.59 (95 CI 0.51, 0.67) and 0.63 (95 CI 0.45, 0.74), respectively. Conditional on being able to classify, the intra-class correlation coefficient for inter- and intra-observer reliability was 0.72 (95 CI 0.66, 0.77) and 0.85 (95 CI 0.82, 0.92), respectively. CLINICAL SCIENCE Conclusions. Inter- and intra-observer reliability were good, suggesting that with little training, dermoscopy is likely to be a useful technique to identify capillary distortions/underlying CTD. Key words: Dermoscopy, Reliability, Raynaud s phenomenon, Systemic sclerosis. Introduction Wide field nail fold capillaroscopy (magnification 12 14) is a well-established technique to assess patients with RP [1, 2], in whom abnormal nail fold capillaries are highly predictive of an underlying SSc, scleroderma -spectrum disorder [3 5]. The typical abnormalities occurring in SSc-spectrum disorders include enlarged capillary 1 Salford Royal NHS Foundation Trust, Salford, 2 Health Methodology Group, School of Community Base Medicine, University of Manchester, Manchester, 3 University of Manchester, Manchester Academic Health Science Centre, Salford Royal NHS Foundation Trust, Salford and 4 Department of Dermatology, Chesterfield Royal Hospital NHS Foundation Trust, Chesterfield, UK. Submitted 31 July 2009; revised version accepted 4 November Correspondence to: Ariane Herrick, University of Manchester, Manchester Academic Health Science Centre, Salford Royal NHS Foundation Trust, Salford M6 8HD, UK. ariane.herrick@manchester.ac.uk loops, areas of avascularity, disruption of the normal capillary bed architecture and areas of haemorrhage. Despite the obvious clinical utility of the technique, wide field microscopy is not widely available, as the equipment is relatively expensive and not easily portable. Therefore, some rheumatologists and dermatologists have used an opthalmoscope [6] or hand-held dermatoscope [7 9] to visualize the nail fold capillaries. The dermatoscope is a small, easily portable piece of equipment and relatively inexpensive. A key question is whether dermoscopy might be as useful as wide field capillaroscopy to assess patients with RP. As a first step in answering this question, the aim of this study was to estimate the inter- and intra-observer variability of dermoscopy among rheumatologists with little or no experience of the technique as good reliability is a necessary prerequisite for a test to be a valid clinical or research tool.! The Author Published by Oxford University Press on behalf of the British Society for Rheumatology. All rights reserved. For Permissions, please journals.permissions@oxfordjournals.org

2 Reliability of dermoscopy in RP Patients and methods Study design The study was carried out in four steps as described below. (i) The first step was to collect a set of images, which would include a broad spectrum of nail fold capillary abnormalities. Images were, therefore, acquired from 16 subjects: 4 healthy controls, 4 patients with minor nail fold capillary abnormalities, 4 with definite abnormalities and 4 with severe abnormalities, as judged by two observers familiar with capillaroscopy (T.L.M. and A.L.H.). One patient had primary RP, whereas 11 patients had a SSc-spectrum disorder. Dermoscopy was performed in all 10 finger nail folds from each subject, which means that 160 images were acquired. The dermatoscope used was a pocket epiluminescence microscopy device with a 10 lens and 32 white light-emitting diodes (LEDs) (DermLite II PRO HR; Schuco International London, London, UK). It was attached, using a 28-mm thread, to a Nikon CoolPix 4500 digital camera. The study was approved by the Salford and Trafford Research Ethics Committee, and all subjects participating in the study signed informed consent. (ii) The 160 images were then made into two PowerPoint presentations, each containing images from eight subjects (two from each of the above four categories). Each PowerPoint presentation in addition contained 16 duplicate images (therefore, 96 images in total) in order that intra- as well as inter-observer reliability could be investigated. The reason for having two presentations was to reduce the risk of observer fatigue from a longer single presentation. (iii) A grading scale was compiled: Grade 0 normal; Grade 1 mildly abnormal/ suspicious ; Grade 2 definitely abnormal; and Grade 3 grossly abnormal. Rheumatologists were also given a fifth grading option, which was poor visualization/ cannot be categorized, to allow for the real clinical situation where some nail fold capillary beds are very difficult to see and, therefore, impossible to grade (as opposed to capillaries being absent due to avascularity). (iv) Rheumatologists were invited to participate in the study, which involved for them: (a) attending a 5-min training session giving examples of different grades of images (Fig. 1); and (b) immediately after this 5-min session, grading one of the sets of 96 images, shown as a PowerPoint presentation lasting 15 min. Each image was shown for 10 s. The purpose of the training session was simply to give participants an insight into the range of abnormality they were likely to see and to orient them as to the layout of the images. As a guideline, participants were told that in the investigators opinion, Grade 0 denoted normal capillary loops (with tortuosity of normal calibre capillaries allowed), Grade 1 essentially normal appearance but allowing one enlarged capillary or one area that looked avascular, Grade 2 several enlarged capillaries and/or areas of avascularity and Grade 3 mostly enlarged capillaries and/ or many areas of avascularity and/or marked loss of capillary architecture. The two presentations were shown immediately one after the other; some rheumatologists participated in the first session and others in the second. Statistical analysis and sample size The grading scale involved five ordered categories: unclassifiable (the clinician could not adequately visualize the image) and Grades 0 3 as described above. Statistical analysis estimated inter- and intra-observer agreement, first, for the ability to visualize the image and, secondly, on the ordinal grading, conditional on being able to visualize the image. For the second analysis, the numbers of ratings differed between images due to some observers being unable to grade a particular image. For multiple observers, inter- and intra-rater repeatability for a binary scale can be measured using a multi-rater k-coefficient [10]. This measures pair-wise agreement across all possible pairs of observers. For ordered categorical scales, agreement can be measured by a multi-rater weighted k-coefficient, but this requires an equal number of ratings per subject. The weighted k-coefficient with quadratic weights can be approximated by the intra-class correlation coefficient (ICC), which is estimated using analysis of variance [10], and can be computed where the number of ratings varies between images. Analysis of repeatability conditional on the ability to classify was, therefore, based on the ICC. CIs were computed using a non-parametric bootstrap resampling by image. Computed in this way, CIs take account of the sample variation due to images, but not that associated with the selection of observers. Statistical analyses were carried out using STATA version 9. As there was no pre-specified hypothesis regarding the level of reliability, a sample size was chosen to give acceptable precision. With six ratings of each of 160 images, the ICC would be estimated with a 95 CI of approximately ±0.1 for values in the range [11]. Results Participants Twenty-eight rheumatologists (17 consultants and 11 trainees) participated in the study. Six participated in one and 22 in the other slide show. Inter-observer repeatability Session 1 was attended by 22 observers and Session 2 by 6 observers who assessed. The discrepancy in numbers was due to Session 1 being at a more convenient time slot for most of the observers. In each session, 96 images 543

3 Tonia L. Moore et al. Fig. 1 Examples of dermoscopy images, as graded by two observers experienced in capillaroscopy; (A) Grade 0 normal; (B) Grade 1 mildly abnormal/ suspicious ; (C) Grade 2 definitely abnormal; (D) Grade 3 grossly abnormal capillaries; and (E) Poor visualization/cannot be categorized. A B C D E were assessed including 16 duplicate images to assess repeatability. One image from Session 1 was subsequently dropped because of being a substitution/duplication (due to one patient having had a right index finger amputation). Table 1 summarizes the data for the first assessment of each image for both sessions. The distribution of scores is given together with the distribution of the pairs of scores for pairs of observers. Table 1 also gives the distribution of ratings. Of those, 23 were classified as normal, whereas 29 of the ratings were unclassified. The main diagonal, highlighted in bold, gives the percentage of occasions in which a pair of observers agreed. For example, on 20 of the occasions, both observers of a pair considered that an image could not be adequately visualized and was, therefore, unclassified. The relative frequencies below the main diagonal give the distribution of disagreements between pairs of observers. Only the lower triangle of the table is used, as the pairs are not ordered. For example, for 13 of the rating pairs, one observer used the normal category, whereas the other used suspicious category. Table 1 also gives summary measures of inter-observer repeatability. Values for each session were similar, justifying that the overall estimation is obtained by combining the values. For ability to classify, the multi-rater k-coefficient showed modest agreement with a combined value of 0.59 (95 CI 0.51, 0.67). The ICC, to measure the repeatability conditional on the ability to classify, was 0.72 (95 CI 0.66, 0.77) (Table 1). Intra-observer repeatability Table 2 summarizes the distribution of scores for the intra-observer repeatability based on the 16 replicate images in each session across all 28 observers. In summarizing agreement between the first and second observations, pairs of ratings are now ordered, so there are now frequencies above and below the diagonal. On making the first assessment, observers were unable to classify 15 of the time. This reduced to 14 on the second assessment. For the decision as to whether an image could be classified or not, values were similar for each session (Table 2) and the combined intra-observer k-coefficient was 0.63 (95 CI 0.45, 0.74). Conditional on being able to classify, the ICC for intra-observer reliability was 0.85 (95 CI 0.82, 0.92). Inspection of the CIs suggests that this was significantly higher than inter-observer repeatability. Variation of grades between fingers. Grades varied across a subject s fingers. ICCs of digits and pairs of 544

4 Reliability of dermoscopy in RP Table 1 The distribution of ratings and inter-observer reliability: Session 1: 22 observers, 79 images; Session 2: 6 observers and 80 images Normal, Suspicious, Abnormal, Grossly abnormal, Unclassifiable, Distribution of ratings Distribution of pairs of ratings Normal 12.7 Suspicious Abnormal Grossly abnormal Unclassifiable Inter-observer reliability Classifiable j (95 CI) Grade ICC (95 CI) Session (0.46, 0.67) Session (0.68, 0.80) Session (0.47, 0.72) Session (0.49, 0.71) Combined 0.59 a (0.51, 0.67) Combined 0.72 b (0.66, 0.77) Numbers in bold give the percentage of occasions on which a pair of observers agreed. a Sessions 1 and 2 values combined using inverse variance weights. b Data combined in a single analysis. Table 2 The distribution of ratings and intra-observer reliability data pooled across 28 observers and both sessions with each observer making 16 repeat assessments Distribution of ratings Normal, Suspicious, Abnormal, Second rating Grossly abnormal, Unclassifiable, Total, First rating Normal 82 (18) 31 (7) 3 (1) 0 (0) 10 (2) 126 (28) Suspicious 28 (6) 61 (14) 15 (3) 0 (0) 8 (2) 112 (25) Abnormal 3 (1) 8 (2) 35 (8) 22 (5) 0 (0) 68 (15) Grossly abnormal 0 (0) 1 (0) 11 (2) 62 (14) 0 (0) 74 (17) Unclassifiable 7 (2) 9 (2) 5 (1) 2 (0) 45 (10) 68 (15) Total 120 (27) 110 (25) 69 (15) 86 (19) 63 (14) 448 (100) Intra-observer reliability Classifiable j (95 CI) Grade ICC (95 CI) Session (0.19, 0.72) Session (0.79, 0.93) Session (0.36, 0.85) Session (0.74, 0.92) Combined 0.63 a (0.45, 0.74) Combined 0.85 a (0.82, 0.92) Numbers in bold give the percentage of occasions on which a pair of observers agreed. a Data combined in a single analysis. digits indicated that the thumb and little fingers may have lower reliability than the middle three digits. Ring fingers, previously believed to provide the best image quality [12], showed no higher reliability than the index and middle fingers. ICCs (95 CI) for finger-pairs (right and left) were as follows: thumbs 0.72 ( ), index fingers 0.79 ( ), middle fingers 0.77 ( ), ring fingers 0.77 ( ) and little fingers 0.58 ( ). Discussion The key finding from this study is that both inter- and intra-observer reliability of nail fold dermoscopy are good, suggesting that even with little training, dermoscopy is likely to be a useful technique to identify capillary abnormalities. As expected, intra-rater reliability was higher than inter-rater. It should be noted that this is in 545

5 Tonia L. Moore et al. contrast to the use of dermoscopy for skin lesions, when the importance of training/previous experience is well recognized. The 4-point grade scale was arbitrary, but was chosen to mirror the clinical situation in a busy outpatient clinic, when clinicians are unlikely to spend time counting specific abnormalities such as the number of dilated capillaries (which in itself is also somewhat subjective). Specifically, a rating of normal in the patient with RP without other features of CTD would most likely allow the clinician to reassure the patient. Conversely, a rating of definitely or grossly abnormal would suggest that the patient most likely had an underlying CTD, and a rating of mildly abnormal/suspicious would probably suggest the need for keeping the patient under review. It must be stressed, however, that this study was not designed to examine the sensitivity and specificity of dermoscopy in the diagnosis of CTD. This question has, to some degree, already been addressed in previous studies. Bergman et al. [8] performed dermoscopy (magnification 9.3) in 106 patients attending dermatology or rheumatology clinics and 170 controls, and found a scleroderma/dermatomyositis pattern in 19/27 (70) of the patients with SSc, 7/11 (64) with dermatomyositis, 4/8 (50) with MCTD, 2/38 (5) with RP alone and in no controls. Patients had to demonstrate abnormalities in at least two of all finger nail folds to be classed as abnormal [8]. This study, albeit in relatively small numbers of patients with CTD, suggests that the dermatoscope may be less sensitive than the wide field microscope in detecting abnormality, as the sensitivity of capillary microscopy for the diagnosis of scleroderma-spectrum disorders has been estimated to be [3]. However, as pointed out by the authors [8], differences in the definition of the scleroderma pattern, which has considerable subjectivity, between studies complicates comparisons. More recently, Beltran et al. [13], using a non-portable dermatoscope with magnification 30 (higher than in our study), studied 56 patients with RP: 5 with primary RP, 51 with secondary RP (26 with SSc) and 10 healthy controls. They examined four nail folds in each subject, and concluded that dermoscopy had 77 sensitivity and 91 specificity for SSc. However, their equipment is not strictly comparable with the 10 magnification dermatoscope, projecting real-time images on to a computer screen [13]. While these previous studies are of interest, relatively small numbers of patients with SSc-spectrum disorders were included and larger studies are now required, given the background provided by our study that the technique is reliable without extensive training. A recent study by Baron et al. [14] also examined the reliability of dermoscopy in assessing nail fold capillary abnormalities. These investigators assessed inter- and intra-observer reliability in identifying the presence or absence of three specific features dilated loops, giant capillary loops and avascular areas. Four fingers in each of six patients with SSc and two controls were examined. For the three features, the k-coefficients for inter-observer variability were 0.63 (dilated loops), 0.40 (giant capillaries) and 0.20 (avascularity), and for intra-observer variability 0.71, 0.55 and 0.40, respectively [14]. The authors concluded that reliability for dilated and giant capillaries was good and that dermoscopy was therefore a useful tool. In our study, we did not ask raters to comment on individual capillary features but rather state their overall opinion. Where observers were able to classify, this was sufficiently reliable to make group-level comparisons but not sufficient for diagnostic purposes on its own. A reliability coefficient of has been suggested as being sufficient for group-level comparisons, but if measures are taken to make individual decisions then values should be [15]. This suggests that our measure, with inter- and intra-observer k-coefficient of 0.72 and 0.85 conditional on the ability to classify images, is adequate for group-level comparisons. A possible limitation of our study is that the reproducibility being tested was the reproducibility of clinicians scoring an image that had been acquired for them and projected (magnified) on a screen. This is not the same as the reproducibility of dermoscopy in clinical practice, which involves image acquisition. This would have involved different clinicians assessing the subjects at the bedside to test. Although feasible, this would have taken very much longer, and we felt that assessing the acquired image was a reasonable surrogate in the first instance, and allowed very much larger numbers of images, by more rheumatologists, to be included. Our study is the first in a programme of research aiming at answering the simple question Should all rheumatologists use a dermatoscope in their routine clinical practice? The key issues remaining to be addressed are as follows: (i) What are the positive and negative predictive values of abnormal and normal dermoscopy findings, respectively, in the assessment of patients with CTD? As discussed above, larger studies are required to adequately address this issue. (ii) How does dermoscopy compare with nail fold video capillaroscopy, in terms of reliability, and prediction of underlying CTD? Video capillaroscopy, which uses much higher magnifications ( ) than dermoscopy, is used in specialist centres and allows detailed visualization of individual capillaries, allowing calculation of nail fold dimensions. Cutola et al. [2] have developed a scoring system to assess the scleroderma pattern of capillary abnormalities, subdividing these into early, active and late patterns. It seems unlikely that the lower magnification of dermoscopy would allow such detailed categorization, but it may be that dermoscopy is equally sensitive and specific in the separation of normal from abnormal. In conclusion, our study has demonstrated the reliability of dermoscopy among rheumatologists with little or no previous experience in the technique. Its findings are therefore supportive of dermoscopy being more widely used in the assessment of the patient with CTD, although 546

6 Reliability of dermoscopy in RP further research is required to evaluate its sensitivity and specificity in comparison with wide field capillaroscopy and video capillaroscopy. Rheumatology key messages. Dermoscopy allows visualization of nail fold capillaries and is relatively inexpensive.. Inter- and intra-observer reliability among rheumatologists is good.. Dermoscopy is likely to be useful in identifying underlying CTD. Acknowledgement We are grateful to all those rheumatologists who participated in the study. Disclosure statement: The authors have declared no conflicts of interest. References 1 Maricq HR. Widefield capillary microscopy. Technique and rating scale for abnormalities seen in scleroderma and related disorders. Arthritis Rheum 1981;24: Cutolo M, Grassi W, Matucci Cerinic M. Raynaud s phenomenon and the role of capillaroscopy. Arthritis Rheum 2003;48: Carpentier PH, Maricq HR. Microvasculature in systemic sclerosis. Rheum Dis Clin North Am 1990;16: Spencer-Green G. Outcomes in primary Raynaud s phenomenon. A meta-analysis of the frequency, rates and predictors of transition to secondary diseases. Arch Intern Med 1998;158: Koenig M, Joyal F, Fritzler MJ et al. Autoantibodies and microvascular damage are independent predictive factors for the progression of Raynaud s phenomenon to systemic sclerosis. Arthritis Rheum 2008;58: Anders HJ, Sigl T, Schattenkirchner M. Differentiation between primary and secondary Raynaud s phenomenon: a prospective study comparing nailfold capillaroscopy using an opthalmoscope or stereomicroscope. Ann Rheum Dis 2001;60: Bauersachs RM, Löbner F. The poor man s capillary microscope. A novel technique for the assessment of capillary morphology. Ann Rheum Dis 1997;56: Bergman R, Sharony L, Shapira D, Nahir MA, Balbir-Gurman A. The handheld dermatoscope as a nail-fold capillaroscopic instrument. Arch Dermatol 2003; 139: Sontheimer RD. A portable digital microphotography unit for rapid documentation of periungal nailfold capillary changes in autoimmune connective tissue diseases. J Rheumatol 2004;31: Fleiss JL. Statistical methods for rates and proportions. 2nd edition. New York: John Wiley, Giraudeau B, Mary JY. Planning a reproducibility study; how many subjects and how many replicates per subject for an expected width of the 95 confidence interval. Stat Med 2001;20: Houtman PM, Kallenberg CGM, Fidler V, Wouda AA. Diagnostic significance of nailfold capillary patterns in patients with Raynaud s phenomenon. J Rheumatol 1986; 13: Beltran E, Toll A, Pros A, Carbonell J, Pujol RM. Assessment of nailfold capillaroscopy by 30 digital epiluminescence (dermoscopy) in patients with Raynaud phenomenon. Br J Dermatol 2007;156: Baron M, Bell M, Bookman A et al. Office capillaroscopy in systemic sclerosis. Clin Rheumatol 2007; 26: Nunnally J, Bernstein I. Psychometric theory. 3rd edition. New York: McGraw-Hill,