Assessing the Reliability, Discriminative Ability, and Validity of Disability Glare Tests

Transcription

1 Assessing the Reliability, Discriminative Ability, and Validity of Disability Glare Tests David B. Elliott* and Mark A. Bullimore\ Purpose. To gather information regarding the reliability, discriminative ability, and validity of disability glare tests. Methods. The following glare tests were evaluated: the Miller-Nadler, Vistech MCT8000, Berkeley, van den Berg Straylightmeter, and the Brightness Acuity Tester used with the Pelli-Robson and Regan charts. Three test evaluation criteria were used: (1) repeatabilitycomparing test scores on two visits; (2) discriminative abilitythe tests' ability to differentiate between young and old subjects and between old normal and cataract subjects; (3) validitycomparing cataract test scores with the reference standard of the van den Berg Straylightmeter. Three subject groups were evaluated: young normals (n = 24, mean age 24.3 ± 3.3 yr), older normals (n = 22, mean age 66.0 ± 6.2 yr), and early cataract (n = 33, mean age 70.6 ±8.1 yr). Results and Conclusions. Data indicate that contrast sensitivity or low contrast acuity measured in the presence of glare are superior to disability glare scores in assessing cataract patients with normal neural function. Under glare conditions, contrast sensitivity and low contrast acuity scores from the Pelli-Robson, Regan, and Berkeley tests provide similarly reliable, discriminative, and valid measures of visual assessment in cataract. The Miller-Nadler glare tester poorly detects and measures subtle changes in the ocular media, such as early cataract, because of its large step sizes at low contrast thresholds. The poor reliability of the Vistech MCT8000 limits its usefulness. The study suggests that unless good chart design and psychophysics are used, the geometry and intensity of the glare source are of little importance. Invest Ophthalmol Vis Sci. 1993;34: v>«ontrast sensitivity (CS) and glare tests are being used more frequently to clinically evaluate patients with media opacities, such as after refractive surgery and in patients with corneal edema, cataract, and capsular opacification. In some cases, they also are used to From the *Centre for Sight Enhancement, School of Optomelry, University of Waterloo, Waterloo, Ontario, Canada, and the-fschool of Optometry, University of California, Berkeley, California. Financial support was provided by Bausch and Lomb (Canada) and the Centre for Sight Enhancement and Centre for Contact lens Research, University of Waterloo. Submitted for publication: May 2, 1992; accepted August 20, Proprietary interest category: N. Reprint requests: David B. Elliott, Centre for Sight Enhancement, School of Oplometry, University of Waterloo, Waterloo, Ontario N2L 3G /, Canada. justify cataract surgery. The decision to operate on a patient with dense cataract that appears to be the sole cause of low visual acuity (VA) is seldom difficult. A significant number of patients with cataract, however, retain relatively good VA, yet report significant visual problems. Many authors have suggested that VA should be complemented by CS and disability glare testing in these patients. 1 " 5 is the reduction in VA or CS resulting from a nearby glare source and is the result of forward intraocular light scatter. In a recent survey of 396 ophthalmologists of the American Society of Cataract and Refractive Surgery, 65% of respondents were routinely using CS or glare testing. 6 Although there has been a proliferation of techniques and commercially available tests, infor- 108 Investigative Ophthalmology & Visual Science, January 1993, Vol. 34, No. 1 Copyright Association for Research in Vision and Ophthalmology

2 Comparing Disability Glare Tests 109 mation about the relative clinical efficacy of such tests is largely unavailable. 5 We advocate that clinical tests, such as glare tests, should be evaluated by three criteria: (1) reliability are the results repeatable?* (2) Discriminative ability does it provide diagnostically useful information? and (3) validitydoes the test measure what it purports to measure? Attributes such as familiarity to the patient and a short testing time clearly are desirable in any clinical test, but are of secondary importance. Furthermore, terms such as "accuracy" should be avoided because they are ambiguous. Reliability merely reflects the consistency in test scores between sessions. Typically, test scores are compared from two occasions separated by a week or so. Correlation coefficients, although frequently used to assess repeatability, give a measure of association, but not necessarily of agreement. A more appropriate indicator of reliability is gained from the coefficient of repeatability (COR). 7 The COR describes the 95% confidence limits for any discrepancy between test and retest data. For normally distributed data, the COR is 1.96 multiplied by the standard deviation of the discrepancy. These confidence limits also provide information for longitudinal assessment of function, so the significance of any change in performance can be assessed. 8 ' 9 Clinicians have no difficulty diagnosing cataract, and glare tests are not used to determine whether cataract is present. However, the discriminative ability of the tests provides a useful assessment of the tests' sensitivity to small amounts of cataract and other media opacities. Increased glare scores can occur with age 4 and in patients with corneal edema, corneal scarring, keratoconus, and capsular opacification. 5 When evaluating the diagnostic performance of a clinical test, it is not enough to show a statistically significant difference between groups of normal and abnormal subjects. In clinical situations, it is individuals, not groups, who need be evaluated. An analysis that determines the test's ability to classify individual patients is superior to one that simply discriminates between two groups. It is unlikely that all subjects will be classified correctly. A proportion of normal subjects will give an abnormal score (false-positive), while some diseased subjects will show normal results (false-negative). Both of these proportions and the percentage of correct classifications (true-positive and true-negative) will depend on the adopted cut-off criterion. A receiver-operating-characteristic (ROC) curve is a plot of sensitivity (number of true positives/total number!! It also is important to keep in mind that the discriminative ability and validity of a given test will depend somewhat on its reliability. of abnormal subjects) against 1-specificity (number of false negatives/total number of normal subjects) using several different test scores as cut-off points. 10 " An example is given in Figure 1. A quantitative summary of the curve can be obtained by determining the area beneath the ROC curve 1011 (termed A z by Swets and Pickett 10 ). A z has a lower limit of 0.50 for an ROC lying along the major diagonal, defining performance at a chance level. The upper limit is 1.0 for an ROC that follows the left-hand and top axes of the graph, defining total discriminative ability. Comparing tests or systems using an ROC curve is superior to determining the discrimination ability of a test using a single point (such as the number of subjects with eye disease who fall outside the 95% confidence limits of normal). 10 Validity describes how well a test measures the entity that it purports to measure. represents something of a challenge because there is no generally accepted reference or gold standard. The American Academy of Ophthalmology (AAO) advocates that disability glare tests be validated by comparing results with a quantification of overall visual performance. 5 One approach would be to use a carefully constructed questionnaire regarding how patients believe their vision affects their performance of everyday activities. This approach has been used before in cataract studies, 12 " 14 but no standard technique is available and this method is time consuming and requires considerable patient cooperation. In addition, most questions relate to habitual, and therefore binocular visual performance, 14 and three of the glare tests assessed in the present study cannot be used for binocular testing. An alternative validity assessment might involve comparing glare test scores with visual performance under "real life" conditions. Typically, comparisons have been made with VA or CS measured outdoors in bright sunlight. 15 " 17 Standardization of this approach is difficult, however, because of environmental variations, such as the time of day and angle of the sun Furthermore, if VA is measured outdoors, disability glare tests that use VA charts naturally perform well, whereas glare tests that measure CS perform less well. 16 Comparing glare tests with subjective measures of cataract density (eg, LOCS II) or automated lens analysis systems (eg, Interzeag Lensmeter or Scheimpflug photography) is inappropriate. These techniques measure back scatter, whereas glare tests assess forward scatter. Previous reports have found a poor correlation between measures of forward and back scatter. 18 " 20 The simplest assessment of validity is to adopt a measure of forward light scatter as a reference standard. 20 " 22 We employed this approach in the present study, measuring forward light scatter using the van den Berg Straylightmeter. The van den Berg method was chosen for the following reasons. 20 " 24

3 110 Investigative Ophthalmology & Visual Science, January 1993, Vol. 34, No (A C 0) (f) Pelli-Robson CS Pelli-Robson CS+BAT Specificity FIGURE l. A receiver-operating characteristic (ROC) curve for the Pelli-Robson chart with and without the BAT. The plotted curve indicates their respective ability to discriminate between a group of 24 young normal and 23 older normal subjects. 1. It provides a direct measure of forward light scatter, not one estimated from contrast or resolution loss resulting from a glare source. 2. It provides measures of light scatter at different glare angles, and therefore can be used to compare disability glare tests using various types of glare geometry. 3. Results are free from neuronal interference. 4. Scores are repeatable and sensitive. For example, the test has been able to show differences in forward light scatter between normal subjects with different eye pigmentation. 5. The amount of contrast loss caused by the light scatter can be derived/calculated. If tests of disability glare are to enjoy widespread acceptance by researchers and clinicians, they must be shown to provide reliable and valid information and be sensitive to slight changes in light scatter. This has yet to be established for most commercially available tests. In the present study, we report on our evaluation of the reliability, validity, and discriminative ability of several disability glare tests, the majority of which are commercially available. MATERIALS AND METHODS Subjects Subjects were recruited from a local ophthalmologist's office and from the staff, student, and patient population of the School of Optometry, University of Waterloo. Three subject groups were recruited: young normal (n = 24, mean age 24.3 ± 3.3 yr), older normal (n - 22, mean age 66.0 ± 6.2 yr), and cataract with VA better than 20/70 (n = 33, mean age 70.6 ± 8.1 yr). Subjects with poor general health, diabetes, a refractive error greater than ±6.00 diopters, any history of amblyopia, or ophthalmic surgery were excluded. To ensure any loss of visual function was the result of lens changes, subjects with intraocular pressure greater than 21 mmhg or with any ocular disease other than cataract were excluded. All cataract and older normal subjects had been screened for ocular disease by ophthalmoscopy, slit-lamp biomicroscopy, and applanation tonometry. Normal subjects had clear lenses and visual acuities of 20/25 or better. All subjects were white to avoid any major differences in light scatter because of ocular pigmentation. 24 Methods The tenets of the Declaration of Helsinki were followed, and the study gained ethical approval from the Office of Human Research, University of Waterloo. Informed consent was obtained after the nature of the study had been fully explained. Subjects were tested using the following glare tests: the Vistech MCT8000, 25 the Miller-Nadler Glare Tester, 4 the Berkeley Glare Test, 26 the van den Berg Straylightmeter, 2122 the Brightness Acuity Tester (BAT) 15 with the Pelli-Robson chart, 27 and the BAT with the Regan charts. 28 Details of the tests are summarized in Table 1. The manufacturer's recommended testing procedures were adopted when provided. All measurements were made monocularly in a dimly lit room, using natural pupils and best optical correction. Full aperture trial lenses were used, and working distance lenses

4 Comparing Disability Glare Tests 111 TABLE i. A Summary of the Target Stimuli, Range of Values, Step Sizes, Psychophysical Methods of Measurement, and Glare Sources of the Six Glare Tests Test Vistech MCT8000 Miller-Nadler Pelli-Robson Berkeley Regan charts Straylightmeter Target Sine-wave gratings; variable contrast 1.7 Landolt's rings; variable contrast 2.8 letters; variable contrast 18% contrast letters; variable size 96, 25, and 11% contrast letters; variable size 1 circle Range/Steps log CS; log steps log CS; 5 and 2.5% contrast steps log CS; 0.05 log steps 1.0 to -0.3 log MAR; 0.02 log steps 0.80 to -0.3 log MAR; log steps log straylight; log steps Psychophysics 4 AFC 10 AFC 10 AFC 10 AFC Criteriondependent Criteriondependent Glare Source Multiple point; 10 and 130 cd/m 2 Broad surround; 30 X 30 approx cd/m 2 around target. BAT*; 345 cd/m 2 Broad surround; 15 X cd/m 2 BAT*; 345 cd/m 2 Multiple point; 3.5, 10, or 28 * The BAT is a hand-held illuminated bowl with a central 12 mm aperture and was used on the medium intensity setting (345 cd/m 2 ). CS, contrast sensitivity. AFC, alternative forced choice. were incorporated when necessary. The order of measurement was randomized, unless stated otherwise, and all subjects were allowed to recover from any glare source-induced after-image before starting the next test. Tests were repeated on 41 subjects (19 young normal, 15 older normal, and 7 cataract) with a testretest separation of at least 1 wk. The Vistech MCT8000 The targets and glare sources of the Vistech MCT8000 are contained within a portable unit. A console provides control of target presentation, luminance, and glare source position. The unit allows CS measurement at 1.5, 3, 6, 12 and 18 cycles/degree under "nighttime" (3 cd/m 2 ) and 'daytime' (125 cd/m 2 ) luminance conditions with or without a central or peripheral glare source. Each target consists of seven circular discs, each containing a sine-wave grating of a fixed spatial frequency. The gratings are either vertical or tilted 15 to the right or left, and the contrast of the gratings progressively decreases from disc one to seven. Starting at disc one, the subject is asked to indicate the orientation of each grating or to respond "blank" when nothing is seen. The last disc whose orientation is correctly identified determines the CS score. Measurement of CS at 6 cycles/degree with and without glare is described as a functional disability test (FDT) and is recommended as an initial screening technique. The chart and glare source luminance levels were checked using the internal calibration feature before measurements were taken for each subject. We measured FDT under nighttime and daytime luminances with and without a central and peripheral glare source, respectively. As recommended, 25 nighttime measurements always were made first. The Miller-Nadler Glare Tester The Miller-Nadler Glare Tester consists of a modified slide projector. The slides present a series of randomly orientated Landolt rings of progressively reduced contrast (80% to 2.5%) surrounded by a broad glare source of constant luminance. The viewing distance is 40 cm. The endpoint of the test is recorded as the last correctly identified slide. CS without glare can be measured using an additional five slides (between 40% and 5% contrast). The Berkeley Glare Test The Berkeley glare test consists of a reduced low contrast Bailey-Lovie letter chart (Weber's contrast = 18%) mounted on a triangular opaque panel in the center of a 30 X 27 cm opal Plexiglass panel. The chart is front illuminated (80 cd/m 2 ), and the glare source is provided by transillumination of the Plexiglass panel at the medium setting (750 cd/m 2 ). Low contrast VA is measured at 1 m with and without the glare source,

5 112 Investigative Ophthalmology & Visual Science, January 1993, Vol. 34, No. 1 with credit (0.02 log MAR units) given for each letter read correctly. The Brightness Acuity Tester (BAT) The BAT is a hand-held instrument that consists of a hemispheric bowl with an internally illuminated surface. The subject holds the device to their eye and views the chart through a central 12 mm aperture. The medium intensity setting (measured to be 345 cd/m 2 using a spot photometer) was used. The high intensity setting has been reported to give inappropriately high predictions of disability glare 1617 and can reduce contrast beyond a chart's limits with some early cataract patients The BAT was used with the Pelli-Robson CS and Regan VA charts. The Pelli-Robson Chart The Pelli-Robson chart is a 86 X 63 cm chart that contains 16 triplets of 4.9 X 4.9 cm letters. At a test distance of 1 m, these letters correspond to spatial frequencies of about 1-2 cycles/degree. Within each triplet, the letters have the same contrast, and the contrast in each successive triplet decreases by a factor of 0.15 log units. A by-letter scoring system that gives credit (0.05 log units) for each letter read correctly was used. This has been shown to provide more reliable test scores than the originally recommended scoring rule. 30 The chart was illuminated to 100 cd/m 2 and the recommended viewing distance of 1 m was used. CS was measured with and without the BAT. The Regan Charts The Regan charts are log MAR acuity charts of varying contrast. In addition to the traditional high-contrast chart (Weber's contrast = 96%), we used the 25% and 11% contrast charts. They have been found to be the most useful for disability glare evaluation in older patients and those with cataract. 28 A by-letter scoring method of log units per letter was adopted. The chart was illuminated to 100 cd/m 2, and the recommended viewing distance of 3 m was used. VA was measured with and without the BAT for each of the three charts. The van den Berg Straylightmeter Subjects position their eye against a cup at the top of a viewing tube. They view a 1 circular target, surrounded by an annulus with an outer radius of 2 of steady luminance of 30 cd/m 2. Concentric with this target and positioned along the inside of the viewing tube are three rings of yellow (lambda max 570 nm) light-emitting diodes. They are positioned at angular distances of 3.5, 10, and 28 from the subject's eye. The LED sources flicker sinusoidally at 8 Hz. The three rings can be illuminated separately to allow measurement of light scatter at each of the three angular positions. The subject is instructed to observe the central target, and one of the three glare rings is switched on. Because of forward light scatter within the eye, a visible flicker is seen on the central target. The investigator then slowly increases the luminance modulation of the central target, which flickers in counterphase to the LED sources. The depth of modulation of this counterphase light that produces zero perceived flicker corresponds directly to the amount of forward light scatter The straylight scores given later are calculated from the equation = log(l[0] X V) L is the compensating luminance of the central target, (f> is the scattering angle, and E is the illuminance of the stray light source measured in the pupil plane. For a given scattering angle, the investigator increased the luminance modulation of the central target and recorded the point at which the central flicker ceased. The modulation then was increased further until the flicker reappeared, corresponding to where the target modulation overwhelms that caused by straylight. This procedure was repeated three times to provide six measures for each scattering angle. These values then were averaged. Subjects viewed the targets without spectacle correction to avoid incurring light scatter caused by their own spectacles and to ensure exact eye position against the instrument's eye cup. Because the central target is large and the task is to perceive a flickering stimulus, refractive blur has virtually no effect on the measurements. 23 ' 24 RESULTS The mean (±1 standard deviation) values for each test for the young and older normal subjects and for cataract subjects are shown in Table 2. The tests are segregated into those that measure CS (Vistech, Miller- Nadler, and Pelli-Robson), those that assess high or low contrast VA (Berkeley and Regan), and those that measure forward light scatter (van den Berg Straylightmeter). Values determined with and without the glare source are given. Because all but one subject could correctly identify the 5% contrast target of the Miller-Nadler glare tester without the glare source, these data are not displayed. For all tests, the older subjects performed worse than the young subjects, whereas the cataract patients showed the poorest values. For example, mean Regan 96% VA for the young normal subjects, older normal subjects, and cataract subjects were (-20/13), (20/16), and 0.17 (~ 20/30), respectively. The introduction of a glare source reduced visual performance for all tests. Only six of the 33 cataract subjects could see any of the gratings on the nighttime Vistech FDT in the pres-

6 Comparing Disability Glare Tests 113 TABLE 2. Mean Test Results (± 1 Standard Deviation) for Young Normal (n = 24), Older Normal (n = 23), and Cataract (n = 33, Unless Stated) (log CS units) Psychophysical Test Young Normal Older Normal Cataract Vistech FDT (day) Vistech FDT (day) with peripheral glare Vistech FDT (night) Vistech FDT (night) with central glare Miller-Nadler glare tester Pelli-Robson chart Pelli-Robson with BAT 2.09 ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ±0.25* 1.31 ±0.36f 0.27 ± 0.36f 1.11 ±0.31f (n = 6) 0.91 ± ± ± ±0.15 (log MAR units) Berkeley Berkeley with glare Regan 96% chart Regan 96% with BAT Regan 25% chart Regan 25% with BAT Regan 11 % chart Regan 11% with BAT 0.19 ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± 0.14* 0.71 ±0.15t 0.17 ± 0.1 It 0.17 ± ± 0.14* 0.10 ±0.07* 0.37 ± 0.15* 0.50± ± ±0.17 (n = 16) (log stray light) Straylightmeter at 3 Straylightmeter at 10 c Straylightmeter at 28 C 0.98 ± ± ± ± ± ± ± ± ±0.26 n = 32. fn = 31. % n = 29. n = 30. FDT, functional disability test. ence of the central glare source. Only 16 cataract subjects could see any letters on the 11% Regan chart with the BAT. A disability glare score also has been computed for each test (see Table 2). This represents the change in CS or VA induced by the glare source.* As expected, disability glare scores were worst for the cataract subjects. Conversely, the disability glare scores for the young normal subjects frequently were close to zero ie, the introduction of the glare source had very little effect on visual performance in these subjects. Repeatability Forty-one subjects were tested on separate occasions, with test-retest intervals ranging between 1 and 12 wk, * Various names have been adopted for this parameter. These include "glare disability," "disability glare index," 26 "glare susceptibility ratio," 28 and "glare sensitivity." with a median interval of 2 wk. Using one-tailed, paired t-tests, there was no significant improvement (95% confidence limits, P < when adjusted for 25 comparisons) in score on retest for any of the tests. The COR 7 and Pearson's correlation coefficients for the test-retest data are shown in Table 3. All the VAbased measures showed good reliability, with COR values of 0.15 log MAR or less. The COR values for the Regan charts increased with decreasing contrast. Of those tests that measured contrast sensitivity, the Pelli- Robson test was by far the most repeatable, with COR values of about 0.20 log CS units. COR values for the Miller-Nadler and Vistech tests were 0.40 log CS units and higher. The Straylightmeter had a COR of 0.18 log straylight units at 3, and the repeatability decreased at the larger angles. Because the disability glare scores depended on values determined with and without glare, their repeatability should have been correspondingly decreased. The lack of such an effect

7 114 Investigative Ophthalmology 8c Visual Science, January 1993, Vol. 34, No. 1 TABLE 3. Repeatability, Discriminative Ability, and Validity Results for Each Psychophysical Test Psychophysical Test Repeatability Discriminative Ability Validity Coefficient of Repeatability (log CS) Correlation Coefficient (r) A z Values From ROC Curves (A z l) (A Z 2) (3.5 C Correlation r Values With Straylight ') (10 ) (28 ) Vistech FDT (day) Vistech (day) with glare Vistech FDT (night) Vistech (night) with glare Miller-Nadler Pelli-Robson chart Pelli-Robson with BAT ±0.36 ±0.48 ±0.42 ±0.48 ±0.64* ±0.68* ±0.36 ±0.18 ±0.20 ±0.22 (logmar) ±0.14 ±0.14 ±0.13 ±0.11 ±0.13 ±0.10 ±0.09 ±0.10 ±0.11 ±0.14 ±0.15f ±0.15f (log straylight) ±0.18 ±0.21 ± * 0.17* _ _ Berkeley Berkeley with glare Regan 96% chart Regan 96% with BAT Regan 25% chart Regan 25% with BAT Regan 11 % chart Regan 11% with BAT f 0.56f ' Straylight at 3.5 Straylightat 10 Straylight at * -N = 35. t-n = 38. Repeatability: coefficient of repeatability and Pearson's correlation coefficient values between test and retest data of 41 subjects. Discriminative ability: how well the scores from each test discriminate between 24 young and 23 older normal subjects (A z l) and between 33 subjects with cataracts and 23 age-matched normals (A Z 2). Validity: correlation coefficient values between forward light scatter at 3.5, 10, and 28 measured with the van den Berg Straylightmeter for 29 cataract subjects. To ease comparison, any nonsignificant correlations (P > 0.05) are not shown. ROC, receiver operating characteristic. FDT, functional disability test. indicates that the with- and without-glare scores are not truly independent measures. Discriminative Ability The ability of each test to discriminate between visual performance in young and older normal subjects and older normal and cataract subjects was evaluated using ROC curves The A z values, representing the area beneath the ROC curve, for young versus older normal subjects (A z l) and cataract versus older normal subjects (A Z 2) for each test are shown in Table 3. The number of criterion points used in each ROC analysis was determined by the range of values used and the step size of the tests. In the example shown in Figure 1, the highest Pelli-Robson and BAT score for the older subjects was 1.85, and the lowest young subject score was Because the Pelli-Robson was measured in 0.05 log unit steps, this limited the number of criterion points to five. The Straylightmeter has a continuous scale, and criterion points were arbitrarily taken at 0.05 log unit steps. As expected, most of the tests discriminated better between older normal and cataract subjects than between young and older normal subjects. In general, measures of CS or VA in the presence of glare had the highest discriminative ability, whereas the disability glare scores had the worst. Many of the discriminative ability values for cataract versus normal subjects (A z ) were greater than Although this implies that many tests have good discriminative ability, the small differences in A Z 2 values depend on

8 Comparing Disability Glare Tests 115 scores from a few subjects and caution should be exercised in comparing tests or drawing conclusions from these data. Validity Validity was assessed by calculating correlation coefficients between test data and corresponding measurements of forward light scatter obtained with the Straylightmeter. Results from the 11% Regan chart with the BAT and the nighttime Vistech with central glare were excluded from the analysis. Four cataract subjects were unable to see anything on the charts in the presence of glare for one or more of the remaining tests. Therefore, correlation coefficient values were calculated only for 29 cataract subjects and are given in Table 3. In general, measures of CS or VA in the presence of glare have the highest validity, whereas the disability glare scores have the worst. A hierarchical multiple-regression analysis was performed for all the tests and each of the three Straylightmeter measurements. 31 High-contrast VA measurements were considered before any of the other tests, because these are traditionally taken in practice. All the other tests then were considered simultaneously for inclusion into the multiple regression equation. In this format, the analysis assessed whether any tests were providing additional significant information about the Straylightmeter scores beyond VA. High-contrast VA only accounted for 15%, 9%, and 8% of the variance of the 3.5, 10, and 28 scores, respectively. The Pelli-Robson scores with the BAT provided most of the additional information beyond VA. They increased the accounted variance to 65% and 52% for the 3.5 and 10 scores. The Miller- Nadler glare score increased the accounted variance of the 28 score to 45%. When just the CS and low-contrast acuity scores without glare are considered, only the Pelli-Robson scores provided any additional significant information beyond VA. The percentage variance explained by VA and Pelli-Robson scores was 42% and 30% for the 3.5 and 10 scores, respectively. DISCUSSION The normal data compare well with previously reported data for these tests " 30 ' 32 The slightly lower thresholds may be attributable to the use of a full spectacle correction with working distance lenses rather than the subject's own spectacles, 32 using different scoring rules 2930 or using more strict exclusion criteria for the older normal subjects. 26 For most tests, the older normals performed worse than the young normals by around 0.1 log units (one line on the VA charts). This decrement usually was accentuated by the addition of a glare source. Repeatability Both VA based measures, the Berkeley Glare Test and the Regan charts with the BAT, showed good reliability, with COR values of 0.15 or less. These results are consistent with previous studies. 11 ' 33 For the Regan/ BAT tests, there was an increase in COR as contrast was diminished. Furthermore, the repeatability for the Berkeley Glare Test is slightly poorer than that for the Regan/BAT tests. This probably is the result of there being fewer letters per line (5 versus 8) with the Bailey- Lovie charts used in the Berkeley test. 9 Of those tests that measure glare using a contrast sensitivity target, the Pelli-Robson/BAT test was by far the most repeatable, with COR values of 0.18 log units, consistent with previously reported values. 30 Conversely, the Vistech and Miller-Nadler tests gave COR values of 0.40 log units and greater. This is particularly alarming given the widespread use of these tests. The use of COR values makes it somewhat difficult to compare the repeatability of the VA and CS measurements, because they measure different variables. In this situation, the correlation coefficient may be the most useful comparison, particularly if the same group of subjects are used for all analyses and likewith-like comparisons are madeie, comparing CSwith-glare values to VA-with-glare values. Using this approach, the VA measures, the Pelli-Robson chart, and the Straylightmeter had similar levels of repeatability. The Miller-Nadler and Vistech tests were much less repeatable (Table 3). It is interesting to speculate about why the Vistech and Miller-Nadler tests are so unreliable. It probably is a combination of poor psychophysical method, single trials at each contrast level, and large contrast increments. The Vistech test uses a nonforced choice, criterion-dependent psychophysical technique; it allows subjects to make a "blank" response. Criterion-dependent methods have been shown to be less reliable than forced-choice ones Furthermore, Bailey and coworkers 9 showed that test reliability is adversely affected by having a test or scale with large step sizes. The Vistech test has only six contrast values compared to the Pelli-Robson's 16. Finer steps are available for the Pelli-Robson chart if it is scored by-letter. This simple protocol modification decreases the COR by a factor of two. 30 The reduced number of steps on the Vistech test resulted in large step sizes, averaging approximately 0.25 log CS units. This compares with increments of 0.05 log units for the Pelli-Robson chart, when scored by-letter. This same criticism can be leveled at the Miller-Nadler test, which uses 2.5% and 5% contrast steps. For example, the 5% to 2.5% contrast step represents 0.30 log CS units, whereas the 80% to 75% contrast step is equivalent to 0.03 log CS units. Most of the subjects in the repeatability study were

9 116 Investigative Ophthalmology 8c Visual Science, January 1993, Vol. 34, No. 1 normal subjects with low contrast thresholds. The step sizes close to threshold tended to be large. The repeatability of the Miller-Nadler may be better for subjects with poorer CS, so log CS step sizes would be smaller. Indeed, for 10 subjects with a test log CS score of or less, the COR was improved to ±0.20 log units. Bland and Altman 7 stated that the main reason the correlation coefficient is a poor analysis of test-retest repeatability is because it is heavily influenced by the range of scores used. This is supported by the results shown in Table 3. Although for any given test the COR values were very similar for the three scores, the correlation coefficient values for the disability glare scores were consistently much lower. This merely reflects a smaller range of scores (shown by SD values in Table 2). This phenomenon is further highlighted by comparing the test repeatability with and without the glare source. Although the COR values for a given test are similar, the r values are larger for the glare scores than for the nonglare scores, reflecting the greater range of values found. For example, the r values for the Berkeley test were 0.87 without the glare source, with the glare source, and 0.57 for the disability glare score. However, the correlation coefficient values were strongly influenced by the range of values used, which were 0.68, 0.82, and 0.28 log MAR units, respectively. The Berkeley COR results all were about ±0.14 log MAR units, showing similar repeatability. Discriminative Ability and Validity In general, measures of CS or VA in the presence of glare have the highest validity and discriminative ability, whereas disability glare scores have the worst. Disability glare scores are inherently more variable, because they involve two measurements in their calculation. The repeatability results in Table 3, however, suggest this is not a major factor. The repeatability of disability glare scores was similar to that of other test scores. The poorer discriminative ability and validity is more likely, because the disability glare scores lose information provided by the CS or VA loss without glare. 21 CS or VA scores of cataract patients were reduced because of the light from the chart being scattered by the opacity, producing a veiling luminance over the retinal image and thereby reducing its contrast. Test scores with-glare include this information and information that describes the further loss caused by light scatter from a glare source. scores (test score with glare-test score without glare) lost the information about light scatter provided by the CS or VA loss without glare. They only describe the loss in CS or VA caused by the glare source. The superior ability to discriminate between the young and older age groups can, in addition, be explained because the measures of CS or VA in the presence of glare are determined by the whole visual system. The disability glare scores probably reflect only the state of the ocular media and are independent of neural loss. Recent reports suggest that the neural system is primarily responsible for the reduction in CS with age Comparison of Glare Tests Vistech MCT8000. The Vistech normative data combines results from all age groups despite large differences in CS between young and old subjects. The test-retest repeatability was much poorer than any of the other tests used. This may explain the test's poorer discriminative ability and it's lack of correlation with Straylightmeter measurements. The Miller-Nadler Glare Tester. The Miller-Nadler test also showed poor repeatability. This may have been the result of relatively low contrast thresholds among the subjects in the repeatability study. For these low thresholds, the Miller-Nadler tests in large step sizes. Repeatability appears to have been better in subjects with lower thresholds, where step sizes are smaller. Virtually all subjects with cataract could see the lowest contrast target without the glare source, and their contrast thresholds thus were truncated to 5%. From the test's luminance specifications, 4 Weber's contrast was 11.25%, which corresponds to a log CS value of Only two cataract subjects recorded a Pelli-Robson CS score below 0.95 log units and most were well above (Table 2). Because of the truncated nature of the CS without-glare data, the test did not provide 'true' disability glare scores. The large step sizes at low contrast levels resulted in the poor discriminative ability results. This may limit the use of the Miller-Nadler Tester when trying to detect or quantify small amounts of light scatter, as in the PERK study. 38 The smaller step sizes at higher contrast levels improved the test's repeatability, and the ensuing data correlate well with straylightmeter measurements. This suggests that the test may provide clinically useful information when scores are in the region where its step sizes are relatively small. Pelli-Robson CS/BAT. The Pelli-Robson/BAT scores were substantially more repeatable than those of the other glare tests using CS. They had similar repeatability to the VA and Straylightmeter measures. CS loss in cataract is primarily found at the high spatial frequencies, and is thought to be the result of narrow angle light scatter. 239 The low spatial frequency content of the Pelli-Robson chart resulted in its poorer discrimination between cataract and older normal subjects compared to VA (see Table 3). Test scores without glare reflect the effect of light scatter from the "white" areas of the charts. The Pelli- Robson letters are 2.8 in size, whereas the letter size calculated from the average VAs of the cataract sub-

10 Comparing Disability Glare Tests 117 jects varied between (96% Regan) and 0.06 (Berkeley and 11% Regan). This suggests that the reduction in Pelli-Robson CS with cataract was due to light scatter that was wider-angled than that that affected VA. This is confirmed by the much higher correlations between Pelli-Robson scores and the 3.5 and 10 Straylightmeter scores than between straylight and any of the VA scores (Table 3). The Pelli- Robson scores were the only nonglare scores that provided additional information about straylight measurements beyond VA. A reduction of CS or VA in the presence of a peripheral glare source reflects wide-angle light scatter. 22 The Pelli-Robson and VA scores in the presence of glare showed similarly high correlation with straylight and excellent discrimination. Because Pelli-Robson CS without glare already was reduced because of wide-angle scatter in cataract, the BAT glare source increased the effect of such straylight only slightly. This resulted in the poor discriminative ability and validity of the disability glare scores with the Pelli-Robson. In normal subjects, where the Pelli-Robson scores without glare were minimally affected by narrow-angle scatter, the discriminative ability of the disability glare scores were comparatively better. Berkeley and Regan VA/BAT Tests. All the VAbased measures showed good reliability, with COR values of 0.15 log MAR units or less. The VA tests all showed very high discriminative ability between cataract and normal, although the cataract patients probably were referred to the ophthalmologist's office on the basis of VA, not CS. The discriminative ability between young and older normal subjects and the correlation with Straylightmeter measurements was best for the 11% Regan and worst for the 96% Regan scores. 28 Results with the Berkeley test were similar to those for the 25% Regan with the BAT. When using the Regan charts and the BAT to measure relatively small amounts of disability glare, such as in normal subjects or for post-refractive surgery, the 11% chart should be preferred over the 96% or 25%. When measuring larger amounts of disability glare, such as in cataract patients, some patients cannot see any of the letters on the 11 % chart in the presence of the BAT glare source. Therefore, for cataract patients, the 25% Regan chart is preferred over the 11%. The Berkeley test appears capable of being used effectively in both situations. Straylightmeter. The 28 Straylightmeter score was less affected by early cataract and^was less discriminative and reliable than the other straylight scores (Tables 2 and 3). Because there was a high correlation between the 3 and 10 cataract scores (r = 0.95), cataract assessment could be made using either of these measurements (or the mean of both to improve reliability). The Straylightmeter approach has been advocated for clinical use 2140 and does have several advantages, as listed earlier. In addition, the scores have been shown to be discriminative and reliable. However, the test is somewhat difficult for the subject to perform. If the test is used clinically, it may be easier to determine threshold just from the "reappearance" of the flicker. This is much easier for cataract patients to perceive than the disappearance point. Further research is needed to assess the suitability of the Straylightmeter for clinical use. The present study confirms the criteria for clinical vision test design suggested by the AAO report The test should use a forced-choice psychophysical method. 2. Test targets should follow a uniform logarithmic progression. 3. Several trials should be used at each level of acuity or contrast. The Pelli-Robson, Regan, and Berkeley tests have incorporated these three principles into their test design. This study shows that they provide reliable measurements of CS/acuity and disability glare with good discriminative ability and a high correlation with Straylightmeter measurements. The Vistech MCT8000 uses none of these test design principles and provides scores that are less reliable and discriminative and that do not correlate with Straylightmeter measurements. The Miller-Nadler tester uses a fouralternative forced-choice method, but does not use design principles 2 and 3. Glare sources other than the Vistech central source produce no great increase in COR value for CS or VA. The effect of this central glare source may be position and adaptation sensitive. 1 ' 5 The reliability of a test appears to be its most important property. Tests that show poor reliability have poor discriminative ability and validity. The large COR values of the Vistech chart without glare is probably due to the poor chart design, as already discussed. Regardless of how wellstandardized the glare sources are, the reliability of the test cannot be improved beyond this point. This suggests that unless good chart design and psychophysics are used, the geometry and intensity of the glare source are of little importance. Glare tests have been proposed as complementary tests to traditional high-contrast VA, because a significant number of patients with cataract retain relatively good VA yet have significant visual problems. 1 " 6 This may be because high-contrast VA is a poor measurement of forward light scatter, as indicated by its low correlation with Straylightmeter scores (Table 3). The simplest disability glare test, which has been used for many years, is to measure high contrast VA when hindered by a glare source, such as a penlight. 15 " 17 Hierarchical multiple regression analysis indicates that the high contrast VA scores with the BAT provided signifi-

11 118 Investigative Ophthalmology & Visual Science, January 1993, Vol. 34, No. 1 cant additional information beyond high-contrast VA about straylight measurements. The percentage of the variance accounted for increased from 15%, 9%, and 8% with VA, to 27%, 23%, and 14%, respectively, with both measurements. The question is whether this type of test is a sufficiently sensitive evaluation of disability glare or whether low-contrast targets or CS charts with a glare source are needed. Multiple regression analysis suggests that significant additional information about straylight beyond both these measurements can be gained. High-contrast VA measurements were considered first in the analysis, followed by high-contrast VA-with-glare. Next, all the other tests were considered simultaneously for inclusion into the multiple regression equation. The percentage of the variance accounted for using the Pelli-Robson and Miller-Nadler scores in addition to the two VA scores increased to 77%, 63%, and 43% for the 3.5, 10, and 28 Straylightmeter scores, respectively. This suggests that CS charts or low-contrast VA under glare conditions are a better assessment of light scatter than high-contrast VA with glare. Our results suggest that the CS and VA scores in the presence of glare from the Pelli-Robson, Regan, and Berkeley tests all provide similarly reliable, discriminative, and valid measures of visual assessment in cataract. The Miller-Nadler glare tester may provide similar efficacy when used with cataract patients, whose CS scores will be in the region of the test where its step sizes are relatively small. The results suggest it would not be as useful for detecting or measuring subtle changes in the ocular media, such as corneal edema. The Vistech's poor reliability limits the usefulness of its results. The results confirm that CS or low-contrast VA measured in the presence of glare are superior to disability glare scores in assessing cataract patients with normal neural function. 21 Such scores provide better discrimination and are more highly correlated with light scatter. The disadvantage of taking CS or VAwith-glare scores is that they are influenced by postoptical factors. Imagine a patient who has age-related macular degeneration (AMD) as well as cataract, for example. A clinician could not be certain how much of the reduction in CS or VA is the result of cataract and how much is the result of the macular disease. Disability glare scores should not be affected by AMD or any other retinal or neural abnormality. 5 CS and glare tests are most useful in assessing cataract patients with good VA. It is generally possible to obtain an ophthalmoscopic view of the fundus and an assessment of central visual fields in such patients. If neural function appears normal, CS or VA-with-glare measurements should be used to help evaluate visual loss. In patients with early cataract and some neural abnormality, disability glare scores should be used. Because Pelli-Robson CS-without-glare already is reduced because of wide-angle scatter in cataract, the BAT glare source increases the effect of such straylight only slightly. This results in the poor discriminative ability and validity of the disability glare scores with the Pelli-Robson and BAT. Thus, scores from cataract subjects are of limited value. The Regan low-contrast charts with the BAT and the Berkeley test provide reliable, discriminative, and valid measures of disability glare. The lowest contrast Regan chart on which the patient can see some letters should be used, because this provides the highest discriminative ability. With patients who have neural abnormality, it also probably is advisable to use a test that can evaluate neural function behind the cataract. 41 Key Words cataract, contrast sensitivity, disability glare tests, reliability, validity. Acknowledgments The authors thank Drs. Hugh Jellie and Dante Pokrnich for providing us with office space and access to their patients; Drs. Ian Bailey, John Flanagan, and John Perofffor the loan of some of the glare tests; Clement Clarke Ltd. for providing us with Pelli-Robson charts; Mr. Jim Cassidy for producing the A z integration program; Drs. David Whitaker, Graham Strong, and Jeff Hovis for valuable comments on an earlier draft of this report; and all the subjects for their participation. References 1. Miller D, Jernigan ME, Molnar S, Wolf E, Newman J. Laboratory evaluation of a clinical glare tester. Arch Ophthalmol. 1972; 87: Hess R, Woo G. Vision through cataracts. Invest Ophthalmol VisSci. 1978; 17: Paulsson LE, Sjostrand J. Contrast sensitivity in the presence of a glare light. Invest Ophthalmol Vis Sci. 1980; 19: LeClaire J, Nadler MP, Weiss S, Miller D. A new glare tester for clinical testing. Arch Ophthalmol. 1982; 100: American Academy of Ophthalmology report. Contrast sensitivity and glare testing in the evaluation of anterior segment disease. Ophthalmology. 1990; 97: Koch DD, Liu JF. Survey of the clinical use of glare and contrast sensitivity testing./ Cataract Refract Surg. 1990;16: Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986; 1: Reeves BC, Hill AR, Aspinall PA. The clinical significance of change. Ophthalmic Physiol Opt. 1987; 7: Bailey IL, Bullimore MA, Raasch TW, Taylor HR. Clinical grading and the effects of scaling. Invest Ophthalmol VisSci. 1991;32:

12 Comparing Disability Glare Tests 119 ] 0. Swets JA, Pickett RM. Evaluation of Diagnostic Systems. Methods From Signal Detection Theory. London: Academic Press; 1982: Massof RW, Drum BA, Rubin GS. ROC analysis applied to multivariate diagnostic tests. Noninvasive Assessment of the Visual System Technical Digest 1988; 3: Bernth-Petersen P. Visual functioning in cataract patients. Ada Ophthalmol. 1981; 59: Koch DD. Glare and contrast sensitivity testing in cataract patients. J Cataract Refract Surg. 1989; 15: Elliott DB, Hurst MA, Weatherill J. Comparing clinical tests of visual function in cataract with the patient's perceived visual disability. Eye 199O;4: Holladay JT, Prager TC, Truillo TC, Ruiz RS. Brightness acuity test and outdoor visual acuity in cataract patients. J Cataract Refract Surg. 1987; 13: Neumann AC, McCarthy GR, Locke J, Cobb B. Disability glare devices for cataractous eyes: A consumers guide. J Cataract Refract Surg. 1988; 15: Prager TC, Urso RG, Holladay JT, Stewart RH. Glare testing in cataract patients: Instrument evaluation and identification of sources of methodological error. J Cataract Refract Surg. 1989; 15: Allen MJ, Vos JJ. Ocular scattered light and visual performance as a function of age. American Journal ofoptometry and Archives of the American Academy of Optometry!967;44:7l ] 9. Elliott DB, Hurst MA. Assessing the effect of cataract: A clinical evaluation of the Opacity Lensmeter 701. Optom VisSci. 1989;66: de Waard PWT, IJspeert JK, van den Berg TJTP, de Jong PTVM. Intraocular light scattering in age-related cataracts. Invest Ophthalmol Vis Sci. 1992; 33: Elliott DB, Hurst MA, Weatherill J. Comparing clinical tests of visual loss in cataract patients using a quantification of forward light scatter. Eye 1991;5: van den Berg TJTP. On the relation between glare and straylight. Doc Ophthalmol. 1991; 78: van den Berg TJTP. Importance of pathological intraocular light scatter for visual disability. Doc Ophthalmol. 1986;61: IJspeert JK, Waard PWT, van den Berg TJTP, De Jong PTVM. The intraocular straylight function in 129 healthy volunteers; dependence on angle, age and pigmentation. Vision Res. 1990; 30: Vistech Consultants Inc. Multivision Contrast Tester (MCT8000). Instruction manual. 26. Bailey IL, Bullimore MA. A new test for the evaluation of disability glare. Optom VisSci. 1991; 68: Pelli DG, Robson JG, Wilkins AJ. The design of a new letter chart for measuring contrast sensitivity. Clinical Vision Science. 1988; 2: Regan D. The Charles F. Prentice Award Lecture 1990: Specific tests and specific blindnesses: Keys, locks, and parallel processing. Optom Vis Sci. 1991;68: Elliott DB, Hurst MA. Simple clinical techniques to evaluate visual function in patients with early cataract. Optom VisSci. 1990;67: Elliott DB, Bullimore MA, Bailey IL. Improving the reliability of the Pelli-Robson contrast sensitivity test. Clinical Vision Science. 1991; 6: Thorpe Davis E, Sherman J, Bass SJ, Schnider CM. Pre-surgical prediction of post-surgical visual function in cataract patients: Multivariate statistical analyses of test measures. Clinical Vision Science. 1991;6: Hirsch RP, Nadler MP, Miller D. Clinical performance of a disability glare tester. Arch Ophthalmol. 1984; 102: Elliott DB, Sheridan M. The use of accurate visual acuity measurements in clinical anti-cataract formulation trials. Ophthalmic Physiol Opt. 1988;8: Higgins KE, Jaffe MJ, Coletta NJ, Caruso RC, De- Monasterio FM. Spatial contrast sensitivity. Importance of controlling the patient's visibility criterion. Arch Ophthalmol. 1984; 102: Vaegan, Halliday BL. A forced-choice test improves clinical contrast sensitivity testing. Br J Ophthalmol. 1982;66: Morrison JD, McGrath C. Assessment of the optical contributions to the age-related deterioration in vision. Quarterly Journal of Experimental Physiology 1985;70: Elliott DB, Whitaker D, MacVeigh D. Neural contribution to spatiotemporal contrast sensitivity decline in healthy ageing eyes. Vision Res. 1990;30: Waring GO, Lynn MJ, Gelender H, et al. Results of the prospective evaluation of radial keratotomy (PERK) study one year after surgery. Ophthalmology. 1985;92: Elliott DB, Gilchrist J, Whitaker D. Contrast sensitivity and glare sensitivity changes with three types of cataract morphology: Are these techniques necessary in a clinical evaluation of cataract? Ophthalmic Physiol Opt. 1989;9: Beckman C, Abrahamsson M, SjostrandJ. Evaluation of a clinical glare test based on estimation of intraocular light scatter. Optom Vis Sci. 1991;68: Fuller DG, Hutton WL. Presurgical Evaluation of Eyes With Opaque Media. London: Grune and Stratton; 1982.