Measuremet Variability i Duplex Sca Assessmet of Carotid Atherosclerosis Kim Sutto-Tyrrell, DrPH; Sidey K. Wolfso Jr., MD; Tria Thompso, BSN, RVT; ad Sheryl F. Kelsey, PhD Backgroud ad Purpose: The reproducibility of duplex sca measures of carotid atherosclerosis was evaluated as part of a study assessig the prevalece of carotid disease i elderly adults. Methods: Doppler measures of blood flow velocity were used to evaluate disease severity, ad extet of carotid plaque was scored from the B-mode image. A reader assiged a grade from to to each of seve segmets i the carotid system, based o the umber ad size of lesios preset. Reproducibility data were obtaied from study participats who uderwet a repeat sca by a secod soographer. Each sca was the scored by two readers. Results: Doppler measures of blood flow velocity were foud to be highly reproducible, with itraclass correlatio coefficiets of.8 for the commo carotid artery,.8 for the iteral carotid artery, ad. for the iteral carotid artery velocity to commo carotid artery velocity ratio. Reproducibility of plaque grade was evaluated usig segmet as the uit of aalysis, ad both soographer ad reader variatio were aalyzed. Whe readers differed perfect agreemet was achieved i 8% of the segmets (K=.), ad whe soographers differed perfect agreemet was obtaied i 8% of the segmets (K=.). Whe both soographer ad reader differed, perfect agreemet was obtaied i % of the segmets (K=.). The plaque idex, created by summig plaque grades from selected segmets, was highly reproducible, with a itraclass correlatio coefficiet of.8. Coclusios: The duplex sca protocol described here provides reliable measures of both extet ad severity of carotid disease that are appropriate for use i cross-sectioal studies. (Stroke 99;:-) Whe coductig a cliical study of carotid atherosclerosis usig duplex scaig, the vast amout of iformatio obtaied from the sca must be summarized ito relevat measures of carotid disease. These measures are subject to variatio from may sources; therefore, the demostratio of their reproducibility is crucial. This paper presets a aalysis of the variatio i measuremet foud i the duplex scaig protocol used i a study desiged to evaluate the prevalece of carotid atherosclerosis i elderly adults. Duplex scaig combies real-time B-mode ultrasoud with Doppler ultrasoud to create a system capable of both imagig of vessels ad From the Departmet of Epidemiology (K.S.-T., S.F.K.), Graduate School of Public Health, the Departmet of Neurosurgery (S.K.W.), School of Medicie, ad Motefiore Uiversity Hospital (S.K.W., T.T.), Uiversity of Pittsburgh, Pittsburgh, Pa. Supported by Natioal Istitutes of Health grat HL-98. Address for correspodece: Kim Sutto-Tyrrell, DrPH, Departmet of Epidemiology, Graduate School of Public Health, Uiversity of Pittsburgh, DeSoto Street, Pittsburgh, PA. Received September, 99; accepted October, 99. aalysis of blood flow characteristics. Iformatio from the Doppler compoet of the duplex sca ca be used to determie the degree to which atherosclerotic plaque causes chages i blood flow ad thus is a measure of the cliical importace or "severity" of the plaque. As a plaque progressively arrows the lume of a vessel, the velocity of blood flow at that site must icrease if a costat flow volume is to be maitaied. There is cosiderable itersubject variatio i blood flow velocity readigs; therefore, the ratio of the iteral carotid artery (ICA) blood flow velocity to the commo carotid artery (CCA) blood flow velocity (ICA/CCA ratio) is ofte used as a measure of disease. - This ratio stadardizes the ICA measuremet so that patiets ca be compared. Images geerated from the B-mode compoet of the duplex sca are valuable i assessig a wider spectrum of carotid atherosclerosis, icludig miimal to moderate disease that has ot yet caused chages i blood flow. This iformatio ca thus be used to assess the overall burde of atherosclerotic plaque or "extet" of disease.
Stroke Vol, No February 99 Subjects ad Methods The duplex sca protocol described here was developed for use i a study acillary to the Systolic Hypertesio i the Elderly Program (SHEP). The study was desiged to estimate the prevalece of carotid atherosclerosis i participats of SHEP at the Uiversity of Pittsburgh ad i a group of ormotesive cotrol subjects. Participats uderwet duplex scaig at the Peripheral Vascular Diagostic Laboratory located i Motefiore Uiversity Hospital, Pittsburgh, Pa. Scas were performed usig a Diasoics DRF duplex scaer (Milpitas, Calif.) with a -MHz imagig probe ad.-mhz Doppler. The soographer measured peak blood flow velocity at the mid-cca ( cm proximal to the begiig of the carotid bulb) ad i the ICA at the poit of highest velocity distal to the flow divider. If the mid-cca measuremet was abormal, the soographer took readigs proximal ad distal to this poit. The lowest readig obtaied was the used. The ICA/CCA ratio was used as a measure of disease severity. Scas were recorded o videotape ad scored by a reader. The carotid system was divided ito seve segmets (Figure ), ad for each segmet the degree of plaque was graded usig the followig criteria: grade, o observable plaque; grade, oe small plaque (<% of the vessel diameter); grade, oe medium plaque (-% of the vessel diameter) or multiple small plaques; ad grade, oe large plaque (>% of the vessel diameter) or multiple plaques with at least oe medium plaque. The grades were the summed to create a variable called the plaque idex, which was used as a measure of the extet of atherosclerosis. To evaluate the reproducibility of the ICA/CCA ratio ad the plaque idex, participats were selected at radom to udergo a repeat sca o oe side by a secod soographer. Because the hypertesive participats had a higher prevalece of carotid disease tha the ormotesive participats, the selectio process was weighted to iclude more hypertesive tha ormotesive subjects so that a sample with a sufficiet rage of carotid disease severity was TABLE. Segmet ICA CCA FIGURE. Schematic drawig of segmets of carotid artery., from take-off of commo carotid artery (CCA) to poit cm proximal to carotid bulb;, from cm proximal to carotid bulb to begiig of carotid bulb (poit where ear ad far walls of artery are o loger parallel);, from begiig of carotid bulb to flow divider;, first cetimeter of iteral carotid artery (ICA), measured from tip of flow divider;, secod cetimeter of ICA, measured from tip of flow divider;, first cetimeter of exteral carotid artery (ECA), measured from tip of flow divider; ad, secod cetimeter of ECA, measured from tip of flow divider. obtaied. For each of the patiets, two scas were produced, oe from each soographer. Each of the scas was the scored by two differet readers, resultig i records for aalysis. Measuremet variatio due to the soographer was assessed by comparig records for which the reader was the same but the soographers differed. Variatio due to the reader was assessed by comparig records for which the soographer was the same but the readers differed. Fially, the combied variatio due to both soographer ad reader was assessed by comparig records for which both the soographer ad the reader differed. Both the soographers ad the readers were uaware of the blood pressure status of the participats. I additio, soographer was uaware of which participats would have a repeat sca for the study of reproducibility, ad soographer was blided to the results of soographer. Readers A ad B scored the scas separately ad were uaware of the other's results. Readigs of videotapes of the same patiet were separated by -week itervals to esure that the reader would ot recogize patiet Distributio of Plaque Grade by Segmet for Primary Soographer ad Primary Reader Oly Missig No. % No. % No. % No. % No. % 8 9. 8............. = carotid systems. :, o observable plaque;, oe small plaque (<% of vessel diameter);, oe medium plaque (-% of vessel diameter) or multiple small plaques;, oe large plaque (>% of vessel diameter) or multiple plaques with at least oe medium plaque......................
Sutto-Tyrrell et al Variability i Ultrasoic Measuremet of Carotid Disease TABLE. Reproducibility of Plaque Grade Reader variatio* Reader B Reader A plaque grade 8 8 8 8 Soographer variatiof Soographer Soographer plaque grade 9 9 8 9 9 9 9 Combied reader/soographer variatio^: Soographer, reader B or soographer, reader A Soographer, reader A or soographer, reader B plaque grade 8 'Perfect agreemet=8%, agreemet withi oe grade=99%, K=.. tperfect agreemet=8%, agreemet withi oe grade=98%, K=.. ^Perfect agreemet=%, agreemet withi oe grade=9%, K=.. aatomy ad thus idetify the videotape as oe previously read. Reproducibility of the ICA/CCA ratio ad the plaque idex (both cotiuous variables) was evaluated by calculatig Pearso ad itraclass correlatio coefficiets. The Pearso correlatio coefficiet is a measure of the liear relatio betwee two readigs, with a high value idicatig a strog liear relatio. While a liear relatio is ecessary for reliability, it is ot sufficiet. For example, if reader A cosistetly reads twice as high as reader B, there would be a strog liear relatio but reproducibility would be poor. To appropriately address this problem, the itraclass correlatio coefficiet was calculated to determie the portio of measuremet variability that could be explaied by differeces betwee subjects. A high value idicates that the majority of variatio comes from itersubject differeces ad little from error i measuremet. 9 8 8 8 The reproducibility of plaque grade (a categorical variable with levels of,,, or ) was assessed usig the K statistic, a measure of the agreemet that occurs over ad above chace. Values from. to. are geerally iterpreted as represetig slight agreemet, those from. to. as represetig fair agreemet, those from. to. as represetig moderate agreemet, ad those above. as represetig substatial agreemet. Results The carotid systems chose to study reproducibility represet a rage of carotid disease. I three arteries (%) o areas of plaque could be idetified. Whe disease was preset, it was most ofte foud i the carotid bulb or the first cetimeter of the ICA ad most ofte received a plaque grade of (Table ). Whe the B-mode image was ot of sufficiet quality for accurate assessmet, the plaque grade was
8 Stroke Vol, No February 99 TABLE. Aalysis of Disagreemets i Plaque Grade Disagreemet withi Oe grade Ovs. vs. vs. Two grades Ovs. vs. Three grades Ovs. Readers differed (=9) No. % 8 9.8..... Disagreemets whe Soographers differed (/i=8) No. %..8.... Both reader ad soographer differed (=8) No. % 8.8....9. coded as missig. This occurred most ofte with the secod cetimeter of the exteral carotid artery ( of, %) ad the secod cetimeter of the ICA (seve of, %). A plaque grade was assiged for each segmet, so reproducibility of this variable was evaluated usig segmet as the uit of aalysis (Table ). Reader agreemet was assessed by comparig the plaque grades assiged whe the soographer was the same but the readers differed. Sixty comparisos were available, from soographer ad from soographer. A total of comparisos were possible (seve segmets times scas); however, because reproducibility could be assessed oly for segmets for which both readers agreed that the B-mode image was adequate, oly 8 segmets were aalyzed. Readers had perfect agreemet i 8.% of the segmets, correspodig to a K of.. The aalysis was the repeated, comparig the plaque grades assiged whe the reader was the same but the soographers differed. Of segmets, perfect agreemet was obtaied i.8%, correspodig to a K of.. Fially, the results were compared whe both the reader ad the soographer differed. Of segmets, perfect agreemet was obtaied i.8%, correspodig to a K of.. TABLE. Reproducibility of Plaque Grade By Segmet Segmet Iterreader variatio % perfect agreemet 9. 8.. 8. 8..8 K.8..... Itersoographer variatio % perfect agreemet 9. 8... 8. 9. K.8.9...9.9 Aalysis of segmet was ot possible because of amout of missig data. Whe disagreemets occurred, they were most ofte withi oe grade (8.8%). The most commo disagreemet was betwee o disease (grade ) ad little disease (grade ) (Table ). The reproducibility of plaque grade varied by segmet (Table ). Whe the soographer was the same but the readers differed, K reflected substatial agreemet for the CCA, bulb, ad first cetimeter of the ICA (.-.8) but oly fair to moderate agreemet for the secod cetimeter of the ICA ad the exteral carotid artery (. ad., respectively). Whe the reader was the same but the soographers differed, K reflected moderate to substatial agreemet for the CCA ad bulb (.9-.8) but oly fair agreemet for the ICA ad exteral carotid artery (. ad.9, respectively). Whe both the reader ad the soographer differed, K also reflected moderate to substatial agreemet for the CCA ad bulb (.-.8) but oly fair agreemet for the ICA ad exteral carotid artery (.-.8). Iitially, the plaque idex was defied as the sum of plaque grades from all seve segmets. Because of the high prevalece of missig data i the distal segmets, this defiitio was revised to iclude plaque grades from the CCA, bulb, ad first cetimeter of the ICA oly. For the carotid systems Combied reader/ soographer variatio % perfect agreemet 9. 8... 88. 9. K.8.8....8
Sutto-Tyrrell et al Variability i Ultrasoic Measuremet of Carotid Disease 9 Nubef of Cfirotid Systems TABLE. Reproducibility of Peak CCA Ratio Pearso Variable CCA velocity ICA velocity ICA/CCA ratio 8 8 correlatio.8.88.8 Blood Flow Velocity ad ICA/ Itraclass correlatio.8.8.8 CCA, commo carotid artery; ICA, iteral carotid artery. Plaque Idex FIGURE. Bar graph of distributio of plaque idex (sum of plaque grades from segmets,,, ad ) as obtaied from primary soographer ad primary reader. Plaque idex could ot be calculated i two cases because of missig data. chose to study reproducibility, the mea plaque idex was. with a rage of - (Figure ). Both Pearso ad itraclass correlatio coefficiets were high for this variable, eve whe both the soographer ad reader differed (.8 ad.8, respectively; Table ). Blood flow velocity measuremets are subject to variability due to the soographer oly. Pearso ad itraclass correlatio coefficiets were high for the CCA ad ICA readigs as well as for the ICA/CCA ratio (Table ). Discussio Data obtaied from a duplex sca ca be processed i umerous ways. The measure of carotid disease chose depeds o the goal of the test. From a cliical perspective, oe is usually iterested i the severity of disease rather tha the extet. The Doppler compoet of the sca is best suited to measure disease severity because it directly measures the effect of carotid disease o blood flow. For this reaso, some authors have cocetrated o developig techiques to quatify carotid disease based primarily o aalysis of the Doppler spectrum. The ICA/CCA ratio obtaied with the protocol described here was foud to be highly reproducible. This ratio idicates the degree to which carotid disease causes chages i blood flow, allowig us to describe our populatio i terms of disease that has cliical importace. Assessig the reproducibility of the B-mode compoet of the scaig protocol was much more TABLE. Reproducibility of Plaque Idex Source of variatio Reader Soographer Reader ad soographer Pearso correlatio.8.8.89 Itraclass correlatio.88.8.8 Plaque idex, sum of plaque grades from segmets,,, ad. difficult because variatio ca come from both the soographer ad the reader. Assessig the variability of oly oe may uderestimate the true error. We foud that most of the variatio i measuremet is due to the soographer ad relatively little to the reader. This reflects the complexity of performig a high-quality sca ad uderscores the ecessity for extesive traiig ad experiece of the soographer. While reproducibility of the plaque idex was high, reproducibility of the plaque grade withi a segmet was lower. This is cosistet with the fidigs of other groups. 8 Some of the variability is due to the decisio o the locatio of the plaque. If a lesio is close to a segmet border or crosses it, the both readers may agree that the lesio exists, but they may score it as occurrig i differet segmets. This type of disagreemet does ot affect the plaque idex because it is idepedet of the locatio of disease. Other groups have also reported that reproducibility decreases as oe moves higher i the carotid system. 9 The positio of the bulb ad ICA with respect to the ski surface results i a less optimal agle of isoatio tha i the CCA ad thus a lower-quality image. I additio, complicated lesios with acoustic shadows are more ofte foud i the bulb ad ICA. Because this fact has bee recogized, research by at least oe group has restricted duplex scaig to the CCA aloe. While it is possible that the extet of disease low i the carotid system may serve as a accurate surrogate for disease higher i the system, the most efficiet method of summarizig the extet of carotid disease remais ukow. I ivestigatios of the etiology of carotid disease, it has bee foud that a measure of the extet of disease such as the plaque idex correlates better with risk factors tha a measure of the severity of disease. 8 This may be because measures of disease extet obtaied from the B-mode compoet of the duplex system iclude a wide spectrum of disease from miimal to severe. Thus, studies of the epidemiology of carotid disease frequetly cocetrate o the B-mode compoet of the duplex system. Measuremet techiques usig B-mode ultrasoud are still evolvig. I vitro studies of huma carotid artery specimes have show that the itimal-medial thickess ca be measured accurately usig B-mode ultrasoud. Such measures have also bee foud to be highly reproducible. This has resulted i the ability to observe the progressio of carotid disease over time ad the iitiatio of studies that use progressio of carotid atherosclerosis as a
Stroke Vol, No February 99 outcome measure. 9 Reproducibility is particularly importat for such studies because if the precisio of the techique is poor, the evidece of disease progressio could easily be lost i measuremet error. Such measuremets require high-quality B-mode images; therefore, missig iformatio ca result whe arteries are tortuous or cotai mieralizatio with resultig acoustic shadows. Missig data due to aatomic variatio are probably radom; however, missig data related to disease severity are ot radom because the B-mode image deteriorates as severity worses. Results could therefore be biased if the populatio uder study has a high prevalece of advaced disease. A score such as our plaque idex is more subjective, but iformatio ca be obtaied i more tha 9% of study participats eve whe the prevalece of complicated lesios is high. Studies are curretly uder way to determie whether this protocol is useful i assessig carotid disease progressio over time. I summary, the duplex sca protocol described here provides two primary measures of carotid disease, the plaque idex (a measure of disease extet) ad the ICA/CCA ratio (a measure of disease severity). Both measures were foud to be highly reproducible. This duplex sca protocol is thus a reliable method for the measuremet of carotid atherosclerosis i cross-sectioal studies. Refereces. Garth K, Carroll B, Sommer EG, Oppeheimer DA: Duplex ultrasoud scaig of the carotid arteries with spectrum aalysis. Radiology 98;:8-8. Blackshear W, Phillips DJ, Chikos PM, Harley JD, Thiele BL, Stradess DE: Carotid artery velocity patters i ormal ad steotic vessels. Stroke 98;ll:-. Keagy BA, Pharr WF, Thomas D, Bowes DE: Objective criteria for the iterpretatio of carotid artery spectral aalysis patters. Agiology 98;:-. Shrout PE, Fleiss JL: Itraclass correlatios: Uses i assessig rater reliability. Psychol Bull 99;8:-8. Ladis JR, Koch GG: The measuremet of observer agreemet for categorical data. Biometrics 9;:9-. Laglois Y, Roederer GO, Cha A, Phillips DJ, Beach KW, Marti D, Chikos PM, Stradess DE Jr: Evaluatig carotid artery disease: The cocordace betwee pulsed Doppler/ spectrum aalysis ad agiography. Ultrasoud Med Biol 98; 9:-. Roederer GO, Laglois YE, Jager KA, Lawrece RJ, Primozich JF, Phillips DJ, Stradess DE Jr: A simple spectral parameter for accurate classificatio of severe carotid disease. Bruit 98;8:-8 8. Crouse JR, Harpold GH, Kahl FR, Toole JF, McKiey WM: Evaluatio of a scorig system for extracraial carotid atherosclerosis extet with B-mode ultrasoud. Stroke 98;: - 9. O'Leary DH, Polak JF, Wolfso SK Jr, Bod MG, Bommer W, Sheth S, Psaty BM, Sharrett AR, Maolio TA, o behalf of the CHS Collaborative Research Group: Use of soography to evaluate carotid atherosclerosis i the elderly: The Cardiovascular Health Study. Stroke 99;:-. Saloe R, Saloe JT: Progressio of carotid atherosclerosis ad its determiats: A populatio-based ultrasoography study. Atherosclerosis 99;8:-. Sutto KC, Wolfso SK Jr, Kuller LH: Carotid ad lower extremity arterial disease i elderly adults with isolated systolic hypertesio. Stroke 98;8:8-8. Pigoli P, Tremoli E, Poli A, Oreste P, Raoletti R: Itimal plus medial thickess of the arterial wall: A direct measuremet with ultrasoud imagig. Circulatio 98;:99-. Bod MG, Isull W, Gardi JF, Wilmoth SK: Reliability of B-mode ultrasoud i measurig small carotid artery plaques (abstract). Circulatio 98;(suppl II):II-. Bod MG, Stricklad HL, Wilmoth SK, Safrit A, Phillips R, Szostak L for the MIDAS Research Group: Itervetioal cliical trials usig oivasive ultrasoud edpoits: The Multiceter Isradipie/Diuretic Atherosclerosis Study. / Cardiovasc Pharmacol 99;(suppl l):s-s. Bod MG, Wilmoth SK, Eevold GL, Stricklad HL: Detectio ad moitorig of asymptomatic atherosclerosis i cliical trials. Ami Med 989;8(suppl A):- KEY WORDS ultrasoics blood flow velocity carotid artery diseases