Results from the Health and Retirement Study Biomarker Validation Project

Transcription

1 Results from the Health and Retirement Study Biomarker Validation Project Eileen Crimmins Jung Ki Kim Heather McCreath Teresa Seeman DRAFT January

2 Table of Contents Aims 4 Approach of the Project 4 Analyses 5 Overview of Results 7 Comparison of Glycosylated Hemoglobin (HbA1c) from DBS and Whole Blood 11 General Information and Overview of Results 11 Descriptive Information 15 Figure 1: Scatter Plots of the Validation Samples 16 Figure 2: Bland Altman Plots 24 Figure 3: Differences between DBS and Venous Blood 30 Comparison of Whole Blood and DBS (including Regression Estimates and 31 Z Converted Values) Figure 4: HbA1C in HRS Samples and NHANES and NSHAP 36 Comparison of Total Cholesterol from Dried blood Spots and Venous Blood 37 General Information and Overview of Results 37 Descriptive Information 38 Figure 1: Scatter Plots of the Validation Samples 41 Figure 2: Bland Altman Plots 46 Figure 3: Differences between DBS and Venous Blood 48 Comparison of DBS and Serum (including Regression Estimates and Z 49 Converted Values) Figure 4: Total Cholesterol in HRS Samples and NHANES 53 Comparison of HDL Cholesterol from Dried blood Spots and Venous Blood 54 General Information and Overview of Results 54 Descriptive Information 55 Figure 1: Scatter Plots of the Validation Samples 58 Figure 2: Bland Altman Plots 63 Figure 3: Differences between DBS and Serum 65 Comparison of DBS and Serum (including Regression Estimates and Z 66 Converted Values) Figure 4: HDL in HRS Samples and NHANES 70 Comparison of Cystatin C DBS and Venous Assays 71 General Information and Overview of Results 71 Descriptive Information 71 Figure 1: Scatter Plots of the Validation Samples 75 Figure 2: Bland Altman Plots 82 Figure 3: Differences between DBS and Serum 86 Comparison of DBS and Serum (including Regression Estimates and Z 88 Converted Values) Figure 4: Cystatin C in HRS Samples and NHANES 94 Comparisons of CRP DBS and Venous Blood Assays 95 General Information and Overview of Results 95 Descriptive Information 96 Figure 1: Scatter Plots of the Validation Samples 102 2

3 Figure 2: Bland Altman Plots 119 Figure 3: Differences in CRP by Serum Level 141 Comparison of DBS and Serum (including Regression Estimates and Z 142 Converted Values) Figure 4: CRP in HRS and NHANES 167 3

4 The aims of this project were to a. Calibrate DBS results against whole blood values. b. Compare DBS across labs c. Compare across filter papers. The Approach of the Project: To accomplish our aims, we a. Collected dried blood spots and venous blood from 92 non-sample subjects aged 50 + during the period from June through December These were volunteers who came to the UCLA Alhambra Research Center. Participants provided signed consent to be part of the study and were paid $25 to participate. b. DBS were sent to a variety of labs: Heritage, the University of Vermont, the University of Washington, Geonostics (using FlexSite assay), and to Thom McDade. c. HbA1c was assayed from fresh whole blood at a local lab. d. For other assays, whole blood was separated into serum and plasma. Serum was frozen and sent to the University of Vermont for assay of total and HDL cholesterol, CRP, and Cystatin C. The DBS were sent to the following Laboratories: a. Heritage DBS Cards. These cards were sent to HRS in the mail (not frozen) and then sent from HRS to Heritage with HRS survey subject cards for assay of HbA1c, Total cholesterol, and HDL cholesterol. These samples were exposed to conditions similar to those of the actual HRS samples. b. DBS cards for CRP and Cystatin C were sent directly to the University of Vermont for assay (these were frozen at the UCLA facility and sent frozen to Vermont in one batch). c. We also collected one blood spot on a Biosafe treated card that we sent to Geonostics (which has acquired FlexSite). These assays were done in the Spring of The cards were sent frozen from UCLA where they were stored at -70 degrees from the time of collection until just before assay. d. We also sent DBS spots on untreated cards (Biosafe card) to the University of Washington for examination of Total cholesterol, HDL cholesterol, HbA1c, CRP and Cystatin C. These cards were frozen after drying and they were sent frozen to Washington. Most of the assays were done in However, washington developed and validated the CRP and Cystatin C assays during the course of this project. The CRP and Cystatin C assays were done in August of e. We sent one frozen untreated Biosafe card to Thom McDade for assay of CRP for comparison with assays done by the University of Vermont. 4

5 Comparisons that can be made a. Comparisons can be made between whole blood values of HbA1c and DBS assays from the University of Washington, Heritage, and Geonostics (using FlexSite assay). b. Serum blood comparisons can be made with DBS assays for Total and HDL cholesterol from the University of Washington and Heritage labs. c. DBS assays for the University of Washington and Heritage can be compared for HbA1c, total, and HDL cholesterol. d. Comparisons of CRP assays from DBS and serum values can be done for the University of Vermont, Thom McDade s lab and the University of Washington. These three DBS assays can also be compared to each other. e. Cystatin C from serum at the University of Vermont is compared to the assays from DBS at the University of Vermont, and the University of Washington. f. Values for DBS assays from two cards - the Heritage Card and what we call the Biosafe card (cards from earlier HRS rounds) - can be compared for CRP and Cystatin C assayed at the University of Vermont. Analyses Presented in this Document: We present detailed information for each of the assays in a separate section: HbA1c, Total cholesterol, HDL cholesterol, CRP, and Cystatin C. For each of the assays we present some general information, descriptive statistics and information, followed by a variety of methods to assess comparability between assays. The specific information included in each section is arranged as follows: 1. Descriptive information on the assays for all the available data followed by descriptive information on matched pairs of assays from DBS and venous assays a. The percent who would be scored in the level of risk using conventional definitions of risk b. The distribution of values on the validation samples c. Scatterplots relating the DBS and venous assays and DBS assays. d. Bland Altman Plots which provide conventional ways of comparing values from multiple assays. This comparison and the one below help to determine whether the difference between assays is similar across the range of values. Bland Altman plots the Difference between two assays on the Y axis against the Average on the X Axis. For our assays the formulas are 5

6 Average = (DBS + Whole Blood)/2; Difference = DBS - Whole blood. We show the 95% confidence intervals for the difference. e. Differences between DBS and Venous Blood by level of venous blood assay 2. We then investigate methods of transforming DBS values. a. Quartile distributions of venous and DBS assays to determine how similarly values from different assays would distribute by quartile. b. Transforming DBS scores into conventional whole blood scores 1) Comparisons of assays when DBS are converted to whole blood values using Z scores. We can use Z scores to transform the values of one assay into those based on some standard and retain the variability. If we compute the Z score for one assay, (Z 1 = x mean 1 / sd 1 ), we can estimate the value of x in a standard assay by substituting the Z 1 value into an equation using the mean and standard deviation of the distribution from the standard assay (Z 1 sd s ) + mean s = x s The x s value reflects the value that would be obtained with the standard assay. 2) Regression equations relating the whole blood and venous blood assays and the multiple DBS assays when available. One could estimate a venous equivalent value using a regression equation based on comparing values from a standard assay and another assay. This may be a good solution if the variance explained is very high. If the variance explained is not high, it results in losing variability as unexplained variance results in estimates closer to the mean. 3) Comparison of assays when the regression equations shown in section 9 are used to estimate DBS values. 3. Comparison of HRS sample assays with assay results from other surveys At the end of each section, we compare HRS sample assay results with those from other surveys with similar assays. 4. Results: We include conclusions throughout these analyses but we summarize the high points below. 6

7 Overview of Results 1. HbA1c All the DBS values are reasonably highly correlated with the venous blood values (R 2 of.74 for FlexSite,.95 for the University of Washington, and.96 for Flexite). The University of Washington generally seems to be a superior assay with the exception that in the Bland Altman results which indicate that the differences between the UWDBS assay for HbA1c and the venous blood assay vary with the level of the assay. The R 2 between the two DBS assays is.71. The Flexite assay also seems to match venous values very well: Flexite is highly correlated with the venous (R 2 of 0.93). The R 2 between Flexite and UWDBS is very high (R 2 of 0.95). The regression estimated equivalent of venous values and the Z score estimated values from the University of Washington and FlexSite have average values very close to whole blood. Because the R 2 is so high the regression transformation is reasonable although we would be cautious in using transformed values. Using cutoffs is an issue even when the R 2 is high. Transforming the Heritage assay values using the regression equation results in a narrowing of the range. We need to note that the Heritage samples were not maintained under the same conditions as the University of Washington samples. The Heritage samples were shipped in the mail like regular HRS samples which could be the reason that the assay does not relate as strongly to serum levels. Examination of the HRS data show that the various assays appear to be relatively similar in distribution. However, where they differ is right at the cutpoint for high risk. This may need to be a special focus of any adjustment. The University of Washington provided whole blood equivalent values differs more from the whole blood values we obtained than any of the DBS assays. It indicates levels of dysregulation that are too high. 2. Total Cholesterol. The assays of total cholesterol produce very different values. The University of Washington value is very high relative to the value from the venous blood and the Heritage DBS. The Heritage value is closer to the venous value, although somewhat lower than venous on average. The University of Washington distribution of DBS assays shows that there were a significant number of very high values in their assay. Using traditional cutpoints will not work with the University of Washington values. While the mean level of the University of Washington differs more from the venous level than the Heritage mean difference, the University of Washington DBS is more closely associated with venous blood values than the Heritage DBS 7

8 values are (R 2.55 versus.30). The relationship between the University of Washington and the Heritage values is very low (R 2.23). The size of the differences between both DBS assays and the value of the venous assay differs by the level of the assays. Using a regression approach to produce serum cholesterol equivalents does not work well because of the low associations. Z scores are more promising for creating equivalent values. The DBS cholesterol assays need to be improved or replaced. The lack of strong association between the cholesterol values for the two HRS labs is going to make it challenging to examine change over time. Examination of HRS data indicates that the distribution from Biosafe assays for 2006 and 2006 are quite different. The 2006 Biosafe assay is similar to NHANES and the 2008 assay is similar to the University of Washington in distribution, with the exception of some very high values in the Washington assay. 3. HDL cholesterol The HDL values are similar to the Total cholesterol values in that the University of Washington assay produces values much higher than the others. On the other hand, the association between the University of Washington DBS values and the venous blood values is stronger (.48) than that of Heritage DBS (.32). The association of the two DBS assay values is low (.20). Z scores may be a better method of transformation than regression estimates of venous equivalents for these measures. The low value of association between the two HRS labs is going to make examining change over time difficult. The distribution of the HRS 2006 values and 2008 values from Biosafe are similar to each other and to NHANES. The University of Washington value is shifted to the right. The Biosafe distribution in 2006 for HRS is lower than that for Biosafe in Cystatin C The mean of the serum values is considerably higher than that of the DBS values but the correlation of the serum and DBS values is fairly high (~ ). The correlation between the DBS assays based on the two cards both done at the Univ of Vermont is even higher (.90) and the means are almost identical. The somewhat smaller value for the Heritage DBS card could be related to either the handling of the sample or to the card itself. The correlation of the DBS asays at the Univ of VT and Washington are relatively high. Assays using the same card and same storage are correlated.91 to.96. The University of Washington lab provided two assays for each sample. Mean values of the two Washington DBS are much closer to that of the serum values, particularly the second Washington DBS. 8

9 The serum value from the Univ of Vermont assay does not have the same range as Cystatin C assays in the literature. It has much lower values and conventional cutoffs for high risk cannot be used. The moderately strong association between the DBS and serum values means that we could use either the regression or the Z score approach to get a value very close to the value produced by the Serum Cystatin C assay; however, because this assay value is quite different from those reported in the literature, the value of doing this is not clear. The distribution of Cystatin C values in HRS is almost the same in 2006 and 2008 but it is very different from that in NHANES. 5. C-Reactive Protein The values of the Vermont serum assays differ from the DBS assays of CRP done at Vermont (called the Heritage and Biosafe for the cards used), the McDade assay, and the Washington values. The Vermont Serum assay has a relatively low value, the VT DBS values are even lower, the Washington values are highest.. The associations between the DBS assays done at Vermont and the venous blood assays are generally very high (correlation coefficients ). The UW2 with a higher range is also realtively highly correlated with the serum values (.86). The differences between the DBS and serum values are larger at higher levels. When both values are log transformed, as is often the case in CRP analysis, the differences are similar across the assay range. The McDade DBS assay and the Vermont DBS assay have very different values and ranges. McDade s assay detects much lower values but it has a severely restricted range at the top. However, the correlation between the two sets of DBS assays is high. When cases are imputed for the Vermont assay, the correlation is.89. The Washington 1 assay tends to have lower association with other values. It has 3 values that are coded as outside of the range (too high and are not assigned an actual assay value). For assay 2, these three samples were diluted 1:2 or 1:4 and then re-assayed. The second Washington assay has a value for those three cases, which are very high, all over 20. The Washington values are differentially correlated with the venous blood assay values because of this; correlation with Washington 2 is high (.87) but that with Washington 1 is considerably lower (.56). 9

10 Because the relationships among many of these assays are reasonably high, various approaches to converting them into similar metrics should work reasonably well. The distribution of the CRP values in HRS in 2006 and 2008 is very similar. The values are lower than those in NHANES and similar to those in NSHAP. Sum of Overall Results: The DBS approach appears to do quite well for indicating levels of HbA1c, Cystatin C, and C-Reactive Protein. Assessing lipid levels continues to be a challenge. 10

11 Comparison of Glycosylated Hemoglobin (HbA1c) from DBS and Whole Blood General Information HbA1c was measured using DBS at Heritage, the University of Washington, and. Geonostics (based on FlexSite assay). Whole blood assays were done for 66 subjects at a local AMA commercial lab. Heritage assays were collected using the Heritage card. Heritage was sent 92 cards (Bloodspots collected as numbers 1-6). Values were returned values for 89 cases (3 ID problems). University of Washington assays were collected using the untreated Biosafe card. UW was sent 91 cards. These samples were from drops No data were returned for 14 cards. There was no sample for 5 and either a very small or a smeared spot for 9 cards. So 9/86 had an unusable sample and there were 77 usable samples. Of these, some were classified as very small, smeared, or of poor quality. Washington suggests doing the comparison without the very small (25), smeared (19), poor quality (1). When we ran the mean for the two groups (37 good and 39 bad), there is no significant difference (when we eliminate 1 outlier with a value of 10). UW produces a whole blood equivalent value which we find not to be very useful but we include it in the tables. FlexSite was sent 90 cards which had been frozen for some time while the company was still recovering from the bankruptcy arrangements. These cards did not have pretreated spots. Overview of Results All of the DBS values are reasonably highly correlated with the venous blood values (R 2 of.74 for Heritage,.95 for the University of Washington and.96 for FlexSite). The means of the three DBS assays are fairly similar but the range of the University of Washington assay is less. The University of Washington and Flexite appear to be marginally superior assays with the exception of the Bland Altman results which indicate that the differences between the UWDBS assay for HbA1c and the venous blood assay vary with the level of the assay. The regression estimated equivalent to venous values and the Z score estimated values from the University of Washington and FlexSite result in 11

12 means very close to whole blood. Because the R 2 is so high the transformation is reasonable although we would be cautious in doing this. Using cutoffs is an issue even when the R 2 is high. Transforming the Heritage assay results in a narrowing of the range. We need to note that the Heritage samples were not kept under the same conditions as the University of Washington samples. 12

13 1. Descriptive Information on All HbA1c Assays HbA1c Average Range #Cases Interquartile Range UCLA Whole Venous Blood Heritage DBS UWDBS UWEQ FlexSite Mean values The DBS means from Heritage, Washington and Flexite are significantly (DBSvenous) higher than the venous (Ns Heritage = 64, UW=58, Flexite=66). UW Whole blood equiv difference from venous is even larger. (N=58) UW Whole blood equiv and Heritage DBS are not significantly different (N=75). 2. %High HbA1c (>=6.4) with no adjustment to data % N UCLA whole blood (N=66) Heritage (N=89) U Washington (N=77) U Washington Equivalent (N=77) FlexSite (N=90) High Values The percentage high in the Univ of Washington and the FlexSite assays is similar to that from whole blood. The Heritage assays produce a level much higher; similar to that in the Univ of Washington Whole blood equivalent. 13

14 . 3. Distribution of HBA1C (%) in the Validation Samples Washington Washington UCLA whole Heritage FlexSite Equivalent blood < There are no very low values in the DBS assays done by the University of Washington and Heritage. The range of is these DBS assays is smaller than that of the whole blood assays. The FlexSite range is more similar to the whole blood range. FlexSite has the most very high assays. 14

15 4. Descriptive Information on matched DBS and Venous blood samples for HbA1c HBA1C Average S.D. Range #Cases Correlation Coeff. UCLA whole blood Heritage <.0001 <.0001 UCLA whole blood UWDBS <.0001 <.0001 UCLA whole blood UWEQ <.0001 <.0001 UCLA whole blood FlexSite <.0001 <.0001 Heritage UWDBS < Heritage FlexSite <.0001 <.0001 Flexite UWDBS <.0001 <.0001 The results for the matched samples are similar to those reported above. The DBS values are higher than the whole blood values and the UWEQ is even higher than the DBS. The two DBS mean vales are statistically the same although the range of the University of Washington value is smaller. 15

16 Figure 1. Scatter Plots of the Validation Samples: Whole Blood (UCLA lab) versus DBS Heritage Heritage against UCLA lab UCLA lab: Hemoglobin A1c Heritage lab: Hemoglobin A1c Number of Observations Read 95 Number of Observations Used 64 Number of Observations with Missing Values 31 R-Square Whole Blood (hrsa1c) = * DBS Heritage (hdba1c) 16

17 Washington against UCLA lab UCLA lab: Hemoglobin A1c UW: A1c (%) Number of Observations Read 95 Number of Observations Used 58 Number of Observations with Missing Values 37 R-Square Whole Blood (hrsa1c) = *DBS Univ of Washington (uwa1c) 17

18 Washington equivalent against UCLA lab UCLA lab: Hemoglobin A1c UW: A1c whole blood equivalent (%) Number of Observations Read 95 Number of Observations Used 58 Number of Observations with Missing Values 37 R-Square Whole Blood (hrsa1c) = * Univ of Wash Equivalent (uwa1ceq) 18

19 Flexite against UCLA lab UCLA lab: Hemoglobin A1c HBA1C_FLEXITE Number of Observations Read 95 Number of Observations Used 66 Number of Observations with Missing Values 29 R-Square Whole Blood (hrsa1c) = * Flexite (hba1c_flexite) 19

20 Washington Equivalent against Heritage Heritage lab: Hemoglobin A1c UW: A1c whole blood equivalent (%) Number of Observations Read 95 Number of Observations Used 75 Number of Observations with Missing Values 20 R-Square Heritage DBS (hdba1c) = * Univ of Wash Equiv (uwa1ceq) 20

21 Comparison of Two DBS Values Washington against Heritage Heritage lab: Hemoglobin A1c UW: A1c (%) Number of Observations Read 95 Number of Observations Used 75 Number of Observations with Missing Values 20 R-Square Heritage DBS (hdba1c) = * Univ Wash DBS (uwa1c) 21

22 Flexite against Heritage Heritage lab: Hemoglobin A1c HBA1C_FLEXITE Number of Observations Read 95 Number of Observations Used 87 Number of Observations with Missing Values 8 R-Square Heritage DBS (hdba1c) = * FlexSite (hba1c_flexite) 22

23 Flexite against Washington UW: A1c (%) HBA1C_FLEXITE Number of Observations Read 95 Number of Observations Used 76 Number of Observations with Missing Values 19 R-Square Washington DBS (uwa1c) = * FlexSite (hba1c_flexsite) 23

24 Figure 2. HbA1c Bland Altman Plots Difference between Heritage and Whole Blood Average= (Heritage + Whole Blood)/2 Difference = Heritage - Whole blood (hdba1c - hrsa1c); Difference 0.7 HBA1C Average Mean 95% CI Average Heritage Whole Blood # of out of range (2SD): 2 The range is wide; few outside the range; the difference appears similar across the range. 24

25 Difference between U Washington DBS and Whole Blood Average= (UWDBS + Whole Blood)/2 Difference = UWDBS - Whole blood (uwa1c-hrsa1c); Difference HBA1C Average Mean 95% CI Average UWash Whole Blood # of out of range (2SD): 3 The range is larger than for Heritage; the number outside the range is similar; however, the difference changes with the level. At low values the difference is positive; at high values it is negative. 25

26 Difference between FlexSite and Whole Blood Average= (FlexSite + Whole Blood)/2 Difference = FlexSite - Whole blood (hba1c_flexsite - hrsa1c); HBA1C Difference Average Mean 95% CI Average FlexSite Whole Blood # of out of range (2SD): 4 26

27 Difference between Heritage DBS and U Washington DBS Average= (Heritage + UWDBS)/2 Difference = Heritage - UWDBS (hdba1c uwa1c); HBA1C Difference Average Mean 95% CI Average Heritage UWash ~ 0.08 # of out of range (2SD): 2 The University of Washington value is greater at lower average levels. 27

28 Difference between Heritage and FlexSite Average= (Heritage + FlexSite)/2 Difference = Heritage - FlexSite (hdba1c - hba1c_flexsite); HBA1C Difference Average Mean 95% CI Average Heritage Flexite # of out of range (2SD): 3 28

29 Difference between U Washington DBS and FlexSite Average= (UWDBS + FlexSite)/2 Difference = UWDBS - FlexSite (uwa1c - hba1c_flexsite); Difference HBA1C Average Mean 95% CI Average UWDBS FlexSite # of out of range (2SD): 4 At low values the difference is positive; at high values it is negative. 29

30 Figure 3. Differences between DBS and Venous Blood by level of venous blood 30

31 We compare a number of ways to use the DBS values as alternatives to the assayed value and compare these to values from whole blood. Quartile of HbA1c: Heritage - Whole blood (N=64) Heritage whole 1 st qt 2 nd qt 3 rd qt 4 th qt 1 st qt (N=15) 6.25 (N=4) nd qt 9.38 (N=6) 4.69 (N=3) rd qt (N=8) 7.81 (N=5) th qt (N=13) Cutpoints for Quartiles 1st 2nd 3rd Heritage ~5.7 ~5.9 ~6.2 Whole Blood ~5.5 ~5.75 ~6.1 About 56% are in the same quartile of the Heritage and whole blood values. High and low are closest. Quartile of HbA1c: Univ Washington DBS-Whole blood (N=58) Wash whole 1 st qt 2 nd qt 3 rd qt 4 th qt 1 st qt (N=16) 5.17 (N=3) nd qt 5.17 (N=3) (N=11) 6.90 (N=4) rd qt (N=3) 6.90 (N=4) th qt (N=10) Cutpoints for Quartiles 1 st 2 nd 3 rd U Washington ~5.8 ~6.0 ~6.2 Whole Blood ~5.5 ~5.8 ~6.2 About 70% of the DBD and whole blood assays are in the same quartile. The University of Washington does better than Heritage in matching quartiles 31

32 Quartile of HbA1c: FlexSite - Whole blood (N=66) Flexite whole 1 st qt 2 nd qt 3 rd qt 4 th qt 1 st qt (N=16) nd qt 7.58 (N=5) (N=7) 6.06 (N=4) 1.52 (N=1) 3 rd qt 1.52 (N=1) (N=10) th qt (N=3) (N=13) Cutpoints for Quartiles 1 st 2 nd 3 rd FlexSite ~5.5 ~5.8 ~6.1 Whole Blood ~5.5 ~5.8 ~6.2 About 70% of the whole blood assays are in the same quartile, similar to the University of Washington. 32

33 Equations relating whole blood HbA1c with DBS and Three DBS values to each other: HbA1c EQ y a b x R 2 N Whole blood UWDBS Whole blood Heritage DBS Whole blood UWEQ Whole blood FlexSite Heritage DBS UW DBS Heritage DBS UWEQ Heritage DBS FlexSite FlexSite UWDBS UWDBS Flexite The University of Washington and FlexSite DBS values match venous blood values better than Heritage values; however the FlexSite and UWDBS (R 2 =.95) and the Heritage and FlexSite DBS values are more highly correlated (R 2 =.87) than the Heritage and UWDBS (R 2 =.71). Conversions of DBS based on regression equations It is possible to use the regression equation to estimate whole blood values based on the DBS value. When this is done the estimated means, SD, and range are closer to those of the whole blood. The percent with high levels is somewhat lower. Mean SD Range %High Risk (>=6.4) Whole Blood Heritage Est of Serum Equiv Mean SD Range %High Risk (>=6.4) Whole Blood UW Est of Serum Equiv Mean SD Range %High Risk (>=6.4) 33

34 Whole Blood FlexSite Est of Serum Equiv Estimating values with the regression equation results in relatively close means for DBS values. The range of values and the S.D. of values for the U of Washington and FlexSite look similar to those in the whole blood. The percentage high is slightly lower. Using the regression on the Heritage values, reduces their range to only 2.2 points; the SD is about half of that in the other distributions; and results in a relatively low estimate of the percent high. 34

35 An alternative approach is to convert DBS values to whole blood equivalent values based on Z scores. We compute z scores in the two distributions and match equivalent Z scores and then transform DBS values into whole blood values. HBA1C Average S.D. Range #Cases %high risk (>=6.4) UCLA whole blood %(N=9) Heritage Converted to UCLA %(N=8) UCLA whole blood %(N=9) Biosafe(Washington) Converted to UCLA %(N=8) UCLA whole blood FlexSite Converted to UCLA %(N=10) 13.6%(N=9) The percentage with high levels of glycosylated hemoglobin (>=6.4%) (N=64) Whole blood 14.1% Heritage DBS assay original 21.9% Heritage DBS used in regression est 10.9% Heritage DBS assay converted using Z scores 12.5% (N=58) Whole blood 15.5% UW DBS assay original 17.2% UW DBS used in regression est 13.8% UW DBS assay converted using Z scores 13.8% (N=66) Whole blood 15.2% FlexSite assay original 13.6% FlexiSte used in regression est 13.6% FlexSite assay converted using Z scores 13.6% 35

36 Figure 4. HbA1C in HRS Samples and NHANES and NSHAP HRS Sample (2006, 2008 two labs), NHANES, NSHAP (Weighted) The distribution of values is generally similar for HRS 2006 done by Biosafe and HRS 2008 by Biosafe and both of these are quite similar to NHANES venous values. There is a shift in the modal value in the FlexSite assays which is similar to the value in NSHAP. 36

37 Comparison of Total Cholesterol from Dried Blood Spots and Venous Blood 1. Dried blood spots were assayed at 1) Heritage labs from DBS on heritage cards (N=87). The mean was 177 (range is ). 2) University of Washington assayed DBS based on Biosafe cards (N= 91). Spot quality essentially the same as for HbA1c: 1 spot is said to be out range; 50 not good quality (mean is 281); 41 with good quality (mean is 261). 3) University of Washington provides a Plasma equivalent (mean 161 range ). Venous blood Serum assayed at the University of Vermont lab (N=94) - range is 110 to 285 mean is 191 Overview of Results Differences between assays in level are significant. The University of Washington value is very high relative to venous and Heritage. Heritage value is closer to venous but lower than venous. The University of Washington distribution of DBS assays shows that there were a significant number of very high values. Using traditional cutpoints will not work with the University of Washington values. While the mean level of the University of Washington differs more from the venous level than the Heritage mean difference, the University of Washington DBS is more closely related to venous blood values than Heritage DBS (R 2.55 versus.30). The relationship between the University of Washington and the Heritage values is very low (R 2.23). Differences between both DBS assays and the venous assays differ by the level of the assays. Using a regression approach with Cholesterol to produce Serum equivalents does not work because of the low associations. Z scores are more promising. 37

38 1. Descriptive Information on All Total Cholesterol Assays Cholesteroltotal Average Range #Cases Interquartile VT Serum DBS Heritage DBS UW DBS UWPlaEQ %High Total Cholesterol (>=240) no adjustments % N Vermont serum (N=94) Heritage (N=87) Washington (N=91) Washington Equivalent (N=91) The mean values for UWDBS are almost 100 points higher than the values for Heritage DBS. The Univ of Washington Plasma Equivalent value is the lowest of all. Using conventional cutoff levels 2/3 of the sample is high according to the Univ of Washington DBS assay and almost no one is using the Heritage assay or the plasma equivalent provided by the UW. 38

39 3. Distribution of Total Cholesterol (%) Washington DBS Washington Equivalent Heritage DBS Vermont Serum The lowest values in the Univ of Washington DBS distribution are in the 160s which is close to the midpoint of the Heritage or the University of Vermont distribution. 39

40 4. Descriptives on matching samples for Total Cholesterol Descriptives Cholesterol Average S.D. Range #Cases Correlation Coeff. VT Serum DBS Heritage <.0001 <.0001 VT Serum DBS UW <.0001 <.0001 VT Serum DBS UWPlaEQ <.0001 <.0001 The differences are the same as those described above. 40

41 Figure 1. Scatter Plots of the Validation Samples for Total cholesterol from Serum and DBS at Heritage Vermont: Cholesterol (mg/dl)-serum 300 Heritage against Vermont Heritage lab: Cholesterol Number of Observations Read 95 Number of Observations Used 86 Number of Observations with Missing Values 9 R-Square Serum (uvchol) = * Heritage DBS (hdbchol) 41

42 Washington against Vermont Vermont: Cholesterol (mg/dl)-serum UW: Cholesterol (mg/dl) Number of Observations Read 95 Number of Observations Used 90 Number of Observations with Missing Values 5 R-Square Serum (uvchol) = * Univ of Washington DBS (uwchol) 42

43 Comparison of two DBS Washington Equivalent against Heritage Heritage lab: Cholesterol UW: Chol plasma equivalent conc (mg/dl) Number of Observations Read 95 Number of Observations Used 86 Number of Observations with Missing Values 9 R-Square Heritage DBS (hdbchol) = * Univ of Wash EQ (uwcholeq) 43

44 Washington Equivalent against Vermont Vermont: Cholesterol (mg/dl)-serum UW: Chol plasma equivalent conc (mg/dl) Number of Observations Read 95 Number of Observations Used 90 Number of Observations with Missing Values 5 R-Square Serum (uvchol) = * Univ of Washington EQ (uwcholeq) 44

45 Washington against Heritage Heritage lab: Cholesterol UW: Cholesterol (mg/dl) Number of Observations Read 95 Number of Observations Used 86 Number of Observations with Missing Values 9 R-Square Heritage DBS (hdbchol) = * Univ of Washington DBS(uwchol) 45

46 Figure 2. Total Cholesterol Bland Altman Plots Difference between Heritage DBS cholesterol and U Vermont Serum Average = (HeritageDBS + UVcholserum)/2 Difference = (HeritageDBS uvcholserum) Difference Total Cholesterol Average Mean 95% CI Average Heritage Vermont Serum ~ # of out of range (2SD): 4 The differences are larger at higher values meaning that at higher values the assay is less precise. At higher values, the Heritage assay underestimates are greater. 46

47 Difference between UWashington DBS cholesterol and U Vermont Serum Average = (UWcholDBS + UVsholserum)/2 Difference = uwcholdbs uvcholserum Difference Total Cholesterol Average Mean 95% CI Average UWash Vermont Serum # of out of range (2SD): 5 If the value of the assay is low, the difference is smaller; if the value if large, the difference is larger. 47

48 Figure 3. Differences between DBS and Venous Blood by Level of Cholesterol in Vermont assay UV-Heritage UV-Washington UV-WashingtonEq Differences are all positive for Univ of Washington and mostly negative for Heritage. The larger differences at higher levels are shown here also. 48

49 Quartile Values for Heritage DBS and UVT Serum: Total Cholesterol (N=86) Heritage uv 1 st qt 2 nd qt 3 rd qt 4 th qt 1 st qt (N=12) 5.81 (N=5) 5.81 (N=5) 1.16 (N=1) 2 nd qt 6.98 (N=6) 6.98 (N=6) 8.14 (N=7) rd qt 1.16 (N=1) 6.98 (N=6) 5.81 (N=5) (N=9) 4 th qt 3.49 (N=3) 4.65 (N=4) 5.81 (N=5) (N=9) Cutpoints for Quartiles 1st 2nd 3rd Heritage ~155 ~171.5 ~193 UV ~168 ~191 ~215 Just over a third (37%) are in the same quartile. Quartile Values for UWcholesterolDBS and UVTSerum (N=90) Uw uv 1 st qt 2 nd qt 3 rd qt 4 th qt 1 st qt (N=15) 6.67 (N=6) (N=1) 2 nd qt 5.56 (N=5) 7.78 (N=7) 6.67 (N=6) 3.33 (N=3) 3 rd qt 3.33 (N=3) 7.78 (N=7) 8.89 (N=8) 6.67 (N=6) 4 th qt (N=7) (N=12) Cutpoints for Quartiles 1st 2nd 3rd UW ~ ~ ~ UV ~167 ~189.5 ~215 Almost a half are in the same quartile (47%). 49

50 Equations Relating Whole blood and DBS values and other assays: Total Cholesterol. Chol tot y a b x R 2 N VT Serum Heritage DBS VT Serum UW chol DBS VT Serum UW chol eq Heritage DBS UW choldbs Heritage DBS UW chol eq The R 2 indicates that the University of Washington values are more highly correlated with the venous blood values (R 2.55 than the Heritage values (R 2.30) The Two DBS values are not highly related (R 2.23) 50

51 Estimated DBS values based on regression equations and Z scores When the regression equation is used to estimate serum blood values from the DBS values, the means for the estimated DBS values are the same as those for the venous values but the SD is quite a bit lower especially for the Heritage values. Using the estimated values the percent high is much lower in the regression estimated values. Mean SD Range Percent high Serum Blood Estimated Serum UWDBS Mean SD Range Percent high Serum Blood Estimated Serum - Heritage DBS We can also make the conversion by changing the DBS values to Z scores and then convert the Z score into a whole blood value using the mean and SD of the whole blood distribution. When this is done, the means and standard deviations are similar. The estimated % with high levels is higher than in the whole blood distribution but it has been substantially reduced from the original untransformed values. Total Cholesterol Average S.D. Range #Cases %high risk (>=240) Whole blood % Washington Converted to Whole blood % Whole blood % Heritage Converted to Whole blood % 51

52 The percentage with high levels of Total Cholesterol (N=90) Whole blood 7.78% DBS assay UW original 75.56% DBS used in regression est 2.22% DBS assay converted using Z scores 8.89% (N=86) Whole blood 8.14% Heritage DBS assay original 4.56% Heritage DBS used in regression est 1.16% Heritage DBS assay converted using Z scores 11.63% Because the association of the DBS assays and venous blood assays are relatively weak, it seems inappropriate to transform the equations using the regression approach. Making the transformation results in a reduced range and relatively small SD for the values. Transforming the values using Z scores seems more appropriate where the associations are low. 52

53 Figure 4. Total Cholesterol in HRS Samples and NHANES HRS Sample (2006, 2008 assayed in 2 sources Biosafe and Univ. of Washington (called HRS 2008 DBS)), NHANES (Weighted) The distribution for HRS assays from 2006 and 2008 both done at Biosafe are quite different. The University of Washington asay is closer to the 2008 Biosafe assay but there are a high number of very high values as in the validation sample. 53

54 Comparison of HDL Cholesterol from Dried Blood Spots and Venous Blood Dried blood spots were assayed at 1) Heritage was assayed using heritage cards (N=90). The mean was 51.9 (range is 30-94) 2) UW assays were done from Biosafe cards (N= 91). The mean was 75.6 (range ). 3) UW provides a Plasma equivalent with a mean of Venous blood was assayed at the University of Vermont lab - serum (N=94) - range is mean is 53.3 Overview of Results The HDL values are similar to the Total cholesterol values in that the University of Washington assay produces values much higher than the others. On the other hand, the association between the University of Washington DBS values and the venous blood values is stronger (.48) than that of Heritage DBS (.32). The association of the two DBS assay values is low (.20). Z scores may be a better method of transformation than regression estimates of venous equivalents. 54

55 1. Descriptive Information on all HDL Cholesterol and Venous Blood Assays Cholesterol- HDL Average Range #Cases Interquartile VT Serum Heritage DBS UW DBS UW Plas Eq % Adverse levels of HDL Cholesterol (<40) with no adjustments % N Vermont serum (N=94) Heritage (N=90) 2 18 Washington (N=91) 0 Washington Equivalent (N=91) As for total cholesterol, the values of HDL from University of Washington DBS are much higher than the venous blood or the Heritage values. The average of the Heritage values is close to the venous mean. The University of Washington Plasma Equivalent mean value is the lowest. Using cutoffs with these values shows that this is problematic. None of the University of Washington values are below the normal cutoff. 55

56 3. Distribution of HDL Cholesterol (%) Washington Washington Equivalent Heritage Vermont The University of Washington has a substantial number of high values. 56

57 4. Descriptives Matched samples for HDL Cholesterol HDL Cholesterol Average S.D. Range #Cases Correlation Coeff. VT Serum DBS Heritage < VT Serum DBS UW <.0001 <.0001 VT Serum DBS UWPlaEQ <.0001 <.0001 DBS Heritage DBS UW <.0001 The differences are similar to those described above. 57

58 Figure 1. Scatter Plots of the Validation Samples Heritage against Vermont Vermont: HDL (mg/dl)-serum Heritage lab: HDL Number of Observations Read 95 Number of Observations Used 89 Number of Observations with Missing Values 6 R-Square Serum (uvhdl) = * Heritage DBS (hdbhdl) 58

59 Vermont: HDL (mg/dl)-serum 140 Washington against Vermont UW: HDL (mg/dl) Number of Observations Read 95 Number of Observations Used 90 Number of Observations with Missing Values 5 R-Square Serum (uvhdl) = * Univ of Washington DBS (uwhdl) 59

60 Washington Equivalent against Vermont Vermont: HDL (mg/dl)-serum UW: HDL plasma equivalent conc (mg/dl) Number of Observations Read 95 Number of Observations Used 90 Number of Observations with Missing Values 5 R-Square Seru (uvhdl) = *Univ of Washington DBS (uwhdleq) 60

61 Heritage lab: HDL 140 Washington Equivalent against Heritage UW: HDL plasma equivalent conc (mg/dl) Number of Observations Read 95 Number of Observations Used 89 Number of Observations with Missing Values 6 R-Square Heritage DBS (hdbhdl) = * Univ of Washington DBS (uwhdleq) 61

62 Washington against Heritage Heritage lab: HDL UW: HDL (mg/dl) Number of Observations Read 95 Number of Observations Used 89 Number of Observations with Missing Values 6 R-Square Heritage DBS (hdbhdl) = * Univ of Washington DBS (uwhdl) 62

63 Figure 2. HDL Bland Altman Plots Total Cholesterol Difference between Heritage DBS HDL and U Vermont Serum Average = (HeritageDBS + UVhdlserum)/2 Difference = HeritageDBS - UVhdlserum HDL Difference Average Mean 95% CI Average Heritage Vermont Serum # of out of range (2SD): 5 Somewhat larger spread at higher levels. 63

64 Difference between UWashington DBS HDL and U Vermont Serum Average = (UWhdlDBS + UVhdlserum)/2 Difference = UWhdlDBS - UVhdlserum Mean 95% CI Average UWash Vermont Serum # of out of range (2SD): 5 Somewhat less precision at higher levels. 64

65 Figure 3. Differences Between DBS value and Vermont Serum by level of Vermont Serum HDL Difference UV-Heritage UV-Washington UV-WashingtonEq HDL Diff Vermont (UV) 65

66 Quartiles for UWhdlDBS and UVhdlserum (N=89) Heritage uv 1 st qt 2 nd qt 3 rd qt 4 th qt 1 st qt 8.99 (N=8) (N=11) 5.62 (N=5) nd qt 7.87 (N=7) (N=9) 3.37 (N=3) 4.49 (N=4) 3 rd qt 5.62 (N=5) (N=9) 4.49 (N=4) 4 th qt 3.37 (N=3) 3.37 (N=3) (N=14) Cutpoints for Quartiles 1 st 2 nd 3 rd Heritage ~41.00 ~5 ~61.00 UV ~41.00 ~51.00 ~ % are in the same quartile. Quartiles for UWhdlDBS and UVhdlserum (N=90) Uw uv 1 st qt 2 nd qt 3 rd qt 4 th qt 1 st qt (N=15) 7.78 (N=7) nd qt 6.67 (N=6) (N=11) 5.56 (N=5) 1.11 (N=1) 3 rd qt 1.11 (N=1) 3.33 (N=3) (N=10) 7.78 (N=7) 4 th qt 1.11 (N=1) 1.11 (N=1) 6.67 (N=6) (N=14) Cutpoints for Quartiles 1 st 2 nd 3 rd UW ~65.02 ~76.44 ~86.27 UV ~41.00 ~51.00 ~ % are in the same quartile. 66

67 Equations Linking Venous Blood and DBS Assays and other Assays HDL Equa y a b x R 2 N VT Serum Heritage VT Serum UW HDL VT Serum UW HDL eq Heritage DBS UW HDL Heritage DBS UW HDL eq The association between the University of Washington DBS values and the venous blood values is stronger (.48) than that of Heritage DBS (.32). The association of the two DBS assay values is low (.20). 67

68 Estimated DBS values bassed on regression equations and Z scores When DBS values are used with the regression equations to estimate venous equivalents, the means for both the University of Washington and the Heritage samples are the same as that for the venous sample but the SDs are smaller and the range is narrower. The percent with adverse levels is twice as high in the whole blood sample as the University of Washington value based on DBS. The Heritage percent high is very low relative to that from comparable venous assays. Mean SD Range Percent Adverse levels Serum Blood % Estimated Serum - Heritage DBS % Mean SD Range Percent high Serum Blood % Estimated Serum UWDBS % We can also make the conversion by changing the DBS values to Z scores and then convert the Z score into a whole blood value using the mean and SD of the distribution. When this is done, the means and SDs for the DBS and the venous blood are essentially the same. The estimated % with adverse levels is higher in the estimated values from the University of Washington DBS and the Heritage DBS than in venous blood. HDL Cholesterol Average S.D. #Cases Range %adverse levels (<40) Whole blood % Washington Converted to Whole blood % Whole blood % Heritage Converted to Whole blood % 68