Author's response to reviews Title: A novel differential diagnostic model based on multiple biological parameters for immunoglobulin A nephropathy Authors: Nan Zhen Dong (dongzn@301hospital.com.cn) Yong Wang (wangyong301@263.net) Jing Gao (gaojingwang@yahoo.cn) Wang Xing Jia (jiaxingw301@yahoo.com.cn) Ping Ya Tian (tianyp61@gmail.com) Version: 2 Date: 14 March 2012 Author's response to reviews: see over
Dr Adiran Aldcroft Executive Editor BMC Medical Informatics and Decision Making 12 March 2012 Dear Dr Aldcroft, Thank you for your suggestions and the reviewers comments concerning our manuscript entitled A novel differential diagnostic model based on multiple biological parameters for immunoglobulin A nephropathy (MS: 1522420505623066). Your suggestions were very valuable and helpful in further improving our manuscript. We have added a statement in the Methods section regarding ethical approval. Furthermore, informed consent was obtained throughout the study, and this was indicated in the Acknowledgments section. Our manuscript was also revised by Edanz for grammar and appropriate English usage. We hope that our revised version meets your requirements. The point-by-point responses to the reviewers concerns are provided below. We hope that our responses to the reviewers comments, as well as the improvements made in the grammar and use of English, have met the requirements necessary for publication in your journal. Please do not hesitate to contact me if you may have any problems or questions regarding our revised manuscript. Best regards, Yaping Tian On behalf of all co-authors: Jing Gao, Yong Wang, Zhennan Dong, and Xingwang Jia.
Responses to Francois Berthoux: Comment: Background: the picture of renal biopsy complications is too dark and does not correspond to the reality!! Response: We thank the reviewer for their helpful suggestion. Accordingly, we have deleted the sentence regarding renal biopsy complications (Please refer to Page 3). Material and methods: The method with two sets of patients is correct: first establishing the model on 58 IgAN (biopsy-proven) and 63 non-igan. We need guaranty that the initial histopathological diagnosis is correct: we need to have the results of immunofluorescence with quantitation of IgA mesangial deposits in all 121 patients; we need to have the exact histopathological diagnosis for the non-igan and the score of the Oxford classification for all IgAN (M 0/1; E 0/1; S 0/1; and T 0/1/2). The supplement 3 with the 57 parameters should be included as Table. Response: As per the reviewer s suggestion, we have provided the immunofluorescence data of all 121 patients, the exact histopathological diagnosis of non-igan, and the scores of the Oxford classification for IgAN in Supplemental Table 1 (Please refer to Page 4 and Supplemental Table 1). Results: You should produce for each parameter (57) the data (mean+/-sd and median with extremes) for the 2 groups and comparisons by T and U tests. After you may keep only those significant (P<0.05) or better highly significant (P<0.001) due to the big number of variables. For each parameter, you may produce in a Table the C statistics for the ROC curves:
AUC with 95% CI and P value. After you may select/keep only those highly significant (P<0.001). For logistic regression, you should present univariate analyses with all the preselected variables. After, all significant or highly significant variables should be included in a multivariate analysis for prediction of IgAN. The really significant and independent variables should be used for the model. Response: We thank the reviewer for their very helpful advice on the appropriate statistical analyses that need to be conducted with respect to our data. According to the reviewer s suggestion, we have added a supplemental table, which includes the mean ± SD, median with extremes, and P-values obtained with the t- and U-tests, of the 57 biological parameters studied (Please refer to Supplemental Table 2). We selected the 16 significant parameters (P<0.05), listed in Table 3 of the manuscript, for further analysis (Please refer to Page 6). We also made a supplemental table containing the area under curve with 95% CI and P-values of each parameter obtained from the ROC curve analysis (Supplemental Table 3). It was found that 17 parameters were significant (Table 4). Of these, six parameters were highly significant (P<0.001). As there were only 6 highly significant parameters, we used all of the 17 significant parameters (P<0.05) for further analysis (Page 6). The findings obtained from the univariate logistic regression analysis ( Enter method) on the 13 pre-selected parameters are presented in Table 5. Parameters with a P<0.2 were chosen to prevent the exclusion of important variables. With the exception of UN, the other 12 variables had a P<0.2 and were all substituted into the multivariate logistic regression, using the forward conditional method of entry. Similar findings were obtained with the multivariate logistic regression. Manifestation, FIB and siga were found to be significant parameters for predicting IgAN and non-igan. Please refer to Page 7, Table 5, and Table 6 in the revised manuscript. For the model, it should be defined what is acceptable as accuracy parameters for a clinician: here specificity is more important than sensitivity; in my view specificity
should not be below 0.90 (1false diagnosis out of 10) and ideally 0.95+. The validation set presented is an internal validation on retrospective cohort; an external independent validation will be needed in the future. Response: We agree with the reviewer that specificity is more important than sensitivity in diagnostic testing and should be pretty high. Usually, when we evaluate the diagnostic level of a test, method, or reagent, we always evaluate normal cases or samples to serve as the control group. Consequently, we are able to acquire pretty high specificity and sensitivity levels. However, this study is comparing and distinguishing two subtypes of kidney disease, which possess a number of similar characteristics. In this case, it would be hard to obtain high specificity. However, we will continue paying close attention to those discerning indicators, and will conduct an external independent validation in the future. Figures: ROC curves without C statistics are useless. Figure 2 and Figure 6 are difficult to understand. Figure 5: no legend for blue color! Response: We thank the reviewer for their comment, and have clarified or made the following changes: - Figure 1 presents the ROC curves of the top three AUCs of the 57 parameters studied. The C statistics of the ROC curves are presented in Supplement Table 2. - The P-values and 95% CI of Figure 3 and Figure 4were added in Pages 7 and 8. Figure 2 presents the correlation coefficients of the two variables. - Figure 6 demonstrates the decision procedure that should be followed when using this equation. When a patient is suspected to have chronic kidney diseases, the predicted probability should be calculated using the variables obtained from blood tests and the equation. When the calculated predicted probability is >0.59, the patient has at least an 85.0% probability of having IgAN. However, when the predicted probability is <
0.26, the patient has at least an 88.5% probability of having non-igan. When the predicted probability is between 0.26 and 0.59, it is hard to diagnose the patient, as the misdiagnosis ratio is high and further examination, including renal biopsy, are warranted. - Figure 5 was adjusted accordingly. Tables: Table 1: all definitions should be given for each item: hypertension, nephritic Sd, nephroticsd, Proteinuria, Haematuria, normal renal function; etc; is Gross/Macroscopic hematuria included? Table 3: what is constant? Table 4: there is not a unique definition of accuracy; you should produce all accuracy parameters: Se, Sp, NPV, PPV, etc Response: As suggested by the reviewer, the definitions of the items used in Table 2 (previously Table 1) were provided in the footnotes of the table. They are as follows: Hypertension was defined as systolic BP 140 mmhg, diastolic BP 90 mmhg, or use of antihypertensive medications. Chronic nephritis syndrome was defined as proteinuria or hematuria with hypertension or edema. Nephrotic syndrome was defined as persistent proteinuria of more than 3.5 g/1.73 m2/24 h, hypoalbuminemia or albumin levels 30 g/l, edema, and varying degrees of hyperlipidemia. Isolated proteinuria or hematuria was defined as a urine protein excretion >0.3 g/1.73 m2/24 h or urine red blood cell (RBC) >3/HP with normal renal function and without hypertension and edema. Normal renal function was defined as estimated glomerular filtration rate (GFR) >90 ml/min/1.73 m2 on at least two occasions. Chronic renal insufficiency was defined as an estimated GFR <60 ml/min/1.73 m2 on at least two occasions with chronic kidney disease. Acute renal insufficiency was defined as an abrupt (within 48 h) reduction in kidney function, according to the Acute Kidney Injury Network (AKIN) criteria. Furthermore, Constant in Table 6 (previously Table 3) refers to the intercept. In the Options setting of the logistic regression, the bottom of dialog box has an option to include
constant in model (Please refer to the picture below). We choose this option, and thus, the constant was included in the following equation: PRE-1 =1-1/ [1+e (-0.648-0.326FIB+0.011sIgA-1.089Manifestation) ]. Thus, the constant in Table 6 is -0.648. Finally, the previous Table 4 was deleted. The evaluation of the model was described in the revised manuscript, and included accuracy, sensitivity, specificity, PPV, NPV, +LR, -LR, α, β, and Youden s Index (Please refer to Page 7). References: Ref 3 is incomplete. Ref to CT/NMR/PET-Scan, and CKD-EPI are useless with comments in background. Response: We thank the reviewer for their comment. Accordingly, Ref 3 was revised. Additionally, the comments and reference to CT/NMR/PET-Scan and CKD-EPI were deleted from the Background (Page 3).
Response to Yasunori Utsunomiya: Comment: 1. The definition of non-iga nephropathy group is unclear. Does this group include focal segmental glomerulosclerosis (FSGS), or postinfectious glomerulonephritis, etc? The kidney diseases recruited in non-igan group should be delineated in more detail. Response: We thank the reviewer for their valuable suggestion. Indeed, the non-iga nephropathy group in the present study included FSGS, membranous nephropathy, minor change disease, and other chronic primary glomerulonephritis. However, acute glomerulonephritis secondary to infectious was not included. According to your suggestions, we added a supplemental table (Supplement Table 1) with the histopathological diagnosis of all 121 patients used for modeling (Please also refer to Page 4). 2. The biochemical indicators in this study may be mainly affected by the difference of clinical manifestation between two groups, especially different incidence of nephrotic syndrome and chronic renal function. Therefore, to explore these effects, biochemical parameters should be evaluated in comparable clinical manifestations. The specificity of the biochemical indicators for IgA nephropathy should be clarified. Response: We thank the reviewer for their valuable suggestion. We agree with the reviewer that the biochemical indicators in this study may be primarily affected by the differences in the clinical manifestation. To our knowledge, currently there are almost no specific biochemical parameters to differentiate between the clinical manifestations of nephrotic syndrome and chronic renal function. Thus, we attempted to analyze the profiles of all related blood tests and explore the relationships between the different biochemical tests for nephrotic syndrome and chronic renal function. The specificities of the combination (i.e. manifestation, siga, and FIB) were 79.4%, with the equation derived from logistic regression analysis, and 74.6%, with the equation derived from the discriminant analysis (Please refer to Pages 7 and 8).
3. Do the biochemical indicators show statistical correlation with levels of renal function or urinary protein excretion in patients with IgA nephropathy? Response: We thank the reviewer for the helpful suggestion. Using the equation of CKD-EPI [1], we calculated the estimated GFR, an indicator of renal function, and further analyzed the correlations between 57 biochemical indicators and renal function in patients with IgA nephropathy. It was found that in patients with IgA nephropathy, renal function correlated with certain biochemical indicators (P<0.05), specifically age, hypertension (HP), β2-microglobulin (B2MG), complement 3 (C3), red blood cell(rbc), hemoglobin (HB), serum creatinine (scr), serum urea (sun), triglyceride (TG), alkaline phosphatase (ALP), magnesium (Mg), carbon dioxide (CO 2 ), and cytokeratin fragment 21-1 (CYFRA21-1) (Table 1). Additionally, age, C3, scr, sun, and ALP had relatively higher correlation coefficients with estimated GFR than all the other parameters. In the present study, we primarily focused on serum biochemical indicators, and did not initially include urinary protein excretion. With your suggestion in mind, we reassessed the cases, and conducted analyses to determine the correlations between urinary protein excretion and the 57 biochemical indicators in patients with IgA nephropathy. We found that 18 out of the 57 variables correlated with urinary protein levels. Of these, B2MG, scr, ALB, and SCC had relatively high correlation coefficients (Table 2). Reference: [1] Levey AS, Stevens LA, Schmid CH, Zhang YL, Castro AF 3 rd, Feldman HI, Kusek JW, Eqqers P, Van Lente F, Greene T, Coresh J: A new equation to estimate glomerular filtration rate. Ann Intern Med 2009, 150(9): 604-612.