Use of neural networks to diagnose acute myocardial infarction. II. A clinical application

Transcription

1 Clinical Chemistiy 42: (1996) Use of neural networks to diagnose acute myocardial infarction. II. A clinical application SUSANNE M. PEDERSEN,l* JORGEN S. J#{216}RGENSEN,2 and J. BOIDEN PEDERSEN2 We investigated the ability of neural networks to diagnose acute myocardial infarction (AM!) from laboratory data only. Several networks were trained with different combinations of data obtained at admission and within the first 12 h and 24 h after admission. The data used included the electrocardiogram (ECG) and the concentrations in serum of potassium, creatine kinase B-subunit (CKB), and lactate dehydrogenase isoenzyme 1 for 250 patients with suspected AM!. Based on admission data, the correct diagnosis was predicted for 76% of the patients in the test group from the ECG data only, and the best combination of ECG results with other variables yielded correct diagnoses for 85% of the test group. Using all of the data available within 24 h, the network predicted the correct diagnosis for 99% of the test data. Almost the same high predictability was obtained by using only two CKB values-recorded at admission and within 12 h after admission-or by using just the latter one. Neural networks and quadratic discriminant analysis performed similarly, but the neural networks were more robust for combinations with many laboratory data. INDEXING TERMS: diagnosis, computer-assisted #{149} discriminant analysis Thrombolytic treatment within 24 h after onset of symptoms in patients with acute myocardial infarction (AMI) reduces mortality and preserves the myocardium [1-3]. Because the beneficial effect is greater the earlier the treatment is given, early diagnosis is quite important. Two previous applications of neural networks to diagnosis of AMI have been published. One was based on analysis of paired sets of cardiac enzymes with the time interval between serial Klinisk Kemisk Afdeling, Svendborg Sygehus, DK-5700 Svendborg, Denmark. 2 Fysisk Institut, Odense Universitet, DK-5230 Odense M, Denmark. *Author for correspondence. Fax tnt Nonstandard abbreviations: AMI, acute myocardial infarction; BBS, bundle branch block; CK, creatine kinase; DFA, discriminant function analysis; ECG, electrocardiogram; LD, lactate dehydrogenase; LDFA, linear discriminant function analysis; and QDFA, quadratic discriminant function analysis. Received July 5, 1995; accepted January 31, determinations as long as 48 h [4]. The other involved a combination of clinical data and an electrocardiogram (EGG) taken at admission [5]. In the present study, we investigated whether a reliable early diagnosis of AMI could be achieved by applying neural networks to laboratory data available at admission, within 12 h, and within 24 h after admission. The data set in the analysis included four markers of myocardial ischemic injury: EGG, serum potassium (K), creatine kinase (EG ) B-subunit (CKB), and lactate dehydrogenase isoenzyme 1 (EC ; LDI). We also used the neural network to determine the predictive values of several combinations of laboratory data and to find which combination(s) yielded the greatest discrimination between AM! and non-am!. The neural networks used in the present study were designed and optimally trained as described in the previous paper [6]. The detailed discussion is based on results from neural networks trained directly on the laboratory data. We found previously [6] that, for a specific combination of laboratory data (EGG and K), a slightly higher performance was obtained by training on a reduced set of the principal components. Here we examined whether this would also be the case for other data combinations and compared the results with those of a quadratic discriminant function analysis (QDFA). Matenals and Methods PATIENTS The data used in this investigation were gathered from 250 patients admitted to the coronary care unit at Svendborg Hospital with acute chest pain suggestive of myocardial infarction within the last 10 h. The procedures followed were in accordance with the Helsinki Declaration of 1975, as revised in Our study group consisted of 125 consecutive patients with confirmed AM! and of 125 consecutive patients for whom the diagnosis of AJ vii was rejected. Thus the prevalence of AMI in the study group was The prevalence of AMI in the group of patients admitted to the coronary care unit was Patients with an insufficient number of laboratory data were excluded from the study. The diagnosis of AM! was based on the WHO diagnostic criteria for AMI [7]: typical history, and unequivocal changes in EGG and (or) in cardiospecific isoenzymes. 613

2 614 Pedersen et al.: Neural networks to diagnose acute myocardial infarction H DATA The data used in the analysis were nine deviations of the EGG; values for K, GKB, and LD1 obtained immediately after admission; and GKB and LD1 at 0800 and CKB at 2000 for the next 24 h. The nine deviations of the EGG were ST segment in leads I, II, III, and V1-V6. Only values numerically 0.1 mv in leads I, II, I!!, and V4-V6 and 0.2 mv in V1 -V3 were recorded; smaller values were set to 0. Serum [KI was analyzed with ion-selective electrodes. CKB was determined at 37 #{176}C as residual activity after specific immunoinhibition of all CK M-subunit activity [8] with reagents from Roche Diagnostic Systems (Nutley, NJ). LDI was determined at 37 #{176}G by the recommended method [9] after immunoprecipitation of all M-subunit-containing LD isoenzymes with the Isomune-LD reagent from Roche. All analyses were performed with use of a Model 761 Monarch 2000 analyzer (Instrumentation Laboratory, Warrington, UK) TRAINING AND TEST DATA A training set of 100 examples was formed by randomly selecting 50 cases of AM! and 50 cases of non-ami. The remaining 150 cases were used as the test set, also with equal numbers of AM! and non-am!. In the AMI group used for training, 7 patients had bundle branch block (BBB) and 8 had normal EGG at admission. In the non-am! group used for training, 7 patients had BBB and 25 had normal EGG at admission. In the AMI group used for testing, 8 patients had BBB and 14 had normal EGG; in the non-am! group, 8 patients had BBB and 39 had normal EGG at admission. NEURAL NETWORKS The neural networks used were designed and optimally trained as described in the previous paper [6]: We used feed-forward networks with the number of input neurons equal to the number of clinical variables (1-15), one hidden layer, and a single output neuron. Before training the neural network, the laboratory data were scaled to be of order 1 [6]. The network was trained with the intention of giving an output of +1 for AMI and -1 for non-am!. The actual output can, however, be any number between -1 and +1, a positive output being interpreted as AM! and a negative output as non-am!. TEST EVALUATION The results of the outputs from the network were classified as follows: true positive if the result was positive for an AMI patient, false positive if the result was positive for a non-ami patient, true negative if a negative output was found for a non-ami patient, and false negative if a negative output occurred for an AMI patient. The performance of a network was described by its sensitivity, specificity, positive and negative predictive value, and diagnostic efficiency. In accordance with usual practice [10, 11], these quantities were defined as follows: Sensitivity, the fraction of AM! patients with a positive test result; an indication of the probability that the network would produce the correct diagnosis for an AM! patient. Specificity, the fraction of non-am! patients with a negative test result; an indication of the probability that the network would produce the correct diagnosis for patients who do not have AMI. Positive predictive value, the fraction of test-positive patients who have AM!; an indication of the probability that A1 vh is actually present if the test result produced by the network is positive. Negative predictive value, the fraction of test-negative patients who do not have AfvH; an indication of the probability that AM! is not actually present if the test result produced by the network is negative. Diagnostic efficiency (of a specific neural network for the diagnosis of AM!), the fraction of all test results that are true predictions (positive or negative). We express efficiency as a percentage; i.e., our displayed values are the fractional values multiplied by 100. Both the predictive value and the diagnostic efficiency of a test depend on the prevalence of the disease in the group NEURAL investigated. NETWORKS Results anddiscussion The training and test results of several neural networks used to diagnose AMI are displayed in Table 1. The results (in %) are presented as the number of true positives, false positives, true negatives, and false negatives of the examples shown to the network. The different neural networks correspond to different combinations of laboratory data, both at admission and within 12 h and 24 h after admission. The diagnostic performance of the networks with the test data is displayed in Table 2. At admission.the following data were available at admission: EGG, K, GKB, and LDI. Table 1 shows that, depending on the network, the training success was 70-97%; i.e., the network was able to learn the correct diagnosis in of the 100 training examples shown to it. The corresponding generalization performance (diagnostic efficiency) was 64-81%; i.e., the network correctly classified of the 150 test examples (Table 2). The performance of the network trained on EGG data alone is discussed in some detail, given that EGG still provides the most convenient and reliable method of early diagnosis in patients with chest pain [12]. Using EGG data alone, the network had a training success of 87% (see Table 1). Among the 13 patients that the network was not able to learn (all false negatives), 4 had nonspecific findings (e.g., borderline ST segment deviation in only one lead or in two leads unconnected to each other), and 8 patients had normal EGG patterns. Only 1 of the 13 had an EGG consistent with possible AMI (0.2 mv ST segment elevation in leads V4-V6), but the output of the network was -0.01, indicating that the non-ami diagnosis predicted by the network was extremely uncertain. Eliminating the EGG data sets with no suspicion of AM! resulted in a training success of 99%. However, this did not improve the diagnostic performance. Table 2 shows that the network trained only on EGG data had a diagnostic efficiency of 71%; i.e., 107 of the 150 test

3 Clinical Chemistry 42, No. 4, Table 1. GeneralIzation and training performance of neural networks used to diagnose AM1. True positive False positive True negative False negative Laboratorydata G T T Admission data CKB(l) K + CKB ECG ECG + K ECG + CKB LD CKB Data at 12 h CKB(lI) CKB Data at 24 h 3CKB CKB + 2LD1 a Results (in %) on 150 test examples, 75 with AMI and 75 without AMI. Results (in %) on the 100 training examples, 50 with AMI and 50 without AMI. examples were correctly classified. The network misclassified 43 Table 2. Diagnostic performance (%) of neural networks for patients: 31 false negatives and 12 false positives. Of the AMI8. false-negative patients, 14 had normal EGG, 4 had nonspecific Sensi- SPed- EGG, and 13 had EGG patterns consistent with possible AMI, Laboratory data tlvlty ficity PV+ PV- Efficiency Admission data including 3 with BBB. Of the false positives, all had EGG CKB(l) patterns consistent with possible AMI-7 due to BBB, 3 due to K + CKB earlier AM!, and 2 due to other causes not compatible with ANtI Our results showed, not unexpectedly, that the network was ECG unable to provide the correct diagnosis for AM! patients with ECG + K normal or nonspecific EGG patterns. The sensitivity of the ECG + K* network for patients without these EGG patterns was 77%. LD1 Furthermore, in accordance with clinical findings [13], we found ECG + CKB that the network was not very efficient at providing the correct diagnosis for EGG data with BBB (only 6 of 16 were correctly classified). However, the ability of the network to correctly CKB diagnose patients is as good as or even better than that obtained by physicians using EGG at admission, where an estimated Data at 12 h 20-50% of diagnoses of ANtI require biochemical confirmation [13-15], usually by measurement of cardiac enzymes on consec- 2 utive days after admission. CKB(ll) If, in addition to EGG, the values for K* and GKB were 2CKB Data at 24 h successively included, the efficiency of the networks was in- 3CKB creased to 75% and 81%, respectively (Table 2). The increase 99 gg gg 99 gg attributable to K is due to the concomitant decrease in the 3CKB + number of both false negatives and false positives, which sug- 2LD1 gests that K could have a diagnostic effect as an early marker of a Results on 150 test examples, 75 with AMI and 75 without AMI (in %). AM!. The large increase from including GKB is due mainly to PV+, positive predictive value; PV-, negative predictive value, a substantial reduction in the number of false negatives. As

4 616 Pedersen et al.: Neural networks to diagnose acute myocardial infarction II shown in Table 2, none of the other combinations of data available at admission had a diagnostic efficiency better than the combination of EGG, K, and GKB data. In particular, adding LD 1, a late enzyme marker of AM!, to this combination did not improve the diagnostic efficiency. A previously published application [5] of neural networks trained on EGG and clinical data (total of 20 variables) was tested on 36 patients with AM! and 295 without AMI. That network had a sensitivity of 97% and a specificity of 96%. Gomparison with our results is difficult because of the different EGG data recorded, and particularly because of the small number of patients with AM! in that study, but apparently the performance may be improved by including the clinical data. The 12-h horizon.within the first 12 h after admission, the following data were available: EGG, K, LD1, and two GKB values. Using only the two consecutive values of GKB, the training success of the neural network was 96% and its diagnostic efficiency was 97%. As Table 2 shows, the efficiencies of the networks that used only the two consecutive CKB values or just the second GKB value were almost identical to that obtained by the network that used all of the available data. The neural network trained on two consecutive GKB values misclassified 5 patients (3%)-3 (false positives) with GKB values 12 U/L (including 1 patient with falsely high GKB values because of macro CKBB), and 2 (false negatives) with GKB values 9 UIL, i.e., within normal limits. The 24-h horizon. Within the first 24 h after admission the following data were available: all data mentioned at 12 h plus one more value for GKB and one more for LD 1. Using the three consecutive GKB values and no other data produced a training success of 100%. However, the diagnostic efficiency of the combination of three consecutive GKB measurements was not significantly different from that obtained with all data available (Table 2). The neural network trained on three consecutive GKB values misclassified only three patients (2%). One patient, with macro CKBB, was falsely positive. Two patients were false negatives: one with three normal GKB values (8 U/L), and one with an initial GKB value of 12 U/L but no further increase in the subsequent GKB values (probably because that patient received thrombolytic treatment). Gompared with admission data alone, two GKI3 values within 12 h or three GKB values within 24 h significantly increased the diagnostic efficiency of the network-to 97% and 98%, respectively. As expected, neither the network nor the physician could correctly classify patients with macro GKBB or patients with insignificant cardiac enzyme release within the time interval tested. In actual practice the physicians used a GKB value of 20 U/L to discriminate between the presence and absence of AM!. With this discrimination limit, serial GKB values within the 12-h and 24-h horizon yielded a diagnostic efficiency of 95% and 98%, respectively, without a neural network. Identical results were obtained with and without a neural network within 24 h. Within 12 h, the results were slightly better with a neural network than without. Interestingly, for diagnoses of AM! confirmed or rejected by all available data, the network trained on two GKB values found a discrimination limit for the second GKB of 12 U/L, i.e., considerably lower than the value used by the physicians; the value of the first GKB appeared to be of no importance. With use of this lower discrimination limit, the diagnostic efficiency increased to 97% without a neural network. Thus a neural network may be used to determine the optimal discrimination limit for an input variable. In another study [4], a neural network was trained on paired sets of 10 cardiac enzymes and the time interval (48 h) between the measurements, i.e., a total of 21 input variables. The network was tested with only a few patients, 9-2 2, and had a sensitivity of % and a specificity of 33-93%, depending on the method used for diagnosing Alvll. However, in the present work we have shown that laboratory data obtained within 12 h after admission can themselves yield a higher performance than that obtained from cardiac enzyme values available within 48 h. The present investigation of the diagnostic value of various combinations of laboratory data is just one method of examining clinical variables. A different method, also using neural networks, has previously been published by Baxt [16]. DISCRIMINANT FUNCTION ANALYSIS AND PRINCIPAL COMPONENTS ANALYSIS One of the traditional methods used to investigate the diagnostic value of a set of predictor variables is discriminant function analysis (DFA). Both linear (L) and quadratic (Q) DFA were applied to the above combinations of laboratory data. As in our previous report [6], LDFA gave inferior results, which we do not include here. On the other hand, QDFA gave results similar to that of neural networks, both for the training set and the test set, for the majority of data combinations (see Table 3). However, for data combinations with several different variables, QDFA had a lower performance than the neural networks. Gorrelation between variables can be eliminated by the use of principal components [6]. The results in the last two columns of Table 3 are optimal values found by using a number of principal components that span % of the variance (see [6]). Except for two cases, the neural networks performed slightly better when trained on principal components than on the original variables. These exceptions were the two largest data combinations of multiple variables, with one strong indicator [GKB(II)] and several weaker indicators of AM!. QDFA performed rather poorly on these two combinations, and the results of QDFA trained on all principal components were identical to those obtained by using the original data. OTHER MARKERS The approach presented in this and the companion paper can be applied to any choice of markers. Recently, several new biochemical markers for the diagnosis of AM! have appeared, including GKMB mass concentration, glycogen phosphorylase BB, and the more heart-specific markers cardiac troponin T and I. It will be interesting to use the techniques to investigate the

5 Clinical Chemistiy 42, No. 4, Table 3. ComparIson of performances of neurai networks (NN) and QDFA. Laboratory Admission CKB(l) K + CKB ECG ECG + K* ECG + CKB data data LD1 ECG + K* + CKB Data at 12 h 2CKB + LOl CKB(ll) 2CKB Data at 24 h 3CKB 3CKB + 2LD1 PIN raw 65/70 66/7 1 70/72 71/87 75/88 79/92 78/93 81/97 81/93 96/98 96/96 97/96 98/99 99/100 Generalization/training, % PIN PCAb 65/70 69/73 71/73 76/83 78/85 80/93 85/95 83/97 83/95 91/98 96/96 99/97 97/100 95/100 Neural network trained on the raw data. b Neural network trained on the principal components. QDFA trained on the principal components. QDFA 67/72 67/75 67/73 76/82 7 7/82 79/91 80/88 diagnostic performance of these markers, alone and in combination with other markers. In conclusion, in investigating the diagnostic values of different combinations of laboratory data by using neural networks trained on the raw data, neural networks trained on principal components, and DFA, we determined that (a) LDFA was not applicable; (b) in general, the highest diagnostic performance was obtained by neural networks trained on principal components; and (c) the performance of QDFA was similar to that of the neural networks when applied to a small number of laboratory data but was lower otherwise. A peculiar behavior was observed for the two largest combinations of laboratory data, which included one very strong and several weaker indicators of AMI. Only the neural network trained on the raw data had the expected high performance; the neural networks trained on principal components had a 5% lower performance; and QDFA had > 10% lower performance. The best diagnostic efficiency for AM! at admission was 85%, obtained by neural networks using a combination of EGG, K*, and LD1 data. If all three kinds of data were not available, the most cost-effective data were EGG alone or the EGG and K combination, for which the respective efficiencies were 76% and 78%. Using only two CKB values-at admission and within 12 h afterwards-had a diagnostic efficiency of 99%, which decreased only slightly (to 96%) when only the second GKB value was used. The increased performance showed up as reductions of both false positives and false negatives. The few patients who were incorrectly diagnosed by the neural network had atypical data, and none was correctly diagnosed by the physicians at the time when only these same data were available. We thank Fyns Amt for financial support. J.Sj. is grateful to the Danish Natural Science Research Gouncil for a fellowship. References 79/91 1. Simoons ML, Serruys PW, Van den Brand M, Res i, Verheugt FW, Krauss XH, et al. Early thrombolysis in acute myocardial infarction: 77/90 limitation of infarct size and improved survival. J Am Coil Cardiol 1986;7: Gruppo ltaliano per lo studio della streptochinasi nell infarto 83/92 miocardlo. Effectiveness of intravenous thrombolytic treatment in acute myocardial infarction. Lancet 1986;i: /95 3. Second international study of infarct survival. Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among cases of suspected myocardial infarction. Lancet 1988; ii: / Furlong iw, Dupuy ME, Heinsimer JA. Neural network analysis of 89/98 serial cardiac enzyme data. A clinical application of artificial machine intelligence. Am J Clin Pathol 1991;96: Baxt WG. Use of an artificial neural network for the diagnosis of myocardial infarction. Ann Intern Med 1991;115: J#{248}rgensen is, Pedersen JB, Pedersen SM. Use of neural networks to diagnose acute myocardial infarction. I. Methodology. Clin Chem 1996;42: World Health Organization, Regional Office for Europe. lschaemic heart disease registers: report of the fifth working group. Copenhagen: WHO, 1971:54 pp. 8. Gerhardt W, Waldenstr#{246}m J. Creatine kinase B-subunit activity in serum after immunoinhibition of M-subunit activity. Clin Chem 1979;25:1274-8O. 9. Scandinavian Committee on Enzymes. Recommended methods for determination of four enzymes in blood. Scand I Clin Lab Invest 1974;33: Gerhardt W, Keller H. Evaluation of test data from clinical studies. Scand I Clin Lab Invest 1986;46(Suppl 181): Werner M, Brooks HS, Wette R. Strategy for costeffective laboratory testing. Hum Pathol 1973;4: Timmis AD. Early diagnosis of acute myocardial infarction. Electrocardiography is still best. Br Med I 199O;3O1: Zarling EJ, Sexton H, Milnor P. Failure to diagnose acute myocardial infarction. JAMA 1983;25O: McQueen M, Holder D. El-Maraghi N. Assessment of the accuracy of serial electrocardiography in the diagnosis of acute myocardial infarction. Am Heart I 1983;1O5: Vusuf S, Pearson M, Sterry H, Parish S, Ramsdale D, Rossi P, Sleight P. The entry ECG in the early diagnosis and prognostic stratification of patients with suspected acute myocardial infarction. Eur Heart I 1984;5: Baxt WG. Analysis of the clinical variables driving decision in an artificial neural network trained to identify the presence of myocardial infarction. Ann Emerg Med 1992;21: